Benchmarks

Reproducible measurements from crucible-bench, a harness that drives a real daemon through the sdk Go package (the same typed client the CLI, TUI, and MCP server use). It reports latency distributions, fork fan-out scaling, the lazy-memory efficiency of fork, and sandbox density.

Read these as one machine's numbers, not a spec. They come from a single host, one sandbox size — enough to show the shape of the runtime, not a cross-hardware guarantee. Everything you need to re-run is at the bottom.

The filesystem under --work-base is the biggest lever

Fork clones the per-child rootfs copy-on-write. On a reflink filesystem (btrfs / XFS reflink=1) that clone is O(1) — extents are shared, nothing is copied. On ext4 (no reflink — still the most common default) it's a full byte-copy of the rootfs. Same box, same daemon, same 1 GiB rootfs; only --work-base/--chroot-base differ:

If you fork a lot, put --work-base on btrfs or XFS. The rest of this page reports both, so you can see the floor (ext4) and the ceiling (reflink).

Setup

Latency

Fork is ~9× faster than a cold create even on ext4, and ~3× faster again on reflink. That's the point of snapshot/fork: pay the ~2 s boot + setup once, then branch cheaply. Exec overhead is ~3 ms (a vsock roundtrip) and is filesystem-independent. Snapshot's btrfs p50 is ~0.4 s with occasional multi-second tails from writing the 512 MiB memory image through writeback; the median is representative.

Fork fan-out

Forking N children from one snapshot in a single call. Per-child cost falls with batch size as fixed per-fork overhead amortizes — and reflink pulls far ahead as N grows:

(On ext4 each child copies a full 1 GiB rootfs, so a 64-way fan-out moves ~64 GiB of disk — the throughput ceiling is the disk, not the runtime.)

Memory efficiency

The lazy-userfaultfd payoff: guest RAM is served on demand from the snapshot's memory file, so forks share pages instead of each copying 512 MiB. Forking 128 children from one warm snapshot:

Both obliterate the naïve cost. reflink pulls ahead because ext4's per-fork rootfs copies add page-cache pressure that reflink avoids. The win also grows with fork count — the more children share a snapshot's pages, the lower the marginal cost per fork.

Density (reflink)

Forking toward a live-sandbox target on btrfs and watching free RAM:

512 concurrent microVMs on a laptop, with 7.4 GiB headroom left — 512 was the test's cap, not a ceiling. On reflink, density is RAM-bound (~14 MiB marginal per fork). On ext4 it's disk-bound instead — each fork writes a full ~1 GiB rootfs — so plan disk accordingly, or use reflink.

Reproduce

make bench                                   # builds ./bin/crucible-bench

# a daemon under jailer; raise the fork cap; point --work-base at your target FS:
CRUCIBLE_MAX_FORK=128 sudo ./crucible daemon \
  --firecracker-bin … --jailer-bin … --kernel … --rootfs … \
  --work-base /path/to/work --chroot-base /path/to/jailer

./bin/crucible-bench \
  --samples 50 --fanout 1,4,16,64,128 --mem-forks 128 --density 512 \
  --json bench-results.json

For the ext4 numbers, point --work-base/--chroot-base at an ext4 path; for the reflink numbers, at a btrfs or XFS (reflink=1) mount. crucible-bench --help lists every knob (sandbox size, phases, profile).