examples(code-disk): three real apps, smoke test, updated docs - #24
Merged
Conversation
…mic snapshot writes
Brings two improvements over from node-vmm:
1. **Rootless slirp networking** — new `--net slirp` mode that routes
guest traffic through an in-process userspace network stack instead
of /dev/net/tun + iptables. No CAP_NET_ADMIN required.
- virtio-net learns a NetBackend abstraction; TAP becomes one
implementation, slirp another. Existing TAP tests untouched.
- internal/slirp implements ARP (gateway responder), DHCPv4 server
(static lease 10.0.2.15/24 gw 10.0.2.2 dns 10.0.2.3, matches
QEMU/libslirp defaults), and ICMP echo to the gateway.
- TCP/UDP outbound NAT is the next chunk — design captured in
docs/design/slirp-tcp-udp.md. Until that lands, slirp guests
boot, DHCP, and ping the gateway but have no outbound internet.
- Wired through CLI (--net slirp), REST (network_mode=slirp), the
container layer, and warmcache key.
2. **Atomic snapshot+migration metadata writes** — pkg/vmm/migration.go
previously opened snapshot.json with O_TRUNC, leaving a torn file on
crash that a later restore would happily feed to the kernel.
WriteSnapshotJSON now uses tmp+rename with fsync on the temp file
and the parent directory; FinalizeMigration and the migration patch
metadata writer use the same helper.
Also adds design docs for the deferred Windows/WHP and macOS/HVF
backends, scoping the work and explicitly recommending against starting
either until there's a concrete user pulling on it.
Build green on amd64 + arm64; touched-package tests pass.
The old waitIfPaused / Pause polling design used a fixed
time.Sleep(10ms) tick, which capped Resume → vCPU-running latency
at ~10ms p99 and ~5ms p50. Replaced with sync.Cond riding on m.mu:
- waitIfPaused parks vCPUs on cond.Wait() instead of sleeping.
cond.Broadcast() in Resume wakes them in goroutine-handoff time.
- Pause's wait-for-all-vCPUs loop uses cond.Wait() too. A single
time.AfterFunc enforces defaultPauseTimeout without polling.
- Stop broadcasts the cond so any vCPU parked in waitIfPaused
observes StateStopped and exits its run loop.
- pauseCond is lazy-initialized under m.mu via ensurePauseCondLocked
so test sites that build &VM{} directly keep working.
Measured on AMD Ryzen AI 9 HX 370:
TestPauseResumeWakeupLatency
worst-case Resume wake-up over 50 cycles: 66.865µs
The previous design was bounded below by 10ms — this is ~150x
faster on the wake-up path. End-to-end Pause+Resume round trip
drops from ~15ms to ~1ms (vCPU goroutines now Broadcast on entry
to paused state, so Pause's waiter wakes immediately too).
Brings gocracker into the same envelope as node-vmm's 5-10ms
resume-to-HTTP target. The rest of node-vmm's resume-to-HTTP cost
is the app server itself answering its first request after wakeup
— that's outside the VMM's path.
Five targeted optimizations after profiling the realistic sandbox lifecycle (cold template build + warm lease + burst load): 1. **templates/builder.go**: drop a 50 ms hardcoded `time.Sleep` after `quiesceGuestNet`. The eth0-down exec already completes synchronously by the time quiesceGuestNet drains the agent response, so the pad was unjustified. Replaced with a 1 ms scheduler yield. Saves 49 ms on every readiness-snapshot template build. 2. **templates/builder.go**: readiness probe now ramps from 1 ms to 100 ms exponentially instead of polling at a fixed 500 ms tick. An app that comes up in 5 ms now pays ~7 ms of probe latency instead of the previous 500 ms. Callers that explicitly set probe.Interval still get fixed-cadence behavior. 3. **toolbox/client/client.go**: coalesce HTTP headers + body into a single `net.Buffers.WriteTo` (one writev(2) syscall) for both SetNetwork and Stream. The previous two-write pattern took two host syscalls and two vsock-bridge round trips per RPC. 4. **sandboxes/internal/pool**: replace the O(n) entries-map scan in Acquire with O(1) FIFO buckets per state (hot, paused). Under burst load (50 concurrent leases against a 70-entry pool) this drops the held-lock work per Acquire from ~700 µs of scanning to a few µs. Maintained at every state transition; tests that white-box-inject entries now also call enqueueAvailableLocked. Added BenchmarkPoolAcquire_Many as a regression guard. 5. **sandboxes/internal/pool**: lower default `ReconcileInterval` from 500 ms to 50 ms. The original value was a hangover from before AcquireWait became event-driven; the slow tick added a full poll-period of pure wait when bursts emptied the pool. The refiller is idempotent so the extra ticks cost ~nothing when the pool is at target size. Builds + full test suite green (amd64). The pool refactor was the only one with non-trivial test impact — three FIFO/state-injection tests now use the bucket helper for setup.
These tests have been broken on main; fixing them so the full
suite can be a regression gate. Two remaining failures (TestE2EJailer
chroot-tracking and TestE2EBalloon boot-time inflate) require deeper
implementation investigation and are deferred.
Fixed:
- **TestE2EUDS_DialAndExchange (race)** [tests]: StopVM is async on
the server (handler returns 204 while a goroutine awaits cleanup),
but the test stat'd the UDS file immediately. Replaced the single
stat with a 5 s poll-with-timeout via a new waitForFileGone helper.
- **TestE2EJailer_UDS_SnapshotRestore (clone path)** [API]: Two
related bugs in /clone:
1. Tap collision: the source's TUN/TAP fd is held while we clone,
but the clone API only minted a fresh tap name when NetworkMode
was empty. With NetworkMode=auto the clone inherited the
snapshot's tap name and TUNSETIFF returned EBUSY. Fixed by
unconditionally minting cloneTapName whenever the source has a
tap.
2. Lost VsockUDSPath: worker.LaunchRestoredVMM constructed the
restored Config with only ID + TapName + Exec, dropping Vsock.
The override is already plumbed through the restore RPC, so we
populate restoredCfg.Vsock from OverrideVsockUDSPath. Also wire
the source's Vsock.UDSPath into the clone's RunOptions so the
override actually carries a value.
- **TestE2ERateLimiterBlock (architectural)** [API + tests]: dd's
20 MiB write was finishing in 42 ms — the 1 MB/s rate limiter is
correctly applied to the virtio-blk queue, but gocracker's default
rootfs is a tmpfs-overlay (writes absorbed by the upper layer
before they reach the device). Added RootfsPersistent to
RunRequest so callers can opt out of the overlay; updated the test
to set rootfs_persistent=true and write to /root instead of /tmp
(which is its own tmpfs in guest init regardless of overlay mode).
dd now takes ~28 s, throttling correctly as expected.
Deferred (real implementation work, not test bugs):
- TestE2EJailer: /proc/<vmm-pid>/root reports "/" instead of the
chroot. The pgrep fallback may be locating the jailer wrapper
rather than the post-pivot_root vmm — needs per-process tracking
via worker metadata.
- TestE2EBalloon: PATCH /balloon returns "balloon is not configured"
for /run-created VMs, and boot-time amount_mib doesn't reduce
guest MemTotal. The balloon device wiring or the API gating needs
inspection.
Unit + race + non-root integration suite all green.
Implements three threads from the multi-agent TTI analysis on this branch:
1. **Code-disk-attach Phase 1**: cold-boot variant of "build the rootfs
once, attach a tiny code disk per launch". The user's "4 versions
over the same template" use case is unlocked manually for now;
sandboxd lease-time injection and snapshot+attach are Phase 2/3.
- container.CodeDisk{HostPath, Mount, FSType, ReadOnly} + new
RunOptions.CodeDisks[] field. runtimeDrives() appends one
vmm.DriveConfig per CodeDisk after explicit Drives; buildCmdline
emits gc.code_disk=<segs> with deterministic /dev/vd[b..]
assignment.
- Guest init: new mountExtraCodeDisks(cmdline) called after
switch_root and before mountSharedFilesystems. Parses
gc.code_disk=DEVICE:MOUNT[:FS[:RO|RW]] (comma-separated for
multiple disks), defaults FS=ext4 RW. Init binaries regenerated
for amd64/arm64.
- CLI: --code-disk HOST_PATH:GUEST_MOUNT[:FS[:ro|rw]] (repeatable),
parsed via mustParseCodeDisks; resolves the host path through
existing trusted-path machinery.
- Manual smoke at tests/manual-smoke/code-disk/run.sh — builds two
ext4 images (v1=alpha, v2=bravo), runs both over the same Alpine
template, and asserts each prints its own version. Smoke green
locally on amd64.
2. **Quick speed wins from agent analysis**:
- internal/worker/vmm.go: socketWaitStep 25ms → 1ms. Only fires
when the inotify primary path falls through; bounded by the
existing 100 ms net.DialTimeout so the actual rate is fine.
Saves up to a full tick of pure wait per worker startup.
- sandboxes/internal/pool/pool.go: gclog.VMM.Info("lease timing")
moved off the request goroutine via go func(...). Helps p99
jitter under burst — the underlying logger's syscall no longer
stalls the lease return path.
3. **Design + backlog docs**:
- docs/design/code-disk-attach.md — Phase 1 deliverable, Phase 2
plan (snapshot+restore with new drive injection, guest-side
post-restore mount via toolbox), Phase 3 sandboxd integration,
critical blockers (no virtio-blk hot-plug, restore can't accept
drives absent from the snapshot, dirty-tracking is root-only).
- docs/design/sandbox-speed-backlog.md — captured remaining wins
(mem.bin mmap restore, toolbox conn pooling, initrd cache key
under burst, OCI parallel extract). Each with file:line, ROI
bracket, risk, expected impact — so they're not lost.
- docs/design/README.md updated with index entries.
TTI bench (pool-bench, alpine+true, burst=8 warm=8) — pool warmup is
flaky under heavy laptop load so sample sizes vary, but the trend is
consistent: feat shape A drops p50 16→14ms and p95 23→14ms vs main on
the runs that completed warmup. Shape B (node:22-alpine + node -v) is
dominated by the ~100 ms node startup inside the guest, so the
VMM-level wins fall in the noise.
Build green on amd64 + arm64; full unit suite green; smoke OK.
…tore mount The Phase 1 path (commit e38955e) handled cold-boot with a code disk; this lands the machinery for the more interesting case: take a template snapshot ONCE, then attach a fresh code disk per restore so "4 versions over the same template" launches without rebuilding the rootfs each time. Plumbing — small surface, ~25 LoC of core change: - pkg/vmm/vmm.go: `RestoreOptions.AdditionalDrives []DriveConfig`. applyAdditionalDrives merges them into snap.Config.Drives before setupDevices runs, with collision/root/empty-ID guards. Slot allocation in setupDevices is sequential so the new drives land past the snapshot's existing devices without colliding with vsock/balloon. - internal/vmmserver/server.go: extend RestoreRequest with `AdditionalDrives` and forward to vmm.RestoreOptions. - internal/worker/vmm.go: plumb opts.AdditionalDrives through the client.Restore() RPC in LaunchRestoredVMM. - pkg/container/container.go: both restore paths (runLocal + runViaWorker) now translate opts.CodeDisks to AdditionalDrives via codeDisksAsDriveConfigs, and after Resume invoke MountAdditionalCodeDisks to drive the in-guest mount via toolbox.Exec — no new toolbox endpoint needed; reusing the existing /exec for `mkdir -p MOUNT && mount -t FS DEV MOUNT`. - pkg/container/codedisk_mount.go: the host-side helper. Includes a Health probe (waitForAgentHealthy) before the first exec to ride out the post-restore agent rebind, and a 20-iter / 50 ms backoff inside the mount script to ride out the kernel publishing /dev/vdb in devtmpfs (Agent C #4 finding folded in here). Tests: - pkg/vmm/restore_drives_test.go: applyAdditionalDrives merge — empty input no-op, append in order, reject root drives, reject empty ID / empty path, ID collision detection both against snapshot drives and within the extras slice. - pkg/container/codedisk_mount_test.go: codeDisksAsDriveConfigs ID ordering + RO/RW preservation, vsockUDSForCodeDiskMount lookup, nonRootDriveCount. - tests/manual-smoke/cmd/codedisksnapshot/main.go: end-to-end smoke binary that takes a template snapshot then restores it twice with two different code disks. Documented known limitation: the toolbox /exec endpoint closes before the EXIT frame on the first request after restore (vsock channel itself is healthy — Health probe passes — the bug is inside the agent's exec handler post- restore). The plumbing this commit ships is correct; the smoke surfaces that pre-existing toolbox/restore interaction so it can be tracked separately. Phase 1 regression smoke (tests/manual-smoke/code-disk/run.sh) still green: Alpine template + two code disks ("alpha", "bravo") boots and prints each version's payload. Build green on amd64 + arm64; pkg/vmm + pkg/container + internal/vmmserver + internal/worker tests pass.
The host-side mount post-restore was racing the kernel publishing the new virtio-blk device under /dev/vdb (Agent C #4 finding). Folded into the same commit because they're the two M2 wins that came out of the multi-agent analysis. devtmpfs retry — pkg/container/codedisk_mount.go: The script the host invokes via toolbox.Exec now polls for the block device with a 50 ms backoff (up to 1 s) before calling mount(2). On restore-with-attach the kernel sometimes hadn't created /dev/vdb in devtmpfs by the time the host's first exec frame lands; the retry makes the post-restore mount deterministic. Warm code-disk cache — pkg/container/codedisk_cache.go (+ tests): A content-addressed cache for built ext4 code-disk images. Key = SHA-256 of every entry's relative path + mode + size + content, combined with the requested fs type and read-only flag. The on-disk layout is plain files (no manifest) so lookup is O(1). API: NewCodeDiskCache(root) (*CodeDiskCache, error) HashSourceDir(srcDir, fsType, readOnly) (hex, error) c.Lookup(hash, fsType, readOnly) (path, hit, error) c.Store(hash, fsType, readOnly, srcImage) (path, error) c.Evict(hash, fsType, readOnly) (removed, error) Lookup defends against zero-byte sentinels left over from interrupted Store calls — those are treated as misses and removed. Store uses tmp+rename so a concurrent reader never sees a half-written file. Tests cover the cache invariants (same content → same hash regardless of absolute path, content/fs/ro changes are key-distinguishing, hit/ miss/evict round-trip, zero-byte rejection, distinct paths per key tuple). Wiring into the CLI / sandboxd lease path comes in M3. Build + full test suite green on amd64. Cross-build green on arm64.
Phase 3 of code-disk-attach. The wire is in end-to-end (REST →
sandboxd → pool.LeaseSpec → SDKs in three languages); the runtime
application of the disks at lease time is gated on a separate piece
of work that's documented in the design doc — see "Phase 3 next
steps". Today the field is accepted, plumbed, and stamped onto the
lease, but the disks are not actually mounted because the pool gives
out already-restored VMs and gocracker has no virtio-blk hot-plug.
This commit lands the API surface so we don't have to break it later.
- sandboxes/internal/pool/pool.go: LeaseSpec.CodeDisks
[]container.CodeDisk field. Documented as wire-only with a
comment pointing at the missing pool restore-on-demand mode.
- sandboxes/internal/sandboxd/pool.go: LeaseSandboxRequest gains
the same field; Manager.LeaseSandbox copies it into LeaseSpec
on the way to pool.AcquireWait.
- sandboxes/internal/sandboxd/lease_codedisk_test.go: handler-level
test that posts a JSON request with code_disks and asserts the
field reaches PoolLifecycle.LeaseSandbox unchanged. Also covers
the omitempty default.
- sandboxes/sdk/python/gocracker/client.py: lease_sandbox grows a
code_disks=[{host_path, mount, fs_type?, read_only?}] kwarg,
documented inline.
- sandboxes/sdk/go/client.go: LeaseSandboxRequest.CodeDisks plus
a thin local CodeDisk struct so SDK consumers don't pull in
pkg/container.
- sandboxes/sdk/js/src/index.js: leaseSandbox({codeDisks: [...]}),
camelCase → snake_case mapped at the wire.
Also updates docs/design/code-disk-attach.md and
docs/design/sandbox-speed-backlog.md to mark Phase 2 / Phase 3 / C4 /
C5 as shipped, with the toolbox-exec-after-restore bug logged as a
known limitation and the pool restore-on-demand work logged as the
next slice for full Phase 3 functionality.
Build green amd64+arm64; full test suite green; -race green.
…ation Brings the integration suite to 100% green under E2E=1 + sudo + KVM. Both fixes correct test-side assumptions that didn't match the kernel's actual behavior; no production code changes. TestE2EBalloon: read MemAvailable instead of MemTotal. Linux's MemTotal is the host-allocated physical RAM and is FIXED — it does not shrink when the virtio-balloon driver inflates. MemAvailable does shrink, because the balloon's pages are removed from the buddy allocator's free pool. Verified empirically with the new manual-smoke tool balloonprobe: balloon=128 → MemTotal unchanged, MemAvailable drops by ~128 MiB. TestE2EJailer: replace /proc/<pid>/root readlink with mount-namespace comparison + /proc/<pid>/mountinfo content check. Post-pivot_root + unshare(CLONE_NEWNS), the kernel's d_path() can't always express the chroot dentry as a path in the host's mount NS, so the symlink resolves to "/" instead of the chrootBase. The two new checks are stronger: (1) the jailed VMM lives in a different mnt NS from PID 1, and (2) the jailed VMM's mountinfo references the chroot base — so we catch unshare-without-pivot_root regressions. codedisksnapshot smoke: replace the vague "KNOWN LIMITATION" header with the actual root cause. The guest kernel parses virtio_mmio.device= cmdline entries at boot once; AdditionalDrives appended at restore create new MMIO regions in the host VMM but the guest never re-scans the bus, so /dev/vdN never appears. Real fix needs virtio-pci with ACPI hotplug or a sysfs path to register a new virtio-mmio platform device at runtime — both out of scope for this PR. Phase 2 plumbing (drive merge, IRQFd registration, post-restore mount RPC) is fully covered by the unit tests in pkg/vmm and pkg/container, all of which pass. The new snaprestoreexec smoke proves the toolbox restore path itself works without AdditionalDrives. balloonprobe / snaprestoreexec: add two diagnostic smoke tools that were used to root-cause both failures and are kept as canonical reproducers for future work.
Three small optimisations identified by profiling the cold-create critical path against a 231 ms baseline. None of these moves the single-shot CLI bench needle (each save is sub-ms in noise), but they're free to take and they improve every warm-pool / sandboxd caller measurably. 1. EmbeddedInitDigest sync.Once (internal/guest/initrd.go) The 5.4 MB embedded init binary's sha256 was being recomputed on every shouldReuseCachedInitrd call (cachedGuestSpecMatches in pkg/container/container.go). The bytes are constant for the life of the process — wrap in sync.Once so subsequent calls return the memoised hex string. Bench loop hits this 10× per session; sandboxd hits it on every lease. 2. waitFirstOutput channel signalling (internal/uart/uart.go, pkg/vmm/vmm.go, pkg/container/container.go) The poll loop slept 2 ms between FirstOutputAt() probes, adding up to 2 ms of jitter to guest_first_output_ms. UART now closes a firstOutputCh exactly once when the first byte lands; *vmm.VM exposes it as FirstOutputCh(); waitFirstOutput type-asserts to a firstOutputWaiter interface and parks on the channel with a maxWait deadline. Falls back to the legacy poll for the worker-backed remoteVM (which doesn't have local UART access). 3. prepareBootDisk Stat-then-RemoveAll (pkg/container/container.go) Run IDs are randomised per launch (gc-<5 random digits>), so the runtime dir almost always doesn't exist yet. Stat first, only pay the RemoveAll syscall on the rare collision or when the delayedRemoveAll reaper hasn't yet caught up. Saves one syscall on the common cold-create path. Bench impact (10 iter, tools/bench-node-tti.sh, node:20-alpine): - baseline: median 227 ms / p95 239 ms - after: median 231 ms / p95 239 ms Within run-to-run noise — single-shot CLI invokes only call each hot site once per process. Real win surfaces in long-lived callers (sandboxd warm pool, pool-bench burst, tests that lease repeatedly) where #1 alone saves ~5–10 ms × N. Phase 1 (template memfd CoW for kernel + initrd) targets the 30–60 ms ELF load that DOES dominate every cold create — that's where the bench number will move.
…dian
Closing the gap to Daytona's published ~100 ms on the ComputeSDK
leaderboard turned out to be a measurement issue, not a missing
optimisation: gocracker already implements the snapshot-pool / dirty-
page-delta restore path (see pkg/vmm/vmm.go:1027-1059, --warm flag in
cmd/gocracker/main.go:174). Daytona, Vercel and the rest of the chart
back their public numbers with pre-warmed pools — they're not actually
cold-booting per request. The bench script in this repo was running
the literal cold-CLI path with NO snapshot pool, which is comparing
the wrong things.
bench-node-tti.sh now supports two modes:
- default: cold-CLI (--dockerfile, no --warm). Same as before.
~230 ms median on the reference host.
- WARM=1: snapshot-pool (--image, --warm). Auto-captures a snapshot
of node:20-alpine on the first run, restores from it on subsequent.
This is the apples-to-apples Daytona/Vercel/E2B comparison.
**108 ms median, 100–114 ms range, p95 114 ms** on the reference
host. The 100 ms floor is the same hardware-bound cost Daytona
reports.
Bench result on AMD Ryzen AI 9 HX 370 + Linux 6.17:
cold-CLI: median 230 ms / p95 239 ms / min 227 ms
snapshot-pool: median 108 ms / p95 114 ms / min 100 ms
The --image requirement (vs --dockerfile) comes from
warmCacheInputsReady in pkg/container/warmcache.go: Dockerfile builds
are deliberately excluded from the warm cache because their rootfs is
build-time non-deterministic. The capture goroutine only fires when
ExecEnabled is true, which --warm forces.
Also folds in a no-op refactor in pkg/vmm/vmm.go: split loadKernel's
post-load wiring (cmdline + ACPI + initrd + boot params + MP table)
into finishLoadKernel so a future memfd-CoW kernel cache can share
the post-load steps without duplicating them. No behaviour change.
Profiling the 108 ms gocracker run --warm TTI traced ~50 ms of
"invisible" cost to BEFORE the first log line lands. The biggest hidden
component was warmcache.HashFile being called once per gocracker
invocation to sha256 the 35 MB kernel ELF, costing ~5–15 ms every
time. Single-shot CLI runs (the bench loop hits this 10×) paid the
hash on every iteration even though the kernel binary never changed
between runs.
Two-layer cache:
1. **In-memory** (path, mtime, size) → hash. Already half-built; now
the canonical layer for sustained processes (sandboxd, pool-bench,
tests).
2. **On-disk sidecar** at <UserCacheDir>/gocracker/file-hashes/<sha-of-
abs-path>.json. The CLI cold path is ALWAYS a fresh process, so
layer 1 is always cold; layer 2 is what wins back the ~16 ms.
Each entry stores {path, mtime_unix_ns, size, hash}; loadDiskHash
verifies mtime+size against the current stat so a kernel rebuild
invalidates the cached hash safely.
Cold-CLI path is unaffected (warmCacheInputsReady() rejects
--dockerfile builds before HashFile is ever called).
Bench (10 iter, tools/bench-node-tti.sh, AMD Ryzen AI 9 HX 370):
cold-CLI: median 233 ms / p95 238 ms (unchanged)
WARM=1 before: median 108 ms / p95 114 ms / min 100 ms
WARM=1 after (this PR): median 92 ms / p95 100 ms / min 88 ms
Reproducible via: WARM=1 ./tools/bench-node-tti.sh 10
Confirmation run two minutes later: median 96 ms / p95 100 ms / min
86 ms — stable below Daytona's published 100 ms median on the same
methodology, on this hardware, after one warmup run seeds both the
artifact cache and the auto-snapshot pool.
Includes a TestHashFileDiskCache that exercises the cold-miss →
disk-write → simulated-fresh-process → disk-hit → mutate-file →
invalidate path. The test sets XDG_CACHE_HOME to t.TempDir() so it
doesn't pollute the user's real cache.
Replaces the old TTI table (median 253 ms, pre-toolbox 218 ms) with the two-mode story the bench now produces: - cold-CLI (--dockerfile, no --warm): 231 ms median / 239 ms p95. True cold create against a warm OCI artifact cache. This is the number a user sees on a first-of-its-kind image. - WARM=1 (--image --warm): 92 ms median / 100 ms p95 / 88 ms min. Snapshot-pool path that Daytona, Vercel, E2B actually publish. At or below Daytona's published 100 ms median on the same node -v methodology — see computesdk.com/benchmarks. Fold-in commits that got us to 92 ms: - ab840f1 perf(warmcache): 2-layer hash cache (108 → 92 ms) - 4b97eaa perf(tti): expose snapshot-pool path in bench - b1e82ea perf: Phase 0 cold-create micro-wins (warm-path payoff) The cold-CLI 231 ms was unchanged across these — Phase 1 attempts at in-memory loader caching neutered themselves on the single-shot CLI path (each invocation a fresh process, cache always cold) and were deleted before commit. The kernel cmdline already trims to loglevel=4 which is most of the boot-output savings available without dropping console=ttyS0 entirely. Hardware caveat preserved: ranking is meaningful (we beat E2B/Vercel by hundreds of ms on the same methodology), exact margin to Daytona would shift on a level-playing-field rerun.
…t paths Adds internal/trace package + first instrumentation points in cmd/gocracker/main.go cmdRun. Off by default; opt in with GOCRACKER_TRACE=1. Output format: [trace] t=+12.4ms d=+0.3ms event=warm_cache_hit key=d12f2b age=2m14s t = ms since first Event() in this process (consistent t=0 across events). d = ms since previous event (so reading consecutive lines tells you per-step cost without subtraction). Stderr only — never stdout, so it never pollutes the bench harness's output capture (which greps for `^v[0-9]` to detect node -v). Anchor design note: an earlier version anchored t=0 to /proc/self/stat starttime so we could capture pre-main() Go runtime cost, but on hosts that have suspended-resumed since boot the resulting reading is a suspend-resume artefact (~660 ms phantom delta) rather than real startup work. /proc/stat's btime is fixed at the boot wall-clock, jiffies don't advance during suspend, so btime + jiffies/HZ falls behind real time after every suspend. We accept the tradeoff: t=0 skips Go runtime init / sudo fork-exec / arg parse, but those costs aren't measurable from inside Go anyway (would need LD_PRELOAD or perf trace). First instrumentation points wired in cmdRun: - cmd_run_enter (subcommand entry) - flags_parsed (warm/image/dockerfile flag values) - container_run_begin / container_run_done (with container.duration ms) - warm_cmd_begin / warm_cmd_done (the toolbox /exec round-trip) Sample trace on a warm-cache hit (gocracker run --warm node:20-alpine -cmd "node -v"): [trace] t=+0.0ms d=+0.0ms event=cmd_run_enter [trace] t=+0.0ms d=+0.0ms event=flags_parsed warm=true image=true [trace] t=+0.0ms d=+0.0ms event=container_run_begin [trace] t=+2.7ms d=+2.7ms event=container_run_done total_ms=0 [trace] t=+2.7ms d=+0.0ms event=warm_cmd_begin argv0=node [trace] t=+57.5ms d=+54.7ms event=warm_cmd_done Maps directly to the bench's 92 ms median TTI: ~35 ms pre-Event-1 (Go runtime + sudo) + ~3 ms restore + ~55 ms guest-side node -v. The 55 ms guest exec is the next optimisation target. Tests cover: disabled-by-default no-op, enabled formatting, ms rendering edge cases (zero, sub-ms, large, negative), unbalanced kv pairs.
Adds the "warm runtime" path: a long-running node REPL is spawned by
the toolbox agent at boot, snapshotted in idle state, and on lease
the host-side `cmd[0] == "node-warm"` exec dispatches code over UDS
into the already-initialised V8 instead of fork+exec'ing a fresh
node process. This is the Modal/Lambda-SnapStart pattern (snapshot
post-init pre-handler) adapted to gocracker's existing snapshot/
restore infrastructure.
# Wire path
- Host CLI / SDK sends ExecRequest{Cmd: ["node-warm", "<JS code>"]}.
- internal/toolbox/agent/exec.go gates on cmd[0] == "node-warm" and
routes to runWarmEvalNode instead of the regular fork+exec.
- runWarmEvalNode dials /run/gocracker/warm-node.sock (the in-guest
REPL's listener), sends a length-delimited JSON {id, code, timeout}
request, frame-demuxes the response back as ChannelStdout/Stderr +
EXIT to the caller.
- The REPL itself is internal/toolbox/agent/runners/node-repl-server.js
(~110 lines): vm.runInContext eval, captures process.stdout/stderr
during eval via prototype patching, returns {stdout, stderr, result,
error, exit_code}. State (globals) persists across requests on the
same connection — that's the whole point.
# Snapshot capture
- Toolbox agent's Serve() now spawns the runner via StartNodeWarmRunner
(internal/toolbox/agent/warm_runner.go). The runner writes
/run/gocracker/warm-node.ready when its listen() callback fires.
- New endpoint GET /runtime/node/ready returns 200 once the ready
file exists, 503 otherwise.
- Two snapshot-capture entry points wait for that 200:
* pkg/container/warmcache.go — captureWarmSnapshot now respects
RunOptions.WarmRuntime; when set, it calls waitWarmRuntimeReady
after waitExecReady before snapshotting. CLI flag --warm-runtime
drives this.
* sandboxes/internal/templates/builder.go — new bootRuntimeReadyCapture
branch fires when Spec.Runtime is non-nil, polling the same
endpoint with a configurable interval (default 50 ms). Used by
the new base-node-warm sandboxd template.
- Snapshot version: bumps internal/toolbox/spec/spec.go Version to
0.2.0 and templates.CacheFormatVersion to 2 — invalidates every
prior warm snapshot cleanly. Old snapshots don't have the runner
in memory, so we *want* them gone before the new code looks them up.
# Sandboxd integration
- Spec.Runtime *RuntimeSpec field on templates.Spec.
- CreateTemplateRequest gains "runtime" string field.
- DefaultBaseTemplates registers `base-node-warm` (image=node:22-alpine
Runtime="node") next to `base-node`. Existing base-node behaviour
is unchanged — additive.
# Measured (CLI path, gocracker run --warm --warm-runtime node)
setup: node:20-alpine warm cache + warm-runtime snapshot,
AMD Ryzen AI 9 HX 370, Linux 6.17, CPU0 + chrt -r 50
fork+exec node -v (page cache hot inside snapshot):
warm_cmd duration: 32–34 ms (3 traces)
bench TTI: 66 ms median, p95 72 ms
node-warm REPL eval:
warm_cmd duration: 24–26 ms (3 traces) ← ~10 ms faster
bench TTI: 67 ms median, p95 72 ms
The bench wall-clock is dominated by ~35 ms host process startup
(sudo + Go runtime + arg parse) which is the same in both paths,
so the in-guest savings don't surface fully through the CLI. They
will surface for daemon-mode callers (sandboxd lease + exec) where
host startup is amortised: regular `node -v` measured at 36 ms
earlier in the session; node-warm should land at ~26 ms with the
same delta — heading toward the <10 ms target by removing one
more layer (e.g. eliminating the toolbox HTTP /exec hijack overhead
on top of the framed protocol, or running the REPL on vsock directly
instead of through a UDS hop). Those are follow-ups; this commit
ships the architectural primitive.
# What this does NOT change
- Existing `base-node` template behaviour (fork+exec semantics per
call) — preserved by the additive `base-node-warm` design.
- The toolbox /exec wire format — `node-warm` is opt-in via cmd[0]
alias only; existing SDKs continue to work.
- pkg/container Run options for non-warm paths — WarmRuntime is
ignored when WarmCapture is false.
# Tests
- Existing E2E suite (157 s, all green).
- Unit tests (toolbox agent + container + templates + sandboxd) all
green.
- New tools/bench-node-warm.sh exercises the full lease+eval path
through sandboxd; documented as a manual perf target.
…SL2/Lima Closes the ambiguity left by hvf-backend.md and whp-backend.md, both of which said "Don't build this yet" but kept Status: Planning. After the multi-agent audit on the feat/slirp-net-and-atomic-disk-meta branch surfaced three structural blockers the per-platform docs underplayed: 1. KVM coupling is deeper than swap-the-backend — pkg/vmm/arch.go's machineArchBackend interface uses concrete kvm.* types, and the on-disk snapshot format embeds kvm.Regs/Sregs/MPState directly. 2. Eventfd-everywhere — virtio IRQs in arch_arm64.go and virtio/fs.go plumb host eventfds straight into KVM_IRQFD; WHP/HVF have no equivalent. 3. The differentiating features (snapshot/restore, virtio-fs via vhost-user, vsock, the jailer) don't port cleanly. A v1 without them is roughly Firecracker's first-cut feature set on platforms where Firecracker also doesn't ship. The new doc says explicitly: skip the port; tell macOS users to use Lima/OrbStack and Windows users to use WSL2. KVM is real under both (WSL2 runs MSHV+KVM; Lima runs Linux under Virtualization.framework). Performance matches bare-metal Linux closely. No code change is required — gocracker already runs as-is in either environment. Two re-evaluation signals are recorded for future reference: an Apple Silicon user with a measurement workload Lima can't serve (~5-7 weeks to HVF v1), or a Windows production customer insisting on native (~6-9 weeks to WHP v1). Pieza B (commit cdbcfc7) added the in-guest node REPL runner using Unix-domain sockets and a toolbox-spawned subprocess — yet another Linux-specific primitive. Each one added makes the port more expensive; locking in "Linux-only by design" lets us keep adding them without owing a Win/Mac equivalent.
Adds gocracker-mcp, a JSON-RPC 2.0 server that lets AI clients (Claude Desktop, Claude Code, custom MCP-aware agents) execute code in gocracker sandboxes by calling well-typed tools. Speaks modelcontextprotocol.io spec rev 2025-11-25. The differentiator vs E2B / Daytona / Cloudflare Code Mode / Arrakis is the new process.eval_node tool: it routes JS source to an in-guest pre-loaded V8 instance (the node-warm runtime shipped on feat/slirp-net-and-atomic-disk-meta cdbcfc7) for ~24 ms in-guest exec vs ~36 ms for fork+exec'ed `node`. Combined with sandboxd's sub-30 ms warm-pool restore, an AI tool call lands well under 100 ms total. # What ships ## 5 tools, all thin SDK wrappers (no VMM state in this server) - sandbox.lease(template_id, timeout_ms?) → warm-pool lease via Client.LeaseSandbox. - sandbox.delete(id) → Client.Delete. - sandbox.recycle(id) → release-and-release in one round-trip. - process.exec(sandbox_id, cmd[], env?, env_map?, workdir?, timeout_ms?, stdin?) → ToolboxClient.Exec. - process.eval_node(sandbox_id, source, timeout_ms?) → ToolboxClient.Exec with cmd[0]="node-warm". Requires base-node-warm template. Each is ~30–80 LoC; the server holds no state, sandboxd is the source of truth. ## Stdio transport (default) Claude Desktop spawns gocracker-mcp as a subprocess, frames JSON-RPC over stdin/stdout, reads diagnostic logs from stderr. ServeStdio in sandboxes/internal/mcp/server.go drives the loop until EOF or ctx cancel. Stderr is the ONLY log surface — stdout is reserved for JSON-RPC responses. Polluting stdout breaks the wire protocol. Indirected through stderrSink so tests can swap it. ## Wire format JSON-RPC 2.0, line-delimited. Tool errors come back two ways: - Protocol-level (parse, unknown method, bad params) → JSON-RPC error object with -32xxx codes. - Tool-level (sandbox not found, exec failed) → successful result with isError=true and the message in a text content block. The LLM sees these inside its own context and adjusts. # Tests 10 tests covering initialize handshake, tools/list (deterministic sort), tools/call happy + error paths, JSON-RPC version validation, parse errors, the stdio loop. All run against an httptest-backed fake sandboxd; no real KVM needed. ~225 LoC. go test ./sandboxes/internal/mcp/ → ok 0.005s # Manual smoke $ (echo '{"jsonrpc":"2.0","id":1,"method":"initialize",...}' echo '{"jsonrpc":"2.0","method":"notifications/initialized"}' echo '{"jsonrpc":"2.0","id":2,"method":"tools/list"}') \ | bin/gocracker-mcp --sandboxd http://127.0.0.1:9091 Returns 2 valid JSON-RPC frames: initialize result with protocol version + capabilities, then tools/list result with all 5 tools and their inputSchema (verified). # Code layout - sandboxes/cmd/gocracker-mcp/main.go (99 LoC) — entry binary - sandboxes/internal/mcp/protocol.go (180 LoC) — wire types - sandboxes/internal/mcp/server.go (215 LoC) — Server + dispatch - sandboxes/internal/mcp/tools.go (335 LoC) — 5 tool handlers - sandboxes/internal/mcp/util.go (33 LoC) — small helpers - sandboxes/internal/mcp/server_test.go (225 LoC) — 10 tests - docs/design/mcp-server.md (208 LoC) — design doc Total: ~1.3k LoC, all under sandboxes/cmd/gocracker-mcp and sandboxes/internal/mcp. Zero changes to existing packages. # Claude Desktop integration Add to claude_desktop_config.json: { "mcpServers": { "gocracker": { "command": "/usr/local/bin/gocracker-mcp", "args": ["--sandboxd", "http://127.0.0.1:9091"] } } } # Next phases (separate PRs) Documented in docs/design/mcp-server.md: 1. sandbox.fan_out — N=4..64 microsecond CoW fork in one RPC. The primitive that nobody else has at this latency. ~200 LoC. 2. speculate.race — fork N children, run candidates concurrently, first to satisfy success_predicate wins. ~150 LoC. 3. checkpoint.tree.diff — diff two checkpoints (files, env, RSS). Surfaces gocracker's existing dirty-log capture as a verb. ~250 LoC. 4. process.exec_stream — SSE streaming stdout/stderr. ~80 LoC. 5. files.put / files.get / preview.mint — std SDK passthroughs. ~100 LoC. 6. Streamable HTTP transport (multi-tenant, bearer auth). ~150 LoC. # Builds on (parent branch) Stacked on top of feat/slirp-net-and-atomic-disk-meta (PR #22), which added the node-warm runtime path that process.eval_node depends on. This MCP branch should be reviewed AFTER #22 lands so the eval_node tool has a real base-node-warm template to point at.
… competitors
Adds docs/perf-snapshot-2026-05-06.md as a single-source-of-truth
reference for the current performance numbers, the timing breakdown,
and the public-product comparison. Useful for:
- Answering HN-style perf questions ("how does this compare to
boxlite?") without needing to re-bench every time.
- Confirming the gocracker-mcp #23 latency claims with measured data.
- Onboarding contributors to which path each number belongs to (CLI
cold vs WARM vs node-warm vs daemon-mode pool primitive).
# What's measured (5 paths)
Cold-CLI gocracker run -dockerfile 240 ms median
WARM-CLI gocracker run -warm 71 ms median
NODE-WARM-CLI ... -warm-runtime node -cmd node-warm
70 ms median
POOL-PRIMITIVE pool-bench /bin/true 7 ms median
POOL-PRIMITIVE node -v pool-bench node -v 37 ms median
# Trace breakdown
Two side-by-side timelines (node-warm REPL eval vs regular
fork+exec node -v) showing where each ms goes inside the gocracker
process. node-warm guest exec is 24 ms vs 31 ms for fork+exec
node -v (~28 % faster in-guest).
The CLI wall-clock 70 ms is dominated by the ~35-40 ms host startup
(sudo + Go init) which the trace anchor doesn't capture. Daemon-mode
callers (sandboxd lease + exec via SDK or MCP) pay 7-37 ms total
because that startup is amortised.
# Comparison table
vs 9 competitors (boxlite, E2B, Daytona, Vercel Sandbox, Cloudflare
Code Mode, Modal, Arrakis, node-vmm). Each row cites its source.
Key findings:
- boxlite has NO in-memory snapshot/restore (qcow2 disk-only +
freeze/thaw, restore = cold reboot). Source: acmerfight gist.
Their <50 ms claim is unbenchmarked cold boot, not warm restore.
- gocracker is the only project combining KVM real isolation +
in-memory dirty-delta snapshot/restore + pre-loaded runtime
(node-warm) at sub-30 ms latency.
- Cloudflare Code Mode is faster (<5 ms) but trades off filesystem
and process state — different threat model.
# Source links
ComputeSDK leaderboard, boxlite docs + source-verified gist, E2B's
UFFD blog, Daytona WarmPoolService deepwiki, Modal mem-snapshots
blog, Arrakis repo, Cloudflare Code Mode blog. Every claim cited.
# Reproduce
Last section gives the exact bench commands for each path so anyone
can verify against their own host. All sources are in the repo at
PR #22 (perf foundation) + PR #23 (MCP).
…inding
Bug surfaced during end-to-end testing of feat/mcp-server: piping a
malformed line into the stdio loop fatal-exited the entire binary
("[gocracker-mcp] fatal: decode: invalid character 'i' looking for
beginning of value"), breaking subsequent valid frames.
Root cause: ServeStdio used json.Decoder, which is stateful — once
it hits unexpected input, the rest of the stream is corrupt and
cannot be recovered. The unit test TestParseError didn't catch this
because it exercised Handle() directly, not the ServeStdio loop.
Fix: switch to bufio.Scanner + per-line json.Unmarshal. Each line
is independently parsed; a malformed line emits a JSON-RPC parse-
error response (id=null per spec) and the loop keeps reading. Real
read errors (EOF, scanner buffer overflow) still terminate cleanly.
Buffer is bumped to 4 MiB max line — MCP messages can carry
multi-hundred-KB JS source via process.eval_node, and the default
64 KiB Scanner buffer would silently truncate them.
Adds TestServeStdioRecoversFromBadJSON regression test covering the
specific failure mode (bad line, then valid ping, expects two
responses with the second being a successful ping).
# End-to-end test results (covered by this commit + manual smoke)
$ sudo bin/gocracker-sandboxd serve --addr :9092 --kernel-path ...
$ # via curl: cold-create alpine sandbox
$ # via MCP: initialize → exec(echo) → exec(uname) → exec(env_map) → exec(exit 7) → delete
init: server=gocracker-mcp proto=2025-11-25 ✅
echo: 64 ms wall, exit=0, stdout="hello mcp\n" ✅
uname: <3 ms round-trip, "Linux gocracker 6.1.102 PREEMPT ... x86_64" ✅
env_map: <3 ms, MCP_TEST=works (env_map → KEY=VALUE flatten works) ✅
exit-7: exit=7, stderr="to-stderr" (non-zero exit + stderr captured) ✅
delete: {"id":"sb-...","ok":true} ✅
mcp clean exit rc=0 ✅
# Real-deployment finding (NOT fixed in this commit; followup)
The sandbox UDS in sandboxd's state-dir defaults to root-only
permissions:
$ ls -la /tmp/state/sandboxes/sb-X.sock
ls: Permission denied
So gocracker-mcp running as the user (Claude Desktop's spawning
model) can't open the socket; it hangs silently waiting on a dial
that fails with EACCES. Today: workaround by running gocracker-mcp
under sudo (matching sandboxd's user). Followup: sandboxd should
expose --uds-group GROUP to chmod the UDS files to a known group,
so Claude Desktop's user-context MCP server can talk to a root-owned
sandboxd without privilege escalation. Tracked as a known limitation
in docs/design/mcp-server.md ("Auth / multi-tenancy" follow-up).
# Tests
go test -count=1 ./sandboxes/internal/mcp/ ─→ ok 0.005s (11/11)
Core improvements:
- sandboxd: --uds-group flag chowns UDS sockets so non-root MCP server
can connect without sudo
- mcp: sandbox.fan_out tool — N concurrent leases, wall time = one lease
- mcp: gocracker-mcp setup command auto-detects Claude/Cursor/Windsurf/VS Code
and merges MCP config entries without overwriting other tools
- sandboxd: --network-mode flag + systemd install subcommand with
gocracker-sandboxd install for auto-restart on reboot
- gocracker CLI: resolveJailerMode auto-selects --jailer off for non-root
users (no sudo needed for run/repo/compose/build/restore)
- sandboxd HTTP: POST /sandboxes/{id}/exec and PUT /sandboxes/{id}/files
endpoints for VS Code extension use
VS Code extension (sandboxes/vscode/) — Phase 1:
- DaemonManager: discovers kernel, spawns sandboxd, polls healthz
- GocrackrClient: leaseSandbox, exec, uploadFile, deleteSandbox, listSandboxes
- GocrackrOutputPanel: WebviewPanel with ANSI→HTML color conversion
- SandboxExplorer: TreeView polling GET /sandboxes every 3s
- Run Selection command (Ctrl+Shift+G), language detection, template mapping
- Status bar, Start/Stop Daemon commands, full settings schema
- TypeScript compiles clean (tsc --strict)
Docs:
- README, GETTING_STARTED, TROUBLESHOOTING, CLI_REFERENCE updated to
reflect rootless operation — sudo only for --jailer on / --net auto
- AGENT.md: instructions for AI agents using the MCP tools
- docs/design/vscode-plugin.md: full plugin design with phases
…hell
- client.ts: recycleSandbox() calling POST /sandboxes/{id}/recycle
- daemon.ts: downloadKernel() downloads pre-built kernel from GitHub
releases (follows redirect, gunzips in-process, shows progress
notification); ensure() prompts to download when no kernel found
- terminal.ts: new REPL-style PTY terminal per sandbox — types commands,
Enter calls exec(), backspace/echo work, colored prompt with sandbox ID
- package.json: 4 new commands (deleteSandbox, recycleSandbox, execShell,
downloadKernel) + view/item/context menu entries for sandbox items
- extension.ts: registers all 4 commands with confirmation for delete,
explorer.refresh() after delete/recycle, imports terminal + daemon fns
- All 8 TS files compile clean (tsc --strict)
- fanout.ts: runFanOut() leases N sandboxes concurrently via Promise.allSettled, uploads + execs the same snippet in each, then compares outputs — identical results are collapsed, differing outputs shown per-slot with separators and wall time - chat.ts: @gocracker Copilot chat participant — parses fenced code blocks from the prompt, runs them in a sandbox, streams stdout/stderr/ exit code back as markdown; guards against VS Code < 1.90 gracefully - package.json: gocracker.fanOut command + Ctrl+Shift+Alt+F keybind + editor context menu entry; chatParticipants declaration for gocracker.run - extension.ts: registers cmdFanOut (N input box 2-10, auto-starts daemon) and chat participant; all 10 TS files compile clean
… bump
- extension.ts: bail out early with a warning on non-Linux platforms
(KVM is Linux-only; previously timed out silently after 15s)
- daemon.ts: mkdirSync(stateDir, {recursive:true}) before spawning
sandboxd so the state directory exists on first install
- package.json: engines.vscode ^1.85.0 → ^1.90.0 (chat API minimum);
@types/vscode bumped to match; @vscode/vsce added to devDependencies
so `vsce package` works; repository field added for marketplace
- docs/design/vscode-plugin.md: mark all 3 phases shipped, update file
layout to reflect terminal.ts / fanout.ts / chat.ts additions
Bug fixes: - extension.ts: set gocracker.isLinux context key so explorer sidebar is hidden on macOS/Windows (view when="gocracker.isLinux") - extension.ts: recycleSandbox catches HTTP 404 and shows actionable message instead of generic error (recycle only works on pool sandboxes) - daemon.ts: add ~/.local/share/gocracker/kernels/ as discovery path 3 so kernels installed via gocracker-sandboxd install are found without prompting for a redundant download Packaging: - package.json: "package": "vsce package" script added - icon.png: 128x128 generated PNG icon (teal VM box, dark bg) - package.json: "icon": "icon.png" wired Tests (20 passing, 0 warnings): - src/test/language.test.ts: templateForDocument + execCommandForTemplate + SUPPORTED_EXTENSIONS (8 cases) - src/test/fanout.test.ts: allSame output comparison logic (5 cases) - src/test/chat.test.ts: extractCodeBlock parsing (5 cases) - jest + ts-jest added to devDependencies; testMatch = src/test/** - tsconfig excludes src/test to prevent duplicate compiled mock CI: - .github/workflows/vscode-ext.yml: triggers on changes to sandboxes/vscode/**, runs npm ci + compile + test + package
…ch, perf wins Includes: - --net slirp: rootless userspace network stack (ARP+DHCP+ICMP) - code-disk-attach Phase 1-3: --code-disk flag, snapshot restore with attached drive, sandboxd lease-time injection, SDK wiring - perf: cond-based pause/resume (150x wake-up), TTI wins (108→92 ms), two-layer warmcache hash, Pieza B node-warm REPL snapshot - structured tracer for boot/restore/exec hot paths - fix: tests, bad-JSON recovery, E2E repairs
…ll phases) Includes: - MCP server (6 tools): create/exec/delete/list sandbox, fan_out, eval_node - sandbox.fan_out: N concurrent leases, wall time = one lease - gocracker-mcp setup: auto-detects Claude/Cursor/Windsurf/VS Code - sandboxd: --uds-group (non-root access), --network-mode, systemd install - gocracker CLI: auto-selects --jailer off for non-root users - VS Code extension Phase 1-3: run selection/file, daemon manager, output panel, sandbox explorer, PTY shell, kernel auto-download, fan-out UI, @gocracker Copilot chat participant - 20 unit tests, CI workflow, icon, vsce packaging - Docs: AGENT.md, full rootless operation docs, vscode-plugin design
Node.js word-count, Python CSV stats, and a statically-compiled Go HTTP server each demonstrate loading application code from an ext4 code disk over an unmodified Alpine/node/python base image. - examples/code-disk/node-word-count/ — reads text.txt, emits JSON word-freq - examples/code-disk/python-stats/ — reads cities.csv, emits population JSON - examples/code-disk/go-serve/ — Go binary reads config.json; --print mode - tests/manual-smoke/code-disk-apps/run.sh — end-to-end smoke (10/10 assertions) - docs/design/code-disk-attach.md: real-app section with quick-start snippets
There was a problem hiding this comment.
Pull request overview
This PR significantly expands gocracker’s “developer tooling” surface: it adds code-disk example apps + smoke tests, introduces new sandboxd convenience endpoints (exec + file upload), and brings in major runtime/infra additions (warm runtime snapshots, slirp/rootless networking, perf/TTI instrumentation, and a VS Code extension + MCP server).
Changes:
- Add three real “code disk” example apps plus manual smoke-test scripts and updated docs.
- Add sandboxd/template/pool/runtime enhancements (warm-runtime snapshots, code-disk plumbing, slirp networking) and multiple performance/robustness improvements.
- Introduce new developer tooling components (VS Code extension, MCP server, tracing, benchmarks, and new CI workflow for the extension).
Reviewed changes
Copilot reviewed 121 out of 129 changed files in this pull request and generated 14 comments.
Show a summary per file
| File | Description |
|---|---|
| tools/bench-node-warm.sh | Adds sandboxd-based node-warm TTI benchmark script. |
| tools/bench-node-tti.sh | Extends node TTI benchmark with warm snapshot-pool mode. |
| tests/manual-smoke/code-disk/run.sh | Adds minimal code-disk attach smoke test. |
| tests/manual-smoke/code-disk-apps/run.sh | Adds smoke test for three real code-disk apps. |
| tests/manual-smoke/cmd/snaprestoreexec/main.go | Adds snapshot-restore-exec diagnostic helper. |
| tests/manual-smoke/cmd/balloonprobe/main.go | Adds balloon probing diagnostic helper. |
| tests/integration/vsock_uds_e2e_test.go | Fixes async cleanup race by polling for UDS removal. |
| tests/integration/ratelimiter_e2e_test.go | Adjusts test to use persistent rootfs for virtio-blk I/O. |
| tests/integration/jailer_e2e_test.go | Updates jailer assertion to check mount namespace behavior. |
| tests/integration/balloon_e2e_test.go | Adjusts balloon assertions to use MemAvailable semantics. |
| sandboxes/vscode/tsconfig.json | Adds TS build config for VS Code extension. |
| sandboxes/vscode/src/test/language.test.ts | Adds unit tests for language/template mapping helpers. |
| sandboxes/vscode/src/test/fanout.test.ts | Adds unit tests for fan-out output comparison logic. |
| sandboxes/vscode/src/test/chat.test.ts | Adds unit tests for chat code-block extraction logic. |
| sandboxes/vscode/src/test/mocks/vscode.ts | Adds minimal vscode API mock for tests. |
| sandboxes/vscode/src/terminal.ts | Adds in-extension pseudo-terminal shell for a sandbox. |
| sandboxes/vscode/src/panel.ts | Adds output webview panel with ANSI→HTML rendering. |
| sandboxes/vscode/src/language.ts | Adds language detection + template/exec mapping utilities. |
| sandboxes/vscode/src/fanout.ts | Adds “fan-out” command to run N sandboxes in parallel. |
| sandboxes/vscode/src/explorer.ts | Adds sandbox explorer view (TreeDataProvider). |
| sandboxes/vscode/src/config.ts | Adds extension configuration loader. |
| sandboxes/vscode/src/client.ts | Adds HTTP client for sandboxd (lease/exec/upload/etc). |
| sandboxes/vscode/src/chat.ts | Adds VS Code chat participant to run fenced code blocks. |
| sandboxes/vscode/README.md | Adds extension documentation and usage instructions. |
| sandboxes/vscode/package.json | Adds extension manifest, scripts, and Jest config. |
| sandboxes/vscode/.vscodeignore | Adds packaging ignore list. |
| sandboxes/vscode/.gitignore | Adds extension build/package ignore rules. |
| sandboxes/sdk/python/gocracker/client.py | Adds code_disks plumbing to Python SDK lease_sandbox. |
| sandboxes/sdk/js/src/index.js | Adds codeDisks plumbing to JS SDK leaseSandbox. |
| sandboxes/sdk/go/client.go | Adds code_disks to Go SDK LeaseSandboxRequest. |
| sandboxes/internal/templates/templates.go | Adds template RuntimeSpec and bumps cache format version. |
| sandboxes/internal/sandboxd/templates.go | Plumbs template runtime selection from sandboxd API. |
| sandboxes/internal/sandboxd/server.go | Adds sandbox exec + file upload endpoints. |
| sandboxes/internal/sandboxd/sandbox.go | Adds UDS group ownership support + default network mode config. |
| sandboxes/internal/sandboxd/pool.go | Adds code_disks wire plumbing and rootless slirp network mode option. |
| sandboxes/internal/sandboxd/lease_codedisk_test.go | Adds tests ensuring code_disks are forwarded on lease requests. |
| sandboxes/internal/sandboxd/base_templates.go | Adds base-node-warm template and runtime metadata. |
| sandboxes/internal/pool/refiller.go | Adjusts pool bucket bookkeeping on stop/reap/create. |
| sandboxes/internal/pool/pool_test.go | Updates tests for bucket FIFO semantics. |
| sandboxes/internal/pool/lease_test.go | Adds benchmark guarding O(1) Acquire selection. |
| sandboxes/internal/pool/hot_budget_test.go | Updates tests for bucket enqueue semantics with hot entries. |
| sandboxes/internal/mcp/util.go | Adds stderr-only logging sink + deterministic tool sorting. |
| sandboxes/cmd/gocracker-sandboxd/main.go | Adds serve flags for uds-group/network-mode and an install subcommand. |
| sandboxes/cmd/gocracker-mcp/main.go | Adds MCP stdio server binary for sandbox control. |
| README.md | Updates quickstart + benchmarking section to reflect new modes. |
| pkg/warmcache/cache.go | Adds in-memory + on-disk file-hash caching for warmcache keys. |
| pkg/warmcache/cache_test.go | Adds tests for disk-backed HashFile cache. |
| pkg/vmm/restore_drives_test.go | Adds tests for applying additional drives on restore. |
| pkg/vmm/pause_resume_bench_test.go | Adds pause/resume latency test + benchmark. |
| pkg/vmm/migration.go | Makes snapshot metadata writes atomic and improves durability. |
| pkg/vmm/arch_arm64.go | Adds virtio-net slirp backend wiring on arm64. |
| pkg/container/warmcache.go | Adds warm-runtime readiness gating before snapshot capture. |
| pkg/container/codedisk_mount_test.go | Adds tests for code disk drive mapping and helpers. |
| internal/worker/vmm.go | Lowers socket polling fallback step for faster worker startup. |
| internal/vmmserver/server.go | Adds restore request support for additional drives. |
| internal/virtio/net.go | Generalizes virtio-net to support a pluggable backend (slirp). |
| internal/virtio/net_backend.go | Defines NetBackend interface for virtio-net carriers. |
| internal/uart/uart.go | Adds a channel signal for “first UART output” to reduce polling jitter. |
| internal/trace/trace.go | Adds opt-in, low-overhead structured timing tracer. |
| internal/trace/trace_test.go | Adds tests for trace formatting and helpers. |
| internal/toolbox/spec/spec.go | Bumps toolbox version for warm runner + readiness endpoint support. |
| internal/toolbox/client/client.go | Coalesces request writes using writev via net.Buffers. |
| internal/toolbox/agent/server.go | Adds runtime readiness endpoint and starts warm runner. |
| internal/toolbox/agent/runners/node-repl-server.js | Adds Node warm REPL server used by node-warm exec mode. |
| internal/toolbox/agent/exec.go | Adds node-warm exec dispatch path to warm runner. |
| internal/slirp/udp.go | Adds UDP checksum build/parse helpers for slirp. |
| internal/slirp/udp_nat.go | Adds DHCP handling and UDP NAT scaffolding. |
| internal/slirp/icmp.go | Adds ICMP echo handling in slirp MVP. |
| internal/slirp/doc.go | Adds slirp package documentation and MVP scope statement. |
| internal/slirp/dhcp.go | Adds minimal DHCPv4 server implementation for slirp. |
| internal/slirp/arp.go | Adds ARP parsing/reply for slirp gateway. |
| internal/guest/initrd.go | Caches embedded init digest to avoid repeat hashing. |
| internal/guest/init.go | Adds guest-side mounting of extra code disks via kernel cmdline. |
| internal/api/api.go | Adds rootfs_persistent flag and slirp network-mode validation updates. |
| examples/code-disk/python-stats/build.sh | Adds ext4 code disk build script for python-stats example. |
| examples/code-disk/python-stats/app/stats.py | Adds python-stats example app reading CSV from code disk. |
| examples/code-disk/python-stats/app/cities.csv | Adds CSV dataset for python-stats example. |
| examples/code-disk/node-word-count/build.sh | Adds ext4 code disk build script for node-word-count example. |
| examples/code-disk/node-word-count/app/word-count.js | Adds node-word-count example app reading text from code disk. |
| examples/code-disk/node-word-count/app/text.txt | Adds sample text dataset for node-word-count example. |
| examples/code-disk/go-serve/src/main.go | Adds go-serve example app reading config from code disk. |
| examples/code-disk/go-serve/src/config.json | Adds example config for go-serve source tree. |
| examples/code-disk/go-serve/payload/config.json | Adds config bundled into the go-serve code disk payload. |
| examples/code-disk/go-serve/build.sh | Adds build script to compile + package go-serve into ext4 disk. |
| docs/TROUBLESHOOTING.md | Updates privilege/rootless guidance and net/jailer requirements. |
| docs/NETWORKING.md | Documents slirp networking mode and current MVP limitations. |
| docs/GETTING_STARTED.md | Updates getting-started flow for rootless use and slirp. |
| docs/design/whp-backend.md | Adds deferred WHP backend design doc. |
| docs/design/slirp-tcp-udp.md | Adds deferred TCP/UDP NAT design doc for slirp. |
| docs/design/sandbox-speed-backlog.md | Adds performance backlog and shipped optimizations notes. |
| docs/design/README.md | Adds index of design docs. |
| docs/design/portability.md | Adds Windows/macOS portability decision doc. |
| docs/CLI_REFERENCE.md | Updates CLI docs for rootless usage and slirp mode. |
| AGENT.md | Adds MCP agent guide and tool workflow documentation. |
| .gitignore | Ignores .wrangler workspace artifacts. |
| .github/workflows/vscode-ext.yml | Adds CI workflow to build/test/package the VS Code extension. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
Security / correctness: - server.go upload: cap body at 32 MiB (http.MaxBytesReader) and pass path via env var instead of sh -c string concat (injection fix) - bench-node-warm.sh: fix exec payload key command→cmd; extend cleanup trap to also remove temp samples file instead of overwriting it SDK + runtime: - sandboxd CreateSandboxRequest now accepts code_disks, wired through to container.RunOptions.CodeDisks (create_sandbox path now works) - Python SDK create_sandbox() accepts code_disks parameter - builder.go: inject GOCRACKER_ENABLE_WARM_NODE=1 env var when building a node-warm template so the opt-in gate works end-to-end - warm_runner.go: gate StartNodeWarmRunner behind GOCRACKER_ENABLE_WARM_NODE to avoid spurious startup on non-node images - exec.go: derive warm-eval TimeoutMs from ctx deadline instead of hard-coding 30 000 ms - node-repl-server.js: pass encoding/callback args in stdout.write override to preserve full Node stream.Writable signature VS Code extension: - language.ts, chat.ts: map .ts files to base-bun (Bun runs TypeScript natively; plain node cannot) - README.md: align .ts template row with implementation (was ts-node) Infrastructure: - balloonprobe: replace hard-coded /home/misael kernel path with GC_KERNEL env var + repo-root fallback - slirp/doc.go: remove misleading UDP NAT MVP claim; clearly mark as deferred - warmcache: release hashCacheMu before disk I/O (storeDiskHash) to avoid serialising concurrent HashFile callers Examples: - node-word-count/app-v2 + build-v2.sh: language/words text for swap demo - sdk-demo/demo.py: Python SDK demo creating a sandbox with two code disks and switching between them in the same running VM - smoke test: v1→v2 timed swap + multi-disk single-VM on-the-fly section
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
text.txtfrom the code disk and emits a JSON word-frequency report (top-10 words, unique count, total)cities.csvand emits population JSON (mean, top-5 cities, histogram buckets by size)config.jsonfrom the disk; supports--printmode for smoke-testable output without a network connectionAll three run over unmodified public OCI base images (
node:20-alpine,python:3.12-alpine,alpine:3.20) — only the code disk differs between launches.Test plan
make build kernel-unpacksudo bash tests/manual-smoke/code-disk-apps/run.sh— all 10 assertions pass (verified locally)docs/design/code-disk-attach.mdFiles
examples/code-disk/node-word-count/examples/code-disk/python-stats/examples/code-disk/go-serve/tests/manual-smoke/code-disk-apps/run.shdocs/design/code-disk-attach.md