Skip to content

ADR-0203 stage 5: transparent on-disk compilation cache (--cache, closes D-508) - #141

Merged
chaploud merged 2 commits into
mainfrom
develop/aot-stage5-cache
Jul 9, 2026
Merged

ADR-0203 stage 5: transparent on-disk compilation cache (--cache, closes D-508)#141
chaploud merged 2 commits into
mainfrom
develop/aot-stage5-cache

Conversation

@chaploud

@chaploud chaploud commented Jul 9, 2026

Copy link
Copy Markdown
Contributor

Summary

Final implementation stage of the AOT-full-fidelity campaign (ADR-0203 D5/D6 stage 5): zwasm run --cache[=DIR] x.wasm keys the module by SHA-256 content hash and transparently reuses the .cwasm artifact a previous run produced — deploy artifact stays .wasm, the cache is invisible plumbing (the wasmtime/wazero model).

  • src/cli/cache.zig (new): versioned cache dir <root>/zwasm-<version>-<arch>-<os>-<bounds>/<sha256>.cwasm; hit = full-fidelity CWAS load (cache-hit == cache-miss by construction, ADR-0203 D2); miss = compile + produce + atomic store.
  • --cache / --cache=DIR / --cache-clear CLI flags; default root = ~/Library/Caches/zwasm (macOS) / $XDG_CACHE_HOME|~/.cache/zwasm (Linux) / %LOCALAPPDATA%\zwasm (Windows).
  • Measured: tinygo_json 9.2ms → 4.1ms wall (2.2x; user CPU 7.2 → 2.2ms = codegen skipped).

Failure discipline (DA critique fixes, second commit)

The independent DA critique (14/20 → fixes applied) reproduced four ways a cache defect could fail the run; all now degrade per the ADR-0203 D5 invariant "the cache can never make run fail":

  • HIT gate + self-heal: header (magic/version/arch) + section-bounds check before trusting an entry; corrupt/truncated/drifted entries are deleted and recompiled (verified E2E: 3 corruption shapes → exit 0, byte-identical output).
  • Cache-path compile refusal = bypass, not exit(1) — restores .auto's interp fallback and ZWASM_DEBUG runnability under --cache.
  • Explicit --engine interp bypasses the cache (D-496: the flag keeps forcing interp; the artifact is JIT code).
  • --cache-clear alone no longer implies caching this run.
  • store(): unique random temp suffix (shared temp name could publish a concurrent writer's partial bytes); artifacts over the 256 MiB read cap are not stored.
  • Trust model documented in the module doc (cache-dir write access = native code execution as the user).

Tests

  • zig build test green; zig build lint -- --max-warnings 0 green.
  • Unit: subdir naming / stable key / miss→store→hit byte-identity / 3-shape corrupt-entry self-heal / --cache-clear idempotency.
  • test-aot-diff 63/63 with new cache lanes (miss + hit vs fresh JIT per fixture, ADR-0203 D6 stage-5 ratchet) + interp-bypass probe + stored-entry existence check.

Closes D-508. Campaign stage V (retrospective + docs) follows in the next PR.

https://claude.ai/code/session_01F2PGseEjGFR1uqid4FaXsm

chaploud added 2 commits July 9, 2026 22:14
@chaploud
feat(cli): ADR-0203 stage 5 — transparent on-disk compilation cache (…
a3c6560 
…closes D-508)

`zwasm run --cache[=DIR] x.wasm` reuses the .cwasm artifact a previous
run produced. Users keep deploying plain `.wasm` (the wasmtime
Config::cache / wazero NewCompilationCacheWithDir model); the runtime
keys artifacts by content hash in a versioned directory:

  <root>/zwasm-<version>-<arch>-<os>-<bounds>/<sha256-hex>.cwasm

- src/cli/cache.zig: SHA-256 content key; versioned subdir (every
  codegen-affecting knob lives in the dir name, so stale entries are
  silently unreachable — no config hash needed); temp + atomic-rename
  store; ANY cache-side error degrades to miss/no-store (EXEMPT-FALLBACK
  per ADR-0203 D5 — the cache can never make `run` fail); --cache-clear
  deletes this build's subdir only.
- main.zig: --cache[=DIR] / --cache-clear flags; default root = platform
  user-cache dir (HOME/Library/Caches, XDG_CACHE_HOME|~/.cache,
  LOCALAPPDATA) from the environ map; a HIT swaps the artifact bytes in
  and the existing CWAS branch runs it through the full-fidelity load
  path — cache-hit == cache-miss by construction.
- Components + explicit .cwasm inputs bypass the cache.

Measured (tinygo_json, ReleaseSafe, hyperfine): 9.2 ms → 4.1 ms cold
start (2.2x; user CPU 7.2 → 2.2 ms — parse/validate/codegen skipped).

Unit tests: subdir naming, stable content key, miss→store→hit
byte-identity, corrupt-entry loud rejection at the load gate,
--cache-clear idempotency + re-populate. zig build test green.
@chaploud
fix(cli): cache failure paths degrade, never fail the run (ADR-0203 D…
4a17ced 
…5, DA critique)

The stage-5 cache implemented the D5 silent-miss discipline for read
errors only; the DA critique reproduced four ways a cache defect failed
the run (corrupt entry x3 shapes, ZWASM_DEBUG instrumentation) plus an
--engine interp override. All degrade now:

- HIT gate: parseHeader (magic/version/arch) + section-bounds check
  before trusting an entry; a bad entry is deleted (self-heal) and
  recompiled — a corrupt cache can never brick the run.
- Compile/produce refusal inside the cache path = cache BYPASS (fall
  through to normal dispatch), restoring `.auto`'s interp fallback and
  ZWASM_DEBUG runnability instead of exit(1).
- Explicit `--engine interp` bypasses the cache entirely (D-496: the
  flag must keep forcing interp; the artifact is JIT code).
- `--cache-clear` alone no longer implies caching this run.
- store(): unique random temp suffix (a shared temp name let one
  writer's atomic rename publish another's partial bytes); artifacts
  above the 256 MiB read cap are not stored (never hittable).
- Module doc states the trust model (cache-dir write access = native
  code execution as the user; wasmtime/wazero posture).
- test-aot-diff: cache lanes (miss + hit vs fresh, 63/63) + interp-
  bypass probe + stored-entry check — the ADR-0203 D6 stage-5 ratchet.

Claude-Session: https://claude.ai/code/session_01F2PGseEjGFR1uqid4FaXsm
@chaploud
chaploud merged commit c4137ee into main Jul 9, 2026
5 checks passed
@chaploud
chaploud deleted the develop/aot-stage5-cache branch July 9, 2026 14:30
@chaploud chaploud mentioned this pull request Jul 9, 2026
Merged
chaploud added a commit that referenced this pull request Jul 9, 2026
@chaploud
ADR-0203 stage V: campaign retrospective — docs, bench record, lesson…
cf5d20d 
…, debt reconcile (#142)

* chore(campaign): ADR-0203 stage V retrospective — docs, bench record, lesson, debt reconcile

Campaign close-out (stages 1-5 = PRs #137-#141):

- ADR-0203 Status -> Implemented (aot-diff 63/63 incl. cache lanes;
  D-508/515(1)/516/517/518/519 closed).
- docs/reference/cli.md: --cache/--cache-clear rows; stale claims fixed
  (.cwasm now sandboxed + full-runtime, artifact no longer "v0.1",
  cache/engine interaction); README CLI block gains --cache lines.
- Explicit `.cwasm` + `--engine interp` is now a loud exit-2 refusal
  (contradictory request: the artifact IS JIT code; D-496 spirit) with
  an aot-diff E2E probe.
- bench/results/history.yaml: per-merge record on the feature branch
  (campaign parity: worst +1%, median -8% vs pre-campaign entry).
- Lesson: failure-path-tests-certified-the-defect (DA critique found the
  stage-5 unit test asserting the D5 violation it should have caught).
- handover.md rewritten to the campaign-closing state (118 lines).

Remaining campaign residual: D-515(2) spec-assert corpus under elision
(harness memory provisioning), tracked in debt.yaml.

Claude-Session: https://claude.ai/code/session_01F2PGseEjGFR1uqid4FaXsm

* chore(release): bump version to 2.2.0 + CHANGELOG (USER-DIRECTED release, ADR-0156)

Release directed by the user 2026-07-09 ("campaign Vが終わったら、タグを
切ってリリースしてください"): after this PR merges, the v2.2.0 tag on the
merge commit triggers release.yml (auto-build + Release + Latest).

Claude-Session: https://claude.ai/code/session_01F2PGseEjGFR1uqid4FaXsm
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

None yet

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

1 participant

@chaploud