feat(bytecode): ADR-0173 C5'-b — flate-compressed .clj sources (-312KB) + binary-size rules/governance hardening

Smell-audited: 2: the sources dual-mode (cache_gen raw / cljw compressed)
required a second options module + a stub import on its own WriteFiles —
the first wiring created a build dependency loop (cache_gen -> cache_wf ->
run cache_gen), probed and documented in the ADR revision.

cache_gen emits a second artifact (bootstrap_sources.bin: label +
uncompressed_len + flate bytes per file); bootstrap.sourceText
decompresses on demand into rt.load_arena; rt.source_resolver (new
runtime hook, lang-installed — zone-legal vtable direction) serves
error-render registry misses and memoizes. The eager ~450KB
registerSource gpa copy at startup is gone. Measured: 7,384,984 ->
7,073,080 B. Campaign total: 9,469,816 -> 7,073,080 (-25.3%).

Per the user directive, size awareness is now mechanized:
.claude/rules/binary_size.md (auto-loads on src/build edits; carries the
campaign's measured lessons — instantiation-count metric, comptime
cross-products, embed-compression pattern, safety-inviolable,
median-of-20 protocol) and binary_size_report.sh --check now fails the
gate when the binary exceeds the ADR-0172 derived ceiling
(BUDGET_CEILING_BYTES=8800000, re-derived post-L2). README 7.1MB;
ADR-0172 budget table refreshed (embedded data 0.76/1.0MB).

Author: chaploud

.claude/rules/binary_size.md

@@ -0,0 +1,73 @@
+---
+paths:
+  - "src/**/*.zig"
+  - "build.zig"
+  - "build.zig.zon"
+---
+
+# Binary-size awareness (ADR-0172 is the SSOT)
+
+Auto-loaded when editing Zig source / the build. cljw's headline claim is
+"one small static binary"; size is budgeted per component and gate-checked.
+This rule carries the operational discipline + the measured lessons from the
+2026-07-16 campaign (9.47 MB → 7.38 MB, −22%) so they are not re-derived.
+
+## The mechanical guards (already wired — do not duplicate)
+
+- **`size_claims` step** (full gate, `test/run_all.sh`): README's headline
+  "<N> MB" must stay within 10% of the freshly built binary — a prose size
+  claim can no longer rot (the 3.8-vs-9.5 MB incident class is closed).
+- **Budget ceiling check** (same step, `scripts/binary_size_report.sh
+  --check`): the built binary must stay under the ADR-0172 **derived
+  ceiling** (`BUDGET_CEILING_BYTES` in the script, kept in sync with the
+  ADR's budget table). A breach fails the gate: attribute with the report
+  tool, then either land a lever or consciously amend the budget **in
+  ADR-0172** (Revision history entry) — never silently.
+- **Measurement tool**: `bash scripts/binary_size_report.sh [BIN]` — total +
+  segment breakdown; on a `-Dprofile=true` build it prints top symbols for
+  attribution.
+
+## When your change plausibly adds size, check it
+
+A change is size-relevant when it adds: a new `@embedFile`/generated blob, a
+new comptime-instantiated generic family, a new std subsystem (crypto, http,
+regex-class tables), a large `switch`/table, or a new dependency. For those:
+measure before/after (`binary_size_report.sh`), and note the delta in the
+commit message when it exceeds ~50 KB. Feature growth is legitimate
+(F-013/F-014) — the budget exists to make it *conscious*, not to forbid it.
+
+## Measured lessons (do not re-litigate; citations in ADR-0172/0173)
+
+1. **Instantiation count predicts recoverable size — symbol size does NOT.**
+   A huge single symbol that aggregates once-called inlined handlers is
+   size-NEUTRAL to out-line (zwasm's 707 KB emitter: table-driven rewrite
+   changed nothing). The real wins are ×N-duplicated bodies (zwasm's
+   host-callback thunks: ~300 B × 3,840 → −1.08 MB collapsed to shared
+   bridges + 23 B forwarders). Before proposing a size refactor, ask "how
+   many call sites / comptime instantiations share this code?".
+2. **Watch comptime cross-products.** `fn(T) × arg-kinds × arity × slot`
+   families explode silently. Prefer a type-erased shared body + thin
+   per-instantiation forwarders (`noinline` on the shared body) when the
+   family exceeds a handful of instances. Known open case: std.sort
+   comparator monomorphization ≈ 224 KB / 93 instances (ADR-0172 L6).
+3. **Embedded assets compress ~4-5×** (bytecode blob, `.clj` text): flate
+   `.raw` + exact `uncompressed_len` + decompress-on-demand into
+   `rt.load_arena` is the established pattern (serialize.zig
+   `flateCompress`/`flateDecompress`; two std pitfalls are documented on
+   those fns — non-empty output buffer for Compress, REAL 32 KB window for
+   Decompress).
+4. **Safety is never traded for size.** ReleaseSafe stays the shipped config
+   (ADR-0132); ReleaseSmall/hybrid rejections are recorded as ADR-0172
+   L3/L4 — don't re-propose them without new facts.
+5. **Startup claims need median-of-20 on a quiet machine.** Single-shot
+   `CLJW_PROFILE_STARTUP` probes under load produced a phantom −32% once
+   (corrected in ADR-0173). Size numbers are stable; time numbers are not.
+
+## Cross-references
+
+- `.dev/decisions/0172_binary_size_budget_and_ledger.md` — budget table +
+  lever ledger + governance (the SSOT).
+- `.dev/decisions/0173_envelope_v7_zero_copy_pool_flate.md` — the envelope
+  v7 arc (pool / zero-copy / flate) with its measurement corrections.
+- `.dev/debt.yaml` D-515 — the standing drain anchor.
+- `docs/works/binary_size.md` — the public measured comparison.

.dev/debt.yaml

@@ -276,7 +276,7 @@ active:
       findings fold into the D-175 floor.)
     last_reviewed: "2026-06-24"
   - id: "D-517"
-    status: "IN PROGRESS 2026-07-16 (ADR-0173): C1+C2'+C3'+C4'+C5'-a LANDED. Measured: blob 698->425KB; shipped binary at this point 7,384,984 B (campaign total from 9,469,816 = -22.0%); startup neutral-within-noise except C4' memoize -165us median. Remaining: C5'-b compressed .clj sources (registerSource lazy-handle surface + cache_gen second blob; est -326KB), C6' has_handlers consumer flip (own bench), then the user-directed FULL cross-language bench + record refresh (task #5). ORIGINAL: open (ADR-0162 step 3)."
+    status: "IN PROGRESS 2026-07-16 (ADR-0173): C1-C5' ALL LANDED (C5'-b: compressed .clj sources, binary 7,073,080 B — campaign -25.3% from 9,469,816; startup 450KB gpa source-copy also gone via rt.source_resolver lazy decompress). Remaining: C6' has_handlers consumer flip (own bench), then L6 sort-dedup, then the user-directed FULL cross-language bench + record refresh (task list). ORIGINAL: open (ADR-0162 step 3)."
     category: "perf"
     barrier: |-
       ADR-0162 step 3. Sequenced AFTER D-516 (so it applies to both eager core + lazy

.dev/decisions/0172_binary_size_budget_and_ledger.md

@@ -104,7 +104,7 @@ components, and a breach localizes itself via the report tool:
 | zwasm (engine + api)   | 1.94 MB               | 2.5 MB                             | thunk collapse landed (v2.2.1); x86_64 emitter is comptime-gated (0 B on arm64) |
 | cljw text              | 2.57 MB               | 3.5 MB                             | F-013/F-014 comprehensiveness growth                            |
 | Zig std text           | 1.17 MB               | 1.5 MB                             |                                                                 |
-| embedded data          | 1.59 MB               | 1.75 MB                            | re-set to **1.0 MB** when L2 lands                              |
+| embedded data          | 0.76 MB               | 1.0 MB                             | L2 LANDED 2026-07-16 (ADR-0173 C2'-C5': pool + flate regions + flate .clj) |
 | unwind + linkedit etc. | 0.23 MB               | 0.3 MB                             | L1 LANDED 2026-07-16 (O-052): tables dropped, budget re-set     |
 | **Derived ceiling**    | **8.73 MB**           | **≈ 11 MB → ≈ 10.3 MB post-L2** |                                                                 |
 
@@ -406,6 +406,20 @@ dispositions grounded, platform binding declared, peer number measured).
 
 ## Revision history
 
+- **2026-07-16 (L2 complete + governance hardening)**: ADR-0173 C5′-b landed
+  compressed `.clj` sources (7,384,984 → **7,073,080 B**; the ~450 KB
+  startup gpa copy of raw text is also gone — `rt.source_resolver`
+  decompresses on demand and memoizes). The embedded-data budget line
+  re-sets to 1.0 MB per this ADR's own clause; derived ceiling ≈ 8.8 MB
+  (measured 7.07). **Campaign total: 9,469,816 → 7,073,080 (−25.3%).**
+  Governance mechanized further (user directive): `binary_size_report.sh
+  --check` now also FAILS when the built binary exceeds the derived ceiling
+  (`BUDGET_CEILING_BYTES`, kept in sync with this table), and the new
+  auto-loaded `.claude/rules/binary_size.md` carries the campaign's
+  operational lessons (instantiation-count metric, comptime cross-products,
+  embed-compression pattern, median-of-20 protocol) for every future
+  src/build edit.
+
 - **2026-07-16 (L5 outcome — same day)**: zwasm accepted the CODEV request
   (their ADR-0204) and shipped **v2.2.1** same-day: the JIT host-callback
   thunk collapse (`api.jit_host_bridge` 1,311 → 232 KB, −82%) — measured

.dev/decisions/0173_envelope_v7_zero_copy_pool_flate.md

@@ -200,6 +200,17 @@ measured per-commit (protocol below), not asserted.
   REAL 32 KB history window is required (the survey's §4 recipe is hereby
   corrected). C5′-b (compressed `.clj` sources, est −326 KB) remains.
 
+- **2026-07-16 (C5′-b landing — flate `.clj` sources)**: the raw sources
+  left the binary (cache_gen emits a second artifact, the compressed
+  sources blob; `bootstrap.sourceText` decompresses on demand into
+  `rt.load_arena`; `rt.source_resolver` serves error renders registry-miss
+  → memoized; cache_gen keeps raw @embedFile via the dual
+  `embed_raw_clj_sources` options module — its own stub `bootstrap_sources`
+  import avoids the WriteFiles dependency loop probed during landing).
+  **Measured**: shipped binary 7,384,984 → **7,073,080 B**. The eager
+  ~450 KB registerSource gpa copy at startup is gone (resolver is lazy).
+  Remaining in the arc: C6′ (has_handlers consumer flip).
+
 ## Measurement protocol (mandatory, per commit)
 
 - VM-hot A/B for C1 (representation change, isolated):

README.md

@@ -33,7 +33,7 @@ still use what other languages have already built.
 
 ## Features
 
-- **Small and quick to start** — one static binary of about 7.5 MB with the
+- **Small and quick to start** — one static binary of about 7.1 MB with the
   Wasm JIT engine included (for scale: babashka's native binary is ~71 MB;
   [measured comparison](docs/works/binary_size.md)), starting in ~5 ms, which
   suits short-lived, start-and-stop workloads (CLI tools, serverless, scripts).

build.zig

@@ -56,6 +56,7 @@ pub fn build(b: *std.Build) void {
     // a version literal. The user owns the value via the release tag.
     const build_zon = @import("build.zig.zon");
     build_options.addOption([]const u8, "version", build_zon.version);
+    build_options.addOption(bool, "embed_raw_clj_sources", false);
 
     // ROADMAP §9.6 / 4.8 / §349 — backend gate (ADR-0005 / ADR-0070 / F-012).
     // `vm` is the PRODUCTION DEFAULT (flipped 2026-06-02 once every D-196
@@ -105,13 +106,23 @@ pub fn build(b: *std.Build) void {
         .target = b.graph.host,
         .optimize = .ReleaseSafe,
     });
-    cache_gen_mod.addOptions("build_options", build_options);
+    // C5'-b (ADR-0173): cache_gen embeds the RAW .clj sources (host tool —
+    // its size is irrelevant) and is the producer of the compressed sources
+    // blob; the shipped cljw embeds only the blob. The flag picks the mode.
+    const cache_gen_options = b.addOptions();
+    cache_gen_options.addOption([]const u8, "version", build_zon.version);
+    cache_gen_options.addOption(Backend, "backend", backend);
+    cache_gen_options.addOption(bool, "wasm", wasm_enabled);
+    cache_gen_options.addOption(bool, "embed_raw_clj_sources", true);
+    cache_gen_mod.addOptions("build_options", cache_gen_options);
     if (zwasm_mod) |zm| cache_gen_mod.addImport("zwasm", zm);
     const cache_gen = b.addExecutable(.{ .name = "cache_gen", .root_module = cache_gen_mod });
     const run_cache_gen = b.addRunArtifact(cache_gen);
     const bootstrap_cache_blob = run_cache_gen.addOutputFileArg("bootstrap_core.cljc");
+    const bootstrap_sources_blob = run_cache_gen.addOutputFileArg("bootstrap_sources.bin");
     const cache_wf = b.addWriteFiles();
     _ = cache_wf.addCopyFile(bootstrap_cache_blob, "bootstrap_core.cljc");
+    _ = cache_wf.addCopyFile(bootstrap_sources_blob, "bootstrap_sources.bin");
     const cache_wrapper = cache_wf.add("bootstrap_cache.zig",
         \\// Generated by build.zig (ADR-0056 Cycle 2). Do not edit.
         \\// ADR-0173 C3': comptime-copy into align(8) rodata so the v7
@@ -123,6 +134,28 @@ pub fn build(b: *std.Build) void {
         \\
     );
     exe_mod.addAnonymousImport("bootstrap_cache", .{ .root_source_file = cache_wrapper });
+    // C5'-b: the flate-compressed .clj sources blob (label -> compressed
+    // text index; produced by cache_gen). The shipped cljw decompresses a
+    // file's text on demand (error-context render / source-replay fallback)
+    // instead of carrying 451KB of raw .clj in rodata.
+    const sources_wrapper = cache_wf.add("bootstrap_sources.zig",
+        \\// Generated by build.zig (ADR-0173 C5'-b). Do not edit.
+        \\pub const data: []const u8 = @embedFile("bootstrap_sources.bin");
+        \\
+    );
+    exe_mod.addAnonymousImport("bootstrap_sources", .{ .root_source_file = sources_wrapper });
+    // cache_gen runs in raw mode and never reads the blob — a stub keeps the
+    // @import("bootstrap_sources") in shared bootstrap code resolvable. The
+    // stub MUST come from its own WriteFiles: cache_wf depends on cache_gen's
+    // output, so wiring cache_gen's import through cache_wf is a dependency
+    // loop (cache_gen -> cache_wf -> run cache_gen).
+    const stub_wf = b.addWriteFiles();
+    const sources_stub = stub_wf.add("bootstrap_sources_stub.zig",
+        \\// Generated by build.zig (ADR-0173 C5'-b). cache_gen stub.
+        \\pub const data: []const u8 = "";
+        \\
+    );
+    cache_gen_mod.addAnonymousImport("bootstrap_sources", .{ .root_source_file = sources_stub });
 
     const exe = b.addExecutable(.{
         .name = "cljw",

docs/works/binary_size.md

@@ -78,8 +78,8 @@ Sizes cluster into three capability layers:
   honest claims are the two above: smallest Clojure-family binary, and a
   full-runtime-plus-Wasm-engine inside the 5-20 MB single-binary layer.
 - cljw's number is moving fast: the v1.3.1 row above is the SHIPPED release;
-  the working tree already measures **7.50 MB** (unwind-table strip, envelope
-  v7 constant pool, and zwasm v2.2.1's JIT-thunk collapse — a same-day
-  cross-repo campaign result), with embedded-data compression still queued
-  (**~7.1-7.2 MB** projected). The README figure is gate-checked against
+  the working tree already measures **7.07 MB** (−25% in one campaign:
+  unwind-table strip, envelope-v7 constant pool + flate-compressed lazy
+  regions and `.clj` sources, and zwasm v2.2.1's JIT-thunk collapse — a
+  same-day cross-repo result). The README figure is gate-checked against
   every release build, so it cannot silently rot.

scripts/binary_size_report.sh

@@ -31,6 +31,11 @@ if [[ "${1:-}" == "--check" ]]; then
 fi
 BIN="${1:-zig-out/bin/cljw}"
 
+# ADR-0172 §2 derived ceiling (per-component budgets sum). Keep in sync with
+# the ADR's budget table — a change here REQUIRES an ADR-0172 Revision
+# history entry (the budget moves consciously, never to silence this check).
+BUDGET_CEILING_BYTES=8800000
+
 if [[ ! -f "$BIN" ]]; then
     echo "binary_size_report: binary not found: $BIN (build first: zig build -Dwasm -Doptimize=ReleaseSafe)" >&2
     exit 1
@@ -49,6 +54,12 @@ if [[ "$MODE" == "check" ]]; then
         echo "size_claims: no '<N> MB' size claim found in README.md — add one (ADR-0172)" >&2
         exit 1
     fi
+    if [[ "$ACTUAL" -gt "$BUDGET_CEILING_BYTES" ]]; then
+        echo "size_claims: built binary ${ACTUAL_MB} MB (${ACTUAL} B) exceeds the ADR-0172 derived ceiling ($BUDGET_CEILING_BYTES B)." >&2
+        echo "  Attribute with 'bash scripts/binary_size_report.sh' (-Dprofile build for symbols), then land a lever" >&2
+        echo "  or consciously amend the budget table in .dev/decisions/0172_binary_size_budget_and_ledger.md." >&2
+        exit 1
+    fi
     DRIFT=$(awk "BEGIN{c=$CLAIM*1000000; d=($ACTUAL-c)/c; if(d<0)d=-d; printf \"%.3f\", d}")
     OK=$(awk "BEGIN{print ($DRIFT <= 0.10) ? 1 : 0}")
     if [[ "$OK" == "1" ]]; then

src/app/builder.zig

@@ -230,10 +230,13 @@ pub fn buildBootstrapEnvelope(
     for (files) |file| {
         // Register the file's bytes so a build-time error frame keeps its
         // per-file SourceContext (mirror of loadCoreFiles; idempotent).
-        try rt.registerSource(file.label, file.source);
+        // cache_gen (raw mode) takes the embedded slice; the in-test
+        // compressed-mode round-trip decompresses on demand (C5'-b).
+        const file_source = file.source orelse (bootstrap.sourceText(rt, file.label) orelse return error.MissingBootstrapSource);
+        try rt.registerSource(file.label, file_source);
         var chunks: std.ArrayList(BytecodeChunk) = .empty;
         defer chunks.deinit(allocator); // chunk slices are arena-owned; only the list is ours
-        var reader = Reader.init(arena, file.source);
+        var reader = Reader.init(arena, file_source);
         var form_idx: usize = 0;
         while (true) {
             const form = (try reader.read()) orelse break;
@@ -583,7 +586,8 @@ test "aot: core.clj round-trips — build envelope, restore into a fresh env, ru
         var table = macro_dispatch.Table.init(A);
         defer table.deinit();
         try bootstrap.setupCorePrefix(&rt, &env, &table);
-        core_bytes = try buildEnvelope(A, &rt, &env, &table, arena, bootstrap.CORE_SOURCE, "<core>");
+        const core_src = bootstrap.sourceText(&rt, bootstrap.SOURCE_LABEL) orelse return error.MissingBootstrapSource;
+        core_bytes = try buildEnvelope(A, &rt, &env, &table, arena, core_src, "<core>");
     }
     defer A.free(core_bytes);
 

src/app/repl.zig

@@ -110,7 +110,11 @@ pub fn run(
     // setupCorePrefix and leaves *ns* unresolved when test.clj loads (ADR-0083).
     try bootstrap.setupCorePrefix(&rt, &env, &macro_table);
 
-    const bootstrap_ctx = error_print.SourceContext{ .file = bootstrap.SOURCE_LABEL, .text = bootstrap.CORE_SOURCE };
+    // C5'-b (ADR-0173): bootstrap sources ship flate-compressed; the REAL
+    // text resolves through rt.source_resolver (registry fallback) at render
+    // time, so this eager fallback ctx carries no source bytes (empty text →
+    // the renderer's extractLine misses → registry path serves the line).
+    const bootstrap_ctx = error_print.SourceContext{ .file = bootstrap.SOURCE_LABEL, .text = "" };
     // ADR-0056 Cycle 2c + Cycle 3 (D-452 Part B): restore the WHOLE eager
     // bootstrap (core + the 23 non-core libs) from the embedded AOT envelope
     // (prefix already done above) — no .clj re-parse on startup.