.pnet cache-hit restore writes registry parameters but not cells — silent stuck terms and wrong answers

[kumavis]

Summary

Loading a module from the .pnet cache silently produces definitions that don't reduce. The registry readers are cell-primary, but the .pnet cache-hit restore path writes only the parameters, so every restored registry entry is invisible to every reader.

No error is raised. The type checker is unaffected (specs come through a different channel), so you get a well-typed stuck term.

Reproduction

minilib/minirepro/color.prologos:

ns minirepro::color

data Color
  red
  green

spec color-name Color -> String
defn color-name
  | red   -> "R"
  | green -> "G"

Driver: (ns t) + (imports (minirepro::color :refer-all)) + (eval (color-name red)), run via run-ns-all (or any process-string once the registry cells exist).

cold (color.pnet deleted):  lookup-ctor 'red  cell=<ctor-meta>  param=<ctor-meta>
                            result: "R" : String
warm (color.pnet present):  lookup-ctor 'red  cell=#f           param=<ctor-meta>
                            result: [reduce minirepro::color::red | red -> "R" | green -> "G"] : String

Root cause

Reader is cell-primary — macros.rkt:6300:

(define (read-ctor-registry)
  (or (macros-cell-read-safe (current-ctor-registry-cell-id)) (current-ctor-registry)))

macros-cell-read-safe returns the cell value whenever the cell-id and the persistent-registry net-box are both non-#f; the parameter is consulted only when there is no cell. So once cells exist, the parameter is dead for reads.

The normal elaboration writer dual-writes — macros.rkt:6294:

(define (register-ctor! name meta)
  (current-ctor-registry (hash-set (current-ctor-registry) name meta))
  (macros-cell-write! (current-ctor-registry-cell-id) (hasheq name meta)))

The .pnet cache-hit restore does not — driver.rkt:2637-2642 (same param-only shape repeats for every registry through :2705):

(current-ctor-registry
 (for/fold ([reg (current-ctor-registry)]) ([(k v) (in-hash d-ctor)])
   (hash-set reg k v)))

The .pnet file itself is fine: pnet-serialize.rkt:557-574 serializes the registries correctly and deserialize-module-state (:616) returns them intact. Nothing is lost in serialization — the restore writes to the wrong place.

It was demoted, not written wrong. Cell-primary readers landed in 7fec3751 (2026-03-18, "Track 7 Phase 6e: read functions cell-primary"); the merge landed in 2ef600ba (2026-03-24, "PM Track 10 Phase 2e: .pnet cache WORKING") — written against the parameter API six days after the readers stopped reading parameters. The sibling spec-propagation path in the same file gets it right (driver.rkt:2969-2976 sets the parameter and calls macros-cell-write!).

Why the reduce stays stuck

reduction.rkt:1294-1298:

(define (try-structural-reduce scrut arms)
  (define-values (head-name all-args) (decompose-app scrut))
  (and head-name
       (let ([meta (or (lookup-ctor head-name)
                       (lookup-ctor (ctor-short-name head-name)))])

lookup-ctor → read-ctor-registry → cell → #f, so meta is #f, try-structural-reduce returns #f, and whnf leaves the expr-reduce node in place.

Three severities from one defect

1. Silent stuck term (above).

2. Silent wrong answer. When a dependency comes from .pnet and the dependent is elaborated fresh, known-name? (macros.rkt:9205) and normalize-pattern (:9495-9510) consult the same cell-primary registries at elaboration time. A miss means the declared type name isn't "known" (→ auto-implicit free type variable, i.e. a phantom leading parameter) and constructor patterns degrade to catch-all variable patterns, so the first arm swallows everything:

   cold:    [fn [x <minirepro::tag::Tag>] [reduce x | t1 -> "one" | t2 -> "two"]]
   depwarm: [fn [x :0 <[Type 0]>] [fn [y <x>] "one"]]
   

   Two-module repro on production process-file (tag.pnet present, use.pnet absent, process-file called twice in one process): [tag-name t1] and [tag-name t2] both return "one".

3. Hard failure. With a partially-populated cache: imports: Error loading module <M>: Type mismatch. Good candidate for previously-unexplained flaky module-load failures.

Scope

General, not confined to any one construct. The repro is 8 lines with one data and one defn — no foreign, no imports, no prelude interaction.

14 of the 17 registries merged on that path are affected — every one with a cell-primary reader: preparse, ctor, type-meta, subtype, coercion, capability, trait, impl, param-impl, specialization, bundle, trait-laws, property, functor. Unaffected (parameter-only, no cell): multi-defn, tycon-arity-extension, defn-param-names. So trait/impl restoration is dropped identically — data + reduce is just the loudest consequence.

Precondition (why it isn't always broken)

The registry cells must already exist when the merge runs. init-macros-cells! (macros.rkt:581) snapshots params→cells, but process-file-inner runs preparse — which is where all module loading happens (macros.rkt:2666-2687) — at driver.rkt:2471/2479, and only calls init-macros-cells! afterwards at :2489.

So the first process-file in a fresh process is safe (cell-ids still #f → parameter fallback, then the snapshot bakes the restored values in). Every later process-file, and every process-string / process-string-ws (which never init the cells), is exposed.

Why CI is green

Three independent accidents, not correctness:

1. tools/pnet-compile.rkt:90 only generates .pnet for what (ns pnet-gen) pulls in — prelude modules only. Non-prelude lib modules never get one in CI.
2. tools/batch-worker.rkt:69 sets (current-pnet-write-enabled? #f), so test runs never create the missing ones either.
3. For the prelude subset that is a cache hit, tests/test-support.rkt:110-115 re-runs init-persistent-registry-network! + init-macros-cells! after the prelude load, re-snapshotting parameters into cells and healing exactly those entries.

And the one test that does exercise a cache hit, tests/test-record-pnet-cache.rkt, uses only def with map literals — no data, no match, no trait — so it asserts run-1 ≡ run-2 over precisely the unaffected surface.

Local dev hits it because raco test run directly (not via batch-worker) has .pnet writes enabled, so run 1 populates the cache and every later run is warm.

Silence is aggravated by driver.rkt:2590 wrapping deserialization in with-handlers ([exn? (lambda (_) #f)]), and preparse Pass -1 wrapping process-ns-declaration / process-imports in with-handlers ([exn:fail? void]) (macros.rkt:2681, :2686).

Ruled out

The unserialized imports field on module-network-ref is not the cause. Name resolution across the cache boundary works — the stuck term carries the fully-resolved FQN. Syncing only the ctor cell fixes the symptom while imports remains unserialized.

Fix options

Option A — dual-write at the merge (validated). Mirror what driver.rkt:2971 already does for the spec store: after each parameter set in driver.rkt:2634-2705, add

(macros-cell-write! (current-ctor-registry-cell-id) d-ctor)

macros-cell-write! is already exported (macros.rkt:369) and no-ops when the cell-id or net-box is #f, so pre-init and module-loading contexts are unaffected. The delta written is the deserialized hash; the cells' merge is merge-hasheq-replace, which preserves the accumulator's hash type, so equal?-keyed registries (subtype / coercion / specialization) are safe (see the comment at macros.rkt:591).

Validated: forcing the param→cell sync flips the repro from [reduce ...] STUCK to "R" : String, with no other change.

Tradeoff: preserves the two-writer duplication, so it needs a checklist entry ("new cell-backed registry ⇒ add to the .pnet restore dual-write") or the 15th registry regresses.

Option B — route the merge through the register-*! helpers, which already dual-write. Removes the duplication at the merge site. Tradeoff: not every registry has a per-entry registrar with matching semantics (subtype/coercion use equal?-keyed hash; capability/property have extra validation), and per-entry writes are more CHAMP operations than one bulk write.

Option C — retire the parameter fallback so cells are the single source of truth (the cells-over-parameters / PM Track 12 direction). The only option that eliminates the bug class. Tradeoff: the fallback is currently load-bearing — macros-cell-read-safe returns #f when there is no net-box, which is the real state during module loading and pre-init. Doing it properly means the registry cells must exist before any module load, i.e. moving init-macros-cells! ahead of preparse in process-file and giving process-string / process-string-ws the same initialization.

Regression coverage needed with any fix

tests/test-record-pnet-cache.rkt is the right template but must be extended (or paralleled) with:

• a data + match module asserting run-1 ≡ run-2 including a non-first constructor arm;
• a two-module case with the dependency cached and the dependent fresh.

Both must arrange for the registry cells to exist before the cache hit — run-ns-* from test-support.rkt does that naturally; the existing test's bespoke parameterize does not.

Adjacent, worth filing separately

pnet-stale? (pnet-serialize.rkt:511-517) keys freshness on "~a:~a" of source path + mtime with no transitive-dependency hashing (the comment admits it). A module's .pnet therefore stays "fresh" when a dependency's source changes — which is what generates the mixed fresh/stale cache states that turn this bug from latent into active.

---

Found while re-auditing the OCapN port against current main. Full write-up with the OCapN-side symptom history is in docs/tracking/2026-04-27_GOBLIN_PITFALLS.md entry #43.

🤖 Generated with Claude Code