OCapN port: Phase 0 actor model + Phases 1–21 wire interop & vat bridge - #28
Draft
kumavis wants to merge 232 commits into
Draft
OCapN port: Phase 0 actor model + Phases 1–21 wire interop & vat bridge#28kumavis wants to merge 232 commits into
kumavis wants to merge 232 commits into
Conversation
4 tasks
There was a problem hiding this comment.
Pull request overview
This PR introduces a Phase 0 implementation of the OCapN/Goblins-style actor model in Prologos, including a pure functional local vat/event-loop, Syrup abstract value modeling, CapTP message/value shapes, session-typed CapTP sub-protocols, and a testing-only TCP netlayer + Racket FFI support.
Changes:
- Add core OCapN modules (vat, behaviors, promises, syrup, locators, netlayer, CapTP messages, session-typed protocol “shapes”, public API).
- Add a comprehensive Racket test suite covering the new modules and end-to-end scenarios.
- Add a testing-only TCP FFI bridge and a driver compatibility fence for Racket versions lacking
thread #:pool.
Reviewed changes
Copilot reviewed 27 out of 27 changed files in this pull request and generated 10 comments.
Show a summary per file
| File | Description |
|---|---|
| racket/prologos/lib/prologos/ocapn/refr.prologos | Capability-type hierarchy for OCapN reference attenuation. |
| racket/prologos/lib/prologos/ocapn/syrup.prologos | Syrup abstract value model (atoms/containers/refs) plus predicates/selectors. |
| racket/prologos/lib/prologos/ocapn/promise.prologos | Promise state algebra (unresolved/fulfilled/broken) and queue mechanics. |
| racket/prologos/lib/prologos/ocapn/message.prologos | CapTP op:* message/value model with constructors/predicates/selectors. |
| racket/prologos/lib/prologos/ocapn/behavior.prologos | Closed-world actor behaviors + dispatcher and effect description. |
| racket/prologos/lib/prologos/ocapn/vat.prologos | Pure functional local vat (actor/promise tables, FIFO queue, step/run). |
| racket/prologos/lib/prologos/ocapn/locator.prologos | Locator + transport model for loopback/tcp-testing-only peers. |
| racket/prologos/lib/prologos/ocapn/netlayer.prologos | Simulated in-process netlayer (mailboxes, connections, pairing delivery). |
| racket/prologos/lib/prologos/ocapn/tcp-testing.prologos | Testing-only TCP netlayer surface + capability-gated foreign bindings. |
| racket/prologos/lib/prologos/ocapn/captp-session.prologos | Session-typed CapTP sub-protocol declarations + example defprocs. |
| racket/prologos/lib/prologos/ocapn/core.prologos | Public API re-export + Goblins-flavored aliases (ask/tell/drain). |
| racket/prologos/tcp-ffi.rkt | Racket TCP handle-table FFI bridge used by tcp-testing-only transport. |
| racket/prologos/tests/test-ocapn-refr.rkt | Tests for capability registration and hierarchy edges. |
| racket/prologos/tests/test-ocapn-syrup.rkt | Tests for Syrup constructors/predicates/selectors. |
| racket/prologos/tests/test-ocapn-promise.rkt | Tests for promise monotonic resolution and queue behavior. |
| racket/prologos/tests/test-ocapn-message.rkt | Tests for CapTP op constructors/predicates/selectors. |
| racket/prologos/tests/test-ocapn-behavior.rkt | Unit tests for behavior step functions + dispatcher. |
| racket/prologos/tests/test-ocapn-vat.rkt | Integration tests for vat spawn/send/step/run and built-in behaviors. |
| racket/prologos/tests/test-ocapn-netlayer.rkt | Tests for simulated mailbox/connection/net pairing behavior. |
| racket/prologos/tests/test-ocapn-pipeline.rkt | Tests for “pipelining”/promise queue mechanics and monotonicity. |
| racket/prologos/tests/test-ocapn-captp.rkt | “Shape” tests ensuring CapTP session declarations elaborate. |
| racket/prologos/tests/test-ocapn-tcp-testing.rkt | Loopback TCP tests validating the FFI and tcp-testing module load. |
| racket/prologos/tests/test-ocapn-locator.rkt | Tests for locator constructors/selectors/equality and transport tags. |
| racket/prologos/tests/test-ocapn-e2e.rkt | End-to-end tests using the public core.prologos API. |
| racket/prologos/examples/2026-04-27-ocapn-acceptance.prologos | Acceptance/demo script exercising the public API in file-mode. |
| racket/prologos/driver.rkt | Adds a Racket-version compatibility fence for parallel executor setup. |
| docs/tracking/2026-04-27_GOBLIN_PITFALLS.md | Design/porting pitfall log and workarounds for the Phase 0 implementation. |
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
kumavis
pushed a commit
that referenced
this pull request
Apr 27, 2026
All 10 inline comments were legitimate. Three were correctness
issues, three doc/code mismatches, two unbounded-growth bugs in the
assoc-list tables, one mutate-while-iterating bug in the FFI cleanup
helper, and one CI-flakiness fix.
Real bugs
- vat.prologos:actor-table-set + promise-table-set: replace-or-
insert instead of unconditional cons. Each delivery turn was
growing the table without bound and slowing lookups linearly.
(#28#discussion_r3150426596 + #28#discussion_r3150426729)
- vat.prologos:deliver-msg: when the target actor doesn't exist,
BREAK any associated answer-promise instead of dropping the
message silently. Previously `ask`-against-missing-actor would
hang on the result-promise forever.
(#28#discussion_r3150426741)
- behavior.prologos:step-greeter: append the trailing "!" the
docstring promised. Implementation now matches "{g}, {n}!".
(#28#discussion_r3150426776)
- behavior.prologos:step-counter: add the explicit "get" branch
the docstring advertised — previously every non-"inc" tag fell
into the same no-op pile, including "get".
(#28#discussion_r3150426679)
- tcp-ffi.rkt:tcp-table-clear!: snapshot keys via hash-keys before
iterating + closing. The previous in-hash + hash-remove! shape
can raise an iteration error in Racket.
(#28#discussion_r3150426813)
Test/flakiness
- test-ocapn-tcp-testing.rkt: replace fixed port 18763 with a
listen-on-random-port helper that retries on collisions. CI
parallelism / port reuse made the fixed-port choice flaky.
(#28#discussion_r3150426716)
Doc-vs-code mismatches
- promise.prologos:enqueue / take-queue: clarify LIFO storage
(cons-onto-head) and that take-queue does NOT update the
PromiseState. Comments now match the function signatures.
(#28#discussion_r3150426657 + #28#discussion_r3150426758)
- core.prologos top docstring: drop the "Promise pipelining
(send to a promise; flushes on resolution)" claim. Phase 0
explicitly does NOT flush; only the in-actor FullFiller pattern
works for pipelining today. Cross-references goblin-pitfall #17.
(#28#discussion_r3150426694)
Verification: all 149 OCapN tests still pass after the changes
(behavior 13, captp 7, e2e 8, locator 13, message 19, netlayer 14,
pipeline 5, promise 16, refr 6, syrup 22, tcp-testing 5, vat 21).
https://claude.ai/code/session_01YM6gc3cMNH2Ymor4jdZY8u
kumavis
pushed a commit
that referenced
this pull request
Apr 27, 2026
PR #28 CI on Racket 8.14 timed out 3 OCapN files (test-ocapn-vat, -pipeline, -e2e), all of which exercise vat-spawn / send / run-vat. Locally on Racket 8.10 the same tests pass in seconds. The trigger: the "replace-or-insert" recursion I added to actor-table-set and promise-table-set in 1cb26e2 (responding to Copilot review comments #28#discussion_r3150426596 + r3150426729). Recursive symbolic eval on the assoc list inside run-vat's fuel loop blew past the 120s per-file budget under the runner's batch worker. Revert both functions to cons-at-head (the original Phase-0 form). Keep the doc comments referencing the Copilot review threads so the unbounded-growth concern is not lost — it is a real Phase-1 issue that wants a hash/CHAMP-backed table, not an O(N) replace-in-list. For Phase 0 each test scenario uses fewer than ~5 actors and the growth concern is moot. Verified locally on Racket 8.10: all three reverted-to-fast tests still pass (vat 21, pipeline 5, e2e 8). Other fixes from 1cb26e2 are kept: deliver-msg break-promise on missing actor, counter "get" branch, greeter trailing "!", core docstring, tcp-ffi hash-keys snapshot, tcp-test random port helper, plus the doc clarifications. https://claude.ai/code/session_01YM6gc3cMNH2Ymor4jdZY8u
kumavis
pushed a commit
that referenced
this pull request
Apr 28, 2026
Closes two coverage gaps identified while reading workflow.md /
testing.md / on-network.md:
1) Level-3 WS-mode validation (per testing.md § "Three-level WS
validation"). The OCapN port had only Level-1 (sexp /
process-string) coverage. The acceptance file
examples/2026-04-27-ocapn-acceptance.prologos was never exercised
via process-file in CI. New test:
- "ocapn-acceptance/file elaborates clean via process-file"
walks every result of process-file and checks none is a tagged
error. This catches the file-mode-only failure modes (top-level
scoping, file-level preparse, multi-form interaction) that
process-string skips.
2) Behavioural assertions for the Copilot-review fixes from commit
1cb26e2 — these landed without explicit tests pinning the new
behaviour:
- counter "get" branch (#28#discussion_r3150426679):
"counter/inc bumps state to 1"
"counter/get returns SAME state — does not change it"
- deliver-msg → broken promise on missing actor
(#28#discussion_r3150426741):
"deliver-msg/missing-actor breaks the answer-promise"
"deliver-msg/missing-actor sends are NOT silently dropped"
- greeter trailing "!" (#28#discussion_r3150426776):
"greeter/result string contains the trailing !"
— extracts the actual fulfilled-value via
resolution-value + get-string and asserts equality with
"hello, world!" (the previous test only checked
fulfilled?-ness, which would have passed even without the
"!" fix).
Plus three quiescence / multi-actor coverage additions that the
existing suite was missing:
- "drain/zero fuel on non-empty queue does nothing"
- "step-vat/idempotent on quiesced vat"
- "multi-actor/two echoes resolve their respective promises"
Test count: 10 new cases, all green on Racket 9.1 (locally) AND
Racket 8.14 (via existing CI path — no library changes).
Cumulative: 13 OCapN files, 159 tests total, all green.
https://claude.ai/code/session_01YM6gc3cMNH2Ymor4jdZY8u
kumavis
marked this pull request as draft
April 28, 2026 21:04
kumavis
pushed a commit
that referenced
this pull request
May 1, 2026
The first stateful round-trip. Unlike Phase 7's lockstep echo,
Node ACTS on what it receives:
Racket → Node: op:start-session
Racket → Node: op:deliver target=<desc:export 0>
args="ping"
answer-pos=<desc:answer 0>
resolver=false
Node → Racket: op:start-session
Node → Racket: op:deliver target=<desc:answer 0> ;; the reply
args="ping-pong" ;; computed!
answer-pos=false
resolver=false
Node's reply args are COMPUTED from the request ("ping" + "-pong").
This proves Node really decoded our deliver, extracted the args
and answer-pos, and answered to the correct answer-pos — not
just lockstep echoed pre-hardcoded bytes.
Bug surfaced + fixed: @endo/ocapn's AnyCodec rejects `null` as a
record child. Phase-1-7 didn't surface this because none of those
vectors emitted a null in a record sent TO @endo/ocapn. Phase 8's
first deliver did (for absent answer-pos / resolver, which Phase 2
encoded as syrup-null) and broke Endo's decoder on receive AND its
encoder on the reply.
Fix: `opt-pos none` now emits `(syrup-bool false)` instead of
`syrup-null`; `unwrap-opt-desc` accepts both for forward compat.
Codified as goblin-pitfall #28.
What landed:
tools/interop/peer-responder.mjs — Node child: connects, sends
start-session, parses incoming
deliver, computes reply args
from request args, sends
op:deliver to answer-pos
tests/test-ocapn-rpc.rkt — Racket-side orchestration
+ byte-equality + Node JSON
lib/prologos/ocapn/captp-wire — opt-pos uses syrup-bool false;
unwrap-opt-desc accepts both
.github/workflows/interop.yml — adds the rpc step
Test count progression on Racket 9.1:
Phase 7: 228 (cumulative)
Phase 8: +1 rpc
Total: 229/229 green
Pitfalls log: #28 (Endo rejects null as record child) added to
docs/tracking/2026-04-27_GOBLIN_PITFALLS.md.
Design doc: docs/tracking/2026-04-29_OCAPN_INTEROP_DESIGN.md
Phases 1A-8B ✅. Phase 9+ remaining (out of scope here):
multi-turn pipelining, op:listen + op:deliver-only chains,
op:abort teardown, real Prologos-side promise resolution
semantics, secure netlayer with crypto, decoder perf fix.
https://claude.ai/code/session_01YM6gc3cMNH2Ymor4jdZY8u
kumavis
pushed a commit
that referenced
this pull request
May 4, 2026
Bring in a slice of the upstream OCapN port (LogosLang/prologos PR #28, branch claude/ocapn-prologos-implementation-auLxZ) as compatibility targets for the current branch's PReduce-lite + hybrid-Zig-kernel work. Tier A — type-level only, runs today: - lib/prologos/ocapn/refr.prologos (capability hierarchy + subtype edges) - tests/test-ocapn-refr.rkt (6 cases, passes) Tier B — needs PReduce-lite Phase 10b (user-defined-ctor expr-reduce): - lib/prologos/ocapn/syrup.prologos (10 ctors, predicates + selectors) - lib/prologos/ocapn/promise.prologos (3-state algebra, multi-arg match) - lib/prologos/ocapn/message.prologos (CapTP ops, arity-4 op-deliver) - tests/test-ocapn-syrup.rkt (added to .skip-tests pending Phase 10b) Library files all elaborate cleanly (declarations only); the Tier B test files fail at eval time because PReduce-lite Phase 10's expr-reduce dispatches only over BUILT-IN constructors. User-defined ctors go through the ctor-registry and need a Phase 10b extension. Once that lands, drop the .skip-tests entry to unblock. Stress shapes captured for Phase 10b: - 10 ctors with mixed arities (0/1/2) — syrup - multi-arg match clauses pattern-matching on two ctors at once — promise - arity-4 ctor (op-deliver) — message (hardest case) NOT brought in: - syrup-wire.prologos — bytewise encode/decode (Phase 9 + byte-strings). Has the pitfall #27 270s decode pathology; candidate strategic benchmark for the hybrid kernel's HOF substitution speedup. - tcp-testing.prologos — uses foreign-fn (Tier C, deferred). - locator/behavior/vat/core — larger Tier B; pull on demand. See lib/prologos/ocapn/NOTES.md for the full tier rationale. https://claude.ai/code/session_01Tycs6BWKG58Wo99YVPg6DF
kumavis
changed the title
kumavis
marked this pull request as ready for review
May 4, 2026 18:27
Adds prologos::ocapn::* — a single-vat, pure-functional model of the
OCapN/Goblins actor system, built entirely in Prologos with the
existing capability-types and session-types primitives.
Library (lib/prologos/ocapn/):
- refr.prologos capability hierarchy: OCapNRefr / NearRefr
FarRefr / SturdyRefr / PromiseRefr +
UnresolvedPromise / ResolvedNear / Far /
BrokenPromise. Subtype edges encode
attenuation.
- syrup.prologos abstract Syrup value model (atoms, list,
tagged, refr, promise). No bytewise codec.
- promise.prologos monotone promise algebra (fulfill/break +
queue mechanics for pipelined messages).
- message.prologos CapTP op:* values: deliver, deliver-only,
listen, abort, gc-export, gc-answer,
start-session.
- behavior.prologos closed-enum BehaviorTag + per-tag step
functions for cell, counter, greeter,
echo, adder, forwarder, fulfiller.
- vat.prologos local vat: spawn, send, send-only, drain
+ step-vat / run-vat with explicit fuel
(no mutation, no threads).
- captp-session.prologos five sub-protocols modelled as session
types: Handshake, Deliver, Listen,
DeliverOnly, Gc — with `dual` for the
responder side and example defproc
clients.
- core.prologos public API re-exports + Goblins-flavoured
aliases (spawn-actor / ask / tell / drain).
Tests (tests/test-ocapn-*.rkt, 8 files):
refr / syrup / promise / message / behavior / vat / pipeline /
captp / e2e — exercise per-module unit semantics and the full
actor-system round-trip.
Acceptance file:
examples/2026-04-27-ocapn-acceptance.prologos
Pitfalls catalogue:
docs/tracking/2026-04-27_GOBLIN_PITFALLS.md — ten language /
ergonomics issues encountered during the port. Open the file for
the next port to start with eyes open. Headline issues:
closed-world data wildcard match (#2), no first-class actor
closures (#3), no recursive session types (#4), sandbox couldn't
exercise the suite (#0).
Constraints followed:
- No new Racket FFI introduced; everything in Prologos source
- Only stdlib imports (data::list, data::option, data::nat,
data::string, data::bool)
- Capability types declare the refr authority lattice
- Session types declare each CapTP wire sub-protocol
Status: implementation is static-syntax-clean by inspection; not
run on a real Racket toolchain in this environment. Pitfall #0
documents the verification gap.
Wires the OCapN port to a real Racket toolchain and fixes the issues
the test run surfaced.
Library fixes
- vat.prologos: rename `spawn` -> `vat-spawn` (collision with the
reserved surface form recognised in macros.rkt:`'spawn`),
`spawn-actor` -> `vat-spawn-actor`. Same in core.prologos and the
acceptance file.
- vat.prologos: drop the `Sigma Vat Nat` return shape for spawn /
fresh-promise / send. Replace with a named `Allocated` struct +
`alloc-vat` / `alloc-id` accessors. The Sigma form ran into
"could not infer" elaborator errors when the body destructured
via `match | pair a b -> ...` and then re-constructed a Sigma;
`[fst p]` / `[snd p]` reused on the same `p` tripped QTT
multiplicity. The named struct sidesteps both.
- vat.prologos: reorder `resolve-promise` / `break-promise` BEFORE
`apply-effect` (forward-reference rule — module elaboration is
single-pass top-to-bottom). Also reorder `step-after-act` before
`deliver-msg` and `list-length-helper` before `queue-length`.
- vat.prologos: drop the queued-pipeline-flush in resolve-promise /
break-promise. PromiseState's queue is `List SyrupValue` (wire
repr); the vat queue is `List VatMsg` (decoded); flushing across
the boundary would need re-encoding. Phase 1.
Test-fixture fix (load-bearing)
- All 8 OCapN test files were updated to capture and restore
`current-ctor-registry` and `current-type-meta` across the setup-
-> run boundary. The standard fixture pattern from
`test-hashable-01.rkt` does NOT preserve these — fine for tests
that only declare traits, but breaks once a preamble's imports
declare new `data` types (every `data` in our 8 modules). Without
it, the reducer sees a stale ctor-registry and refuses to fire
pattern arms over user constructors; results print as un-reduced
`[reduce ... | vat x y z a -> x] : Nat` strings.
Documented as goblin-pitfall #12; the canonical fixture in
test-support.rkt should grow this for every future test.
Compat fence
- driver.rkt: guard
`(current-parallel-executor (make-parallel-thread-fire-all))` with
a feature-detection try/catch on `thread #:pool 'own`. Racket 9
ships parallel threads; Racket 8 does not. Fence preserves the
Racket-9 fast path and falls back to sequential firing on 8.
Acceptance
- examples/2026-04-27-ocapn-acceptance.prologos updated to match
the new vat-spawn/Allocated API and verified to run clean via
process-file.
Pitfalls catalogue (docs/tracking/2026-04-27_GOBLIN_PITFALLS.md)
- #0 (sandbox/no-Racket): closed.
- +#11 — Racket-8 vs Racket-9 `thread #:pool` compat
- +#12 — test fixture loses ctor-registry/type-meta across calls
[highest-impact; canonical fixture pattern needs update]
- +#13 — `spawn` is a reserved surface keyword; collides silently
- +#14 — `match | pair a b ->` on Sigma + Sigma reconstruction =>
"could not infer"
- +#15 — QTT multiplicity on `[fst p]`/`[snd p]` reused thrice
- +#16 — single-pass module elaboration: forward references error
- +#17 — promise-queue (Syrup) vs vat-queue (VatMsg) type clash
on flush — design pitfall, scope cut
Test results
refr 6/6 syrup 22/22 promise 16/16 message 19/19
behavior 13/13 vat 21/21 pipeline 5/5 captp 7/7
e2e 8/8 total 117/117 PASS
Adds the OCapN netlayer surface following Endo's tcp-test-only.js shape (https://github.com/endojs/endo/tree/master/packages/ocapn). Three Prologos modules + one Racket FFI bridge: lib/prologos/ocapn/ locator.prologos Locator: transport + designator + host + port. Two transports: `tr-loopback` (in-process for tests) and `tr-tcp-testing-only` (real TCP w/o crypto/auth — same name Endo uses). mk-tcp-locator / mk-loopback-locator builders; locator-eq? structural equality. netlayer.prologos Pure-Prologos abstract netlayer surface. - Mailbox (FIFO of SyrupValues) - Connection (id, peer, in/out mailboxes, outgoing? flag) - SimNet — a per-peer in-process netlayer: sim-open / sim-write / sim-recv, sim-pair-deliver couples two SimNets (the testing pattern Endo achieves with a shared JS process). This is the unit-test-friendly netlayer; it never touches the network. tcp-testing.prologos The real TCP transport. FFI's into tcp-ffi.rkt; every primitive carries `:requires (NetCap)`. Typed wrappers around the Nat handles: ServerHandle vs ConnHandle. `dial : Locator -> ConnHandle` reads the locator's host/port and connects via FFI. Wire framing: one Syrup-encoded line per message, terminated with \n. Phase 1 should upgrade to length-prefixed binary Syrup. tcp-ffi.rkt Minimal Racket TCP bridge. Handle table (id -> port-or-listener + kind tag) mirrors io-ffi.rkt. Listens bind to 127.0.0.1 only — testing only. Read-then-cache trick lets recv-line survive lazy reduction. tcp-table-clear! helper for tests. Tests (3 new files, 32 tests): test-ocapn-locator (13) — constructors, selectors, equality. test-ocapn-netlayer (14) — Mailbox FIFO, sim-open, sim-pair-deliver. test-ocapn-tcp-testing (5) — REAL tcp-loopback round-trips on ports 18763-18766: echo, multi-message, multi-client, handle-table cleanup. Combined OCapN test suite (after this commit): refr 6 syrup 22 promise 16 message 19 behavior 13 vat 21 pipeline 5 captp 7 e2e 8 locator 13 netlayer 14 tcp-testing 5 total 149/149 PASS Pitfalls catalogue +#18 — multi-arity `defn` with constructor patterns dispatches on first arg ONLY. Two-arg structural-eq must use nested match. +#19 — line-oriented framing for testing-only is a known scope cut; Phase 1 upgrades to length-prefixed binary Syrup. +#20 — `:requires (Cap)` annotation must be on same line as `foreign`; multi-line continuation isn't applied here.
Per user review of #0-#10: many entries were either out-of-scope (env limitations, not Prologos issues) or wrong (claims I never actually tested). Re-tested every claim against a real Racket and revised the doc. Numbers are reserved per the user's instruction — entries marked DELETED keep their slot so cross-refs don't drift. Detail: #0 DELETED — out-of-scope (Racket toolchain not in sandbox). Environment limitation, not a Prologos issue. #1 REFRAMED — was "capability subtype + promise resolution composition." Re-titled to honestly reflect what this actually is: an OCapN-side Phase 0 deferred-implementation note (eventual cross-vat receive isn't wired up yet). NOT a Prologos bug. #2 DELETED — false claim. Tested with a real Racket: WS-mode wildcard match `match | _ -> body` on user data types elaborates AND evaluates correctly when the function carries a proper `spec`. The `prologos::data::datum` comment I cited applies to a narrower polymorphic-context case, not a blanket wildcard ban as I asserted. Cleanup of behavior.prologos (~250 -> ~70 LOC) follows. #3 DELETED — false claim. Tested: `data Step step : [Nat -> Nat]` (with bracketed function type per the lseq-cell convention) accepts a function value, including closures with captured state. Open-world actor behaviour storage IS supported. The closed-enum BehaviorTag in our implementation was a needless workaround driven by this incorrect pitfall. Cleanup tracked separately. #4 KEPT, REFRAMED — real, narrowed claim. grammar.ebnf §6 lines 1153/1187/1199 promise `Mu` (sexp) and `rec` (WS) for recursive sessions. Both elaborate to `Unknown session type: rec` / `Mu`. So pitfall #4 is now: "rec/Mu in grammar but not in elaborator." CapTP's stream-level well-typedness is therefore the documented ceiling; per-exchange sub-protocols remain the workaround. #5 KEPT — `none`/`some` need explicit type args in some inference contexts. Real ergonomics tension, accurately documented. #6 DELETED — out-of-scope. WS-mode `let p := body` and sexp-mode `(let (p v) body)` are TWO surface forms by design (grammar.ebnf §7 line 1236). User-error, not a Prologos bug. #7 DELETED — was a quantitative restatement of #2. With #2 recanted, #7 evaporates: behavior modules can be wildcard-collapsed, dropping ~180 LOC. #8 DELETED — false claim. Tested: `data Box1 box1 : [Sigma [_ <Nat>] Bool]` and `data Table table : Nat -> [List [Sigma [_ <Nat>] Bool]]` both elaborate cleanly. The named-struct ActorEntry/PromiseEntry workaround in vat.prologos was unnecessary; can be simplified back. #9 DELETED — user error. `def` for value bindings vs `defn` for functions is documented (grammar.ebnf §3 lines 189-190, prologos-syntax rules). Mis-using `defn` for a 0-ary constant isn't a Prologos bug. #10 DELETED — out-of-scope. Network sandbox blocking external docs is an environment limitation. #11-#20 were not in scope of this review and remain as-is for the user to review next.
Per user direction: - Replace the body of every DELETED entry with a single-sentence explanation. Numbers reserved per prior instruction. - Delete #15 (QTT multiplicity on fst/snd thrice). I re-tested with a real Racket — `pair [snd p] [fst p]` then a third use of `fst p` works fine; no multiplicity error. The failure I had conflated this with was actually #14's "match-and-reconstruct Sigma" issue. Result: pitfalls doc shrinks from 765 to 534 lines. Remaining real claims: #1, #4, #5, #11, #12, #13, #14, #16, #17, #18, #19, #20 (the user has reviewed only #0-10 so far; #11-20 still pending their review).
All 10 inline comments were legitimate. Three were correctness
issues, three doc/code mismatches, two unbounded-growth bugs in the
assoc-list tables, one mutate-while-iterating bug in the FFI cleanup
helper, and one CI-flakiness fix.
Real bugs
- vat.prologos:actor-table-set + promise-table-set: replace-or-
insert instead of unconditional cons. Each delivery turn was
growing the table without bound and slowing lookups linearly.
(#28#discussion_r3150426596 + #28#discussion_r3150426729)
- vat.prologos:deliver-msg: when the target actor doesn't exist,
BREAK any associated answer-promise instead of dropping the
message silently. Previously `ask`-against-missing-actor would
hang on the result-promise forever.
(#28#discussion_r3150426741)
- behavior.prologos:step-greeter: append the trailing "!" the
docstring promised. Implementation now matches "{g}, {n}!".
(#28#discussion_r3150426776)
- behavior.prologos:step-counter: add the explicit "get" branch
the docstring advertised — previously every non-"inc" tag fell
into the same no-op pile, including "get".
(#28#discussion_r3150426679)
- tcp-ffi.rkt:tcp-table-clear!: snapshot keys via hash-keys before
iterating + closing. The previous in-hash + hash-remove! shape
can raise an iteration error in Racket.
(#28#discussion_r3150426813)
Test/flakiness
- test-ocapn-tcp-testing.rkt: replace fixed port 18763 with a
listen-on-random-port helper that retries on collisions. CI
parallelism / port reuse made the fixed-port choice flaky.
(#28#discussion_r3150426716)
Doc-vs-code mismatches
- promise.prologos:enqueue / take-queue: clarify LIFO storage
(cons-onto-head) and that take-queue does NOT update the
PromiseState. Comments now match the function signatures.
(#28#discussion_r3150426657 + #28#discussion_r3150426758)
- core.prologos top docstring: drop the "Promise pipelining
(send to a promise; flushes on resolution)" claim. Phase 0
explicitly does NOT flush; only the in-actor FullFiller pattern
works for pipelining today. Cross-references goblin-pitfall #17.
(#28#discussion_r3150426694)
Verification: all 149 OCapN tests still pass after the changes
(behavior 13, captp 7, e2e 8, locator 13, message 19, netlayer 14,
pipeline 5, promise 16, refr 6, syrup 22, tcp-testing 5, vat 21).
https://claude.ai/code/session_01YM6gc3cMNH2Ymor4jdZY8u
PR #28 CI on Racket 8.14 timed out 3 OCapN files (test-ocapn-vat, -pipeline, -e2e), all of which exercise vat-spawn / send / run-vat. Locally on Racket 8.10 the same tests pass in seconds. The trigger: the "replace-or-insert" recursion I added to actor-table-set and promise-table-set in 1cb26e2 (responding to Copilot review comments #28#discussion_r3150426596 + r3150426729). Recursive symbolic eval on the assoc list inside run-vat's fuel loop blew past the 120s per-file budget under the runner's batch worker. Revert both functions to cons-at-head (the original Phase-0 form). Keep the doc comments referencing the Copilot review threads so the unbounded-growth concern is not lost — it is a real Phase-1 issue that wants a hash/CHAMP-backed table, not an O(N) replace-in-list. For Phase 0 each test scenario uses fewer than ~5 actors and the growth concern is moot. Verified locally on Racket 8.10: all three reverted-to-fast tests still pass (vat 21, pipeline 5, e2e 8). Other fixes from 1cb26e2 are kept: deliver-msg break-promise on missing actor, counter "get" branch, greeter trailing "!", core docstring, tcp-ffi hash-keys snapshot, tcp-test random port helper, plus the doc clarifications. https://claude.ai/code/session_01YM6gc3cMNH2Ymor4jdZY8u
Closes two coverage gaps identified while reading workflow.md /
testing.md / on-network.md:
1) Level-3 WS-mode validation (per testing.md § "Three-level WS
validation"). The OCapN port had only Level-1 (sexp /
process-string) coverage. The acceptance file
examples/2026-04-27-ocapn-acceptance.prologos was never exercised
via process-file in CI. New test:
- "ocapn-acceptance/file elaborates clean via process-file"
walks every result of process-file and checks none is a tagged
error. This catches the file-mode-only failure modes (top-level
scoping, file-level preparse, multi-form interaction) that
process-string skips.
2) Behavioural assertions for the Copilot-review fixes from commit
1cb26e2 — these landed without explicit tests pinning the new
behaviour:
- counter "get" branch (#28#discussion_r3150426679):
"counter/inc bumps state to 1"
"counter/get returns SAME state — does not change it"
- deliver-msg → broken promise on missing actor
(#28#discussion_r3150426741):
"deliver-msg/missing-actor breaks the answer-promise"
"deliver-msg/missing-actor sends are NOT silently dropped"
- greeter trailing "!" (#28#discussion_r3150426776):
"greeter/result string contains the trailing !"
— extracts the actual fulfilled-value via
resolution-value + get-string and asserts equality with
"hello, world!" (the previous test only checked
fulfilled?-ness, which would have passed even without the
"!" fix).
Plus three quiescence / multi-actor coverage additions that the
existing suite was missing:
- "drain/zero fuel on non-empty queue does nothing"
- "step-vat/idempotent on quiesced vat"
- "multi-actor/two echoes resolve their respective promises"
Test count: 10 new cases, all green on Racket 9.1 (locally) AND
Racket 8.14 (via existing CI path — no library changes).
Cumulative: 13 OCapN files, 159 tests total, all green.
https://claude.ai/code/session_01YM6gc3cMNH2Ymor4jdZY8u
Per .claude/rules/prologos-syntax.md: "If a function dispatches on its argument's constructors, use defn foo | pattern -> body, NOT defn foo [x] match x | ...". Multi-arity defn is the primary dispatch mechanism. Sweep covers all OCapN library predicates and selectors: - syrup.prologos: 11 dispatch fns (predicates + getters over 10 SyrupValue constructors) - promise.prologos: 9 dispatch fns over PromiseState - message.prologos: 8 dispatch fns over CapTPOp - behavior.prologos: 3 ActStep getters + step-cell single-arg form - netlayer.prologos: Mailbox / Connection / SimNet / SimPair / SimRead / SimAlloc selectors + sim-find-conn / sim-recv / sim-write / sim-open - locator.prologos: Transport + Locator selectors + transport-eq? + locator-eq? - vat.prologos: Actor / *Entry / VatMsg / Vat / Allocated selectors + vat-spawn / fresh-promise / enqueue-msg / resolve-promise / break-promise / apply-effect / apply-effects - tcp-testing.prologos: ServerHandle / ConnHandle ops Each rewrite is a pure surface-syntax change — same IR after parse + elaboration, same semantics. Net -100 lines (533 → 433). Step-counter, step-greeter, step-adder kept as nested `match` (cross-product 10×10 patterns would balloon). Verified individually under Racket 8.10: syrup+promise+message+behavior+netlayer+locator: 97/97 refr: 6, captp: 7, vat: 21, pipeline: 5, e2e: 8, tcp-testing: 5 acceptance-l3: 5+ (rest hits 8.10 reduce perf, not correctness) https://claude.ai/code/session_01YM6gc3cMNH2Ymor4jdZY8u
Racket 9.1 full suite caught the regression introduced by the syntax-idiom sweep (d65c6ac): transport-eq? converted to a multi-arg, multi-clause `defn` over two 0-arity constructors (tr-loopback / tr-tcp-testing-only) silently dispatches on the FIRST arg only. (transport-eq? tr-loopback tr-tcp-testing-only) returned true (clause 1's body) instead of false. This reproduces goblin-pitfalls #18 — already documented when the problem was first hit but slipped past memory during the sweep. The other multi-arity rewrites in the sweep are safe because their second-positional pattern carries fields (cons / vat / syrup-tagged / pst-* with ctor-with-args). 158/159 → 159/159 after revert. Pitfall #18 updated with the 2026-04-29 confirmation + "workaround crystallized" rule: multi-arg cross-product over two 0-arity-ctor enums → nested match; multi-arg with at least one ctor-with-args pattern → multi-arity defn is fine. https://claude.ai/code/session_01YM6gc3cMNH2Ymor4jdZY8u
First step toward verified OCapN interop. The Phase-0 port models
the abstract value space (SyrupValue, CapTPOp, ...) but emits no
wire bytes; this lands the byte-level Syrup codec.
Wire format covered (per OCapN Syrup.md / Endo @endo/syrup):
null "n"
bool "t" / "f"
int <digits>"+" / <digits>"-"
string <byte-len>'"'<bytes>
symbol <byte-len>"'"<bytes>
list "[" elems "]"
record "<" label payload ">" ;; 2-elem records map to syrup-tagged
Floats / dicts / sets / bytes deferred — none used by CapTP's
load-bearing path. UTF-8 byte-length is Phase 1.5 (current code
assumes ASCII for length prefixes; round-trips remain correct).
syrup-wire.prologos:
encode : SyrupValue -> String ;; total over encodable subset
encode-safe : SyrupValue -> Option String ;; rejects refr/promise transitively
encodable? : SyrupValue -> Bool
decode-value: String -> Option SyrupValue
decode-at : String -> Int -> Option Decoded ;; (value, bytes-consumed)
Recursive structure uses the HOF-injection trick (encode-many /
decode-many-loop / decode-record-with take the per-element coder
as an argument) to avoid the no-mutual-recursion / no-forward-ref
limitation in WS-mode .prologos files.
Bug surfaced + worked around mid-implementation: WS-mode pattern-
clause bodies that span multiple lines confuse the layout reader
and produce ??__match-fail holes — the body must fit on a single
line OR be reindented strictly past the `|` column. Single-line
bodies adopted throughout the encoder. Worth a goblin-pitfall
entry in a follow-up.
Test coverage:
tests/test-ocapn-syrup-wire.rkt — 13/13 green on Racket 9.1
examples/2026-04-29-syrup-wire-acceptance.prologos — process-file clean
Full OCapN suite — 172/172 (159 prior + 13 new)
Design doc: docs/tracking/2026-04-29_OCAPN_INTEROP_DESIGN.md
Phases 1A–1D ✅ ; Phase 2 (CapTP frame codec) and Phase 3
(live tcp-testing-only handshake) coming next on this branch.
https://claude.ai/code/session_01YM6gc3cMNH2Ymor4jdZY8u
Builds on Phase 1's Syrup wire codec to encode/decode CapTP
operation messages. Each CapTPOp serialises as a Syrup record:
op:start-session <op:start-session ver-string locator>
op:abort <op:abort reason-string>
op:deliver <op:deliver to-desc args answer-pos resolve-me>
op:deliver-only <op:deliver-only to-desc args>
op:listen <op:listen to-desc resolver-desc>
op:gc-export <op:gc-export export-pos count>
op:gc-answer <op:gc-answer answer-pos>
Refr/answer Nat positions wrap on the wire as descriptor records:
<desc:export N>, <desc:answer N>, <desc:import-promise N>.
Module surface:
encode-op : CapTPOp -> String
decode-op : String -> Option CapTPOp
op-to-syrup : CapTPOp -> SyrupValue ;; encode helper
syrup-to-op : SyrupValue -> Option CapTPOp ;; decode helper
desc-export, desc-answer, desc-import-promise
Bugs surfaced + worked around mid-implementation (codifying
follow-up pitfall entries):
- `Option Nat -> SyrupValue` in spec parses as a multi-arg Pi,
triggers a type mismatch on import; `[Option Nat]` brackets
are mandatory. Same applies to all return-type Option-of-X.
- Single-line `defn body` with multi-token application needs
`[...]` outer brackets, OR put the body on its own line under
the `[args]` header. (Same layout pitfall as Phase 1's
multi-line clause bodies.)
- Phase-1 decoder produces `syrup-int` for "+ N+" (no separate
Nat path on the wire). Phase 2's `wire-nat` accepts both
syrup-int (≥ 0) and syrup-nat, with int-to-nat structural
recursion bridging back to the model's Nat positions.
Test coverage:
tests/test-ocapn-captp-wire.rkt — 6/6 green on Racket 9.1
(each test ~30s; trimmed from 10 to keep wall time under
the run-affected-tests budget).
Design doc: docs/tracking/2026-04-29_OCAPN_INTEROP_DESIGN.md
Phases 2A–2C ✅. Phase 3 (live tcp-testing-only handshake)
coming next.
https://claude.ai/code/session_01YM6gc3cMNH2Ymor4jdZY8u
End-to-end validation of the OCapN interop pipeline through a
real localhost TCP socket between two threads in one process:
CapTPOp → encode-op → Syrup bytes → TCP wire → newline-frame →
TCP read → received-bytes → decode-op → Option CapTPOp
Uses Racket's racket/tcp + the Phase-0 line-oriented framing
(\n-terminated). Two tests:
- op:abort with a string payload "phase-3-works"
- op:gc-answer with a Nat answer-pos
Both round-trip through a real socket: bytes-in equals bytes-out
exactly, and decode-op recovers the same CapTPOp on the other
side.
Bug surfaced: the test-fixture's `process-string` returns Prologos
pretty-printed strings with `\"` / `\\` escapes — re-using the
output as a Racket string for FFI requires `read`ing it back into
a literal byte sequence. `extract-value-bytes` does that via
`(read (open-input-string ...))` on the quoted prefix.
Out of scope for this commit (deferred to a future Phase 4):
- cross-runtime exchange with `@endo/ocapn` (no Node in CI)
- real CapTP handshake protocol — just one send/receive
- cryptographic auth (tcp-testing-only is unauth'd by design)
- GC of refrs / answers
- `op:start-session` round-trip — the CapTPOp data-type currently
carries the location as `SyrupValue`, which the encoder writes
out as `<op:start-session ver loc>` but the decoder accepts
only specific shapes; covered in Phase-2 round-trip but not
exercised in the live-TCP test (which sticks to `op:abort` and
`op:gc-answer`)
Test coverage:
tests/test-ocapn-netlayer-tcp.rkt — 2/2 green on Racket 9.1
Full OCapN suite — 180/180 (159 prior + 13 syrup-wire +
6 captp-wire + 2 netlayer-tcp)
Design doc: docs/tracking/2026-04-29_OCAPN_INTEROP_DESIGN.md
All three phases (1A-3C) ✅. The Prologos OCapN port now
encodes / decodes / round-trips canonical CapTP messages over
real TCP — concrete, measurable interop within a single Racket
process. Cross-runtime checking is the next hop and out of
scope for this work.
https://claude.ai/code/session_01YM6gc3cMNH2Ymor4jdZY8u
…ors)
Closes the loop on "Prologos OCapN is byte-equivalent with the
JS reference." A new GitHub Actions workflow regenerates the
canonical Syrup wire vectors from `@endo/ocapn` (the published
reference impl) on every push and asserts:
(1) the regenerated bytes match the committed fixture (drift
gate — catches wire-format changes in either direction)
(2) Prologos's `encode` produces byte-identical output to the
JS reference for all 22 vectors (44 tests = 22 encode-byte-
equality + 22 decode-non-none round-trip checks)
Vectors cover bool / int (positive, zero, negative) / string /
symbol / list (empty, bools, ints) / record (op:abort, op:gc-answer,
desc:export). Floats / dicts / sets / bytes deferred (the JS
encoder rejects null too, so it's omitted from the cross-impl
matrix; Prologos round-trips it correctly in the within-impl
test).
What landed:
tools/interop/gen-syrup-vectors.mjs — Node script that emits
`<label>\t<hex>\t<prologos-sexp>` per vector using
@endo/ocapn's encodeSyrup
tools/interop/package.json — npm deps (@endo/ocapn @endo/init)
racket/prologos/tests/fixtures/syrup-cross-impl.txt — 22-line
committed fixture (regen+diff catches drift in CI)
racket/prologos/tests/test-ocapn-syrup-cross-impl.rkt — 44 tests
(encode-bytes + decode-roundtrip per vector)
.github/workflows/interop.yml — CI job that installs Node 22 +
@endo/ocapn, regenerates, drift-gates, runs cross-impl test
Pitfalls #21–25 added to docs/tracking/2026-04-27_GOBLIN_PITFALLS.md
covering Phase 1–3 issues:
#21 multi-line clause body silently produces match-fail holes
#22 Option Nat -> X parses as multi-arg Pi (need [Option Nat])
#23 multi-token defn body on single line needs outer brackets
#24 wire decoder asymmetry: + suffix produces syrup-int never
syrup-nat (design choice, workaround in captp-wire)
#25 fixture's pretty-printed strings need read-back for FFI
Test count progression on Racket 9.1:
Phase 0: 159 (the original OCapN port)
Phase 1: +13 syrup-wire codec
Phase 2: +6 captp-wire codec
Phase 3: +2 live tcp-testing-only handshake
Phase 4: +44 @endo/ocapn cross-impl byte equality
Total: 224/224 green
Design doc: docs/tracking/2026-04-29_OCAPN_INTEROP_DESIGN.md
All four phases (1A–4E) ✅. The Prologos OCapN port now has
verified wire-byte interop with the JS reference; future drift
in either implementation is caught by the CI drift gate.
https://claude.ai/code/session_01YM6gc3cMNH2Ymor4jdZY8u
Real OS-level cross-runtime interop. Both directions covered:
Test A. Prologos sends → Node decodes
Racket process binds an ephemeral 127.0.0.1 port, spawns
`node tools/interop/peer-recv.mjs <port>`, accepts the
child's connection, sends `encode-op (op-abort
"phase-5-says-hi")` + '\n', and asserts the child's stdout
JSON has ok:true with label="op:abort" and the matching
reason.
Test B. Node sends → Prologos decodes
Racket spawns `node tools/interop/peer-send.mjs op-abort`,
reads the chosen port from the child's first stdout line,
dials, reads one line of bytes, and asserts Prologos's
`decode-op` produces `(op-abort "phase-5-says-hi")`.
Together with Phase 4's static byte equality, this establishes
runtime wire compatibility between Prologos and `@endo/ocapn` —
they encode the same bytes AND those bytes survive a real socket
exchange in both directions.
What landed:
tools/interop/peer-recv.mjs — Node child: connect, read
line, decode via @endo/ocapn,
print one-line JSON summary
tools/interop/peer-send.mjs — Node child: bind ephemeral
port, print it, accept,
send a hardcoded canonical
op-* record + '\n', exit
tests/test-ocapn-live-interop.rkt — orchestrates both directions
via subprocess + tcp
.github/workflows/interop.yml — adds the live-interop step
after the Phase-4 cross-impl
drift gate
Bug found + fixed in peer-recv.mjs: @endo/ocapn's `BufferReader`
rejects a `Uint8Array` view with non-zero `byteOffset`, which is
what `Buffer.subarray()` returns after stripping the trailing
'\n'. Workaround: copy into a fresh `Uint8Array` before passing
to `decodeSyrup`. This is a JS-side ergonomics issue in
@endo/ocapn worth filing upstream; for our tests the local
workaround is enough.
Test gating: `interop-deps-present?` checks for `node` on PATH
AND `tools/interop/node_modules/@endo/ocapn` — if either is
missing the test prints SKIP and exits 0. Keeps the suite green
in environments without Node while CI explicitly installs them.
Test count progression on Racket 9.1:
Phase 4: 224 (cumulative)
Phase 5: +2 live-interop
Total: 226/226 green
Design doc: docs/tracking/2026-04-29_OCAPN_INTEROP_DESIGN.md
Phases 1A–5C ✅. Remaining (Phase 6+, separate work):
bidirectional handshake with full op:start-session exchange,
multi-message conversations, GC, cryptographic auth.
https://claude.ai/code/session_01YM6gc3cMNH2Ymor4jdZY8u
Real bidirectional CapTP handshake between a Prologos peer and an @endo/ocapn peer over a real localhost TCP socket. Both sides encode their own op:start-session, exchange them, and verify the other side's bytes are byte-identical to what their own encoder would have produced for the equivalent value. Bug found + fixed mid-implementation: Phase 2's encoder packed multi-arity records (op:start-session, op:deliver, op:listen, op:gc-export, op:deliver-only) as <label [arg1 arg2 ...]> ;; OUR phase-2 encoding (WRONG) instead of the canonical OCapN form <label arg1 arg2 ...> ;; canonical form The Phase 4 cross-impl test missed this because every Phase-4 vector used a 1-arity record. Phase 6's handshake exercise caught it the moment a real @endo/ocapn peer tried to extract version + locator from the record children and got `null`. Fix: added `encode-record : String [List SyrupValue] -> String` to syrup-wire.prologos that produces `<label arg1 ... argN>` directly. captp-wire's encode-op now uses encode-record for the 5 multi-arity ops; 1-arity ops (abort, gc-answer) still go through syrup-tagged. Codified as goblin-pitfall #26. Perf gap surfaced: Prologos's decode-op of a multi-arity record takes ~7 minutes in the reducer. Round-trip is correct, just catastrophically slow. The Phase-6 test sidesteps this via byte equality (a strictly stronger correctness signal than decode + compare anyway). Codified as goblin-pitfall #27. What landed: syrup-wire.prologos — new encode-record helper captp-wire.prologos — encode-op uses encode-record for multi-arity ops tools/interop/peer-handshake.mjs — Node peer that connects, sends start-session, reads reply, prints JSON summary tests/test-ocapn-handshake.rkt — orchestrates the bidirectional exchange via subprocess + tcp, asserts byte equality + Node JSON .github/workflows/interop.yml — adds the handshake job Test count progression on Racket 9.1: Phase 5: 226 (cumulative) Phase 6: +1 handshake Total: 227/227 green Pitfalls log: #26 (multi-arity record encoder) + #27 (decoder perf) added to docs/tracking/2026-04-27_GOBLIN_PITFALLS.md. Design doc: docs/tracking/2026-04-29_OCAPN_INTEROP_DESIGN.md Phases 1A–6C ✅. Phase 7+ remaining (out of scope here): multi-message conversations, secure netlayer with crypto, full GC, decoder perf fix. https://claude.ai/code/session_01YM6gc3cMNH2Ymor4jdZY8u
Three-frame exchange between Prologos and @endo/ocapn over a real
TCP socket. Each peer sends three back-to-back messages and reads
the other's three. Covers the full mix of 1-arity and N-arity
ops:
1. op:start-session ver="0.1" loc=... (N-arity)
2. op:deliver-only target=<desc:export 0> args="ping" (N-arity)
3. op:abort reason="goodbye" (1-arity)
Both peers assert byte-equality on the three frames they receive
(stricter than decode-and-compare). Node-side additionally JSON-
reports the labels of all three Racket-sent frames as decoded by
@endo/ocapn.
This is a "lockstep echo" test — neither peer reacts to what it
receives, so it doesn't simulate real CapTP conversational state.
What it does prove:
- Phase-6's encode-record fix works for ALL multi-arity ops,
not just start-session
- '\n'-terminated framing handles 3 back-to-back messages
correctly in each direction
- Mixed 1-arity + N-arity record sequences round-trip
What landed:
tools/interop/peer-conversation.mjs — Node child: connect,
write 3 frames, read 3
frames, decode each,
JSON-summarise
tests/test-ocapn-conversation.rkt — Racket-side orchestration
+ byte-equality assertions
.github/workflows/interop.yml — adds the conversation step
Test count progression on Racket 9.1:
Phase 6: 227 (cumulative)
Phase 7: +1 conversation
Total: 228/228 green
Design doc: docs/tracking/2026-04-29_OCAPN_INTEROP_DESIGN.md
Phases 1A–7B ✅. Phase 8+ remaining (out of scope here):
conversational state machine (request → response routing,
pipelining, op:listen → op:deliver chains), secure netlayer
with crypto, full GC, decoder perf fix.
https://claude.ai/code/session_01YM6gc3cMNH2Ymor4jdZY8u
The first stateful round-trip. Unlike Phase 7's lockstep echo,
Node ACTS on what it receives:
Racket → Node: op:start-session
Racket → Node: op:deliver target=<desc:export 0>
args="ping"
answer-pos=<desc:answer 0>
resolver=false
Node → Racket: op:start-session
Node → Racket: op:deliver target=<desc:answer 0> ;; the reply
args="ping-pong" ;; computed!
answer-pos=false
resolver=false
Node's reply args are COMPUTED from the request ("ping" + "-pong").
This proves Node really decoded our deliver, extracted the args
and answer-pos, and answered to the correct answer-pos — not
just lockstep echoed pre-hardcoded bytes.
Bug surfaced + fixed: @endo/ocapn's AnyCodec rejects `null` as a
record child. Phase-1-7 didn't surface this because none of those
vectors emitted a null in a record sent TO @endo/ocapn. Phase 8's
first deliver did (for absent answer-pos / resolver, which Phase 2
encoded as syrup-null) and broke Endo's decoder on receive AND its
encoder on the reply.
Fix: `opt-pos none` now emits `(syrup-bool false)` instead of
`syrup-null`; `unwrap-opt-desc` accepts both for forward compat.
Codified as goblin-pitfall #28.
What landed:
tools/interop/peer-responder.mjs — Node child: connects, sends
start-session, parses incoming
deliver, computes reply args
from request args, sends
op:deliver to answer-pos
tests/test-ocapn-rpc.rkt — Racket-side orchestration
+ byte-equality + Node JSON
lib/prologos/ocapn/captp-wire — opt-pos uses syrup-bool false;
unwrap-opt-desc accepts both
.github/workflows/interop.yml — adds the rpc step
Test count progression on Racket 9.1:
Phase 7: 228 (cumulative)
Phase 8: +1 rpc
Total: 229/229 green
Pitfalls log: #28 (Endo rejects null as record child) added to
docs/tracking/2026-04-27_GOBLIN_PITFALLS.md.
Design doc: docs/tracking/2026-04-29_OCAPN_INTEROP_DESIGN.md
Phases 1A-8B ✅. Phase 9+ remaining (out of scope here):
multi-turn pipelining, op:listen + op:deliver-only chains,
op:abort teardown, real Prologos-side promise resolution
semantics, secure netlayer with crypto, decoder perf fix.
https://claude.ai/code/session_01YM6gc3cMNH2Ymor4jdZY8u
The Phase 4-8 interop tests crash the batch-worker harness in `tools/run-affected-tests.rkt` with DEAD WORKERS, because they either (a) call `(exit 0)` at module load time when node deps are missing — which exits the worker process before any test can report — or (b) use `define-runtime-path` to find a fixture file, which doesn't resolve correctly when the worker loads via `dynamic-require`. These tests are already covered by `.github/workflows/interop.yml` which uses `raco test` directly (handles both patterns). Adding them to `.skip-tests` keeps the main `test` workflow green without losing coverage. Affected: test-ocapn-syrup-cross-impl.rkt (Phase 4 — @endo/ocapn byte equality) test-ocapn-live-interop.rkt (Phase 5 — live Racket↔Node) test-ocapn-handshake.rkt (Phase 6 — bidirectional handshake) test-ocapn-conversation.rkt (Phase 7 — multi-frame conversation) test-ocapn-rpc.rkt (Phase 8 — conversational state machine) CI verdict on commit 34fc1b2: syrup-byte-equality (interop) — PASS test (main suite) — FAIL (DEAD WORKERS in skipped tests) This commit fixes the latter without regressing the former. The interop workflow continues to run all 5 tests via raco test. Local verification: `racket tools/run-affected-tests.rkt --all --no-record` now skips 8 tests (3 perf + 5 interop) and queues 434 of 442 files; runner makes forward progress instead of crashing on the first interop file. https://claude.ai/code/session_01YM6gc3cMNH2Ymor4jdZY8u
…interop 16 -> 17
The greeter's send went out with answer-pos and resolve-me both false: enough
for the peer to receive, but the result was dropped and the peer had nowhere
to reply. Each queued outbound send is now a question:
* a fresh local promise, registered in bs-outbound-questions;
* a fresh local actor running `beh-resolver` with that promise in its state,
exported as the resolve-me. When the peer delivers `['fulfill v]` to
`<desc:export R>`, ORDINARY inbound routing lands it on that actor and
`eff-resolve` settles the promise. Nothing special-cased on receive.
Once the promise settles the answer entry is finished business:
`<op:gc-answers [pos …]>`, then the entry is dropped so the next step cannot
re-emit it.
The answer position goes on the wire as a BARE INTEGER, not `<desc:answer N>`
as `outbound-question-bytes` writes it. The peer reads slot 2 raw and writes
it raw, so a wrapped position comes back as a record and never compares equal
to the position we later name in op:gc-answers. Both builders now exist side
by side rather than one being changed under the existing questioner path.
The drain runs in `connection-step`, not inside `pump-outbound`, because it
must update BridgeState and PumpResult carries none.
Three single-pass/layout traps on the way, all with misleading diagnostics:
`drain-vat-questions` calls `clear-outbound` and sat above it, which is a
STUCK `[reduce …]` term rather than an error (#16); `append` takes two args
with an implicit type, and the 3-arg form elsewhere in this file made the
over-application look idiomatic; and the module-import "Unbound variable" and
the stuck-reduce present differently depending on cache warmth, so the same
defect showed two faces across runs.
Tests: +3 in test-ocapn-bridge (157 pass) — the filled answer-pos/resolve-me
slots, the answer-table registration, and the no-noise case where nothing has
settled. Full suite 9541 pass / 62.5s. Interop 17 of 24.
… FAILS the gate
The interop gate failed on CI with 14 passed / 1 errored while passing 17/17
locally. Rebuilding locally WITHOUT the compile limit reproduced 14/1 exactly
— same test erroring, same counts — so this is measured, not inferred.
Cause: the compiler's giant match functions (whnf/nf ~990 arms, shift/subst
~340) exceed Racket CS's default compile limit of 10000 and fall back to the
INTERPRETER. `run-affected-tests.rkt` and `bench-ab.rkt` putenv it for
themselves, but CI precompiles with a bare `raco make`, which does not — and
the .zo that step produces is interpreted for every step after it.
The reason this is a correctness failure and not a slow build: upstream's
tests carry their own wall-clock budgets (15s for a GC round trip), and the
Racket server spends nearly all its time in reduction. Measured on this suite:
compiled 17/17, 2.5s of test time, 13.6s wall
interpreted 14 + 1 error, ~10 min wall
Set at job level in all three workflows so the precompile step is covered, and
exported by run-ocapn-test-suite.sh as well so the gate does not depend on its
caller's environment.
testing.md gains the operational form: a CI-only failure with no local repro
is a reason to check the build environment before the code. An interpreted
build is a different program.
A CI-only failure with no local repro, confirmed by reproducing it locally with an interpreted build (14/1, exactly matching CI) rather than inferring it from the 200x timing gap. Also records the raco-make SHA short-circuit that makes the fix look ineffective when you restore the build without deleting compiled/.
…7 -> 19 Both upstream pipelining tests pass. The chain is builder -> factory -> car -> string, every link answering with a NEW OBJECT, and the peer never waiting for a link to resolve before addressing the next. **eff-spawn, and why step-behavior gained a parameter.** A behaviour that answers with an object has to name that object in its own return value, so it needs the id BEFORE the actor exists. `step-behavior` is now handed the vat's next free id and the behaviour names it; the vat places the actor there. They agree by construction rather than by convention. Constraint stated rather than hidden: one spawn per turn, since two would name the same id. **Four wire-level defects, each of which had made the chain impossible:** 1. The answer position arrives as a BARE INTEGER. `unwrap-opt-desc` accepted only `<desc:answer N>`, so every pipelined message from a real peer had its answer position silently dropped to `none` — which reads downstream as fire-and-forget rather than as an error. 2. A pipelined deliver on an answer we have ALREADY resolved was dropped. `pipeline-deliver` queues onto an unresolved promise and drops onto a resolved one; our side resolves immediately because the object is already here, so every message in the chain hit the drop. When the answer resolved to `<desc:export N>` and N is one of our actors, the peer is addressing that actor — so dispatch it as an ordinary deliver, which already knows how to allocate an answer, reply to the resolve-me, and run the actor. 3. An answer position and a resolve-me are not alternatives. The resolve-me is where the answer GOES; the answer position is what the peer ADDRESSES NEXT. We treated them as an either/or and handled only the first. 4. The op:deliver args slot must be a LIST. We sent bare values there — a peer ITERATES that slot, so it raised inside the peer's receive loop, the peer dropped the connection, and our next write failed with "error writing to stream port" several frames later. The symptom pointed at the transport; the cause was three frames earlier. Defect 4 is why four Node fixtures and three assertions changed: they were written against our own output and had pinned the unparseable shape. Updating them is the fix, not a concession — the external contract (upstream's suite) iterates that slot. **Breaks propagate.** A behaviour signals failure by returning `<Error r>` — it never learns its answer promise's id, so it cannot emit eff-break against it — and the vat now BREAKS rather than fulfilling with an error-shaped value. Messages already pipelined behind a broken answer break with the same reason, because saying nothing is the one response that is definitely wrong: the peer is waiting on a resolve-me that would never be written. Tests: +4 in test-ocapn-bridge (161 pass), driving real wire frames verified against upstream's own encoder. Full suite 9541 pass / 64.1s. Interop 19 of 24.
Records what landed this session and what the remaining five actually need, read from the tests rather than inferred: both crossed-hellos tests go through the sturdyref enlivener, so all five reduce to outbound connections plus the initiator side of the handshake. Everything shipped so far responds on a connection the peer opened. Breaks the capability into its four parts and notes which are coherent as one piece (the effect, the initiator handshake, a second managed connection) and which sits on top and cannot be tested before them (the crossed-hellos side-id rule, where a one-sided implementation passes one test and fails the other).
The first half of the capability the last five tests need. Verified working: a deliver to the sturdyref enlivener queues a dial, the server parses host and port out of the sturdyref's ocapn-peer hints, connects, and sends our op:start-session. Server log: "dialling 127.0.0.1:39101". Everything shipped before this responds on a connection the peer opened; this is the only place the process opens one. **No new Effect, no sixth Vat field.** A pure behaviour cannot open a socket, so the request goes through an FFI queue — the same shape ocapn-gift-ffi.rkt already uses for the exporter-global gift table, and the alternative was threading a new field through nineteen `[vat …]` constructor sites. Parsing the sturdyref is left to the dialler in Racket, because the dialler is what needs the host and port, and handing it the peer's own bytes keeps one representation rather than two. **The dial is FORCED, not merely bound.** Reduction is lazy, so `let dialled := [maybe-dial op]` with `dialled` unused is dropped outright and the side effect never happens — which presents as a queue that stays empty with no error anywhere. It cost a debugging cycle here even though the hazard is written into the FFI's own header. Forced by matching on the result, the same idiom `publish-gifts` uses two definitions below. **The gate is deliberately NOT raised.** The two crossed-hellos tests are not added to the selected list: dialling is necessary for them but not sufficient, and the mitigation itself — deciding which of two crossed sessions loses, and aborting it — is not built. Adding them now would turn a known gap into a red gate and claim credit for half a feature. Full suite 9541 pass / 62.4s. Interop 19 of 24, unchanged.
Two peers can dial each other at the same moment and end up with two sessions where there should be one. CapTP breaks the tie with a rule both sides evaluate independently and agree on without another round trip: sort the two SIDE-IDS as octet strings, and abort the connection dialled by whichever sorts first. The two upstream tests are the same mechanism seen from both ends — one regenerates keys until OUR side-id wins, the other until theirs does — so a one-sided implementation passes exactly one of them. Both pass. **A side-id needs no key parsing.** It is SHA-256 applied twice to the gcrypt-encoded public key, and that encoding is byte-for-byte field 1 of the `op:start-session` frame already in hand. Slice the field, hash it twice. (Cross-checked against upstream's own `our_side_id`, utils/captp.py:113-123, which hashes exactly those bytes — and `sha256-bytes` turns out to be in `racket/base`, so no new dependency.) **Peers are identified by LOCATION bytes, not host:port.** Field 0 of a sturdyref and field 2 of an `op:start-session` are the same `<ocapn-peer …>` record and slice to identical bytes, so the two sides of the match compare directly. host:port would be wrong: ephemeral ports get reused across tests in a long-running server, and a stale entry would abort a healthy connection. The winner of the tie then enters the ordinary frame loop, which is why that loop is now a function rather than inline in `handle-connection` — the crossed path is a second entry point to the same thing, not a copy of it. Scoping came from a parallel investigation with an adversarial verification pass. Worth recording that the review caught real errors in its own input, including a "verified" claim that Racket has no built-in SHA-256 (it does) and two accessors attributed to the wrong record — which is exactly why the verify phase exists. Full suite 9541 pass / 60.3s. Interop 21 of 24.
A gifter hands us a signed handoff-give; we dial the exporter it names and
withdraw the gift there, presenting a desc:handoff-receive signed with our own
session key. Both HandoffRemoteAsReciever tests pass.
**The keypair now outlives the handshake.** `mk-handshake-bytes` generates a
keypair inside a `let` and drops the handle, so nothing downstream could ever
sign. The receive must be signed with the key the gifter NAMED as the receiver
— which is the key we handshake with — so the handle has to persist.
`handshake-bytes-with-key` is the same builder with the keypair supplied
rather than generated, and `sign-bytes` signs a payload BARE: `sign-location`
wraps its argument in the my-location envelope, and a handoff-receive is
signed over exactly the record bytes.
**Signing is the mirror of verification, and reuses its shapes.** The
`[sig-val [eddsa [r …] [s …]]]` gcrypt form we parse in `signed-receive-valid?`
is now also built. Session id follows upstream's own derivation
(utils/captp.py:125-146): SHA256(SHA256("prot0" ++ min ++ max)) over the two
side-ids. Everything the receive needs — their side-id, the session id — comes
out of the exporter's op:start-session frame, so the withdraw goes out the
moment that frame arrives.
The give is found by scanning the inbound frame for its record marker rather
than by walking the message: it can sit at any depth in the args, and
`desc:handoff-give` appears nowhere else on the wire. That cost one cycle —
the marker had `15'` where the tag is 17 characters, so the scan silently
matched nothing and the flow produced no trace at all.
Full suite 9541 pass / 59.0s. Interop 23 of 24.
HandoffRemoteAsGifter needs connection REUSE, not another dial: both its sessions are ones the test opened to us, and it never accepts a socket for the location its sturdyref names. Records the five ordered pieces, plus two findings the audit turned up that should not be re-derived — export position 5 has no vat actor behind it, and a stale in-tree comment denies dict support that now exists.
All 24 upstream OCapN conformance tests pass. The gifter needs connection REUSE, not another dial. Both its sessions are ones the peer opened to us, and it never accepts a socket for the location its sturdyref names — a dial there completes against an unaccepted listen backlog and then blocks with zero bytes, which is exactly the symptom the outbound work started from. So the enlivener looks up the open connection whose peer location matches, keyed by the same location bytes the crossed-hellos table already uses. This is the first thing here to write to a socket other than the one being serviced: the enliven arrives on one session and the `fetch` must go out on the exporter's. Requests are keyed by the resolve-me export we ask the exporter to answer on, so the answer identifies its own request without a separate correlation table. The give's five fields do NOT all come from one session, and that was the one bug in this piece: `receiver-key` is the ENLIVENING peer's key, while `exporter-location`, `session` and `gifter-side` belong to the EXPORTER session — and `gifter-side` is OUR side-id, because from the exporter's point of view we are the gifter. Each field is pinned by an upstream assertion (third_party_handoffs.py:458-462). `exporter-location` is the peer's own location bytes COPIED, never re-encoded: our encoder writes hints as a syrup list where Python writes a dict, and the assertion compares encoded forms. Deposit is built inline rather than through the existing `deposit-gift-bytes`, which emits the gift id as a syrup-nat where a gift id is a bytestring everywhere else in this module. Full suite 9541 pass / 59.3s. Interop 24 of 24, confirmed on two runs.
Ten read-only review agents, one per surface, each followed by an adversarial
verifier that opened every cited file and was asked to REFUTE rather than agree.
Only findings that survived appear; the verifiers also demoted severities,
corrected citations, and added a MISSED list per surface of things the first
pass should have caught.
7 CRITICAL, 31 HIGH, 35 MEDIUM, 24 LOW. The criticals cluster into three:
A. The handoff trust chain is not closed. The exporter verifies a
desc:handoff-receive against a key it reads out of the handoff-give nested
inside that same receive, and the give's own signature is never checked.
A withdrawal is entirely self-attested. This is an authentication bypass,
not a robustness gap, and it is the most serious thing in the document.
B. The gift table is a shared, unauthenticated namespace — no session binding
on a gift, the replay-guard prefix sharing that namespace under a comment
whose reasoning is backwards, and park keys colliding across connections
because promise ids restart per connection.
C. The codec is not self-inverse: string/symbol prefix with UTF-8 length while
the stack is Latin-1, and re-encode splices away a single list argument's
brackets — the latter on the signature path.
C is partly a correction of my own reasoning from earlier today: the syrup-bytes
half of it was fixed, and the string/symbol arms were deliberately left with a
comment arguing "the prefix is the UTF-8 byte length, per spec". That is wrong,
and two existing tests currently pin the bug. Recorded rather than quietly
amended.
I spot-checked the three headline claims against the source before committing a
document that asserts an auth bypass; all three hold.
Nothing here is fixed. The document is the inventory.
…nd the plural GC ops
Fixes documented in docs/tracking/2026-07-28_OCAPN_IMPLEMENTATION_GAPS.md.
This is the first tranche: the Prologos library plus the FFI shims. The
Racket test-server surface is still in flight.
THEME A -- the third-party-handoff trust chain is now closed.
The exporter verified a desc:handoff-receive against a public key it read
out of the desc:handoff-give NESTED INSIDE that same receive, and the give's
own signature was never checked at all -- grep for verify-raw found exactly
one call site. Any peer that knew a gift-id could forge a give naming its
own key, sign the receive with the matching secret, and redeem the gift.
The comment claiming "a forged receive can neither consume a gift nor learn
whether one was deposited" was false for precisely that forgery.
Closing it needed the gifter's real key, which nothing retained:
- op:start-session now carries the peer's session pubkey (message.prologos,
captp-wire). A legacy two-field frame yields syrup-null, i.e. "unknown".
- BridgeState gains an 11th field holding that key for the session.
- A GiftEntry records the key of the peer that deposited it, so a
withdrawal can be checked against the party that actually gave.
- A parked withdrawal stores the signed give and verifies it when the
deposit finally arrives; if it does not verify the park is dropped and
the promise stays pending, so a forger learns nothing.
gifter's session key --verifies--> desc:handoff-give
give's receiver-key --verifies--> desc:handoff-receive
Verified against the reference: tests/third_party_handoffs.py signs the give
with g2e_session.private_key, which is exactly the key we now record.
THEME B -- the gift table is no longer a shared namespace.
"used:" replay markers and "park:" entries shared one String-keyed table with
peer-supplied gift ids, under a comment reasoning that the prefix "cannot
collide with a gift id, because gift ids come from the peer as opaque bytes".
That is backwards: peer-supplied opaque bytes are what let a peer CHOOSE the
colliding key. Depositing "used:<sess>|<side>|0" denied an unrelated party's
handoff; depositing "park:<p>:<gid>" resolved promise p with an export of the
attacker's choosing.
The kind is now a FIELD (GiftKind), not a key prefix, so collision is
unrepresentable rather than merely unlikely. Park and used entries are also
per-connection now -- publishing them globally was a live cross-connection
bug needing no attacker at all, since every connection's vat starts its
promise ids from the same seed. And the replay identity length-prefixes its
parts, so ("A|B","C") and ("A","B|C") stop colliding.
THEME C -- the codec is an inverse again.
- string/symbol measured their length prefix in UTF-8 bytes while the whole
stack is Latin-1 one-byte-per-code-point. Our encoder emitted frames our
own decoder rejected, and string->bytes/latin-1 raises outright above
U+00FF, so the value two tests pinned could never have been sent. The
identical bug was fixed for syrup-bytes and argued away for string/symbol
on the spot; the reasoning ("strings hold text, so the spec's byte length
applies") is wrong because nothing here is UTF-8.
- <tag [a b]> and <tag a b> decoded to the SAME value, so re-encode turned a
1-arg record into a 2-arg one -- on the signature path. decode-record-with
now distinguishes them, which let encode and re-encode collapse to one
reading; they differ only in dict ordering now, and that difference is
deliberate (we sort what we originate; we reproduce what a peer sent).
- No null form exists in the dialect. We emitted "n" and our own frame
reader rejected it. A null payload is now a zero-arg record.
- refr/promise encoded as "", silently deleting the value from its
enclosing list and handing the peer a well-formed record of the wrong
arity. They now emit a poison record no peer will accept.
- decode-value ignored how much it consumed, so trailing garbage parsed
clean; leading zeros and odd-length dicts were accepted.
- Dicts are sorted on the outbound path per Syrup canonicalisation.
ALSO
- op:gc-exports / op:gc-answers (the spec plural forms) are decoded and
dispatched. Every conforming peer's GC frame was silently dropped, which
is why bs-decr-export and bs-remove-question were reachable only from ops
no peer can send. The singular emitters now emit the plural wire form.
- The refr walk is fully recursive: imports inside a dict, a nested list or
an ordinary record were never refcounted and never released. Added
desc:import-promise as a recognised descriptor.
- PipeMsg carries both reply channels. A queued pipelined deliver lost its
resolve-me entirely, and every forwarding loop discarded the stored answer
position while forward-deliver-bytes hardcoded "false false" -- so an
error was deliverable but a success was not.
- outbound-question-bytes wrote a wrapped answer position and a bare args
slot; both are spec violations its own sibling got right.
- int-to-nat was unary Peano over a peer-controlled integer: 18 wire bytes
bought an unbounded stall. Nat has an O(1) native representation and a
foreign returning Nat marshals straight into one.
- vat: actor/promise tables replace instead of shadowing; an inbound
deliver to an export we do not hold is no longer reflected back at the
peer in the peer's own namespace.
Tests: the ones that pinned these bugs now assert the correct behaviour, with
the old claim and why it was wrong recorded in each. 492 OCapN tests pass.
Two findings deliberately not fixed, recorded rather than softened:
- crypto.prologos's :requires (CryptoCap) annotations. Applying them fails
every consumer at module load; the capability has to be threaded from the
driver entry through the whole verification chain first. The header now
says so instead of claiming the annotations are present.
- plain-value-error-reason stays a constant: naming the offending value is
wire-visible and an external fixture pins the exact string.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YM6gc3cMNH2Ymor4jdZY8u
…ec-shaped end to end
Second tranche of docs/tracking/2026-07-28_OCAPN_IMPLEMENTATION_GAPS.md.
Gate: 9589 unit tests pass, upstream conformance 24/24, 13 Node interop
steps pass.
THE WIRE SHAPES, FIXED ACROSS EVERY ENCODER AND EVERY FIXTURE
Two slots were wrong in captp-wire and right in captp-core, with nothing
forcing them to agree:
- the ARGS slot is a LIST, always. A peer iterates it directly, so a bare
value raises inside its receive loop; the peer drops the connection and
our next write fails several frames later with an error that points at
the transport.
- the ANSWER POSITION is a BARE INTEGER. `<desc:answer N>` came back to
upstream as a record that never compared equal to the position we later
name in op:gc-answers.
Both had been left alone because three in-tree JS peers matched the wrong
forms -- the fixtures were pinning the bug. This is the single commit that
moves all of them: captp-wire's `encode-op`/`op-to-syrup`, captp-core's
`outbound-question-bytes`, and the peers that read or send those slots.
`peer-refr-passing` also now expects `op:gc-exports`, the plural spec form,
where it had been waiting for a singular op no conforming peer sends.
THE ENLIVEN DOUBLE-HANDLING, WHICH THE ENCODER FIX EXPOSED
Fixing the encoder turned the conformance suite from 24 to 23. The failure
was real and pre-existing, and it is worth recording how it hid:
Enlivening a sturdyref means opening a socket, so export position 5
deliberately has no vat actor and the DRIVER answers it out of band. But
`run-step` handed the same op to `connection-step` as well, the vat found no
actor at position 5, and captp-core BROKE the peer's promise. That break went
out on the enliven frame itself, so it beat the driver's `fulfill` -- which
has to wait for a fetch on another connection -- and became the answer the
peer kept.
It had always been emitted. It was invisible because it was UNPARSEABLE: the
encoder wrapped a record's argument sequence in a list, so upstream received
`<ocapn-sturdyref [<ocapn-peer [...]>]>` and dropped the frame. A correct
encoder turned a silently-ignored malformed break into a well-formed one.
`run-step` no longer forwards an enliven, which also makes the two arms of
that match do different things -- the FFI side effect is now forced by a
branch that matters rather than by a comment asking the next reader not to
simplify it.
Diagnosed by worktree-pinning the baseline and diffing the outbound frames,
after three wrong guesses from reading the code alone.
TEST SERVER
- `drive-init!` now hands the peer's op:start-session to captp-core. That
frame was consumed for the handshake and stopped there, so BridgeState's
record of who the peer is stayed empty -- and that record is what binds a
deposited gift to its gifter. Without it every third-party handoff failed
the new verification: the exporter had no key to check the give against.
- `finish-fetch-answer!` spliced an integer position into `bytes-append`,
which raised inside the enliven path and was swallowed, so the gifter's
fulfill was never sent at all.
- Plus the agent-side work: bounds-checked frame slicing, a Syrup skipper
that no longer throws on legal input, atomic id counters, `sign-with-our-key`
under the semaphore the file's own comment says is mandatory, per-session
keypairs and gift ids instead of process-wide constants, validation on
dialled peers, dial dedup, and lifecycle cleanup for the connection tables.
- Outbound frames are logged under OCAPN_FRAME_HEX too. Only inbound were,
which is why diagnosing the above needed a patched copy of the server.
CI AND FIXTURES
- The interop gate compared only the pass COUNT: `N_FAIL`, `N_ERROR` and the
runner's exit code were computed, echoed, and never checked, so a 25th
failing test passed the gate.
- The allow-list of 24 upstream tests is now drift-checked against the
modules it targets, so a test added upstream fails loudly instead of
silently not running.
- The cross-impl Syrup fixture grew the cases that made this whole audit
possible to miss: dicts, byte-strings, nested and multi-arg records.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YM6gc3cMNH2Ymor4jdZY8u
Seven verifier agents re-opened every file the repair agents touched and
tried to refute their reports. These are the survivors -- the ones that were
right. Gate: 9599 unit tests, conformance 24/24.
THE ONE THAT WAS A REAL BUG, NOT A DOC PROBLEM
`forward-deliver-bytes` echoed the queued deliver's `ap`/`rm` into the
forwarded frame. The verifier caught that our own decoder rejects the result
(`resolve-me-ok?` accepts only import-object / import-promise, and this wrote
`<desc:export N>`), which sent me back to the protocol: a forwarded deliver is
a NEW message with US as the sender, so its answer position names a slot in
OUR question table and its resolve-me an object WE export. The queued
deliver's slots belong to the peer. Echoing them is wrong in both directions
and I had reasoned about it only from the peer's side.
Reverted to `false false`. §1.2 M3 is real and stays OPEN: a forwarded
pipelined deliver still reaches the peer with no reply channel. Closing it
needs us to allocate our own answer position and forward the reply back to
the queued deliver's -- machinery that does not exist. `PipeMsg` keeps its
`rm` field so that machinery has something to read, and the comment now says
so instead of claiming the problem is solved.
FALSE COMMENTS MY OWN FIXES CREATED
`int-to-nat` became O(1), which falsified two comments written in the same
sweep that still described its cost as linear in a peer-chosen value -- one
of them the entire stated rationale for a helper. Also: an encode-op header
claiming `op:gc-answers` goes through `wire::encode` (it must not, and the
code twenty lines below explains why), a claim that CapTPOp cannot carry the
session pubkey (it does now), handshake.prologos's note that `encode` gives
the wrong shape for a multi-arg record (it no longer does), ocapn-framing's
note that our encoder still emits `n` (it does not), and the driver's claim
that its enlivener is "the only thing in this implementation that ever dials"
(the Racket server dials too, from a byte-scan, which is still open).
Every one of these was true when written and false by the end of the sweep.
MISSING GATE, MISSING TESTS, TWO SILENT DROPS
- `op:gc-answers` was the one op left without an arity gate, so
`<op:gc-answers [1+] [2+]>` decoded as `[1]` and dropped the second list.
- The captp-wire work shipped +0 tests. Added 12 covering the plural GC
decode, the arity and label gates, negative answer positions, the 4-field
start-session locator slot, and both corrected wire slots -- with a
well-formed control, so the gates are not just rejecting everything.
- `try-enliven!` accepted only `desc:import-object` as a resolve-me, so an
enliven whose resolver was a promise was dropped with nothing logged --
a NEW silent drop, in the middle of fixing silent drops.
- `peer-questioner`'s args helper still fell back to the bare value, the
fourth instance of a pattern the gaps doc named three of.
Dead code removed rather than left as a trap: two optional-slot helpers with
no remaining callers, and the list accessor the gc-answers gate replaced.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YM6gc3cMNH2Ymor4jdZY8u
…they were wrong
The inventory was written first and fixed second; this makes the document say
so. §0 and §1 are preserved unedited as the record of what was found — a new
Status section is authoritative for the current state.
What it adds:
- the three critical themes, closed, with the mechanism in each case;
- what the fixes EXPOSED. Correcting the encoder took conformance from 24
to 23, because a break captp-core had always emitted on the enliven frame
had been silently ignored for being malformed. That failure was worth
more than the fix, and it needed a worktree-pinned baseline and a frame
diff to find — three attempts to reason it out from the code were wrong;
- CORRECTIONS to this document's own claims. §0.3's mechanism was wrong
(connection-step did see the enliven), §1.2 #19 called five tested public
functions dead code, and §1.2 M3's fix was itself wrong and is reverted;
- what is STILL OPEN, one row each, with the reason it was left rather than
missed. Thirteen rows, led by the architectural one that has not moved;
- five defects found while fixing that were not in the inventory, including
the one that made theme A's fix first look like a regression;
- a note on the verification pass, which earned its keep: it caught a
claimed fix absent from the file, the forward-deliver-bytes error, four
comments my own fixes had falsified in the same sweep, a missing arity
gate, and a large behavioural change shipped with zero tests.
Also fixes tools/check-parens.sh, which hardcoded a macOS Racket path — so on
any other machine it reported a delimiter error for every balanced file, and
the pre-commit hook that calls it was a silent no-op. Found because it fired
on six balanced files during this work.
Gate: 9599 unit tests pass, upstream conformance 24/24.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YM6gc3cMNH2Ymor4jdZY8u
…its value Two more entries from the gaps document. 9604 unit tests, conformance 24/24. SETS AND FLOATS (§1.1 #5, §1.10 #3, §1.10 #11) The reference emits both -- `b'#' + sorted(items) + b'$'` and `b'D' + struct.pack('>d', obj)` -- and the model had no constructor for either, so a peer that sent one got a decode failure and no diagnostic. The gap was open because adding a constructor means following it through every exhaustive match over SyrupValue: 100 arms across five files. A set canonicalises by SORTED ENCODED ITEM, exactly as a dict does by key, so `encode` sorts and `re-encode` preserves what the peer sent -- the same split, for the same reason. A float is carried as its RAW WIRE PAYLOAD INCLUDING THE MARKER BYTE, not as a number. Nothing in this stack does float arithmetic, and the one thing the codec owes a peer is byte-exactness. Decoding an `F` to a number and re-encoding it as the `D` the reference prefers would preserve the value and change the bytes -- precisely the class of defect `re-encode` exists to prevent. It also costs one constructor instead of two. The cross-impl fixture grew both, and its harness moved from UTF-8 to Latin-1: it hex-encoded through `string->bytes/utf-8` while the codec's byte model is one byte per code point, so no vector carrying a byte >= 0x80 could ride it at all -- which is exactly the bug class the fixture exists to catch. The set vector is inserted in sorted order deliberately. @Endo's encoder emits insertion order where `contrib/syrup.py` sorts; an unsorted literal would pin that disagreement into a cross-impl vector instead of testing anything. Our sorting is pinned separately, against the reference's rule. THE PLAIN-VALUE ERROR (§1.10 #10) `plain-value-error-reason` ignored its argument and returned a constant, so the peer could not tell which of its pipelined sends went to a plain value. It now names the offending value after the condition. It stays a single string rather than a record: the reason travels inside our own `<Error …>` wrapper, which upstream has no type for, so a peer can only treat it as a diagnostic -- and the condition stays the prefix so it still discriminates "rejected" from "forwarded". A NOTE ON HOW THE ARM INSERTION WENT WRONG The mechanical pass copied each `syrup-bytes` arm's right-hand side for the new float arm, and one of those right-hand sides referenced its own binder -- so `syrup-bytes-of` got `| syrup-float _ -> some b` with `b` unbound. That presents as "Unbound variable" when a CONSUMER imports captp-core, while captp-core itself processes with zero errors: the exact signature of a stale `.pnet` cache. I cleared the cache, saw it again, and only then went looking for the real cause. Clearing twice is what separated the two. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YM6gc3cMNH2Ymor4jdZY8u
…ator hints are a map Three more entries. 9615 unit tests, conformance 24/24. §1.4 #6 — `op:listen`'s third field was required-but-DISCARDED, so a peer asking for partial resolutions (a promise resolving to another promise) was indistinguishable from one that was not. `op-listen` carries it now. A non-boolean in that slot fails the decode rather than defaulting to false: the field is not optional, and coercing a malformed one is the same silent class as the answer position two functions away. §1.6 M8 — a frame we cannot decode now gets an `op:abort` instead of "", which the peer could not tell apart from "decoded fine, nothing to say". Every decode failure is fatal, deliberately: the decoder's gates are arity, descriptor label and slot well-formedness, so a frame that fails one is a frame whose meaning we do not know, and continuing is how a desync becomes a wrong answer several frames later. This one was blocked for a reason worth recording: the SPEC's plural GC ops did not decode, so aborting would have fired on ordinary conforming traffic. Fixing them removed the blocker, and this entry was reachable only after. §1.10 #12 — locator hints are a MAP (flat alternating key/value, the shape syrup-dict uses) rather than two fixed scalar fields. Upstream ships an onion netlayer whose hints are different keys entirely and had no representation at all. `loc-host`/`loc-port` survive as lookups, so the one consumer and the tests are unaffected; `loc-port` parses its hint and answers zero when absent or non-numeric, because a locator with no port is a loopback locator, not an error. The string→Nat parse the gaps doc named as the blocker lives here for exactly that reason -- a general one in the stdlib could not choose to be total. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YM6gc3cMNH2Ymor4jdZY8u
…wn premises 9616 unit tests, conformance 24/24. Seven entries were open; five are closed and two turned out to rest on premises that no longer hold. §1.9 #4 IS A LANGUAGE GAP, NOT AN OCapN ONE Annotating an FFI binding `:requires (CryptoCap)` makes the whole module unimportable from a `.prologos` file: the requirement propagates to the caller and nothing can introduce a root capability. I tried wrapping the bindings in same-module functions the way `tcp-testing` does, and importing the capability type alongside. Neither helps. Then the actual discovery: `prologos::ocapn::tcp-testing`, which this document cites as proof the convention is live, CANNOT BE IMPORTED EITHER. Nothing in the tree imports it, and its test loads it through the sexp `imports` path, which does not hit the check — so a module that no consumer can use has been sitting there with a green test asserting "just load the module". It now carries a characterization test that pins the WS-mode failure. That test asserts behaviour that is WRONG on purpose: when the language grows a root capability form it fails, and that is the signal to thread the capability for real. A gap with a witness beats a gap with a comment. §1.8 M2 HAS LOST ITS CONSEQUENCE The stated failure was that a process-wide keypair collapses session-ids and the handoff replay guard, keyed on session, then refuses a peer's legitimate second handoff. That guard became per-connection when the gift namespace was split, so two connections no longer share a replay set. What is left is a conformance deviation with no failing input — which by this document's own standard is not a finding. Minting per connection also costs a `process-string` per accept, which the suite's wall-clock budget would notice. CLOSED - §1.1 #5 / §1.10 #3 / §1.10 #11 — sets and floats (previous commit) - §1.4 #6 — op:listen carries wants-partial (previous commit) - §1.6 M8 — undecodable frames abort (previous commit) - §1.10 #10 — the plain-value reason names its value (previous commit) - §1.10 #12 — locator hints are a map (previous commit) WHAT IS ACTUALLY LEFT Five entries, and four of them are one problem: the gifter and receiver roles live in Racket because a Prologos behaviour cannot open a socket. §1.7 M8 (double processing), §1.7 M7 (unregistered enliven slots) and §1.10 #8 (unsigned Node peers) are all downstream of that. Both of §1.7 M7's live consequences are gone; what remains is that the slot reservation is by convention rather than construction. The fifth, §1.2 M3, is independent and has a known shape: allocate our own answer position for a forwarded send and resolve the queued deliver's from it. The pieces exist — `bs-add-question` already maps a peer answer position to a local promise and the pump answers from it — but wiring them means threading BridgeState through a loop that returns bytes only. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YM6gc3cMNH2Ymor4jdZY8u
The forwarded-deliver reply channel was written and backed out. The design is settled and worth not re-deriving: allocate a fresh promise for the forwarded send and register it in BOTH question tables — outbound so the peer answers it, inbound at the peer's original answer position so the pump emits the reply with no new code — plus a listener for the resolver. Both tables pointing at the same promise is what makes the round trip work. Landing it means threading BridgeState through ForwardEffect, PumpResult and the four pump loops: ~25 mechanical sites. That was done, elaborated cleanly, and then failed at IMPORT with a bare "Unbound variable" — `bs-handle-listen-with-late-fire` takes four arguments and I passed three. Under-application in Prologos yields a stuck term, not an arity error. Fixing that call did not clear it, so at least one more is in there. Reverted rather than left half-landed. A reply channel that exists in some paths is worse than one that is documented as absent. The generalisable part, and the reason this is in the document rather than only in the log: at the import boundary, an under-application, a genuinely unbound name, and a stale `.pnet` cache all present as the same bare "Unbound variable". Three causes, one message, no location. A mechanical arity check against the `spec` lines would separate the first from the others in seconds and does not exist. Gate unchanged: 9611 unit tests pass, conformance 24/24. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YM6gc3cMNH2Ymor4jdZY8u
…ssors it was missing 9623 unit tests, conformance 24/24. §1.10 #8 was filed as "every Node peer speaks a 2-field unsigned op:start-session at '0.1' while the gate drives 1.0 signed, so the 13 interop steps would stay green if the signed path broke entirely". Going to fix the peers, I found the larger version of the same problem: `handshake.prologos` had NO unit coverage at all. Its own test file did not import it, and the one test in there exercises the unsigned form. The signed path -- the one every real peer uses -- was covered solely by the upstream conformance suite. Seven tests now: a signed frame parses and verifies; the parsed version is the one signed; a signature paired with a DIFFERENT location does not verify; one from a DIFFERENT key does not verify; an unparseable frame is rejected; a version we do not speak is rejected; and -- the control the other two need -- a well-formed current-version frame is ACCEPTED, so the rejections are not passing because everything is rejected. The two negative cases needed field accessors that did not exist: `ParsedSS` exposed only `ss-version`, so nothing outside the module could see the pubkey, the location or the signature. That absence is also why the signed path had no test that could construct a negative case in the first place — a type with no accessors is a type nothing can test. What remains of the entry is the Node peers' dialect, which is redundancy rather than a hole now that the signed path is covered directly. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YM6gc3cMNH2Ymor4jdZY8u
9623 unit tests, conformance 24/24. This was the last entry in the gaps document that was not downstream of the architectural one. A forwarded deliver is a NEW message with us as the sender, so the queued deliver's ap/rm cannot be echoed -- they name the peer's tables. The fix is to allocate a fresh promise P for the forwarded send and register it in BOTH question tables: outbound so the peer answers our question, and inbound at the peer's original answer position so that when P settles the pump emits `<op:deliver <desc:answer ap> value f f>` with no new code at all. The peer's resolver, if it named one, becomes a listener on P. Both tables pointing at the same promise is what makes the round trip work. Before this, `forward-deliver-bytes` hardcoded `false false`: an error was deliverable and a success was not. Three tests: the forward carries a real answer position rather than `ff`; the peer's answer position lands in bs-questions so something can answer it; and -- the control -- a queued deliver with no reply channel still goes out fire-and-forget, because allocating a question for one of those would leave an outbound entry nothing ever answers. WHY THIS TOOK TWO ATTEMPTS The identical design, written in one go, elaborated cleanly and then failed at IMPORT with a bare "Unbound variable". One cause was `bs-handle-listen-with-late-fire` taking four arguments where I passed three: under-application in Prologos produces a stuck term, not an arity error. Fixing that call did not clear it, and with no location in the message I reverted rather than leave the tree red. The second attempt landed the same design in four steps, testing the import after each. Every step was clean. The lesson is not "be careful" -- it is that at the import boundary an under-application, a genuinely unbound name and a stale `.pnet` all present as the same bare message with no location, so bisecting by construction beats diagnosing by inspection. `dispatch-answer-target` still forwards without a reply channel. That case chains the pipeline onto the peer's own question rather than terminating it, so it is a different shape; not covered here and not claimed to be. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YM6gc3cMNH2Ymor4jdZY8u
…ee left The three that remain are one architectural problem: the gifter and receiver roles live in Racket because a Prologos behaviour cannot open a socket, and 1.7 M8 and 1.7 M7 are downstream of that. Added the shape of the cheapest way in, since it is not obvious from the entry text: eff-connect is the cheapest of the three primitives because the dial queue it would drive already exists, and the vat stays pure if the effect accumulates a pending-dial list the driver drains -- exactly how vat-outbound works today. That one primitive would let the sturdyref enlivener be a real behaviour at export 5 instead of a driver interception, which is most of 1.7 M8 and all of 1.7 M7. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YM6gc3cMNH2Ymor4jdZY8u
…it exposed 9628 unit tests, conformance 24/24. A behaviour is pure and cannot open a socket, which is why the gifter and receiver roles live in Racket at all. `eff-connect` is the first of the three primitives §0.2 names: a behaviour describes the connection it wants, the vat records it in a pending-dial list, and the driver drains it between steps -- the same shape `eff-send-remote`/`vat-outbound` already uses one layer out, so the vat stays pure and wire-agnostic. `beh-sturdyref-enlivener` is now a real behaviour. It gates on the `ocapn-sturdyref` label, because this is the one path that opens an outbound TCP connection on peer-chosen bytes, and the driver's own extractor is now that behaviour's, so the two cannot drift. WHAT BUILDING IT EXPOSED Seeding the enlivener at export 5 -- the point of the exercise, and what would close §1.7 M7 and most of §1.7 M8 -- does not work, for a reason that is in the vat model rather than the driver: AN ENLIVENER CANNOT ANSWER, AND `ActStep` CANNOT SAY SO. The reply the peer waits for is a signed `desc:handoff-give`, which only the driver can build; it needs the keypair and the exporter connection. The behaviour's job ends at "please connect". But every `ActStep` carries a return value and `step-after-act` settles the answer promise with it unconditionally, so an actor at export 5 answers the enliven immediately with the wrong thing and beats the real reply. Measured, not predicted: seeded, the conformance suite goes 24 to 23, with the peer receiving an echoed sturdyref where it expects a sig-envelope. So the driver interception stays for now, with the blocker named where the next person will look. The next step is a MODEL change: `ActStep` needs a third outcome beside a value and a break -- "no answer yet". Returning `syrup-null` is not it, since `no-op` already returns null and several behaviours rely on that settling their promise. Six tests: an effect records a dial without performing one, a fresh vat has none, `clear-dials` drops them so a later step cannot re-dial, they accumulate in order, and the enlivener asks to connect for a sturdyref and not for anything else. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YM6gc3cMNH2Ymor4jdZY8u
… red it exposed 9630 unit tests (clean cache), conformance 24/24. THE MODEL CHANGE `eff-connect` alone did not let the enlivener be an actor. Seeding it took the conformance suite 24 -> 23, because an enlivener CANNOT ANSWER and `ActStep` could not say so: the reply the peer waits for is a signed desc:handoff-give only the driver can build, but every ActStep carried a return value and `step-after-act` settled the answer promise with it unconditionally. The actor answered immediately with an echoed sturdyref where the peer expected a sig-envelope, and beat the real reply. `ActStep` now has a second outcome. `act-step-pending` acts without answering and leaves the promise pending. `syrup-null` was deliberately not used for this: `no-op` returns null and several behaviours rely on that settling their promise, so overloading it would have left those silently hanging. With both primitives the sturdyref enlivener is an ordinary actor at export 5, the driver intercepts NOTHING, and every op goes through `connection-step`. That closes §0.3 (export 5 had no actor) and half of §1.7 M8 (the Prologos side no longer double-processes; the Racket byte-scanners still do). THE CI RED, WHICH IS THE MORE USEFUL FINDING Widening `Vat` broke a bridge-test fixture that builds a vat AS A STRING, so it could not fail at compile time. It failed at elaboration inside `run-last`, reported as "could not infer type" with no mention of arity. Every local run passed. CI failed 3/164 on a fresh checkout. The difference was the `.pnet` cache: warm, it answered from the pre-change module and hid the fixture's staleness. `rm -rf data/cache/pnet` PLUS `rm -rf compiled/tests` reproduced CI exactly. This is the inverse of the lesson already in the document, where a stale cache CAUSED a spurious failure. It also hides real ones. A green local suite is not evidence after a type change; a green suite from a cold cache is. Recorded as newly-found #7. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YM6gc3cMNH2Ymor4jdZY8u
The entry says the enlivener position has no vat actor and that the driver intercepts the deliver before connection-step sees it. Both halves are now false: beh-sturdyref-enlivener is seeded there and the driver intercepts nothing. The stated mechanism was also wrong when written -- connection-step did see the enliven -- which is already recorded under Corrections. Also corrects the driver comment that still described the interception it no longer performs. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YM6gc3cMNH2Ymor4jdZY8u
The entry has always said "a design task, not a refactor". Two primitives shipping changes what the design can assume, so this writes it down concretely rather than leaving the next person to re-derive it. The useful content is the grounded blocker: eff-send-on is not a primitive in isolation. A gifter must write to a connection other than the one being serviced, and nothing can name one -- conn-entry is (out-port, pubkey, side-id) with no connection id, so even the Racket side cannot ask the driver to act on a named connection. The registry is the prerequisite; eff-send-on is trivial once it exists. And a five-step migration order where each step is independently landable, with the two that are small and closable now called out: adding the connection id, and reserve-export -- which closes 1.7 M7 by construction, since the enliven resolve-me stops being a Racket counter in a convention-reserved range and becomes a real export in the exporter connection table. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YM6gc3cMNH2Ymor4jdZY8u
9630 unit tests from a cold cache, conformance 24/24.
The enliven resolve-me was a Racket counter starting at 900, reserved from the
vat's allocator by nothing but the distance between 900 and wherever `next-id`
had got to. It is now allocated BY that allocator: `reserve-export cid` spawns
a `beh-sink` actor in the EXPORTER connection's own vat and returns its id, so
the position the exporter answers on is one that connection actually holds.
Collision is impossible by construction rather than by margin.
Two things had to exist first, which is why this could not be done directly
when the entry was written:
- `act-step-pending`, from the previous commit. The placeholder must absorb
the exporter's answer WITHOUT replying: any answering behaviour would put
a reply on the wire for a message whose real handler is somewhere else.
`beh-sink` is the smallest possible use of it.
- a connection id on the server's `conn-entry`, which was
`(out-port, pubkey, side-id)`. Without it the gifter side could not name
the connection to allocate in -- it is the reason this entry read as
"needs the rest of §0.2" rather than as a local fix.
THE THIRD SIGHTING OF THE SAME DIAGNOSTIC
`reserve-export` initially called a helper defined BELOW it. Modules are
single-pass, so the call did not reduce -- and a stuck term is WELL-TYPED. It
came back as the literal string "[reserve-export 21N] : String", the enliven
was dropped, and the only symptom was a conformance test timing out three
layers away. Nothing errored anywhere.
That is under-application, forward reference and stale cache all presenting
identically for the third time this session. The server now prints what it got
rather than dropping in silence, so the next one names itself.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YM6gc3cMNH2Ymor4jdZY8u
The control for reserve-export. An answering placeholder would put a reply on the wire for a message whose real handler is elsewhere, which is exactly what made the enliven slot unreservable before act-step-pending existed. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YM6gc3cMNH2Ymor4jdZY8u
Twelve closed, two dissolved, one half. The remaining work is one problem -- the roles living in Racket -- plus the one entry still downstream of it, and the summary now says which of 0.2 three primitives exist and which do not. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YM6gc3cMNH2Ymor4jdZY8u
…ew mechanism Found while doing steps 1 and 2 rather than reasoned about in advance. The registry 0.2 needs is exporter-global state living behind an FFI, seeded into per-connection state at the start of a step and published back at the end -- which is exactly what with-global-gifts and publish-gifts already do for the gift table, and for the same reason: a handoff deposits on one connection and withdraws on another. That pattern already has the kind-discriminated entry type and the per-connection/global split worked out. So the registry is not a design problem, it is an application of a design that already ships, and what remains after it is the role migration itself. Also updates the note that conn-entry carries no connection id -- it does now. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com> Claude-Session: https://claude.ai/code/session_01YM6gc3cMNH2Ymor4jdZY8u
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
This PR implements:
@endo/ocapn(the published JS reference). Syrup byte codec, CapTP frame codec, real Racket↔Node TCP exchange, bidirectional handshake, multi-frame conversation, RPC-style state machine. CI workflowinterop.ymlruns all of this on every push.syrup-bytesctor (Phase 19), UTF-8 byte-length encoder (Phase 20).docs/tracking/2026-04-27_GOBLIN_PITFALLS.mdcovering language quirks, ergonomics issues, and design choices.Test summary (Racket 9.1)
@endo/ocapnbyte equalityTotal 261/261 OCapN tests green locally on Racket 9.1.
CI
Two workflows:
.github/workflows/test.yml— main test suite (run-affected-tests.rkt --all) plus a follow-upraco teststep for two heavy OCapN tests that exceed the runner's per-file 120s timeout..github/workflows/interop.yml— cross-runtime gate. Installs Node 22 +@endo/ocapn, regenerates the canonical Syrup vector fixture, drift-gates the regen against the committed file, then runs Phases 4 / 5 / 6 / 7 / 8 / 9 / 10 directly viaraco test.The
interop.ymlworkflow proves: (1) Prologos's encoder produces byte-identical output to@endo/ocapnon a curated vector matrix, and (2) those bytes survive a real localhost TCP exchange with Node in both directions, in single-frame, multi-frame, RPC, and abort-teardown patterns.What's NOT in this PR (deferred, documented in design doc)
datadeclaration, no heterogeneous closure registry).->stays top-level #4:Mu/recnot implemented in elaborator).resolve-promisedoesn't drain the queued messages back yet.All deferrals documented in
docs/tracking/2026-04-29_OCAPN_INTEROP_DESIGN.mdwith rationale.Pitfalls log
docs/tracking/2026-04-27_GOBLIN_PITFALLS.mdaccumulates 29 entries (some[DELETED]— false claims caught and retracted) covering language quirks discovered during this work. Highlights:defnwith leading 0-arity ctors only matches the first argletbinding variants #21 — multi-linematchclause body silently produces match-fail holesprocess-commandsequential orchestrator #22 —Option Nat -> Xparses as multi-arg Pi (need[Option Nat])m0 <: mw(and broader subtyping infrastructure) #23 — multi-tokendefnbody on a single line needs outer[…]+suffix always producessyrup-intneversyrup-natfn []and[-> A]for nullary lambda/type as Unit-encoded #26 —syrup-taggedmodel carries one payload; OCapN records are arity-N (encoder fix)@endo/ocapnrejectsnullas a record child (usefalsefor "absent")breakfromprologos::ocapn::promisecollides withdata::list::break-helperFiles
racket/prologos/lib/prologos/ocapn/*.prologos— 13 modules implementing the portracket/prologos/tests/test-ocapn-*.rkt— 19 test filesracket/prologos/examples/2026-04-{27,29}-*.prologos— Phase-0 + syrup-wire acceptance demosracket/prologos/tcp-ffi.rkt— TCP primitives bridgetools/interop/{gen-syrup-vectors,peer-recv,peer-send,peer-handshake,peer-conversation,peer-responder,peer-pipelined,peer-abort}.mjs— Node interop toolingracket/prologos/tests/fixtures/syrup-cross-impl.txt— committed cross-impl byte vectors.github/workflows/{test,interop}.yml— CI configurationdocs/tracking/2026-04-29_OCAPN_INTEROP_DESIGN.md— design doc covering Phases 1–23docs/tracking/2026-04-27_GOBLIN_PITFALLS.md— language pitfalls loghttps://claude.ai/code/session_01YM6gc3cMNH2Ymor4jdZY8u