Skip to content

parser: disambiguate bare-token compound patterns in defn (eigentrust pitfall #7) - #16

Merged
hierophantos merged 1 commit into
mainfrom
claude/fix-eigentrust-pitfall-7-marg
Apr 27, 2026
Merged

parser: disambiguate bare-token compound patterns in defn (eigentrust pitfall #7)#16
hierophantos merged 1 commit into
mainfrom
claude/fix-eigentrust-pitfall-7-marg

Conversation

@kumavis

@kumavis kumavis commented Apr 25, 2026

Copy link
Copy Markdown
Contributor

Fixes pitfall #7 from docs/tracking/2026-04-23_eigentrust_pitfalls.md.

Summary

defn sum-rows
  | nil            -> nil
  | cons r nil     -> r
  | cons r rest    -> [add-vec r [sum-rows rest]]

failed with Unbound variable sum-rows::1 — the parser treated cons r rest as three separate arguments instead of one compound pattern.

Approach (a) — auto-pack constructor patterns

In parse-defn-clause, when the leading token after | names a known constructor whose field count matches the remaining tokens, pack them into one compound pattern. Falls back to the existing N-arg interpretation when the leading token is not a constructor (e.g., defn add | x y -> [+ x y] still works).

Local change in one branch of the cond, ~24 lines, no other files touched.

Why (a) over (b) "raise a clear error": defn is the primary dispatch mechanism per prologos-syntax.md, and ML/Haskell users naturally read cons r nil as one compound. Detecting via lookup-ctor is local and preserves multi-arg defns unchanged.

Files changed

  • racket/prologos/parser.rkt (+24)
  • racket/prologos/tests/test-defn-multiarg-patterns.rkt (new, 315 lines, 11 cases)

Test plan

  • 11/11 new tests pass
  • 43/43 existing tests in test-pattern-defn-01 / test-pattern-defn-02 / test-multi-body-defn pass
  • Full affected suite: 4646 tests in 255 files; 1 unrelated pre-existing failure (stale tracking entry for non-existent test-constraint-retry-propagator.rkt)
  • CI fix included (benchmarks/micro/info.rkt skip; redundant once PR Migrate bench-bsp-le-track2.rkt to current ATMS API (CI unblock) #10's actual migration lands first)

Latent bug surfaced (NOT introduced) — worth tracking

The fix exposed a latent compile-match-tree bug: variable patterns named in a sub-position get bound via let v := __cons_1, referring to a param that's been destructured by a later dispatch column. This corrupts recursive bodies like cons r rest -> [+ r [recurse rest]].

Reproducible on main with the bracketed [[cons r rest]] form (same internal AST). The tests cover the slices unaffected (empty/singleton inputs, wildcards, all-var multi-arg). The commit message documents the latent issue for future tracking — it should be filed as a separate bug.

Commits

  1. 1595905 — primary fix in parser.rkt + test file
  2. dd26ad4 — CI fix (skip stale bench)

https://claude.ai/code/session_01MbncYJnrvjzhbVWw4xGi5x


Generated by Claude Code

@hierophantos hierophantos left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM. Surgical scope, uses existing lookup-ctor / ctor-meta-field-types infrastructure (no new APIs), N-arg fallback preserved.

Two non-blocking observations:

  1. The auto-pack rule means a user who picks a parameter name matching a constructor — defn foo | cons x y -> ... with cons as a variable — would now get the compound interpretation. ML/Haskell-aligned default is right, but worth a release note when a batch of these pitfall fixes lands.
  2. Filed the latent compile-match-tree variable-binding bug as #18 with your reproducer + diagnosis attributed. That tracks the recursive eigentrust case independently — your parser fix here can land without it.

Approving — ready to merge once you rebase to drop the skip commit.

@kumavis
kumavis force-pushed the claude/fix-eigentrust-pitfall-7-marg branch from dd26ad4 to 43017b6 Compare April 26, 2026 01:03
kumavis added a commit that referenced this pull request Apr 26, 2026
The 2026-04-23 eigentrust pitfalls memo (forthcoming branch) enumerated 16
items hit during the EigenTrust implementation. Items #1-7 and #11-15 are
language/elaboration defects with their own PRs. Items #8, #9, #10, and #16
are observations rather than Prologos defects; no compiler change is needed
for them but the memo deserves a parallel disposition note so a future reader
does not double-count them as open work.

#8 (exact-Rat slow on deep iter): intrinsic to exact rational arithmetic;
benchmark-scope guidance, not a fix.

#9 (Posit32 literals work): positive observation; `~` literal prefix is
unambiguous unlike `0/1`. No action.

#10 (PVec preserves where List does not): subsumed by pitfall #3 fix. After
#3 lands, both literal forms preserve element type uniformly. Close as
duplicate.

#16 (column-stochastic vs row-stochastic): algorithm/spec clarification, not
a Prologos defect. The eigentrust implementation branch already takes
column-stochastic M directly and validates via col-stochastic?.

https://claude.ai/code/session_01MbncYJnrvjzhbVWw4xGi5x

Co-authored-by: kumavis <1474978+kumavis@users.noreply.github.com>
@kumavis

kumavis commented Apr 26, 2026

Copy link
Copy Markdown
Contributor Author
defn sum-rows
  | nil            -> nil
  | cons r nil     -> r
  | cons r rest    -> [add-vec r [sum-rows rest]]

expected to see an argument body following the function name.

…defn

Pre-fix: `defn f | cons r nil -> r | cons r rest -> ...` parsed each
bare token after `|` as a separate arg pattern, splitting the function
into clauses with mismatched arities (1 nil clause + 2 cons-3-arg
clauses). This produced a per-clause helper (`f::1`, `f::3`) and the
recursive call site `[f rest]` failed with `Unbound variable f::1`
because the arity-1 helper only knew the nil clause and any non-empty
list hit __match-fail.

Fix shape (a) — implement general pattern matrices by auto-packing
when the leading bare token names a known constructor whose field
count matches the remaining tokens. `cons r nil` → one compound
pattern `pat-compound 'cons (var-r, var-nil)` (then normalize-pattern
converts var-nil to compound-nil since nil is a known nullary ctor).

Fallback preserved for genuinely multi-arg defns:
  defn add | x y -> [+ x y]
The leading `x` is a variable (lookup-ctor returns #f), so falls
through to the old N-arg interpretation.

Trade-off vs (b) (raise an error): (a) makes the syntax do what
ML/Haskell users expect — `defn` IS the primary dispatch mechanism
per .claude/rules/prologos-syntax.md, so making `cons r nil` mean
"compound pattern" is the natural reading. The detection is local and
does not change semantics for any pattern that didn't have a known
ctor as its leading token.

Scope: 23 lines in parser.rkt's parse-defn-clause + 11 new tests in
test-defn-multiarg-patterns.rkt. No changes elsewhere.

Test results: 11/11 new tests pass. 43/43 related tests pass
(test-pattern-defn-01, test-pattern-defn-02, test-multi-body-defn).
Full affected-suite: 4646 tests in 255 files, 1 unrelated pre-existing
failure (stale tracking entry for non-existent
test-constraint-retry-propagator.rkt).

Latent issue exposed (NOT introduced by this fix): compile-match-tree
binds variable patterns to outer-param names when those params get
destructured by a later dispatch column. E.g., `cons r rest` after
outer-cons specialization tries `let rest := __cons_1` while `__cons_1`
is being destructured into `__cons_1_0` and `__cons_1_1`. This affects
the recursive bodies of the eigentrust example with multi-element
inputs, but is a pre-existing bug in compile-match-tree —
reproducible on main with the bracketed `[cons r rest]` form, which
parses to the same internal representation. The new tests cover the
slices unaffected by this bug (empty + singleton inputs, wildcard
patterns, all-var multi-arg) and explicitly call out the latent issue.

Co-authored-by: kumavis <1474978+kumavis@users.noreply.github.com>
@kumavis
kumavis force-pushed the claude/fix-eigentrust-pitfall-7-marg branch from 43017b6 to 458bfcf Compare April 26, 2026 03:03
kumavis added a commit that referenced this pull request Apr 26, 2026
The 2026-04-23 eigentrust pitfalls memo (forthcoming branch) enumerated 16
items hit during the EigenTrust implementation. Items #1-7 and #11-15 are
language/elaboration defects with their own PRs. Items #8, #9, #10, and #16
are observations rather than Prologos defects; no compiler change is needed
for them but the memo deserves a parallel disposition note so a future reader
does not double-count them as open work.

#8 (exact-Rat slow on deep iter): intrinsic to exact rational arithmetic;
benchmark-scope guidance, not a fix.

#9 (Posit32 literals work): positive observation; `~` literal prefix is
unambiguous unlike `0/1`. No action.

#10 (PVec preserves where List does not): subsumed by pitfall #3 fix. After
#3 lands, both literal forms preserve element type uniformly. Close as
duplicate.

#16 (column-stochastic vs row-stochastic): algorithm/spec clarification, not
a Prologos defect. The eigentrust implementation branch already takes
column-stochastic M directly and validates via col-stochastic?.

https://claude.ai/code/session_01MbncYJnrvjzhbVWw4xGi5x

Co-authored-by: kumavis <1474978+kumavis@users.noreply.github.com>
@hierophantos
hierophantos merged commit 5c4c00c into main Apr 27, 2026
1 check passed
@hierophantos
hierophantos deleted the claude/fix-eigentrust-pitfall-7-marg branch April 27, 2026 04:07
kumavis pushed a commit that referenced this pull request Apr 27, 2026
Wires the OCapN port to a real Racket toolchain and fixes the issues
the test run surfaced.

Library fixes
  - vat.prologos: rename `spawn` -> `vat-spawn` (collision with the
    reserved surface form recognised in macros.rkt:`'spawn`),
    `spawn-actor` -> `vat-spawn-actor`. Same in core.prologos and the
    acceptance file.
  - vat.prologos: drop the `Sigma Vat Nat` return shape for spawn /
    fresh-promise / send. Replace with a named `Allocated` struct +
    `alloc-vat` / `alloc-id` accessors. The Sigma form ran into
    "could not infer" elaborator errors when the body destructured
    via `match | pair a b -> ...` and then re-constructed a Sigma;
    `[fst p]` / `[snd p]` reused on the same `p` tripped QTT
    multiplicity. The named struct sidesteps both.
  - vat.prologos: reorder `resolve-promise` / `break-promise` BEFORE
    `apply-effect` (forward-reference rule — module elaboration is
    single-pass top-to-bottom). Also reorder `step-after-act` before
    `deliver-msg` and `list-length-helper` before `queue-length`.
  - vat.prologos: drop the queued-pipeline-flush in resolve-promise /
    break-promise. PromiseState's queue is `List SyrupValue` (wire
    repr); the vat queue is `List VatMsg` (decoded); flushing across
    the boundary would need re-encoding. Phase 1.

Test-fixture fix (load-bearing)
  - All 8 OCapN test files were updated to capture and restore
    `current-ctor-registry` and `current-type-meta` across the setup-
    -> run boundary. The standard fixture pattern from
    `test-hashable-01.rkt` does NOT preserve these — fine for tests
    that only declare traits, but breaks once a preamble's imports
    declare new `data` types (every `data` in our 8 modules). Without
    it, the reducer sees a stale ctor-registry and refuses to fire
    pattern arms over user constructors; results print as un-reduced
    `[reduce ... | vat x y z a -> x] : Nat` strings.
    Documented as goblin-pitfall #12; the canonical fixture in
    test-support.rkt should grow this for every future test.

Compat fence
  - driver.rkt: guard
    `(current-parallel-executor (make-parallel-thread-fire-all))` with
    a feature-detection try/catch on `thread #:pool 'own`. Racket 9
    ships parallel threads; Racket 8 does not. Fence preserves the
    Racket-9 fast path and falls back to sequential firing on 8.

Acceptance
  - examples/2026-04-27-ocapn-acceptance.prologos updated to match
    the new vat-spawn/Allocated API and verified to run clean via
    process-file.

Pitfalls catalogue (docs/tracking/2026-04-27_GOBLIN_PITFALLS.md)
  - #0 (sandbox/no-Racket): closed.
  - +#11 — Racket-8 vs Racket-9 `thread #:pool` compat
  - +#12 — test fixture loses ctor-registry/type-meta across calls
           [highest-impact; canonical fixture pattern needs update]
  - +#13 — `spawn` is a reserved surface keyword; collides silently
  - +#14 — `match | pair a b ->` on Sigma + Sigma reconstruction =>
           "could not infer"
  - +#15 — QTT multiplicity on `[fst p]`/`[snd p]` reused thrice
  - +#16 — single-pass module elaboration: forward references error
  - +#17 — promise-queue (Syrup) vs vat-queue (VatMsg) type clash
           on flush — design pitfall, scope cut

Test results
  refr      6/6   syrup    22/22  promise   16/16  message  19/19
  behavior 13/13  vat     21/21   pipeline   5/5   captp     7/7
  e2e       8/8                                  total  117/117 PASS
kumavis pushed a commit that referenced this pull request Apr 27, 2026
Per user direction:
- Replace the body of every DELETED entry with a single-sentence
  explanation. Numbers reserved per prior instruction.
- Delete #15 (QTT multiplicity on fst/snd thrice). I re-tested
  with a real Racket — `pair [snd p] [fst p]` then a third use of
  `fst p` works fine; no multiplicity error. The failure I had
  conflated this with was actually #14's "match-and-reconstruct
  Sigma" issue.

Result: pitfalls doc shrinks from 765 to 534 lines. Remaining
real claims: #1, #4, #5, #11, #12, #13, #14, #16, #17, #18, #19,
#20 (the user has reviewed only #0-10 so far; #11-20 still
pending their review).
kumavis pushed a commit that referenced this pull request Apr 28, 2026
Builds on the lambda-FFI track (PR #35): the affine combination
  out[j] := bias[j] + Σ_i  weight[j][i] · prev[i]
is now a Prologos `defn affine-step` lambda passed across the FFI on
each `net-add-prop` install. The Racket shim's broadcast item-fn
invokes it once per peer per fire.

What stays in Racket (the truly irreducible core):
  * cell-value carrier (gen-tagged immutable Posit32 vector)
  * the propagator's fire-fn (a Racket closure that reads input cell,
    invokes the Prologos kernel via the FFI bridge, and writes the
    output cell — pure plumbing, NO algorithmic content)
  * FFI marshalling glue (cons/nil chain walking, Posit32 bit-pattern
    extraction, per-row IR list construction)
  * handle/cell registries

What's now in Prologos (the entire algorithm):
  * matrix transpose, decay scaling, bias computation
  * the per-row affine kernel itself
  * iteration driver
  * initial-zero vector

`net-add-prop` is purpose-AGNOSTIC — it's a generic broadcast affine
propagator wired to a domain-specific Prologos kernel. The same shim
would serve any algorithm whose per-row update is `bias + Σ w·x`.

CI integration:
  * New `tests/test-eigentrust.rkt` runs the .prologos file via
    `process-file` and asserts the converged scores match the Python
    reference within 1e-2 — auto-picked up by `tools/run-affected-tests
    --all` (the CI test command).
  * 5 power iterations lands within ~6e-3 of the steady-state
    eigenvector; runs in ~16s, under the test runner's 30s first-result
    guard.

New pitfalls captured (docs/tracking/2026-04-28_ETPROP_PITFALLS.md):
  * #0 — FFI-call AST caching collapses identical side-effecting calls
    onto the same physical cell. Fixed via a "freshness tag" arg on
    `net-new-cell`. Surfaced when distinct layer cells turned out to
    be the same physical cell (self-loop).
  * #16 — FFI-callback overhead per fire is the bottleneck. Each
    kernel invocation runs `nf` on the lambda body. Documented as
    expected scaffolding cost of the off-network FFI bridge; future
    work is propagator-native callbacks via cell subscription.

Test results:
  * tests/test-eigentrust.rkt: 2/2 pass (15.4s)
  * regression set (foreign / foreign-block / pvec / foreign-callback
    / eigentrust): 111/111 pass (21s)
  * Final converged scores match Python reference within 1e-2:
      [0.0712, 0.4288, 0.0712, 0.4288]   (steady state ≈ [0.0652, 0.4348])
kumavis pushed a commit that referenced this pull request May 4, 2026
Wires the OCapN port to a real Racket toolchain and fixes the issues
the test run surfaced.

Library fixes
  - vat.prologos: rename `spawn` -> `vat-spawn` (collision with the
    reserved surface form recognised in macros.rkt:`'spawn`),
    `spawn-actor` -> `vat-spawn-actor`. Same in core.prologos and the
    acceptance file.
  - vat.prologos: drop the `Sigma Vat Nat` return shape for spawn /
    fresh-promise / send. Replace with a named `Allocated` struct +
    `alloc-vat` / `alloc-id` accessors. The Sigma form ran into
    "could not infer" elaborator errors when the body destructured
    via `match | pair a b -> ...` and then re-constructed a Sigma;
    `[fst p]` / `[snd p]` reused on the same `p` tripped QTT
    multiplicity. The named struct sidesteps both.
  - vat.prologos: reorder `resolve-promise` / `break-promise` BEFORE
    `apply-effect` (forward-reference rule — module elaboration is
    single-pass top-to-bottom). Also reorder `step-after-act` before
    `deliver-msg` and `list-length-helper` before `queue-length`.
  - vat.prologos: drop the queued-pipeline-flush in resolve-promise /
    break-promise. PromiseState's queue is `List SyrupValue` (wire
    repr); the vat queue is `List VatMsg` (decoded); flushing across
    the boundary would need re-encoding. Phase 1.

Test-fixture fix (load-bearing)
  - All 8 OCapN test files were updated to capture and restore
    `current-ctor-registry` and `current-type-meta` across the setup-
    -> run boundary. The standard fixture pattern from
    `test-hashable-01.rkt` does NOT preserve these — fine for tests
    that only declare traits, but breaks once a preamble's imports
    declare new `data` types (every `data` in our 8 modules). Without
    it, the reducer sees a stale ctor-registry and refuses to fire
    pattern arms over user constructors; results print as un-reduced
    `[reduce ... | vat x y z a -> x] : Nat` strings.
    Documented as goblin-pitfall #12; the canonical fixture in
    test-support.rkt should grow this for every future test.

Compat fence
  - driver.rkt: guard
    `(current-parallel-executor (make-parallel-thread-fire-all))` with
    a feature-detection try/catch on `thread #:pool 'own`. Racket 9
    ships parallel threads; Racket 8 does not. Fence preserves the
    Racket-9 fast path and falls back to sequential firing on 8.

Acceptance
  - examples/2026-04-27-ocapn-acceptance.prologos updated to match
    the new vat-spawn/Allocated API and verified to run clean via
    process-file.

Pitfalls catalogue (docs/tracking/2026-04-27_GOBLIN_PITFALLS.md)
  - #0 (sandbox/no-Racket): closed.
  - +#11 — Racket-8 vs Racket-9 `thread #:pool` compat
  - +#12 — test fixture loses ctor-registry/type-meta across calls
           [highest-impact; canonical fixture pattern needs update]
  - +#13 — `spawn` is a reserved surface keyword; collides silently
  - +#14 — `match | pair a b ->` on Sigma + Sigma reconstruction =>
           "could not infer"
  - +#15 — QTT multiplicity on `[fst p]`/`[snd p]` reused thrice
  - +#16 — single-pass module elaboration: forward references error
  - +#17 — promise-queue (Syrup) vs vat-queue (VatMsg) type clash
           on flush — design pitfall, scope cut

Test results
  refr      6/6   syrup    22/22  promise   16/16  message  19/19
  behavior 13/13  vat     21/21   pipeline   5/5   captp     7/7
  e2e       8/8                                  total  117/117 PASS
kumavis pushed a commit that referenced this pull request May 4, 2026
Per user direction:
- Replace the body of every DELETED entry with a single-sentence
  explanation. Numbers reserved per prior instruction.
- Delete #15 (QTT multiplicity on fst/snd thrice). I re-tested
  with a real Racket — `pair [snd p] [fst p]` then a third use of
  `fst p` works fine; no multiplicity error. The failure I had
  conflated this with was actually #14's "match-and-reconstruct
  Sigma" issue.

Result: pitfalls doc shrinks from 765 to 534 lines. Remaining
real claims: #1, #4, #5, #11, #12, #13, #14, #16, #17, #18, #19,
#20 (the user has reviewed only #0-10 so far; #11-20 still
pending their review).
kumavis pushed a commit that referenced this pull request May 6, 2026
…o-increment imports-refcount

Wires the bridge dispatcher to recognize desc:export / desc:answer
in inbound op:deliver args and increment imports-refcount per
occurrence. Combined with Phase 34c (refcount table) and Phase 34e
(release wired to bs-decr-import, next), this gives us the auto-
tracking half of distributed reference counting.

  captp-bridge.prologos:

    + match-payload-as-export SyrupValue -> List Refr
    + match-payload-as-answer SyrupValue -> List Refr
        Pattern-match SyrupValue payloads (typically syrup-nat).

    + extract-refrs-from-tagged String SyrupValue -> List Refr
        Tag-based dispatch: "desc:export" → match-payload-as-export,
        "desc:answer" → match-payload-as-answer, anything else → nil.

    + walk-toplevel-list [List SyrupValue] -> List Refr
        One-level walk over a SyrupValue list, calling shallow-refr
        per element.

    + shallow-refr SyrupValue -> List Refr
        Per-element check: top-level desc:* tag yields a refr;
        anything else (including nested syrup-list) yields nil.

    + extract-refrs-from-args SyrupValue -> List Refr
        Top-level entry point: handles atomic, single tag, and
        one-level-deep list.

    + bs-incr-import-by-refr Refr BridgeState -> BridgeState
    + bs-incr-imports [List Refr] BridgeState -> BridgeState

    captp-incoming-with-state op-deliver / op-deliver-only /
    op-deliver-to-answer arms: extract refrs from args + bulk-
    increment imports-refcount before dispatching.

Limitation (documented inline): refrs nested deeper than one list
level aren't extracted. Common OCapN args shapes (top-level refr OR
list of refrs+primitives) are covered; deeper nesting (refrs inside
records inside lists) deferred to a future phase that adds
generic-walker recursion.

Lessons learned (codified):

1. Pitfall #16 (forward references / mutual recursion): true mutual
   recursion (`extract-refrs-from-args` ↔ `walk-toplevel-list`)
   isn't supported. Worked around by making the inner walker call
   only `shallow-refr` (which doesn't recurse into syrup-list).
   Trade-off: the walker is one level deep instead of full tree.

2. Multi-arity defn with constructor cons-pattern + multi-line body:
   `defn name | nil -> ... | [cons hd tl] -> body` triggered
   ??__match-fail. Worked around by using bracketed-arg + inline
   `match`: `defn name [xs] (match xs | nil -> ... | cons hd tl -> body)`.
   Pattern is the same shape as data/list's `concat`. Codified.

Tests:
  + 5 new unit tests (69 total in test-ocapn-bridge.rkt):
    - extract-refrs-from-args returns nil for atomic args
    - extracts a top-level desc:export
    - walks one-level into syrup-list (2 refrs in mixed list)
    - extracts top-level desc:answer
    - captp-incoming-with-state op-deliver auto-increments
      imports-refcount[7] when args = <desc:export 7>

Verification:
  - 73 OCapN tests pass across 5 files (40 s combined)
  - bridge tests: 69/69 (5 new + 64 prior)
  - All interop tests still pass — auto-increment is non-disruptive
    for tests that don't pass refrs in args (count stays 0)
kumavis pushed a commit that referenced this pull request May 6, 2026
…it, `->` in identifiers silently fails

Two new Prologos elaborator/reader pitfalls discovered during the
refr-import track (Phase 34a + 34d). Both cost ~15 minutes each
to debug because the failure modes were silent or had misleading
error messages.

  Pitfall #34 — `data` constructor signatures have IMPLICIT return type.

    Writing `ctor : T1 -> T2 -> Result` is INTERPRETED as "takes
    3 args (T1, T2, Result), returns Result." The user usually
    means "takes 2 args, returns Result." The convention in this
    codebase (Listener, QEntry, etc) is to drop the trailing
    `-> Result`: `ctor : T1 -> T2`.

    Discovered Phase 34a (commit 3d8c069). My initial
    `refr : Nat -> Nat -> Refr` caused smart constructors to fail
    with "Type mismatch [Pi Nat Refr -> Refr]". Fix: drop the
    trailing `-> Refr`.

  Pitfall #35 — Function names containing `->` silently fail.

    Identifiers with `->` (like `refr->syrup`, common in ML/Lisp
    converter naming) are SILENTLY DROPPED by the elaborator. The
    WS-mode reader parses `->` as the function-arrow type operator
    inside the identifier, splitting the symbol. spec/defn forms
    can't bind anything sensible and produce no output at all —
    callers get "Unbound variable" downstream.

    Discovered Phase 34d (commit 7c797e6). My `refr->syrup` was
    silently dropped; renamed to `refr-to-syrup` worked. Codebase
    convention: use `-to-` for converters.

  Also documented (no new pitfall, recurrence-only):
    - #18 (multi-arity defn cons-pattern) hit on add's
      `suc a b -> ...`; fix: bracket as `[suc a] b -> ...`.
    - #21 (multi-line clause body → ??__match-fail) hit on
      extract-refrs-from-list; fix: `defn name [arg] (match arg ...)`
      same shape as data/list's `concat`.
    - #16 (mutual recursion) hit on extract-refrs-from-args ↔
      extract-refrs-from-list; fix: one-way recursion via
      shallow-refr.
    - Issue #60 (multi-constructor cross-module inference) hit
      on 5-arg helpers; fix: 2-3 arg shape via BridgeStep.

  Updated .claude/rules/prologos-syntax.md:
    - Added "NEVER use `->` in identifiers" to the Naming section
    - Added a new "Data type definitions" section with the
      implicit-return-type convention

These two are filed in the pitfalls doc but not as GitHub issues
yet — both are reasonably easy elaborator/reader fixes that
upstream Prologos work could close. Triage decision deferred.
kumavis pushed a commit that referenced this pull request May 9, 2026
Three new entries in 2026-04-27_GOBLIN_PITFALLS.md:

- #36: Multi-line constructor / function application — continuation
  args on a separate line are eaten as an inner application. Hit
  twice in this branch (Phase 41 `bs-add-pipeline-msg`, Phase 48
  `bs-gc-listeners-by-notified`); the second occurrence triggered
  codification per the workflow rule "codify a 2-occurrence pattern
  within a track immediately."

- #37: Single-arg multi-arity `defn` over `data` patterns sometimes
  infers a phantom 2nd parameter. Hit on `resolution-syrup-of-pst`
  (Phase 48); the fix was to switch to `defn name [arg] match arg`
  shape, which pinned the inferred type back to the spec.

- #38: `let X := EXPR` value can't span multiple lines. Hit on
  `drive-break-with-two-ops` (Phase 49); same workaround family as
  #21 and #36 — collapse to one line. Recorded separately because
  the error message ("missing value after :=") points at a
  different line than the actual broken `let`.

Plus a "Recurrences during Phase 47-49" section noting that
pitfall #16 (forward references) was hit again on `member-nat?` —
existing entry confirmed correct.

Also updated `.claude/rules/prologos-syntax.md` § "Application
style" with two new bullets cross-referencing #36/#37/#38, so
future implementations catch these at write time rather than at
load-time error. Per the workflow rule "if a workaround is needed
twice in the same track, add it to the pitfalls log AND to the
relevant rule file immediately."

https://claude.ai/code/session_01YM6gc3cMNH2Ymor4jdZY8u
kumavis pushed a commit that referenced this pull request Jul 27, 2026
The Phase 59b commit (366f85f) placed `incoming-deliver` BEFORE
`dispatch-deliver`, which it calls. Module elaboration is single-pass
top-to-bottom, so the callee must already be in scope; the forward
reference failed at import with

  imports: Error loading module prologos::ocapn::captp-core: Unbound variable

turning the interop job red. This is goblin-pitfalls #16, in the log I had
just re-audited — and precisely the failure the "read your own pitfalls log
before each phase" rule exists to prevent.

Fix: move only `incoming-deliver` to sit after `dispatch-deliver`; the
fetch helpers reference nothing later than themselves and stay put. Added a
comment at the definition recording the constraint and the exact error, so
the next edit does not reintroduce it.

Verified definition-before-use across the whole new block:
maybe-fetch@1272 -> used@1314, dispatch-deliver@1291 -> used@1316,
incoming-deliver@1313 -> used@1428, and each fetch helper precedes its
caller.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YM6gc3cMNH2Ymor4jdZY8u
kumavis pushed a commit that referenced this pull request Jul 27, 2026
…ound variable"

Bisected from the Phase 59b failure that cost two CI round-trips and a
revert (366f85f -> 0070f1e -> b62288c). The block had THREE
independent bugs, all producing the identical, nameless error:

  imports: Error loading module prologos::ocapn::captp-core: Unbound variable

  1. definition order (pitfall #16) — genuinely wrong, fixed in 0070f1e
  2. missing [prologos::data::string :as str :refer []] — genuinely missing
  3. [some 1] against a spec of [Option Nat] — bare literals are Int

Each fix looked like it had failed because the next bug printed the same
thing. That is the actual finding, and it is a language/tooling bug, not
an OCapN one: the same source elaborated through process-file reports

  (type-mismatch-error ... "[... Option Nat]" "[... Option Int]" ...)

with both types in hand, while the module `imports` path reports the
wrong error CLASS with no name and no location. The structured error
exists upstream and is discarded at the import boundary; a genuine
unbound reference in the same path does carry symbol + srcloc. So the
ask is error propagation at `imports`, not "add more detail".

Recorded the cheap instrument too: re-elaborate the suspect module
through process-file to recover the true error (~1 min) instead of
bisecting definitions (~2 min/cycle) or via CI (~6 min/cycle).

Also aligns test-ocapn-bridge's shared fixture with main's authoritative
form — capture (global-env-snapshot), re-inject via
(module-network-add-import (make-module-network)
  (module-network-from-snapshot ...)) — rather than capturing the raw
mnr and re-parameterizing it, which is what my 4ae90b3 migration did.
Main's own sweep in 9d166ce was a delete-line because the mnr binding
already sat beside the retired params; our OCapN fixtures predate it, so
the substitution I wrote was never equivalent. NOTE: this did NOT change
the failure count (33/149 before and after), so it is a correctness
alignment, not the fix for those failures — still under investigation.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YM6gc3cMNH2Ymor4jdZY8u
kumavis pushed a commit that referenced this pull request Jul 27, 2026
Re-applies the block reverted in b62288c, now with all three bugs fixed
and the load VERIFIED LOCALLY before pushing (the step whose absence
caused the two red-CI round-trips and the revert).

Every upstream ocapn-test-suite test begins by fetching an object from
the peer's bootstrap object by swiss-num and awaiting the answer on the
RESOLVE-ME descriptor, not on an answer-pos:

  <op:deliver <desc:export 0> ['fetch <swiss>] false <rm>>
  <op:deliver <desc:export rm> ['fulfill <desc:import-object N>] false false>

captp-incoming-with-state discarded `rm` and dispatch-deliver routes only
via answer-pos, so that reply path did not exist. incoming-deliver tries
the fetch path first and falls through to the pre-existing routing, so
dispatch-deliver and its callers are untouched.

The three bugs, all of which printed the identical nameless
"Error loading module ...: Unbound variable":
  1. incoming-deliver defined before dispatch-deliver (pitfall #16)
  2. missing [prologos::data::string :as str :refer []] for str::eq
  3. [some 1] .. [some 5] against [Option Nat] — bare literals are Int,
     so these are now 1N..5N (pitfall #44, filed in 11b15c3)

Bisected locally with a 3-line probe in the real module (~2 min/cycle);
each fix was confirmed in isolation with controls, so the attribution is
not guesswork:
  [some 1] -> FAIL   [some 1N] -> OK
  [none Nat] -> OK   [str::eq s "x"] -> OK

swiss-num-export is scaffolding, and named as such: mapping swiss-nums to
export positions is a REGISTRY and belongs on the network as a hash-union
cell keyed by swiss-num. It is closed here because the objects themselves
are not yet vat actors — Greeter and Echo exist as beh-greeter/beh-echo,
the other three do not, and BehaviorTag is a closed `data` enum so they
cannot be added without widening it. Retirement plan: replace with a cell
once the behaviours exist.

Verified: module loads clean against post-merge HEAD. Behaviour is NOT
yet verified end-to-end — the upstream fetch_object path needs the
objects to answer, which is part 2 and blocked on BehaviorTag above.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YM6gc3cMNH2Ymor4jdZY8u
kumavis pushed a commit that referenced this pull request Jul 27, 2026
Part 1 built the resolve-me reply for `fetch` only. Every other inbound
op:deliver dropped `rm` at dispatch-deliver and, with no answer-pos,
discarded the behaviour's return value at step-after-act — so a peer using
the upstream convention (answer_position=false + resolve_me_desc) waited
forever. This adds the general path.

CORRECTS THE PART-2 WORKLIST. That doc (written from a 14-agent audit)
listed "resolver table mapping local promise-id -> peer resolver export
position" and "a pump stage that emits for settled promises not in the
inbound question table" as two of seven mechanisms that had to be BUILT.
Both already exist — they are the op:listen machinery. A resolve-me
descriptor IS a listener on the answer promise:

  bs-add-listener            records pid -> resolver export position
  pump-outbound              already emits listener-notify-bytes on settle
  listener-notify-bytes      already produces exactly the reply shape
                             <op:deliver <desc:export RM> ['fulfill v]>

So this is a composition of existing parts, not new infrastructure. The
audit's own facets missed it because the machinery is named for op:listen
rather than for resolve-me. Two of seven mechanisms were phantom; the doc
will be corrected in a follow-up.

deliver-resolve-me allocates a local answer promise, registers the peer's
resolver position as a listener, and delivers. It routes through
bs-handle-listen-with-late-fire rather than bs-add-listener directly: the
already-settled case cannot arise for a promise allocated one line
earlier, but that helper is the correct-by-construction entry point and
costs nothing.

incoming-deliver now tries fetch, then answer-pos, then resolve-me, then
plain pass-through. It MOVED below bs-handle-listen-with-late-fire because
single-pass elaboration needs the callee in scope first (pitfall #16); a
breadcrumb sits at the old location.

Verified: captp-core loads clean at this commit via a module-import probe
(~2 min), BEFORE pushing. This does NOT yet move an upstream test — the
server still has no actor seeded at the export position `fetch` hands out,
so there is nothing to deliver TO. That seeding is the next step, and it
is genuinely small (actor-table-set + conn-state are both public).

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YM6gc3cMNH2Ymor4jdZY8u
kumavis pushed a commit that referenced this pull request Jul 27, 2026
…ly deliver path

Upstream's test_send_deliver_no_answer_or_response HUNG rather than failed:
the server logged `62 in / 0 out` and the peer timed out after 62s. Two
independent defects, both silent (no error, no diagnostic, no bytes):

1. A descriptor's table position arrives as a Syrup POSITIVE INTEGER —
   `<18'desc:import-object1+>`, note the `+` — so it decodes to syrup-int,
   not syrup-nat. `nat-payload` accepted only syrup-nat and mapped syrup-int
   to none, so the greeter found no target in the descriptor it had just
   been handed: right tag, length-1 list, and still none.

   Fixed by delegating to `wire-nat`, which already handled both spellings
   with a negative guard. It lived in captp-wire; it reads a SyrupValue as a
   table position, which is a SyrupValue concern and not a codec one, and
   `behavior` must not depend on the whole wire codec to use it — so it moves
   to prologos::ocapn::syrup. captp-wire picks it up via its existing
   :refer-all; captp-core's require is repointed. One definition, not two.

2. The greeter's reply target is the PEER's export position. That namespace
   is independent of our local actor-table ids and collides with them: the
   upstream test delivers `desc:import-object 1` to a greeter that itself
   sits at local actor id 1. `deliver-msg` disambiguated by heuristic — "no
   local actor at this id, therefore remote" — so the reply was delivered
   back to the greeter instead of going out on the wire.

   Fixed by making the namespace part of the effect: a new `eff-send-remote`
   variant goes straight to the outbound queue with no actor-table lookup.
   The heuristic is no longer load-bearing for behaviour-originated sends.
   `enqueue-outbound` moved above `apply-effect` (single-pass modules,
   goblin-pitfalls #16).

Diagnosis note: `connection-step` with a HAND-CONSTRUCTED op emitted the
correct frame, while the same frame decoded off the wire emitted nothing.
That gap is what isolated bug 1 — the constructed op used syrup-nat, the
wire gives syrup-int. Earlier sessions had blamed the FFI stash and the
post-fetch state; both are exonerated (a fresh connection with no fetch
reproduces identically).

Interop: 6 of the 6 selected upstream tests pass (was 5). The greeter is
added to the allow-list and EXPECTED_PASS raised to 6.

Tests: +6 in test-ocapn-behavior.rkt pinning both bugs — int/nat/negative
payload spellings, the descriptor reader, the wire arg-list shape, and that
the greeter emits eff-send-remote (not eff-send-only). Existing OCapN suite
green (83 tests across 8 files).

Also: an env-gated `OCAPN_FRAME_HEX` dump of inbound frames in the test
server. Capturing the real frame is what made this diagnosable — the
previously-recorded probe used `desc:import-object 7`, but the wire sends 1,
and the collision only exists at 1.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YM6gc3cMNH2Ymor4jdZY8u
kumavis pushed a commit that referenced this pull request Jul 27, 2026
…record the exporter blocker

Two things.

1. test-ocapn-bridge's "late op:listen ... broken promise" asserted an
   `<Error _>` wrapper and NO verb on the listener channel. 05b119a made
   resolution-syrup-of-pst carry the verb (['break r]), which is what
   upstream asserts on, so that assertion was pinning the OLD contract. It
   now checks for the `break` verb AND that the reason survives.

   Process note: I missed this at 05b119a because test-ocapn-bridge was not
   in that commit's targeted set, even though the change touched a function
   with three call sites across the bridge. Grep the callers, then pick the
   test set from the callers — not from the files I happened to edit.

2. HandoffRemoteAsExporter (4 tests) was implemented and then REVERTED,
   because the blocker turned out to be upstream of the handler.

   First the good news, which settles the netlayer question empirically: all
   7 third_party_handoffs tests RUN over plain TCP — no ModuleNotFoundError,
   no Tor — and fail on protocol behaviour. Correction 4 is confirmed.

   The implementation covered all four gaps: gift table re-keyed to the id
   BYTES (gift ids are b"my-gift", and the old Nat-keyed dispatch ran
   wire-nat on them, returned none, and bridged through unchanged so no gift
   was EVER recorded); withdraw-gift moved off the answer-pos gateway onto
   the resolve-me one (upstream never sends an answer-pos for it, so the
   reply was always dropped); the five-level descent
   sig-envelope → handoff-receive → sig-envelope → handoff-give → gift-id;
   and fulfill/break reply builders.

   It still emitted 0 bytes, because `decode-op` returns NONE for the
   716-byte withdraw frame. Verified directly: deliver-target on the decoded
   op yields the none-branch sentinel. Nothing downstream of the decoder can
   matter until that is fixed. The frame is the first to carry nested
   desc:sig-envelopes, a gcrypt s-expression signature
   ([sig-val [eddsa [r …] [s …]]] — nested lists, not a record), and
   desc:handoff-give's public-key and location sub-records.

   Reverted rather than landed: the byte-keying breaks ~20 assertions in
   test-ocapn-bridge that deposit Nat ids, and the channel move breaks the
   ones asserting an answer-pos reply. Rewriting 20 assertions to unlock 0
   tests, while the decoder still blocks all 4, would leave the gift table
   half-migrated for no gain — the same call made earlier for the Vat
   struct. The analysis is the deliverable; the handler is a short redo once
   the frame decodes.

   Also worth recording: the stuck-`[reduce …]` signature that has shown up
   repeatedly is the single-pass forward-reference failure mode
   (goblin-pitfalls #16). Placing the new handler AFTER its caller produced
   exactly that — an unreduced term, no error — and reordering fixed it.

Suite: OCapN 229 tests green across 8 files. Interop unchanged at 9 of 24.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Claude-Session: https://claude.ai/code/session_01YM6gc3cMNH2Ymor4jdZY8u
kumavis pushed a commit that referenced this pull request Jul 28, 2026
…interop 16 -> 17

The greeter's send went out with answer-pos and resolve-me both false: enough
for the peer to receive, but the result was dropped and the peer had nowhere
to reply. Each queued outbound send is now a question:

  * a fresh local promise, registered in bs-outbound-questions;
  * a fresh local actor running `beh-resolver` with that promise in its state,
    exported as the resolve-me. When the peer delivers `['fulfill v]` to
    `<desc:export R>`, ORDINARY inbound routing lands it on that actor and
    `eff-resolve` settles the promise. Nothing special-cased on receive.

Once the promise settles the answer entry is finished business:
`<op:gc-answers [pos …]>`, then the entry is dropped so the next step cannot
re-emit it.

The answer position goes on the wire as a BARE INTEGER, not `<desc:answer N>`
as `outbound-question-bytes` writes it. The peer reads slot 2 raw and writes
it raw, so a wrapped position comes back as a record and never compares equal
to the position we later name in op:gc-answers. Both builders now exist side
by side rather than one being changed under the existing questioner path.

The drain runs in `connection-step`, not inside `pump-outbound`, because it
must update BridgeState and PumpResult carries none.

Three single-pass/layout traps on the way, all with misleading diagnostics:
`drain-vat-questions` calls `clear-outbound` and sat above it, which is a
STUCK `[reduce …]` term rather than an error (#16); `append` takes two args
with an implicit type, and the 3-arg form elsewhere in this file made the
over-application look idiomatic; and the module-import "Unbound variable" and
the stuck-reduce present differently depending on cache warmth, so the same
defect showed two faces across runs.

Tests: +3 in test-ocapn-bridge (157 pass) — the filled answer-pos/resolve-me
slots, the answer-table registration, and the no-noise case where nothing has
settled. Full suite 9541 pass / 62.5s. Interop 17 of 24.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants