FFI: retire lambda-passing scaffolding via propagator-native callbacks (cell subscription)

[hierophantos]

Summary

Retire the off-network FFI lambda-passing scaffolding (introduced in PR #35 / wrap-prologos-fn-as-racket) in favor of propagator-native callbacks. Replace the procedure-shaped Racket-side bridge with cell-subscription semantics: a Prologos lambda exposed to a Racket consumer becomes a cell handle the consumer reads/observes, not a procedure they invoke.

Origin

PR #35 added function-typed FFI parameters by wrapping Prologos lambdas as Racket procedures. The contributor explicitly self-labeled the marshaller as "off-network scaffolding by design" with this retirement direction named in the design doc (docs/tracking/2026-04-28_FFI_LAMBDA_PASSING.md). This issue captures the retirement work as a tracked obligation rather than verbal-only intent.

Per .claude/rules/workflow.md adversarial framing rule and on-network.md mantra discipline:

• The current bridge is off-network: Racket call-and-return shape (procedure invocation), not information flow through cells
• Marked scaffolding with explicit retirement direction — exactly the disclosure pattern the project asks for
• Building the bridge now was the right tradeoff: blocked work (EigenTrust on propagators, every Racket HOF reachable from Prologos) cannot wait for the propagator-native path to mature

Current state (post PR #35 merge)

foreign.rkt's wrap-prologos-fn-as-racket produces a Racket procedure that, on each Racket-side call:

1. Marshals each Racket arg back into Prologos IR via marshal-racket->prologos
2. Builds (((pf arg1) arg2) ... argN) via left-folded expr-app
3. Reduces with nf (off-network procedural call)
4. Marshals the result back to Racket

This works but is structurally a Racket procedure boundary — not a propagator network observation.

Target state (propagator-native callbacks)

Replace the procedure-shape boundary with cell-shape:

• A Prologos function value exported to Racket is registered as a cell handle (cell-id + observation contract)
• Racket consumers subscribe to the cell rather than invoking a procedure
• Each call site contributes inputs as cell writes; the result is read from the output cell
• Propagator architecture handles the rest: lattice merges, BSP scheduling, worldview, retraction

This integrates the FFI surface with the propagator network rather than maintaining a parallel mechanism.

Design questions to resolve (when scheduled)

• Cell identity for callbacks: each call site gets fresh cell ids? or reusable handles?
• Multi-arg dispatch: how do N-ary callbacks map to cell topology? (single compound cell vs N input cells + 1 output cell)
• Synchronization: Racket callers expect synchronous return; propagator network is conceptually parallel + asynchronous. Reconcile via BSP barrier? Or eager fire-and-wait?
• Worldview semantics: which worldview does the Racket call observe? (per-call binding? caller's current worldview?)
• Retraction: can a callback fire under retracted assumptions? Stratification handling.

These are real design questions, not implementation details. The issue tracks the obligation; the design happens in a future Stage 3 cycle.

Cross-references

• PR foreign: pass Prologos lambdas across the FFI as live closures #35 — introduced the off-network scaffolding being retired here
• FFI series master: docs/tracking/2026-04-28_FFI_MASTER.md (this issue tracked there as Track N)
• Design doc: docs/tracking/2026-04-28_FFI_LAMBDA_PASSING.md — section on retirement direction
• Mantra discipline: .claude/rules/on-network.md, .claude/rules/workflow.md (adversarial framing, "off-network state is debt against self-hosting")
• Forward vision: Prologos as polyglot hub. Propagator-native callbacks are foundational for FFI to scale to multiple languages without each requiring its own off-network bridge.

Out of scope for this issue

• Implementation timing (gates on broader propagator-native infrastructure maturity)
• Detailed design (deferred to phase-time mini-design when scheduled)
• Reverse-direction marshaling (Racket procedure → Prologos lambda) — separate concern, also currently scaffolded with explicit error in PR foreign: pass Prologos lambdas across the FFI as live closures #35

Priority

Medium-high. The off-network scaffolding works correctly and unblocks downstream work, but every additional FFI consumer that uses it accumulates more dependence on the procedure-shaped boundary. Earlier retirement = less migration burden later.