Skip to content

Track: Events — in-system pub-sub for cross-context decoupling #936

Description

@accuser

The theme

A context declares a typed fact with the event keyword; a handler in that
context emits it with given Events; zero or more subscriber services in
other contexts, declared from Events(SomeEvent), receive each emission as an
ordinary typed handler invocation. The event type is the topic — no string
topic names — and a subscriber may refine which emissions it receives with a
structural pattern on the payload, mirroring the auth pattern already on
services. Emission is fire-and-forget and releases at handler commit; an
aborted handler emits nothing. Delivery is at-least-once; effectful
subscribers dedup via the already-shipped Idempotency capability keyed on
the runtime envelope's eventId. The end state when this track retires:
event declarations, given Events emission, and from Events(...)
subscription (pattern-refined, with envelope metadata and additive
versioning) are real, checked surface with their one first-party provider,
and the design notes' PaymentConfirmed worked example compiles and runs end
to end — minus replay/backfill, which this track splits out to a
separate, future, currently-unfiled track.

  • Realises: design/bynk-design-notes.md §7 ("Events in depth", lines
    162–288) and the emit-lowering note (line 1321); the §12 event-subscriber
    idempotency idiom (lines 641, 656–666), whose substrate — the Idempotency
    capability keyed on env.eventId — already shipped ahead of this track
    (#554,
    idempotency-capability.md).
  • Track doc (added by the settling PR): design/tracks/events.md

Nothing in the compiler implements event/Events today —
bynk-syntax/src/keywords.rs has no event token; bynk-check/src/firstparty.rs
lists the first-party capabilities as Clock, Random, Logger, Fetch,
Secrets, Locale, Idempotency — no Events.

Why a track (the ADR 0076 trigger)

  • Multi-increment — the core is naturally four-to-five separable
    slices: the emit/subscribe loop, pattern refinement, the envelope +
    idempotency idiom, and versioning. Each lands as its own increment proposal.
  • Surface not yet settled — despite the design notes' richness, the
    substrate it lowers onto is an explicit open fork ("Queues with
    topic-as-queue routing, or a custom event-fanout DO"), and the
    per-publisher FIFO guarantee is asserted against a platform whose queues do
    not provide it for free — both need settling, one empirically.
  • Correctness/safety boundary — events cross context boundaries
    carrying domain payloads to an open set of subscribers. A pattern-filter bug
    over-delivers; an owner-emission-enforcement bug lets a context emit
    another's facts; a broken ordering or dedup story double-applies effects.

Open design questions

1. The fan-out substrate on Cloudflare (the load-bearing fork). The design
notes' lowering note offers a fork without choosing it: emit "maps to Queues
with topic-as-topic routing, or a custom event-fanout DO for higher-fanout
scenarios." Three asserted properties all hang off this choice: per-publisher
FIFO ordering, subscriber failure isolation with independent transactions, and
(for the follow-on track) replay from a durable log. Queue-per-topic is cheap
and reuses the shipped on queue consumer path, but a single queue fanning out
to N independent subscribers needs a per-subscriber offset/ack a shared queue
doesn't natively give — failure isolation is the specific casualty. A fanout DO
gives per-subscriber delivery state and a natural home for per-publisher
ordering (single-threaded) and the future replay log, but is new runtime
machinery on the emission hot path and a scaling chokepoint at high fan-out.
This is the ADR to land up front; everything else composes above it.

2. Admitting Events to the closed protocol set. Events is a sixth service
protocol, and the protocol set is closed
(ADR 0079),
so admitting from Events(...) is a deliberate extension. Mostly mechanical —
from <protocol> on the service header already ships and from WebSocket
already proves a from-protocol carrying associated surface — but from Events(E) parameterises the protocol by an event type, and no shipped
protocol currently takes a type argument on the header.

3. Subscription pattern refinement. from Events(E { region: Domestic, .. })
filters emissions by a structural pattern, type-checked against the event
shape and enforced by the runtime before the handler runs. Leans hard on
precedent (multi-actor sum dispatch, authorisation invariants, the shipped
refined-pattern/nested-payload machinery). The one thing to settle: where the
filter runs — the design notes want "server-side filtering where the platform
supports it, deliver-and-filter as a transparent fallback." Slice 1 should
ship deliver-and-filter only; server-side pre-filtering is a later,
semantics-preserving optimisation.

4. Per-publisher FIFO: the contract vs. what Cloudflare delivers. The
design notes assert a specific ordering contract — events from the same
publishing agent are delivered to each subscriber in emission order; across
publishers, no ordering — but this is a claim about runtime behaviour on a
platform whose queues do not guarantee cross-message ordering. Needs the same
empirical treatment the deploy track gave binding-resolution ordering
(ADR 0193):
an integration fixture that emits an ordered burst and asserts delivery order,
run against the chosen substrate, before the guarantee is documented as
shipped.

5. The cross-build schema registry. Additive versioning needs
env.schemaVersion; the design notes specify a compiler-maintained schema
registry across builds, computing the version from the type's structural
shape, with an evolution report and optional @schema(N) pins. Where the
registry lives, how a version is computed (reuse the ADR 0200 contract-hash
normal form, or a distinct hash?), and what counts as a version-bumping
change all need settling. A slice-3 concern, not a slice-0 blocker.

Not an open question — replay/backfill is split out. Event replay — a new
subscriber backfilling from log history, upgrading old wire events to the
current schema on read — moves, together with the inherited actors
Q8 (replay/ordering) (#260),
to a separate, future, currently-unfiled track: it depends on the durable
Idempotency provider, which does not exist yet (only the in-memory provider
has shipped), and it needs its own durable-event-log substrate design as heavy
and unspecified as question 1 above. The live core — emit → subscribe →
pattern-filter → envelope → additive-version — is coherent and shippable
without it. This track still owes replay its seams: the envelope carries
eventId/schemaVersion from day one, and the §1 substrate leaning toward a
fanout DO is chosen partly because it is the natural future log-owner.

Candidate slice decomposition

  • Slice 0 — the emit/subscribe loop. event declarations (owner-only),
    the Events capability with emit (release-at-commit), and pattern-less
    from Events(E) subscriber services delivering via the chosen substrate,
    deliver-and-filter. Opens the closed protocol set to Events and adds the
    event-type-parameterised from Events(E) header grammar. Proven against a
    real tsc --strict project fixture (a two-context emit/receive), not
    golden-diffed alone.
  • Slice 1 — subscription pattern refinement. from Events(E { field: X, .. }),
    reusing the refined-pattern machinery; statically-known fields available in
    the body; deliver-and-filter enforcement.
  • Slice 2 — the envelope + the idempotency idiom. EventEnvelope
    (eventId, publisherId, emittedAt, schemaVersion reserved for slice 3);
    the documented env.eventIdIdempotency.dedup/remember pattern for
    effectful subscribers. Settles whether the idempotency track's parked
    "event-subscriber sugar" is bespoke syntax or documented convention.
  • Slice 3 — additive versioning. Default expressions on event-type fields,
    env.schemaVersion, the cross-build schema registry with its evolution
    report and optional @schema(N) pin.
  • Slice 4 — via version-aware dispatch. The via schema(...) envelope
    pattern clause: literal versions, ranges, _; a no-via subscriber receives
    any version.

Not slices of this track: replay/backfill-from-log, the durable event log
substrate, and the inherited actors Q8 (#260) —
moved to a future, currently-unfiled track.

Slice status

Per-publisher FIFO ordering (question 4) is now also settled, empirically,
scoped narrower than originally asserted:
holds within one emission batch
and across successive non-overlapping calls to one agent, not across
concurrent invocations of the same agent. See
design/tracks/events.md §3.4 and
design/pending/events-per-publisher-fifo-scope.md (pre-stamp ADR).

Front-loaded ADR candidates

  • The Events fan-out substrate (question 1) — records the
    fanout-DO-vs-queue choice and why; per-publisher ordering, subscriber
    failure isolation, and the future replay log all depend on it, and it is the
    most expensive decision to reverse. Must land before slice 0.
  • Events joins the closed protocol set; the header takes a type argument
    (question 2) — records extending
    ADR 0079
    to a sixth protocol and the new event-type-parameterised from Events(E)
    header shape.
  • Per-publisher FIFO is a verified guarantee, not an assertion
    (question 4) — records the empirical fixture that backs the ordering claim
    on the chosen substrate.
  • Subscription pattern dispatch reuses auth/refined-pattern machinery
    (question 3) — records that no bespoke matching engine is introduced;
    deliver-and-filter is committed semantics.
  • Event schema versioning + the cross-build registry (question 5) —
    slice-3 ADR; records where the registry lives and whether it reuses the
    ADR 0200 normal form.
  • Replay is out of scope — records the split decision itself, so it is
    durable and citable, and names the future track and its dependency on the
    durable Idempotency provider.

Threat model

Asset. The event payload in transit from a publishing context to an open
set of subscribers — potentially carrying domain data, including opaque
fields a foreign subscriber may hold but not introspect — and the two
correctness guarantees the mechanism makes: that only the owning context
emits a given fact, and that a subscriber receives exactly the emissions its
pattern admits.

Adversary / failure modes.

  • Over-delivery via a pattern-filter bug. A subscriber whose generated
    guard admits emissions its declared pattern should exclude sees data outside
    its intended slice. Mitigation: the filter is generated from the
    type-checked pattern and enforced before the body runs; slice 1's fixtures
    must include negative cases (an emission that must not arrive).
  • Forged/cross-context emission. A context emitting another context's
    event type would let it fabricate facts attributed to a peer. Mitigation:
    owner-only emission is statically enforced — a compile error, not a runtime
    check, and the primary boundary guarantee.
  • Duplicate delivery double-applying effects. At-least-once means a
    subscriber can receive the same event twice. Mitigation: the shipped
    Idempotency capability keyed on env.eventId — but the in-memory
    provider does not survive a crash, so a duplicate across an isolate restart
    is not deduped until the durable provider (the future track) exists. Named
    here, not hidden — the same accepted-gap posture idempotency itself shipped
    with.
  • Mis-ordered delivery corrupting state. A subscriber that assumes
    emission order breaks if the substrate reorders. Mitigation: the
    per-publisher FIFO guarantee — but only to the extent it is empirically
    verified on the chosen substrate (question 4). Cross-publisher ordering is
    explicitly not guaranteed.
  • Payload validation at the boundary. An event is a value crossing a trust
    boundary; refined fields are validated on receipt, malformed events routed
    to the platform dead-letter policy. Subscribers never defensively re-check a
    refinement the type already established.

Metadata

Metadata

Assignees

No one assigned

    Labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions