feat(serder): #79 pluggable serialization backend — seam + DirectJson zero-copy writer#141
Merged
Merged
Conversation
…n behind EventSerializer Implements the #79 design (docs/superpowers/specs/2026-07-10-79- serialization-backend-seam-design.md §3.2): the SAID/version orchestration moves into serialize_with — render once with a placeholder SAID and zero-size version string, backpatch the measured size in place, hash, splice the SAID into the reported EventLayout slots. Backends implement EventSerializer and only control how bytes are produced. The serde_json path becomes the SerdeJson reference backend: each per-ilk module keeps its build_*_json field logic as render_json, and the public serialize_* signatures are unchanged. Byte identity is proven by the untouched round-trip, kel-chain, and keripy differential suites (1862 tests green). As a side effect the reference backend itself drops from three full JSON renders per event to one render + two fixed-width in-place patches. New public surface (additive): EventSerializer, EventRef, EventLayout, SerdeJson, serialize_with. New error variant InvalidEventLayout (breaking per error-variant policy; called out in CHANGELOG). Refs #79 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Second half of the #79 seam: DirectJson renders the five fixed event grammars straight into the caller's buffer (field names and framing as compile-time constants, values through an RFC 8259 escaper), recording slot offsets as it writes — no serde_json::Value tree, no intermediate String per render. Zero new dependencies. Also fixes a panic found while mirroring tholder_to_json: a (0, 0) weight hit 0/0 integer division, and parse_weight accepted "0/0" from the wire — so a crafted event with a valid SAID deserialized fine and panicked on re-serialization. parse_weight now rejects zero denominators (typed error) and weight rendering is division-free (weight_to_string, shared by both backends). Conformance gates in-module: per-ilk cross-backend proptests assert byte-identical output between DirectJson and the SerdeJson reference (boundary sn/bt/threshold values, all five seal variants, both identifier derivations); the escaper is property-tested against serde_json's own string encoder; a cross-path test SAID-verifies direct output through the unchanged read path. Refs #79 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
- tests/serder_allocation.rs: counting-allocator safeguard asserting the direct backend allocates strictly less than the serde_json reference for the same event (the win is behaviorally invisible, so conformance tests cannot catch an allocation regression). - benches/serder.rs: serialize benches through both backends on identical deterministic fixtures (representative icp, 16-anchor ixn). Local numbers (aarch64-darwin): icp 4.08us -> 2.32us, ixn16 7.33us -> 4.20us (~1.75x, including the fixed Blake3 cost in both). Refs #79 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Open
4 tasks
Merging this PR will not alter performance
Performance Changes
Comparing Footnotes
|
…nds, framing, boundaries Closes the untested-invariant gaps in the #79 seam, each as a failing-when-broken test: - Hostile-backend boundary: EventSerializer is public, so serialize_with must survive any reported layout — out-of-bounds size/SAID/prefix slots, wrong-width slots, reversed ranges all surface as typed InvalidEventLayout, never a panic or corrupt frame (5 tests). - extend_with_layout framing: rejects renders without the zero-size version head, with a nonzero size, missing the SAID placeholder, or missing the second placeholder on double-SAID events; base-offset arithmetic proven by rendering into a prefilled buffer (5 tests). - Append semantics: both backends report absolute slot offsets when the caller's buffer is non-empty (2 tests). - weight_to_string exact mapping table incl. division-free 0/0, 1/0, unreduced 2/4, and u64::MAX collapse. - EventRef: ilk / is_double_said / From<&KeriEvent> mappings for all five ilks (previously untested public API). - Escaper: deterministic probe covering every escape class (quote, backslash, five short escapes, \u00xx fallbacks, DEL boundary, multi-byte UTF-8) plus an any::<String>() oracle proptest — the ".*" regex strategy under-samples control characters. - Tholder boundaries: empty clause list / empty clause byte-identical across backends; proptest strategy widened to generate empties. Refs #79 Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Open
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Implements the #79 design (reviewed in #138): serialization becomes a pluggable component behind the
EventSerializerseam, withserde_jsonas the unchanged default and a hand-rolledDirectJsonbackend selectable per call viaserialize_with. Zero new dependencies.The seam (commit 1)
SAID/version orchestration is written once in
serialize_with: render once with a placeholder SAID + zero-size version string, backpatch the measured size in place, hash, splice the SAID into theEventLayoutslots (both fixed-width). Backends only control how bytes are produced. The old per-ilk pipelines refactor behind the seam with unchanged public signatures; as a side effect even the serde_json reference drops from 3 full JSON renders per event to 1 render + 2 in-place patches.DirectJson backend (commit 2)
Writes the five fixed event grammars straight into the caller's buffer — field names/framing as compile-time constants, values through an RFC 8259 escaper, slot offsets recorded while writing. No
Valuetree, no intermediateString.Also fixes a panic found while mirroring the threshold renderer:
tholder_to_jsonhit0/0integer division on a(0, 0)weight, andparse_weightaccepted"0/0"from the wire — so a crafted event with a valid SAID deserialized fine and panicked on re-serialization.parse_weightnow rejects zero denominators; weight rendering is division-free and shared by both backends. Bug probes red-before/green-after.Conformance gates (commits 2–3)
DirectJsonoutput must be byte-identical toSerdeJsonover generated events (boundary sn/bt/thresholds, all five seal variants, both identifier derivations)serde_json's own string encodertests/serder_allocation.rs): counting allocator asserts direct < serde_json allocationsBenchmarks (
benches/serder.rs, CodSpeed-wired)Local aarch64-darwin, both figures include the fixed Blake3 hashing cost:
Breaking change (0.x minor)
New
SerderError::InvalidEventLayoutvariant (backend-reported layout inconsistencies surface as typed errors, never a corrupt frame). Everything else is additive:EventSerializer,EventRef,EventLayout,SerdeJson,DirectJson,serialize_with.Follow-ups (per design §3.3/§3.5)
Closes #79
🤖 Generated with Claude Code