Skip to content

docs(serder): #79 research write-up — pluggable serialization backend seam#138

Merged
joeldsouzax merged 3 commits into
mainfrom
research/79-serialization-backend-seam
Jul 10, 2026
Merged

docs(serder): #79 research write-up — pluggable serialization backend seam#138
joeldsouzax merged 3 commits into
mainfrom
research/79-serialization-backend-seam

Conversation

@joeldsouzax

Copy link
Copy Markdown
Contributor

Summary

Research-phase deliverable for #79 (research-first card: no production code lands until this write-up is reviewed). Adds docs/superpowers/specs/2026-07-10-79-serialization-backend-seam-design.md covering the five required research sections:

  • Status-quo evaluation — serialize renders each event through serde_json::to_string() three times (measure → size-correct → SAID-splice) with per-field Value clones; deserialize parses 2–3 Value trees and re-renders the whole event to verify SAIDs. No serde derives exist on any domain type, and every event-body scalar is a JSON string (hex sn/bt, qb64), so the serde data model is incidental — the load-bearing requirements are spec field order and byte determinism.
  • Seam design — orchestration (placeholder, backpatch, hash, splice) stays central; the trait renders canonical JSON into a caller-provided &mut Vec<u8> and reports the fixed-width version-size + SAID slot ranges (EventLayout). Both slots are fixed-width, so the optimized backend does one render + two in-place backpatches + one hash. Three options analysed; body-encoder seam recommended.
  • Crate survey — hand-rolled direct writer, zero new dependencies (struson experimental/std-bound; escaping surface is qb64/hex/ASCII so a small in-house RFC 8259 escaper suffices; itoa/zmij unneeded — no JSON numbers on the write path).
  • Conformance strategy — cross-backend byte-identical proptest as the primary gate, keripy differential + round-trip suites parameterized per backend, allocation-budget test on the existing counting allocator, CodSpeed benches, no_std/wasm.
  • Migration planserialize() keeps its signature with serde_json as the unchanged default; the seam + serialize_with are additive (PATCH under 0.x).

Notable research finding

VersionString::to_str() (cesr/src/serder/version.rs:161–170) formats the size with an unguarded {:06x}: a size above 0xFF_FFFF silently emits ≥7 hex digits, breaking the documented 17-byte version-string invariant. The design requires a runtime Err guard + bug-probe test, landing with the seam work.

Scope decisions called out for review

  • Read-path SAID verification is not in the seam — it's a backend-independent optimization; the write-up commits to filing a follow-up card.
  • CBOR/MGPK serialization kinds are a different axis (different bytes, not different backends) and stay out of scope.

Refs #79

🤖 Generated with Claude Code

… seam design

Research-first deliverable for #79: status-quo allocation audit (3x
to_string per serialize, 2-3x from_slice + re-render per deserialize),
body-encoder seam design with fixed-width slot backpatching, crate
survey (zero new deps), cross-backend conformance strategy, and a
latent version-string size-overflow defect to fix under this card.

Refs #79

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@codspeed-hq

codspeed-hq Bot commented Jul 10, 2026

Copy link
Copy Markdown

Merging this PR will not alter performance

✅ 21 untouched benchmarks
⏩ 21 skipped benchmarks1


Comparing research/79-serialization-backend-seam (b64b7f8) with main (f274048)

Open in CodSpeed

Footnotes

  1. 21 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

…d in #139

Answers the review question 'why do we need serde at all?': deserialization
happens once at the edge and the write path uses none of serde's data model,
so the end state is a serde-free production path with serde_json demoted to
a dev-dependency oracle. Read path becomes a strict canonical parser
(borrowed fields + SAID offsets, feeding C-a #129); strictness is itself a
conformance feature — the whitespace-acceptance defect fixed in #139 is the
proof case. Staged migration spelled out in §6.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@joeldsouzax joeldsouzax enabled auto-merge (squash) July 10, 2026 14:38
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant