Skip to content

feat(serder)!: #142 strict canonical read-path parser — offset-based SAID verification#146

Merged
joeldsouzax merged 21 commits into
mainfrom
feat/142-strict-read-path
Jul 11, 2026
Merged

feat(serder)!: #142 strict canonical read-path parser — offset-based SAID verification#146
joeldsouzax merged 21 commits into
mainfrom
feat/142-strict-read-path

Conversation

@joeldsouzax

Copy link
Copy Markdown
Contributor

Closes #142. The read-path half of the #79 seam (design §3.3/§3.5).

What

  • Strict single-pass canonical parser for the five fixed event grammars (serder::deserialize::canonical): compact JSON, spec field order, no escapes/duplicates/whitespace — rejected by construction with the new typed SerderError::NonCanonical { offset, expected, found }.
  • Offset-based SAID verification: one scratch copy + span fill + hash, replacing 2–3 Value trees and a full re-render per ingested event.
  • Borrowed &str field views (ParsedIcp<'a> …) — the substrate for C-a C-a · Zero-copy event input — borrow-ify KeriEvent across serder #129's borrow-ified KeriEvent<'a>.
  • keripy intive=True integer kt/nt/bt still accepted (their SAIDs are computed over the integer form; rejecting them is a parity gap).
  • Bonus fix, pre-existing DoS on main: VersionString::parse panicked on multi-byte UTF-8 straddling its fixed slice offsets (non-char-boundary &str slicing), reachable through deserialize_event via the old tolerant path. Fixed at the root with failing-first regression tests.

Breaking (MINOR under 0.x)

  • New SerderError::NonCanonical variant; non-canonical inputs now fail with it instead of assorted Json/MissingField errors (the strict path never returns MissingField).
  • said::verify_said is strict and returns Result<(), SerderError> (was Result<bool, _>); its digest-code trust contract is now documented.
  • Per-ilk deserializers require their exact ilk (deserialize_rotation no longer accepts drt bytes; deserialize_inception no longer silently accepts dip bytes and drops the delegator; deserialize_interaction no longer accepts rot/drt bytes as a truncated interaction).

Verification

  • Strict-vs-reference differential proptests over the builder-reachable event space (all five ilks), byte-identical re-serialization; mutation-subset property (anything strict accepts, the tolerant oracle accepts identically).
  • Deterministic variant × invariant matrix (26 tests): every Seal/Tholder/Count/Config/Identifier variant, both SAID branches (single d≠i, double d==i), all five ilk-dispatch arms, and one reachability probe per reachable read-path error variant — guaranteed coverage, not probabilistic.
  • Rejection probes: whitespace (size-consistent), duplicate keys, reordered fields, escapes, trailing bytes, every-prefix truncation, oversized/unknown ilks, non-ASCII version windows.
  • keripy corpus through the strict path via keri/tests/differential.rs.
  • Allocation count pinned (38 allocs per icp deserialize, exact, deterministic); CodSpeed serder_deserialize/icp bench added; bolero+AFL fuzz target serder_deserialize_event (idempotence oracle) wired into the deep-fuzz matrices.
  • nix flake check green (wasm + no_std included).

Discovered during this work (filed separately)

The old serde_json::Value read path survives only as the #[cfg(test)] differential oracle (deserialize::reference). serde demotion to dev-dependencies is the follow-up card per design §6 staging.

🤖 Generated with Claude Code

joeldsouzax and others added 21 commits July 10, 2026 21:23
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…roperty tests (#142)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…#142)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ation (#142)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…#142)

VersionString::parse sliced the input &str at fixed byte offsets; a
multi-byte UTF-8 char straddling a field boundary (offsets 4, 5, 6, 10,
or 16) is not a char boundary, so the slice panicked — a DoS on
untrusted input. The panic predates #142: it is reachable on main via
validate_version_string in deserialize.rs, which feeds serde_json-
unescaped arbitrary UTF-8 into VersionString::parse. Closed at the
root: every fixed-offset slice now goes through checked str::get and
returns InvalidVersionString.

Defense in depth at the strict-parser seam: canonical.rs::head() now
rejects any non-ASCII byte in the 17-byte version window with a
NonCanonical error reporting the precise violating byte, before the
window ever reaches VersionString::parse.

Also: document require_ilk's error-offset convention, and add five
grammar rejection probes (wrong first byte, oversized ilk, delegator
field on icp, missing delegator on dip, corrupt version terminator
seam).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
… hash (#142)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…142)

BREAKING: non-canonical input (whitespace, reordered/duplicate fields,
escapes, trailing bytes) now fails with SerderError::NonCanonical; per-ilk
deserializers require their exact ilk (deserialize_rotation no longer
accepts drt bytes, deserialize_inception no longer accepts dip bytes).
The tolerant serde_json path survives as the #[cfg(test)] differential
oracle (deserialize::reference).

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…rror> (#142)

BREAKING: verify_said now parses with the strict canonical parser and
returns Ok(()) / Err(SaidMismatch) instead of Result<bool, _>; non-canonical
input fails with NonCanonical instead of being tolerated.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…-subset properties (#142)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…142)

typos gate flagged "abd" as a misspelling of "and"/"bad"; the actual
source test in canonical.rs uses "abX" for this mismatch scenario —
fix the plan doc's transcription to match.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…t read path (#142)

Pins every ParsedSeal/Tholder/Count/Identifier variant, both SAID branches
(single d!=i and double d==i), all five ilk-dispatch arms, and each
reachable read-path error variant through strict==oracle==roundtrip,
replacing probabilistic proptest coverage of these variants with guaranteed
deterministic checks.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
…ey extend (#142)

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@codspeed-hq

codspeed-hq Bot commented Jul 11, 2026

Copy link
Copy Markdown

Merging this PR will not alter performance

✅ 25 untouched benchmarks
🆕 1 new benchmark
⏩ 21 skipped benchmarks1

Performance Changes

Benchmark BASE HEAD Efficiency
🆕 icp N/A 38.2 µs N/A

Comparing feat/142-strict-read-path (dedb299) with main (d8d1cf3)

Open in CodSpeed

Footnotes

  1. 21 benchmarks were skipped, so the baseline results were used instead. If they were deleted from the codebase, click here and archive them to remove them from the performance reports.

@joeldsouzax joeldsouzax merged commit e9df476 into main Jul 11, 2026
34 checks passed
@joeldsouzax joeldsouzax deleted the feat/142-strict-read-path branch July 11, 2026 00:11
@devrandom-release-plz devrandom-release-plz Bot mentioned this pull request Jul 10, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

serder: strict canonical read-path parser — offset-based SAID verification, serde-free endgame

1 participant