Skip to content

Harden JSON parser byte reads - #103

Open
MesTTo wants to merge 2 commits into
trueagi-io:mainfrom
MesTTo:pr/mork-json-byte-guards
Open

Harden JSON parser byte reads#103
MesTTo wants to merge 2 commits into
trueagi-io:mainfrom
MesTTo:pr/mork-json-byte-guards

Conversation

@MesTTo

@MesTTo MesTTo commented Jun 23, 2026

Copy link
Copy Markdown
Contributor

This PR is stacked on #61.

It turns the JSON parser byte-read and index-advance invariants into runtime assertions. read_byte now checks index < length before the pointer read, and bump checks the same invariant before advancing. The pointer read uses add(index) after the bound check.

The added tests are public parser contract checks for truncated string inputs. They verify that normal EOF paths still return UnexpectedEndOfJson after the internal byte-read hardening.

Validation:

  • git diff --check
  • rustfmt --check frontend/tests/json_parser.rs
  • jscpd --reporters ai --min-lines 5 --min-tokens 50 frontend/src/json_parser.rs frontend/tests/json_parser.rs
  • CARGO_HOME=/home/user/Dev/.cargo-isolated RUSTFLAGS='-C target-cpu=native -Awarnings' cargo +nightly test -p mork-frontend --test json_parser
  • CARGO_HOME=/home/user/Dev/.cargo-isolated RUSTFLAGS='-C target-cpu=native -Awarnings' cargo +nightly test -p mork-frontend --tests

Measured cost. Interleaved release A/B on a 28.5MB JSON corpus: best 2016 MB/s on main vs 1864 MB/s with this branch (which includes #61), about 7.6% on the frontend bulk-load path — the kernel is untouched. A 30-document valid-input differential (deep nesting, big mantissas, near-boundary exponents, escapes, unicode) produces byte-identical transcriber event streams, so the asserts change no accepted input. If the cost matters, read_byte can stay a debug_assert (every call site is eof-guarded) and only bump keeps the hard assert, recovering most of it.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant