Skip to content

release(v0.49.0): finalize parser recovery and performance ratchet - #666

Merged
odvcencio merged 11 commits into
mainfrom
codex/post-v10-perf-ratchet
Aug 11, 2026
Merged

release(v0.49.0): finalize parser recovery and performance ratchet#666
odvcencio merged 11 commits into
mainfrom
codex/post-v10-perf-ratchet

Conversation

@odvcencio

@odvcencio odvcencio commented Aug 9, 2026 •
edited
Loading

Copy link
Copy Markdown
Owner

Release candidate: v0.49.0

This PR completes the parser performance ratchet and fixes the C enum recovery regression.

What changes

  • Fixes C: enum with exactly three enumerators and no trailing comma loses the third (zero-width MISSING, not ERROR) #667. C enum lists with three or more values now keep all enumerators.
  • Confirms a recovered tree with the forest only after the result is complete and has no ERROR or MISSING node.
  • Keeps C preprocessor input on its existing token-source path.
  • Adds Node.HasErrorOrMissing for strict parse-health checks.
  • Makes grammargen parse -strict reject both recovery node forms.
  • Includes the parser stop, GLR recovery, GSS, and memo-layout improvements from this branch.
  • Finalizes the v0.49.0 changelog and README release boundary.

Correctness evidence

  • The locked C reference matrix passes ten enum forms.
  • The matrix covers values, comments, typedefs, trailing commas, and adjacent declarations.
  • Normal, token-source, incremental, profiled, and C-reference routes have focused regression coverage.
  • The focused C race test passes in Docker.
  • The generated C real-corpus floor is unchanged from the branch base: 22/25 clean and 19/25 deep-parity samples.
  • The generated C corpus has known grammar gaps. They are not caused by this PR.

Performance evidence

The accepted performance bundle compares e1ce651 with 94a628b on a C2 Spot worker.

Target Change
Combined timing geomean -1.77%
Elixir recovery corpus -15.21%
KDL recovery -9.12%
Go full parse -1.16%
Go incremental, no edit -6.51%
FactProgram parse and extraction -1.23%

The final candidate adds recovery-only work. Clean benchmark inputs do not take that path.

A paired final randomized C2 measurement is pending a Cloud credential refresh. The existing 20-seed receipt remains attached below.

Release path

Merge this PR into main. Run hosted CI on the exact merge commit. Publish through the governed v0.49.0 workflow after its required soak.

Fixes #667

@odvcencio
optimize(parser): reduce wall clock reads during materialization
dc87633 
- Check the parse deadline every 64 materialization checkpoints instead of every checkpoint.
- Continue to check cancellation flags and sticky stops at every checkpoint.
- Reduce timed Swift recovery by 3.70% and timed Elixir recovery by 3.52% in 20-seed combined benchmarks.
- Include the optional single-file recovery lane in randomized comparisons.
@odvcencio odvcencio added area/parser Parser runtime, recovery, and GLR behavior area/performance Performance measurement and optimization correctness Parser correctness and reference parity labels Aug 9, 2026
@odvcencio
optimize(recovery): Combine error cost and visible count walks
19e4171 
- Add cNodeErrorCostAndVisibleSubtreeCount to compute both C subtree aggregates in one walk
- Update recovery and prefix aggregate call sites to use the combined function
- Prevent duplicate tree traversals when the recovery stack requires both values
- Add test to verify combined results match independent walks and cache correctly
@odvcencio
optimize(recovery): Reduce memo index cost
65c9472 
Replace the two multiplication-based mix steps with two XOR folds.

Use the node alignment to discard constant address bits.
@odvcencio odvcencio changed the title perf(parser): reduce deadline polling during materialization perf(parser): ratchet materialization and recovery hot paths Aug 9, 2026
@odvcencio
refactor(glr): move clean-zero cache to GSS nodes
84f0271 
- Move clean-zero error cache from scratch-local map to on-node state to reduce allocations and improve locality
- Track cost and visibility validity independently with an aggValid bitmask on GSS nodes
- Check primary cache way before set-associative slot in cNodeMemo lookups to reduce indirection
- Terminate clean-zero traversal on cycles by tracking visiting state on nodes
@odvcencio
fix(glr): invalidate clean-zero cache on extra-link mutation
eaf5359 
- Set cleanZeroState to gssCleanZeroUnknown in setExtraLink and appendExtraLink so extra-link writes discard the stale all-links cleanliness result
- Prevent gssNodesCanMerge from reading a cached clean-zero answer after an extra-link rewrite changes the link set
- Add a test assertion that confirms the clean-zero cache entry is absent after setExtraLink replaces a link
@odvcencio odvcencio added the memory Allocation, retention, and memory budgets label Aug 9, 2026
@odvcencio odvcencio changed the title perf(parser): ratchet materialization and recovery hot paths perf(parser): reduce materialization and GLR recovery costs Aug 9, 2026
@odvcencio
optimize(parser): skip inactive parser stop callbacks
12ea32f 
- Skip callback polls when no timeout or cancellation is active
- Keep the common stop path direct for the parser loop callback
- Inline the deadline check and remove the checkDeadline parameter
- Make materialization self-contained with throttled wall-clock reads
- Add tests for unbudgeted parser and between-budget cancellation paths
@odvcencio odvcencio mentioned this pull request Aug 10, 2026
Open
@odvcencio
odvcencio marked this pull request as ready for review August 10, 2026 00:44
@odvcencio
optimize(recovery): Promote error stack to GSS in C-recovery
8868817 
- Promote the error stack to the graph-structured stack before reduction forks.
- Share the immutable prefix across deep recovery branches instead of copying each deep linear stack.
- Reduce Swift recovery time by 9.96%, bytes by 59.65%, and peak RSS by 22.09%.
@odvcencio
optimize(parser): Reduce allocations in hidden field repeat tracking
d6b5d61 
- Replace heap-allocated map with fixed-size scratch buffer for tracking repeated hidden fields.
- Pool scratch buffer in nodeArena to eliminate allocation overhead between parses.
- Track field repetitions using inline begin/record/end methods instead of map lookups.
- Update materializeHiddenNodeForAlias to reuse pooled scratch for consistent memory usage.
- Lower B/op and allocs/op metrics by removing temporary allocations from the hot path.
@odvcencio
optimize(glr): Reduce cStackSummaryEntry struct size
94a628b 
- Change depth field in cStackSummaryEntry from int to uint8.
- Update all comparisons and assignments to handle the narrower type.
- Add a struct size test to prevent future growth.
- Save memory per stack entry during GLR recovery.
@odvcencio

odvcencio commented Aug 10, 2026

Copy link
Copy Markdown
Owner Author

Validation update for 94a628b3.

This change packs cStackSummaryEntry from 24 bytes to 16 bytes. It stores the bounded recovery depth as uint8.

Validation:

  • Focused Docker tests passed without out-of-memory or timeout.
  • Swift diagnostics matched the prior baseline: 25/25 no-error, 22/25 semantic parity, and 22/25 deep parity.
  • Eight randomized seeds ran on a C2 Spot VM.
  • Recovery B/op improved 17.48% across the two recovery benchmarks.
  • Recovery allocations improved 15.67%.
  • Time changed by -0.37%, without statistical significance.
  • All GitHub checks passed.

The Swift diagnostic differences remain the existing requireParity=false floor.

@odvcencio
fix(parser, tree): fix forest recovery fallback for C enum lists
2ae9d30 
- Add Node.HasErrorOrMissing() to detect trees with ERROR or MISSING nodes.
- Run forest fallback when DFA parse recovers partial trees.
- Update grammargen --strict flag and JSON output to reject MISSING nodes.
- Add regression and parity tests for C enum parsing across all paths.
- Gate forest fallback on preprocessor presence to avoid invalid guesses.

Refs #667
@odvcencio
update(docs): Finalize changelog and readme for v0.49.0 release
c609a18 
- Move untracked improvements to the v0.49.0 section
- Add release notes for enum list and error node fixes
- Clarify release status and consolidation goals in roadmap
@odvcencio odvcencio changed the title perf(parser): reduce materialization and GLR recovery costs release(v0.49.0): finalize parser recovery and performance ratchet Aug 11, 2026
@odvcencio odvcencio added area/api Public APIs and consumer-facing interfaces area/grammars Grammar, scanner, and query support release/v0.49.0 The governed v0.49.0 release train labels Aug 11, 2026 — with ChatGPT Codex Connector
@odvcencio
odvcencio merged commit 4a15cf1 into main Aug 11, 2026
44 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Reviewers

No reviews

Assignees

No one assigned

Labels

area/api Public APIs and consumer-facing interfaces area/grammars Grammar, scanner, and query support area/parser Parser runtime, recovery, and GLR behavior area/performance Performance measurement and optimization correctness Parser correctness and reference parity memory Allocation, retention, and memory budgets release/v0.49.0 The governed v0.49.0 release train

Projects

None yet

Milestone

No milestone

Development

Successfully merging this pull request may close these issues.

C: enum with exactly three enumerators and no trailing comma loses the third (zero-width MISSING, not ERROR)

1 participant

@odvcencio