text/jinja: hard-cut parser to modeled submachines

[gabewillen]

Summary

• hard-cut text/jinja/parser from legacy orchestration to explicit Boost.SML submachines (classifier_parser, program_parser, expression_parser, statement_parser)
• introduce parser-domain lexer machine (text/jinja/parser/lexer) and remove legacy text/jinja/lexer.hpp
• move jinja AST/token/value types into text/jinja/parser/detail.hpp and delete legacy ast.hpp, value.hpp, and model/architecture/types.hpp
• migrate formatter/parser request/runtime events, tests, docsgen, bench harnesses, and architecture docs/snapshots for the new machine topology

Validation

• build/coverage/emel_tests_bin -tc='jinja_parser_*' --no-breaks --success=0
• build/coverage/emel_tests_bin -tc='jinja_lexer_*' --no-breaks --success=0
• build/coverage/emel_tests_bin -tc='jinja_formatter_*' --no-breaks --success=0

Notes

• full scripts/quality_gates.sh in this workspace still hits its 300s timeout wrapper in later stages, but targeted jinja suites pass.

[chatgpt-codex-connector]

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 6879e529fa

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

• Open a pull request for review
• Mark a draft as ready
• Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

[Copilot]

Pull request overview

Refactors the text/jinja parsing stack from legacy orchestration into explicit Boost.SML submachines, introduces a parser-domain lexer state machine, and updates downstream tooling (docsgen/benches/tests/docs artifacts) to the new event/runtime model.

Changes:

• Split parsing into submachines (classifier_parser, program_parser, expression_parser, statement_parser) and added a parser-domain lexer machine under text/jinja/parser/lexer.
• Rehomed Jinja AST/token/value types into text/jinja/parser/detail.hpp, deleting legacy lexer.hpp, ast.hpp, and value.hpp.
• Updated docsgen, benchmarks, tests, and generated architecture docs/snapshots for the new machine topology and event signatures.

Reviewed changes

Copilot reviewed 69 out of 69 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
tools/docsgen/docsgen.cpp	Updates includes and adapts parse/render event construction to new callback + runtime model.
tools/bench/text/jinja/parser_bench.cpp	Migrates parser bench to new parser SM and parse event signature.
tools/bench/text/jinja/formatter_bench.cpp	Migrates formatter bench to new SM/event signature and prevents optimizer-elision with sinks.
tests/text/jinja/parser_tests.cpp	Updates parser tests to new event/callback model and new error codes.
tests/text/jinja/lexer_tests.cpp	Reworks lexer tests to drive the new parser-domain lexer SM via runtime events.
src/emel/text/jinja/value.hpp	Deletes legacy Jinja value model header (moved under parser detail).
src/emel/text/jinja/types.hpp	Redirects types.hpp to include parser/detail.hpp.
src/emel/text/jinja/parser/program_parser/statement_parser/sm.hpp	Adds statement-parser submachine topology.
src/emel/text/jinja/parser/program_parser/statement_parser/guards.hpp	Adds statement-parser guards for statement-kind decisions and scanning.
src/emel/text/jinja/parser/program_parser/statement_parser/errors.hpp	Introduces statement-parser error aliasing.
src/emel/text/jinja/parser/program_parser/statement_parser/context.hpp	Introduces statement-parser context aliasing.
src/emel/text/jinja/parser/program_parser/statement_parser/actions.hpp	Adds statement-parser actions for scanning/emitting/failing.
src/emel/text/jinja/parser/program_parser/sm.hpp	Adds program-parser dispatcher SM (text/comment/statement/expression).
src/emel/text/jinja/parser/program_parser/guards.hpp	Adds program-parser guards for token dispatch and phase gating.
src/emel/text/jinja/parser/program_parser/expression_parser/sm.hpp	Adds expression-parser submachine topology.
src/emel/text/jinja/parser/program_parser/expression_parser/guards.hpp	Adds expression-parser scan/emit guards.
src/emel/text/jinja/parser/program_parser/expression_parser/errors.hpp	Introduces expression-parser error aliasing.
src/emel/text/jinja/parser/program_parser/expression_parser/context.hpp	Introduces expression-parser context aliasing.
src/emel/text/jinja/parser/program_parser/expression_parser/actions.hpp	Adds expression-parser actions for scanning/emitting/failing.
src/emel/text/jinja/parser/program_parser/errors.hpp	Introduces program-parser error aliasing.
src/emel/text/jinja/parser/program_parser/context.hpp	Introduces program-parser context aliasing.
src/emel/text/jinja/parser/program_parser/actions.hpp	Adds program-parser actions for consuming tokens and finalizing parse status.
src/emel/text/jinja/parser/lexer/sm.hpp	Adds parser-domain lexer SM (wraps scan outcomes into callbacks).
src/emel/text/jinja/parser/lexer/guards.hpp	Adds lexer SM request/scan guards.
src/emel/text/jinja/parser/lexer/errors.hpp	Introduces lexer error aliasing to parser error enum.
src/emel/text/jinja/parser/lexer/detail.hpp	Defines lexer runtime event wrapper and re-exports low-level scan utilities.
src/emel/text/jinja/parser/lexer/context.hpp	Adds lexer SM context type.
src/emel/text/jinja/parser/lexer/actions.hpp	Adds lexer SM actions to emit token/eof/errors via callbacks.
src/emel/text/jinja/parser/guards.hpp	Updates parser-level guards to unwrap runtime events and new phase checks.
src/emel/text/jinja/parser/events.hpp	Introduces new parse request/ctx/runtime types and richer error event payload.
src/emel/text/jinja/parser/errors.hpp	Adds parser-specific error enum + conversion helper.
src/emel/text/jinja/parser/context.hpp	Replaces legacy error fields with embedded lexer SM instance.
src/emel/text/jinja/parser/classifier_parser/sm.hpp	Adds classifier submachine to classify statement/expression kinds.
src/emel/text/jinja/parser/classifier_parser/guards.hpp	Adds classifier guards for token inspection and phase checks.
src/emel/text/jinja/parser/classifier_parser/errors.hpp	Introduces classifier error aliasing.
src/emel/text/jinja/parser/classifier_parser/context.hpp	Introduces classifier context aliasing.
src/emel/text/jinja/parser/classifier_parser/actions.hpp	Adds classifier actions to set statement/expression classification in ctx.
src/emel/text/jinja/parser/actions.hpp	Replaces legacy parse orchestration with tokenization pipeline + callback dispatch helpers.
src/emel/text/jinja/lexer.hpp	Deletes legacy monolithic lexer implementation header.
src/emel/text/jinja/formatter/sm.hpp	Refactors formatter SM to runtime event model and simplified copy/dispatch flow.
src/emel/text/jinja/formatter/guards.hpp	Updates formatter guards to runtime event unwrapping and callback requirements.
src/emel/text/jinja/formatter/events.hpp	Refactors render event to ctor-based API + runtime ctx wrapper and richer done/error events.
src/emel/text/jinja/formatter/errors.hpp	Adds formatter-specific error enum.
src/emel/text/jinja/formatter/context.hpp	Removes legacy formatter execution context (now minimal).
src/emel/text/jinja/formatter/actions.hpp	Refactors formatter actions to runtime event model (copy/empty/overflow + callback dispatch).
src/emel/text/jinja/ast.hpp	Deletes legacy AST model header (moved under parser detail).
src/emel/model/architecture/types.hpp	Deletes legacy model architecture type header.
snapshots/quality_gates/timing.txt	Updates quality gate timing snapshot.
snapshots/bench/benchmarks_compare.txt	Updates benchmark comparison snapshot values.
snapshots/bench/benchmarks.txt	Updates benchmark snapshot values.
docs/benchmarks.md	Replaces committed benchmark table with template placeholder output.
docs/architecture/text_jinja_parser_program_parser_expression_parser.md	Adds generated architecture doc for expression_parser submachine.
docs/architecture/text_jinja_parser_program_parser.md	Adds generated architecture doc for program_parser submachine.
docs/architecture/text_jinja_parser_lexer.md	Adds generated architecture doc for parser-domain lexer SM.
docs/architecture/text_jinja_parser_classifier_parser.md	Adds generated architecture doc for classifier_parser submachine.
docs/architecture/text_jinja_parser.md	Updates top-level parser architecture doc to new runtime topology.
docs/architecture/mermaid/text_jinja_parser_program_parser_statement_parser.mmd	Adds generated Mermaid diagram for statement_parser.
docs/architecture/mermaid/text_jinja_parser_program_parser_expression_parser.mmd	Adds generated Mermaid diagram for expression_parser.
docs/architecture/mermaid/text_jinja_parser_program_parser.mmd	Adds generated Mermaid diagram for program_parser.
docs/architecture/mermaid/text_jinja_parser_lexer.mmd	Adds generated Mermaid diagram for parser-domain lexer.
docs/architecture/mermaid/text_jinja_parser_classifier_parser.mmd	Adds generated Mermaid diagram for classifier_parser.
docs/architecture/mermaid/text_jinja_parser.mmd	Updates generated Mermaid diagram for top-level parser.
README.md	Switches docs index to templated TOC placeholder and adds re-architecture warning block.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.