Skip to content

Add an opt-in native Lean 4 grammar - #664

Closed
odvcencio wants to merge 1 commit into
mainfrom
codex/lean-native-grammar
Closed

Add an opt-in native Lean 4 grammar#664
odvcencio wants to merge 1 commit into
mainfrom
codex/lean-native-grammar

Conversation

@odvcencio

Copy link
Copy Markdown
Owner

Summary

  • Add a native Lean 4 grammar with the Go grammar domain-specific language.
  • Package the generated blob in the opt-in grammars/lean module.
  • Add nested comments, highlights, outline tags, and incremental tests.
  • Extend RegisterExtension with optional source and tags metadata.
  • Keep the default 206-language fleet unchanged.

Scope

Lean modules can extend their parser at runtime. This grammar gives core declarations stable nodes. It preserves unknown extension syntax as line-scoped custom_command nodes.

The grammar targets Lean 4.32.2 at commit f3b06c705e6c85f5314019d5d3baab0fec5b580c.

Validation

  • Docker package and grammar tests pass.
  • Docker race tests pass.
  • Docker registry isolation tests pass.
  • The official corpus gate parses 3,699 of 3,699 files cleanly.
  • The corpus covers 15,540,447 bytes with no recovery or early stops.
  • The corpus process used 100,900 KiB maximum resident memory.
  • The largest 259,520-byte file used 45,440 KiB maximum resident memory.
  • The packaged blob matches regeneration from the Go grammar definition.

Graduation

This pull request keeps Lean opt-in. It does not change the default registry or its 206-language receipts. A later promotion can update the fleet gates to 207 languages.

- Add opt-in Lean 4 grammar authored with the Go DSL. The grammar targets Lean 4.32.2 and provides stable nodes for core declarations. Extension-specific syntax remains in line-scoped custom_command nodes because Lean modules extend the parser at runtime.
- Add external scanner for nested block comments, documentation comments, and module documentation comments.
- Add highlight captures and outline tags queries for editor integration.
- Extend RegisterExtension to accept GrammarSource and TagsQuery fields. An empty GrammarSource defaults to GrammarSourceGrammargen. This lets external packages declare blob-based grammars and symbol extraction without calling Register directly.
- Add corpus characterization test that validates 3,699 files from the official Lean 4.32.2 source without recovery nodes or early stops.
- Keep the grammar opt-in to preserve the default 206-language graduation gates.

Copy link
Copy Markdown
Owner Author

Integrated into codex/linkerscript-normalizer-retirement at ad4ee003. Regenerated the Lean blob at e47f12b0 for the current bytecode format. Focused Docker tests pass. Closing this source PR as superseded.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant