Skip to content

feat: forbid non-reserved identifiers via withForbidden#14432

Merged
sgraf812 merged 2 commits into
masterfrom
sg/parser-forbid-idents
Jul 21, 2026
Merged

feat: forbid non-reserved identifiers via withForbidden#14432
sgraf812 merged 2 commits into
masterfrom
sg/parser-forbid-idents

Conversation

@sgraf812

@sgraf812 sgraf812 commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

This PR makes withForbidden stop a term at a non-reserved word used as an identifier, not just at a registered token. A parser like withForbidden "invariant" termParser now ends the term when it reaches a bare invariant, while invariant remains usable as an identifier everywhere else.

identFn rejects an identifier whose text is in forbiddenTks, mirroring the forbidden-token check in mkTokenAndFixPos. The check is guarded by forbiddenTks.isEmpty, so the common identifier path is unchanged.
We considered putting a unified check in tokenFn or mkIdResult, but decided against that. The former would move the check outside the cache and regress some benchmarks by ~1%. The latter forces a flag on mkIdResult and identFnAux because nameLitAux and rawLitFn must not forbid any tokens.

@sgraf812 sgraf812 added the changelog-no Do not include this PR in the release changelog label Jul 17, 2026
@github-actions github-actions Bot added the toolchain-available A toolchain is available for this PR, at leanprover/lean4-pr-releases:pr-release-NNNN label Jul 17, 2026
@mathlib-lean-pr-testing

mathlib-lean-pr-testing Bot commented Jul 17, 2026

Copy link
Copy Markdown

Mathlib CI status (docs):

  • ❗ Batteries/Mathlib CI will not be attempted unless your PR branches off the nightly-with-mathlib branch. Try git rebase a104164801f9cdf71dc671801a01c74c1f3bf8ab --onto 79bec6d064d8672cb10864cad3d9fd054b6917ae. You can force Mathlib CI using the force-mathlib-ci label. (2026-07-17 12:41:39)
  • ❗ Batteries/Mathlib CI will not be attempted unless your PR branches off the nightly-with-mathlib branch. Try git rebase a104164801f9cdf71dc671801a01c74c1f3bf8ab --onto 25ba8c3d3bcb1dded7ff5a6f3b6044b0a7970198. You can force Mathlib CI using the force-mathlib-ci label. (2026-07-20 15:46:16)

@leanprover-bot

leanprover-bot commented Jul 17, 2026

Copy link
Copy Markdown
Collaborator

Reference manual CI status:

  • ❗ Reference manual CI will not be attempted unless your PR branches off the nightly-with-manual branch. Try git rebase a104164801f9cdf71dc671801a01c74c1f3bf8ab --onto a4b639c5a1e545677b36b09a4770b7b559df805b. You can force reference manual CI using the force-manual-ci label. (2026-07-17 12:41:41)
  • ❗ Reference manual CI will not be attempted unless your PR branches off the nightly-with-manual branch. Try git rebase a104164801f9cdf71dc671801a01c74c1f3bf8ab --onto 49ff95727f98d43984726b26742d17a1ceea9dd5. You can force reference manual CI using the force-manual-ci label. (2026-07-20 15:46:18)

@sgraf812
sgraf812 force-pushed the sg/parser-forbid-idents branch from 675571f to 44667e1 Compare July 17, 2026 12:52
@sgraf812

Copy link
Copy Markdown
Contributor Author

!bench

@sgraf812
sgraf812 marked this pull request as ready for review July 17, 2026 13:39
@leanprover-radar

leanprover-radar commented Jul 17, 2026

Copy link
Copy Markdown

Benchmark results for 44667e1 against a104164 are in. No significant results found. @sgraf812

  • build//instructions: -4.2M (-0.00%)

Small changes (1✅, 2🟥)

  • 🟥 build/module/Lean.Parser.Basic//instructions: +60.7M (+0.79%)
  • 🟥 compiled/parser//instructions: +11.5M (+0.03%)
  • elab/bv_decide_mod//maxrss: -96MiB (-4.76%)

@Rob23oba

Copy link
Copy Markdown
Contributor

Shouldn't this be something handled directly in mkIdResult? Also we should do this on the basis of the raw string, not the name, we don't want to forbid «invariant».

This PR makes `withForbidden` stop a term at a non-reserved word used as an identifier, not just at a registered token. A parser like `withForbidden "invariant" termParser` now ends the term when it reaches a bare `invariant`, while `invariant` remains usable as an identifier everywhere else.

`identFn` rejects an identifier whose text is in `forbiddenTks`, mirroring the forbidden-token check in `mkTokenAndFixPos`. The check is guarded by `forbiddenTks.isEmpty`, so the common identifier path is unchanged, and it does not affect `rawIdent` (field projections), the parenthesized escape via `withoutForbidden`, or reserved-token forbidding.
@sgraf812
sgraf812 force-pushed the sg/parser-forbid-idents branch from 44667e1 to 79bbf4e Compare July 17, 2026 14:54
@sgraf812
sgraf812 force-pushed the sg/parser-forbid-idents branch from 70fb0ea to 648cffa Compare July 20, 2026 14:38
@sgraf812

Copy link
Copy Markdown
Contributor Author

Thanks for the review. Yes, we should do it on the raw string; the new test case tests for that now. While doing the check in mkIdResult means we can have just one check, it also means that mkIdResult (and its caller identFnAux) needs to be passed a flag for whether or not the forbidden set should be tested (it shouldn't be tested when called from rawIdentFn and nameLitAux). It's simpler to just duplicate the test in identFn.

@sgraf812

Copy link
Copy Markdown
Contributor Author

!bench

@leanprover-radar

leanprover-radar commented Jul 21, 2026

Copy link
Copy Markdown

Benchmark results for 648cffa against a104164 are in. No significant results found. @sgraf812

  • build//instructions: -749.7M (-0.01%)

Small changes (1✅, 2🟥)

  • 🟥 build/module/Lean.Parser.Basic//instructions: +53.1M (+0.69%)
  • 🟥 compiled/parser//instructions: +14.1M (+0.04%)
  • elab/delayed_sharing//wall-clock: -23ms (-6.12%)

@sgraf812
sgraf812 added this pull request to the merge queue Jul 21, 2026
Merged via the queue into master with commit aacd612 Jul 21, 2026
21 checks passed
@sgraf812
sgraf812 deleted the sg/parser-forbid-idents branch July 21, 2026 12:22
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

changelog-no Do not include this PR in the release changelog toolchain-available A toolchain is available for this PR, at leanprover/lean4-pr-releases:pr-release-NNNN

Projects

None yet

Development

Successfully merging this pull request may close these issues.

5 participants