standard: revisit locator validation mechanism (regex + examples vs. alternatives)

[maehr]

Affected spec section

/standard/system-profiles/ — locator validation contract for citation-system profiles.

Motivation

Two concrete registry issues filed today both tweak locator_regex to fix concrete bugs, but they share a deeper question that is worth surfacing at the spec level before we keep patching individual profiles:

• data: Bekker locator regex rejects valid pages 1–99 registry#2 — Bekker page regex rejects valid pages 1–99 and currently admits non-canonical leading-zero spellings.
• data: Bible book-chapter-verse regex rejects numbered books; vocabulary unpinned registry#3 — Bible book-chapter-verse regex rejects digit-initial numbered books; canonical vocabulary (OSIS codes vs. full names) and case are unpinned.

The question: is locator_regex + examples the right validation contract for a citation-system profile at all?

Reasons to revisit:

1. The deterministic UUID seed uses the exact normalized locator bytes, so any regex-valid spelling difference (leading zeros, case, abbreviation form) mints a distinct permanent identity. A regex alone is a weak fence against identity splits.
2. Some axes that profiles actually need to validate cannot be expressed in a regex at all — most obviously Bible book vocabulary, which is an enumerated set, not a character pattern.
3. ECMAScript regex is backtracking; allowing arbitrary contributor regexes is a latent DoS/CI-stall risk (see review notes on RE2-compatible subsets).
4. Tightening the regex to enforce canonical forms (no leading zeros, canonical case) tangles syntactic validation with normalization rules in a way that is hard to read and easy to get wrong.

This isn't a request to ship a new design today — it's a request to decide what we want the contract to be before more profiles land.

Proposed change

Open the design question for discussion. Sketch options without picking one:

1. Status quo, tightened. Keep locator_regex but constrain it to an RE2-compatible / linear-time subset, and require profiles to declare canonical digit and case forms enforced by rejection rather than folding.
2. Regex + structured fields. Keep locator_regex for the grammar around named capture groups, and add structured fields beside it for things regex can't express — e.g. vocabulary: for enumerated tokens (Bible books, Stephanus columns), canonical_case:, leading_zeros: forbid.
3. Declarative validator schema. Replace the single regex with a small declarative shape: named components with per-component types and constraints (integer min=1 no-leading-zeros, enum: [Genesis, …], regex: [ab]). Compiler derives the regex.
4. Drop syntactic validation from the profile. Rely on minting-time review and on the resolver. Cheapest spec, riskiest data.

Alternatives considered

The four options above are the alternatives. Reasonable hybrids exist (e.g. option 2 with option 1's RE2 constraint).

Compatibility impact

additive (no breaking change)

(The discussion is additive. A chosen outcome might later require breaking changes to existing profiles; that would be handled in a follow-up proposal.)

Target spec version

v0.2.0-draft

[maehr]

Sub-proposal: restrict locator_regex to a linear-time, RE2-compatible subset

Folding in review-doc Issue 5 here, because it is the explicit content of Option 1 above and it bites regardless of which of the four options is ultimately picked: whatever the validation contract becomes, contributor-supplied regex pieces should not be allowed to introduce catastrophic backtracking.

Normative shape

Each locator_regex MUST be matchable in time linear in input length. Forbidden ECMAScript constructs:

• backreferences (\1–\9)
• lookarounds: (?=…), (?!…), (?<=…), (?<!…)

Named capture groups ((?<name>…)), Unicode property escapes (\p{…}), and standard character classes remain allowed. The constraint is intentionally framed in terms of forbidden constructs, not "must compile under RE2", so it can be enforced syntactically without adding an RE2 dependency.

Current-state verification

All 10 live profiles in data/systems/*.yaml are already clean:

grep -rE "\\[1-9]|\(\?=|\(\?!|\(\?<=|\(\?<!" data/systems/*.yaml
# (no matches)

So adopting this constraint is a no-op for existing data; it only bites future contributions.

Suggested lint

Add a check in the registry validation pipeline (likely alongside scripts/compile.ts, or in a CI step that runs against data/systems/*.yaml) that scans each locator_regex for the forbidden constructs and fails the build with a clear message pointing at the offending profile. Implementation is a few lines of regex-over-regex; no new dependencies.

Why now

• Defends against catastrophic-backtracking patterns reaching main before any profile ships one. Backtracking validator stalls are silent until they aren't.
• Preserves the option to later compile profiles against RE2 / re2j / Hyperscan-style engines without having to retrofit syntax.
• Cheap to enforce while the registry is small.

Scope

Intentionally narrow: just the linear-time constraint + lint. Canonical digit/case form rules belong with review-doc Issue 4 (separate sub-thread) and with the broader option ladder above.