chore(security): scrub internal identifier examples from public docs and tests

[lyonzin]

Summary

Scrub of internal identifier examples from public docs, config samples, tokenizer docstrings, and test fixtures. The FTS5 fast-path materials had been using MDR-AD002 / MDR-AD003 as the canonical identifier examples across README, config, docs, benches, in-code comments, and test seed data. Those match the shape of an internal blue-team detection rule taxonomy and should not leak into public code samples.

This PR replaces every MDR-AD00x mention on the public surface with MS17-010 (the well-known public EternalBlue / MS Security Bulletin MS17-010 identifier) — same tokenization behavior, same routing coverage, zero internal context. MDR-AD999 (mock-only) becomes MS99-999.

What changed

• Docs & config (README.md, config.example.yaml, docs/features/fts5_fast_path.md)
• Bench (bench/test_bench_fts5_lexical.py)
• Tokenizer docstring (mcp_server/server.py:847)
• Test seed data + parametrize (tests/conftest.py, tests/test_fts5_index.py, tests/test_query_router.py, tests/test_search.py, tests/test_e2e_fts5.py, tests/test_config_fts5.py)

Router coverage expansion

MS17-010 is a letter+digit compound (MS<year>-<seq>) that the shipped default patterns [A-Z]{2,}-\d+ do not match (they require ≥2 letters before the hyphen). Rather than pick a narrower replacement, mcp_server/config.py fts5_patterns gains a new default pattern [A-Z]+\d+-\d+ so MS-style Security Bulletin identifiers route to the FTS5 fast-path just like CWE-79 and CVE-2021-4034. This is additive on the default pattern list (4 → still additive, users with custom patterns unaffected).

tests/test_config_fts5.py::test_ut031_defaults_when_section_empty bumped from assert len == 3 to assert len == 4. tests/test_query_router.py::ROUTER_TEST_PATTERNS gains an anchored ^[A-Z]+\d+-\d+$ counterpart for the boundary tests.

LEI 1 preserved

Zero change to the 13 frozen MCP tool signatures. scripts/check_api_surface.py --check returns no diff.

Test plan

• 66/66 pass locally across tests/test_fts5_index.py, tests/test_query_router.py, tests/test_v483_hotfix.py, tests/test_config_fts5.py
• ruff check + ruff format --check clean on mcp_server/, tests/, bench/
• API surface unchanged (scripts/check_api_surface.py --check)
• 9-cell CI matrix green
• 7 Quality Gate pillars green (may skip perf-gate — no runtime behavior change beyond the default pattern list expansion, RSS jitter unlikely)

Summary by CodeRabbit

• New Features

  • Expanded lexical search pattern recognition to support identifiers such as MS17-010.
  • Added support for letter-and-digit vulnerability codes in default FTS5 matching patterns.

• Documentation

  • Updated FTS5 examples and configuration guidance with current identifier formats.

• Tests

  • Updated search, routing, indexing, fallback, and configuration coverage for the revised identifier patterns.

Greptile Summary

The PR replaces internal-looking identifier examples with public MS bulletin identifiers and expands the runtime router defaults to recognize letter-plus-digit compounds.

• Scrubs identifier examples across documentation, benchmarks, docstrings, and fixtures.
• Adds [A-Z]+\d+-\d+ to the programmatic FTS5 pattern defaults.
• Updates routing and configuration tests for the expanded default list.

Confidence Score: 4/5

The sample configuration must be synchronized with the new runtime pattern before merging, otherwise users following it will not get the advertised MS-style FTS5 routing.

The runtime default recognizes MS17-010, but the explicit pattern list shipped for users to copy replaces that default without the new regex, causing identifiers covered only by that regex to remain on the semantic path.

Files Needing Attention: config.example.yaml, docs/features/fts5_fast_path.md

Important Files Changed

Filename	Overview
mcp_server/config.py	Adds the MS-style identifier regex to the programmatic FTS5 defaults.
config.example.yaml	Updates an example identifier but omits the new runtime pattern, so copied configurations do not receive the expanded routing behavior.
docs/features/fts5_fast_path.md	Scrubs the old identifier but leaves the documented default pattern list inconsistent with runtime defaults.
tests/test_query_router.py	Adds MS17-010 coverage to the router’s anchored boundary-test pattern set.
tests/test_config_fts5.py	Updates assertions for the four programmatic default patterns.

Prompt To Fix All With AI

### Issue 1
config.example.yaml:289-292
**Sample omits new routing pattern**

When a user copies this explicit `patterns` list and enables the lexical fast path, it replaces the programmatic defaults without `[A-Z]+\d+-\d+`, causing identifiers covered only by the new pattern to remain on the semantic path; the feature guide repeats the same incomplete default list.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

_{Reviews (1): Last reviewed commit: "chore(security): scrub internal identifi..." | Re-trigger Greptile}

Greptile also left 1 inline comment on this PR.

[coderabbitai]

📝 Walkthrough

Walkthrough

The FTS5 default patterns now recognize letter-plus-digit identifiers such as MS17-010. Fixtures, tests, benchmarks, and documentation use updated identifier examples. Tokenization executable logic remains unchanged.

Changes

FTS5 identifier support

Layer / File(s)	Summary
Default pattern contract mcp_server/config.py, tests/test_config_fts5.py	The default FTS5 patterns now include letter-plus-digit identifiers. Configuration tests expect the revised pattern and four defaults.
Query and index validation tests/conftest.py, tests/test_query_router.py, tests/test_fts5_index.py, tests/test_e2e_fts5.py, tests/test_search.py, mcp_server/server.py	Fixtures and tests use MS-style identifiers. Matching, case handling, token separation, fallback metrics, and empty-result behavior remain covered.
Documentation and benchmark alignment README.md, config.example.yaml, docs/features/fts5_fast_path.md, bench/test_bench_fts5_lexical.py	Examples and benchmark queries use updated vulnerability identifiers. Regex behavior and executable tokenization logic remain unchanged.

Estimated code review effort: 2 (Simple) | ~10 minutes

Possibly related PRs

• lyonzin/knowledge-rag#158: Introduced related FTS5 configuration, tokenizer patterns, fixtures, and tests.
• lyonzin/knowledge-rag#160: Updated related FTS5 query classification and tokenization behavior.
• lyonzin/knowledge-rag#168: Updated related FTS5 examples, identifier patterns, benchmarks, and tests.

🚥 Pre-merge checks | ✅ 5

✅ Passed checks (5 passed)

Check name	Status	Explanation
Docstring Coverage	✅ Passed	Docstring coverage is 100.00% which is sufficient. The required threshold is 80.00%.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Description Check	✅ Passed	Check skipped - CodeRabbit’s high-level summary is enabled.
Title check	✅ Passed	The title clearly summarizes the primary change: replacing internal identifier examples in public documentation and tests.

✨ Finishing Touches

📝 Generate docstrings

• Create stacked PR
• Commit on current branch

🧪 Generate unit tests (beta)

• Create PR with unit tests
• Commit unit tests in branch chore/scrub-internal-identifier-examples

---

_{Comment @coderabbitai help to get the list of available commands.}

[greptile-apps]

Sample omits new routing pattern

When a user copies this explicit patterns list and enables the lexical fast path, it replaces the programmatic defaults without [A-Z]+\d+-\d+, causing identifiers covered only by the new pattern to remain on the semantic path; the feature guide repeats the same incomplete default list.

Prompt To Fix With AI

This is a comment left during a code review.
Path: config.example.yaml
Line: 289-292

Comment:
**Sample omits new routing pattern**

When a user copies this explicit `patterns` list and enables the lexical fast path, it replaces the programmatic defaults without `[A-Z]+\d+-\d+`, causing identifiers covered only by the new pattern to remain on the semantic path; the feature guide repeats the same incomplete default list.

---

For each issue above, determine whether it is valid and should be fixed. If so, fix it directly.

[coderabbitai]

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@config.example.yaml`:
- Line 290: Synchronize the public pattern examples with the runtime default in
config.example.yaml lines 290-290 and docs/features/fts5_fast_path.md lines
60-60: add the [A-Z]+\d+-\d+ pattern, retain the CWE-79 pattern, and use
MS17-010 and MS99-999 as examples in both locations.

🪄 Autofix

Fix all unresolved CodeRabbit comments on this PR:

• Push a commit to this branch (recommended)
• Create a new PR with the fixes

---

ℹ️ Review info

⚙️ Run configuration

Configuration used: defaults

Review profile: CHILL

Plan: Pro Plus

Run ID: ac45ccac-b153-4308-8aa4-7897b03b1d4d

📥 Commits

Reviewing files that changed from the base of the PR and between 3f91f7b and a1f969d.

📒 Files selected for processing (12)

• README.md
• bench/test_bench_fts5_lexical.py
• config.example.yaml
• docs/features/fts5_fast_path.md
• mcp_server/config.py
• mcp_server/server.py
• tests/conftest.py
• tests/test_config_fts5.py
• tests/test_e2e_fts5.py
• tests/test_fts5_index.py
• tests/test_query_router.py
• tests/test_search.py

[coderabbitai]

🗄️ Data Integrity & Integration | 🟠 Major | ⚡ Quick win

Synchronize both public pattern examples with the runtime default.

Both snippets claim that MS17-010 is supported while showing only "[A-Z]{2,}-\\d+", which cannot match that identifier. Add "[A-Z]+\\d+-\\d+" in both locations and retain the CWE-79 pattern.

• config.example.yaml#L290-L290: update the copyable configuration block and use MS17-010 and MS99-999 as examples.
• docs/features/fts5_fast_path.md#L60-L60: update the documented default pattern block with the same pattern list.

📍 Affects 2 files

• config.example.yaml#L290-L290 (this comment)
• docs/features/fts5_fast_path.md#L60-L60

🤖 Prompt for AI Agents

Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@config.example.yaml` at line 290, Synchronize the public pattern examples
with the runtime default in config.example.yaml lines 290-290 and
docs/features/fts5_fast_path.md lines 60-60: add the [A-Z]+\d+-\d+ pattern,
retain the CWE-79 pattern, and use MS17-010 and MS99-999 as examples in both
locations.