feat(cache): add a pinned-version vocabulary rule to the pool allowlist

[myselfsiddharth]

Closes #126. ADR-0017. Related: #118 (ADR-0014, read order — already landed).

The pool captures 1 of 12 rows on the only real compiled bundle in the repo. The issue's diagnosis: isChromeName() is an exact match against ~50 generic words, and locator names like Add new panel / toggle-viz-picker / Plugin visualization item Stat are not tenant data — they're Grafana's own UI vocabulary at a pinned open-source version, reproducible by anyone who runs docker run grafana/grafana-oss:9.5.21.

The issue asked for three things: measure yield properly, decide whether product vocabulary is pool-safe, and record it. Measuring it properly changed the question.

The measurement changed the question

The bundle's stamped 1/12 is the compiler's own pre-check (src/compiler/pool.ts), which docs/gate/compiler.md already documents as deliberately stricter than the authority. Routed through the actual authoritative path instead (buildPoolRow, src/cache/write.ts) — verified with git stash against the pre-PR code, so this is the number before this PR changes anything:

rows: 12
pool_eligible (via src/cache/write.ts, authoritative B5 path): 7
reasons: { literal_in_assertion: 5 }

7 of 12, not 1 of 12, before this PR does anything. Six of those seven pool through the existing structural-locator allowance (boundary-spec.md rule 2) — no vocabulary rule needed. The gap between "1" and "7" is the compiler's pre-check disagreeing with its own authority in the conservative direction (safe, per its own doc — "a pre-check may be stricter than the authority; it may never be looser" — but currently misleading anyone who reads the artifact's stamped field as the real number).

This PR's vocabulary rule changes that 7/12 by exactly zero, for two reasons, neither of which is isChromeName's narrow list:

1. The remaining five rows are blocked by literal_in_assertion, not locator taint — a url-matches assertion whose template residue is a URL path refuses the row unconditionally, before any locator is considered. docs/gate/compiler.md already named this "the single biggest reason" and deferred it to B5 as a separate decision. Still deferred here — reopening it means bundling two decisions into one PR.
2. The recorder blanket-tags every role_name / label / text candidate tenant_scoped: true, independent of content — confirmed directly in the source trajectory JSON. caller_marked_tenant correctly honors that upstream claim ahead of any vocabulary match (that's the fail-closed behavior the boundary spec requires, not a bug), but it means a content-based rule is structurally inert on every locator this recorder has ever produced. src/recorder/ is out of this PR's scope.

Full writeup, including the per-locator (not row-level) effect the rule does have on this bundle's one testid, in docs/gate/pool-vocabulary.md.

What shipped anyway, and why

src/cache/vocabulary.ts — a committed snapshot of five strings (the four the issue named, plus one testid found while verifying them), each independently checked against the public grafana/grafana GitHub repo at tag v9.5.21 today, with file + line + access date. "Apply" (the fifth locator on the live bundle) is not in the snapshot: not independently verified in the time available, and it wouldn't have mattered anyway — that exact locator is already tenant_scoped: true in the source trajectory.

No live fetch in the shipped code — the fail-closed guarantee can't depend on GitHub being reachable at write time or in CI. Network access was used once, by hand, to build the citations.

Wired additively: allowlist.ts gained isPoolSafeAccessibleName / isPoolSafeTestId (existing isChromeName / isAllowedTestId untouched, still used unqualified in assertionHasTenantLiteral). taint.ts's aria_label_or_name_tenant, role_text_tenant, and non_vocab_testid rules call the composed functions; caller_marked_tenant and every other rule are unchanged.

Why ship a rule that measures zero yield improvement: the evidence indicts the compiler pre-check and the recorder's tagging policy, not the principle that pinned-version product vocabulary is poolable. Concluding "pooling doesn't work" from a measurement that names the wrong culprit would be a worse error than shipping a currently-quiet, architecturally real mechanism. It also isn't inert everywhere: repair-proposed locators (ADR-0009 / #64) carry no blanket tenant_scoped tag — nothing in update.ts / confidence.ts sets one — so a repair proposing toggle-viz-picker was refused before this PR and pools after it, purely on content. That path has no model behind it yet (#27), so no yield number is claimed for it either — only that the mechanism is real and tested.

The collision guarantee

The issue asked how the rule tells "Grafana's own accessible name" apart from "a tenant string that happens to collide with one." It uses a signal the boundary already had: tenant_scoped, honored unconditionally by caller_marked_tenant, which runs independently of the vocabulary check and is never overridden by it. tests/canary/vocabulary.test.ts (merge-blocking) proves both directions:

• an untagged locator named toggle-viz-picker (not in UI_CHROME_NAMES) pools, shaped the way a repair proposal is shaped
• the exact same string, Alias, tagged tenant_scoped: true, is still refused — and a mutation test (removing caller_marked_tenant) shows the guarantee actually depends on that rule, not on rule ordering by accident

Multi-version yield: no_data

The issue's checklist wants yield per version across ADR-0003's 8 pins. This environment has no running Docker daemon (verified: docker ps fails with no daemon socket present) and no ANTHROPIC_API_KEY, so recording fresh trajectories against the other seven versions isn't possible here. docs/gate/pool-vocabulary.md states this as a gap, not a zero, and names exactly what a follow-up run needs.

Docs touched

• New: docs/gate/pool-vocabulary.md (measurement), docs/decisions/ADR-0017-pool-vocabulary-rule.md (decision)
• docs/gate/cache.md and docs/gate/compiler.md: the stale "1 of 12" references annotated against the 7/12 authoritative recount, not deleted
• docs/decisions/ADR-0014-cache-read-path.md: open questions updated with what the pool actually contains, per Cache read path: resolve a program from the cache and record hit/miss — the amortization claim has no mechanism without it #118
• docs/privacy/boundary-spec.md: the "Open, and shared with Pool captures 1 of 12 rows on the live bundle: the allowlist cannot tell product vocabulary from tenant data #126" note now points at the answer
• docs/pitch/proof-points.md (F4, F6), narrative.md, objections.md: footnoted with the measured number and a link to ADR-0017 — touched only those specific lines, nothing else, since a parallel agent is reconciling Track-2 status in the same five files right now

Scope, explicitly

Not touched: src/intent/, src/recorder/cli.ts, src/session/ — out of scope per the task split, and two of the three findings above (literal_in_assertion, recorder blanket-tagging) live partly in that territory anyway; naming them here is not fixing them here.

Before you open the PR checklist, honestly:

• Measured yield recorded in docs/gate/ with version and date — both numbers, and why they disagree
• ADR recording the decision — vocabulary rule shipped, with the zero-yield-on-this-bundle finding stated plainly
• New canary covering the collision case, merge-blocking
• docs/pitch/ claims reconciled with the measured number
• Cache read path: resolve a program from the cache and record hit/miss — the amortization claim has no mechanism without it #118's read-order doc updated with what the pool actually contains
• Full ADR-0003 matrix measurement — explicitly deferred, no_data, no Docker testbed here

npm run ci            # green — 405 unit, 26 integration, secret-scan clean, lint-docs clean (59 docs)
npm run test:canary   # 52 pass (was 48), including the 4 new vocabulary canaries

🤖 Generated with Claude Code

[myselfsiddharth]

Review — independently verified, not taken on faith

Reviewed at 7988717, in a clean worktree off origin/track1/b4-pool-vocabulary. This PR makes a self-undermining-sounding claim about its own change ("the issue's number is wrong, and my rule changes it by zero"), so I reproduced the measurement myself rather than reading the writeup. The claim holds up completely. Details below, then the one thing that blocks merge (a mechanical conflict, not a substantive problem).

---

1. The 1/12 vs 7/12 claim — verified, the PR is right and the issue is wrong

I wrote my own measurement script (deliberately not reusing the _measure.ts left in a scratch worktree) and ran it against the parent commit 35145f9, i.e. before this PR changes anything. It builds candidates that strip the artifact's own stamped verdict, so nothing the compiler decided can leak into the recount, and it runs the full writeCacheRow authority with fail-closed on — not just buildPoolRow:

rows: 12
A) stamped in committed artifact : 1  { literal_in_assertion: 1, tenant_locator_text: 10 }
B) compiler pre-check (re-run)   : 1  { literal_in_assertion: 1, tenant_locator_text: 10 }
C) buildPoolRow (write.ts)       : 7  { literal_in_assertion: 5 }
D) writeCacheRow AUTHORITY       : 7  { literal_in_assertion: 5 }

Line B is the load-bearing one: re-running decidePoolEligibility on the same rows reproduces the artifact's stamped field exactly, which confirms the 1/12 in issue #126 is the compiler pre-check and not a write-time result. Line D is the authority, and it says 7.

These are genuinely different code paths, and I confirmed why they disagree rather than just observing that they do:

• decidePoolEligibility (src/compiler/pool.ts:37-42) rejects the whole row if any locator in the chain carries tenant_scoped === true.
• buildPoolRow (src/cache/write.ts:153-162) instead partitions the chain via classifyLocators — tainted locators are dropped from the pool copy and kept on the tenant twin, and the row still pools if at least one clean locator survives.

So on the six rows (steps 2, 3, 4, 5, 8, 9) whose role_name/label candidates are tenant-tagged but whose structural fallback is clean, the pre-check refuses and the authority accepts. Plus step 6, already pooling → 7. That matches the PR's per-step list exactly. The pre-check is stricter than its authority, never looser, which is the direction docs/gate/compiler.md requires, so this is a documentation/perception bug, not a leak — and annotating the stale numbers rather than deleting them is the right call.

2. "The vocabulary rule changes that by exactly zero" — verified, both reasons check out

Same script at the PR head: still 7/12, still { literal_in_assertion: 5 }. Delta zero, confirmed.

Both stated causes are real, and I checked them at the source rather than accepting the writeup:

(a) literal_in_assertion on 5 rows — steps 0, 1, 7, 10, 11, every one a url-matches whose expected.template leaves a URL path after typed holes are stripped (http://{host}:{port}/dashboard/new?orgId=1 → http:///dashboard/new?orgId=1). assertionHasTenantLiteral refuses the row at write.ts:157, before any locator is considered. Genuinely unrelated to locator vocabulary.

(b) Recorder blanket-tagging — the PR says this was confirmed in the source trajectory; I confirmed it in the recorder source, which is stronger. src/recorder/locators.ts hardcodes tenant_scoped: true on role_name (:106), label (:110), text (:127) and placeholder (:134) with zero content inspection, and false on testid (:117) and structural (:124). A content-based rule therefore cannot rescue any name-bearing locator this recorder has ever emitted. That is a structural fact, not an excuse.

The narrower claims hold too:

• Per-locator effect is real. Step 7's testid data-testid Apply changes and go back to dashboard moves TAINT[non_vocab_testid] → SAFE under the new rule. Step 11's data-testid Dashboards breadcrumb (not in the snapshot) stays tainted — so the rule is genuinely narrow, not a blanket testid pass.
• Repair path carries no blanket tag. No tenant_scoped assignment exists anywhere in src/cache/update.ts or src/cache/confidence.ts. Confirmed.

I'll say plainly that I went in expecting this section to be a rationalization for shipping something that doesn't work. It isn't. The measurement is honest, the culprits named are the actual culprits, and the reasoning for shipping anyway — that killing the mechanism over a measurement indicting two other subsystems would be the worse error — is sound. Reporting a zero delta on your own PR is the right behaviour and I'd rather see more of it.

3. Sourcing — 4 of 5 exact; 1 off-by-one found and fixed

I fetched all cited files from raw.githubusercontent.com at tag v9.5.21 (all HTTP 200) and checked every cited line number:

String	Citation	Result
Add new panel	DashboardEmpty.tsx:42	✅ exact
toggle-viz-picker	VisualizationButton.tsx:45 + components.ts:139	✅ exact
Plugin visualization item Stat	PanelTypeCard.tsx:42 + components.ts:273 + stat/plugin.json "name": "Stat"	✅ exact
Alias	QueryEditor.tsx:204	✅ exact
data-testid Apply changes and go back to dashboard	PanelEditor.tsx:362 + components.ts:137	⚠️ primary exact; sub-citation off by one

applyButton is on line 138 of components.ts, not 137 — line 137 is the closing brace of the preceding DataPane block. The primary citation (PanelEditor.tsx:362) and the snapshot string itself are correct. Trivial and unambiguous, so I fixed it directly in 08156f6 (both src/cache/vocabulary.ts and docs/gate/pool-vocabulary.md) and pushed. Nothing else about the sourcing is wrong — no fabricated citation, no string that isn't where it's claimed to be.

One nuance worth recording: the PR excludes "Apply" as unverified. It's doubly harmless — isChromeName("Apply") is already true, and my step-7 trace shows that locator fires only caller_marked_tenant, never aria_label_or_name_tenant. Excluding it was correct.

4. Collision guarantee — genuine, and it survives harder cases than the canary tests

I read tests/canary/vocabulary.test.ts rather than trusting the description. It does what it says, including the mutation test. Worth noting the guarantee is structurally stronger than "rule ordering": createTaintChecker runs all rules and collects every reason — there is no short-circuit — so ordering cannot be what's doing the work, and the mutation test is the correct way to prove caller_marked_tenant is load-bearing.

I then probed cases the canary does not cover. All pass:

• Tenant-tagged Alias as the only locator, no structural fallback → row refused outright. (The shipped canary only covers the case where a clean fallback rescues the row, so this was the gap worth checking.)
• Tenant-tagged vocabulary testid, no fallback → refused.
• Vocabulary string via text strategy, untagged → still refused (free_text_strategy correctly kept on unqualified isChromeName).
• Vocabulary string inside a CSS attribute value → still refused.
• Exact-match discipline: "toggle-viz-picker ", "Toggle-viz-picker", "toggle-viz-picker-2", "alias", "Add new panel from panel library" all correctly not pool-safe. No substring, case, or whitespace leniency.
• Arbitrary tenant string ("Acme Corp Q3 Invoice") → refused as before.

5. Fail-closed posture — intact, additive only

• src/cache/vocabulary.ts has zero imports — a pure static data module. No fetch/http anywhere in src/cache/. The snapshot is a committed artifact, so the boundary never depends on GitHub being reachable. Correct.
• isChromeName / isAllowedTestId are unmodified and still used unqualified in free_text_strategy, non_vocab_css_attr, and write.ts's assertionHasTenantLiteral. Confirmed by reading, and by probes 3–4 above.
• The total widening is exactly five source-cited exact strings. No existing taint rule weakened.
• The canary is genuinely merge-blocking: .github/workflows/ci.yml:110 runs npm run test:canary as its own privacy-canary job. (Note npm run ci alone does not include it — pre-existing, and the workflow covers it.)

6. Checks I ran myself

npm run ci          → exit 0
                      secret-scan clean · contracts ok · eslint clean
                      lint-docs clean (59 docs) · typecheck clean
                      405 unit passed · 26 integration passed
npm run test:canary → 52 passed (8 files), incl. 4 new vocabulary canaries

Both match the PR's stated numbers exactly. Re-ran both after my citation fix; still green.

---

⛔ Blocks merge: conflict with main

mergeable: CONFLICTING, mergeStateStatus: DIRTY. This branch hasn't been rebased since #157 and #158 landed. Exactly one file conflicts:

docs/README.md — the ADR index table (merged-result lines 65–71). Both sides edited the same region:

• main (via feat(intent): resolve a natural-language goal to a task_key before requiring one #158) added the ADR-0015 row and left ADR-0014's row dated 2026-08-11.
• This PR rewrote ADR-0014's row (date → 2026-08-12, plus the "open questions updated… (Pool captures 1 of 12 rows on the live bundle: the allowlist cannot tell product vocabulary from tenant data #126 / ADR-0017)" note) and appended the ADR-0017 row.

There is no semantic disagreement — the resolution is purely additive: keep this PR's rewritten ADR-0014 row, keep main's ADR-0015 row, keep this PR's ADR-0017 row, in that order. I diagnosed it but deliberately did not resolve it.

Good news on the files you'd expect to be worse: docs/pitch/proof-points.md, docs/pitch/narrative.md, docs/pitch/objections.md and docs/gate/cache.md all auto-merge clean. I checked semantically too, not just textually: #157 touched row F11 in proof-points.md; this PR touches F4 and F6. Disjoint. The claimed discipline of "touched only those specific lines" paid off exactly as intended.

ADR numbering is also clean: 0015 on main (#158), 0016 claimed by the still-open #156, 0017 here. No collision.

---

Verdict

The central claim is correct. 1/12 is the compiler pre-check; 7/12 is the authoritative write-time number; the vocabulary rule moves it by zero on this bundle for exactly the two reasons stated, both of which I verified at the source. Sourcing is real, the collision guarantee is real and survives harder probes than the shipped canary, and the fail-closed posture is genuinely additive.

No substantive blocking issues. The only thing standing between this and merge is the mechanical docs/README.md ADR-table conflict — rebase onto current main, take both sides' rows, and re-run npm run ci + npm run test:canary.

Non-blocking, for the record: version-blind matching is a real imprecision, but it's disclosed in both the ADR and the module docstring, and it can only ever widen the allowlist with source-cited vendor strings — it cannot admit tenant content. Agreed that it doesn't weaken the guarantee, and agreed it needs a contract change to fix properly.

Reviewed with Claude Code. All numbers above were reproduced independently at 35145f9 (pre-PR) and 7988717 (post-PR); citations spot-checked live against raw.githubusercontent.com at tag v9.5.21 on 2026-08-12.