asvs: the engine-side half of the ASVS gate (#1202) + the tally (#1204) and residual-citation (#1205) lints

[wshallwshall]

Three engine-side pieces of the ASVS tracking rework: the half of the vault gate that can live here
(#1202), a forward-only lint refusing a new hard-coded ASVS tally (#1204), and a forward-only
lint refusing a new file:line citation in scorecard prose (#1205).

A note on evidence throughout: the assessment corpus is a private security record and its
tallies are the posture, so no corpus text, no per-document tally and no verdict distribution
appears in this repo or in this description. Where the measurement is over that corpus, the
aggregate is reported and the line-level output is not.

---

Part 1 -- #1202: the engine-side half of the ASVS gate

asvs-scorecard.yml lives in the vault, fires on vault paths, and measures the engine tree.
The gate makes three claims about this tree, and they split on whether they need the private
record:

#	Claim	Inputs	Checked before
1	recorded evidence anchors still resolve	needs the private scorecard	vault only, daily cron
2	scripts/asvs/scorecard.py exists at that literal path	public	neither repo
3	it runs -- bare script, bare interpreter, nothing installed	public	neither repo

Claim 1 admits no engine-side trigger and none is added. Both inputs must be co-located, only
the vault holds the record, and GitHub's only credential-free cross-repo trigger is polling -- which
is already the cron. repository_dispatch needs a credential, and a credential in the public repo
that reaches the private vault is a worse trade than the latency it buys, for a detector that is
post-merge either way (the gate checks out engine main). That half is vault-side.

Claims 2 and 3 belong here. The vault runs the verifier twice and neither job installs anything:
asvs-scorecard.yml after a bare setup-python, and asvs-verifier-drift.yml's preflight, which
runs the incoming engine copy from the vault checkout so sys.path[0] is scripts/asvs and a
first-party import has nothing to resolve against. The invariant is stated in asvs-scorecard.yml's
own comment -- "No install step and no dependency: the verifier is stdlib-only ... precisely so this
job cannot rot on a lockfile it does not own" -- and was enforced nowhere. This repo's tests import
the module with the repo root on sys.path and full extras installed, so the invocation that matters
was never exercised.

Breaking it does not red the gate. The mirror opens as a draft when the incoming verifier
will not run, and a draft is never merged, so the vault keeps verifying its record with the previous
copy. That is the recurring condition -- six hand-made mirror commits, one found 326 lines behind --
that splitting asvs-verifier-drift.yml out was written to end.

The old configuration was blind

A used import httpx appended to scripts/asvs/scorecard.py (httpx is a real engine dependency,
so not dead code a linter rejects anyway). Every pre-existing gate returned a verdict identical to
its unmutated one:

                                                        unmutated   mutated
ruff check .                                            PASS        PASS
ruff format --check .                                   unchanged   unchanged
mypy (strict) messagefoundry messagefoundry_webconsole  unchanged   unchanged   <- types messagefoundry, not scripts/
pytest tests/test_asvs_scorecard.py (54 tests)          PASS        PASS
pytest the new guard                                    PASS        FAIL
python -I -S -c "import httpx"                          rc=1        rc=1        <- models the vault runner

The new guard reds, and each injection landed

Applied, confirmed present by sha256 change, run, restored byte-exact:

                                          landed  path  ast-scan  bare-run
A  module-level import httpx              yes     PASS  FAIL      FAIL
B  import httpx DEFERRED into a function  yes     PASS  FAIL      PASS   <- only the scan sees it
C  import messagefoundry (first-party)    yes     PASS  FAIL      FAIL
D  verifier renamed away                  yes     FAIL  FAIL      FAIL
restored                                          PASS  PASS      PASS

B is why the ast scan is load-bearing: a --help smoke test is structurally blind to a
deferred import. Re-run against the refactored guard in Part 2 and still red.

The first run of that harness reported INJECTION LANDED: False for A-C: the tree is CRLF under
core.autocrlf=true and the \n anchors matched nothing. The confirm-it-landed step is the only
reason that was not read as "the guard cannot see it."

---

Part 2 -- #1204: a forward-only lint refusing a NEW hard-coded ASVS tally

ADR 0156 made the count computed. It did not stop anyone writing one down: 44 documents assert a
whole-corpus tally, roughly fifty distinct tallies exist, and approximately one is correct.

The idiom set was rebuilt from the corpus, and the old recall reproduced first

The previous attempt matched two shapes: an N / N / N / N tuple and N of 345. Re-implementing
exactly those two and running them over the corpus reds 2 of the 8 chapter reports -- which
matches the register's independent finding that it missed 6 of 8, so the reproduction is calibrated
against a number I did not choose. The five idioms here red 8 of 8:

idiom	what it catches	old set
SLASH_RUN	a slash run whose integers sum to the pinned corpus total	caught
LABELLED_VERDICTS	3+ distinct verdict classes each carrying a count, markdown-emphasis tolerant	partly
AGAINST_TOTAL	a count stated against the corpus total	partly
TABLE_ROW	a Markdown table row whose cells close to the total	invisible
ARITHMETIC	a sum asserted to close to the total	invisible

TABLE_ROW is the shape the one correct record is written in, which is why a grep for the
corpus's own tuple idiom does not find it. Markdown-emphasis tolerance is worth naming separately:
one chapter report writes its tally with the verdict words in backticks, and a detector using a bare
\s* walks straight past it -- that alone was 1 of the 8.

The discriminator is that a run of integers is a tally only when it sums to the pinned corpus
total. That is what keeps out HL7 field notation, X12 transaction sets, status-code lists and
config defaults; the previous attempt produced 73 false hits on exactly those. Corpus structure
is deliberately not flagged -- the requirement count and the level split are pinned constants that
cannot go stale, and flagging them is what made the previous attempt red the method document's own
worked example, which is a literal letter rather than a count.

Acceptance, measured: 8 of 8 chapter reports red (V1, V3, V4, V6, V10, V13, V15, V16); the old
two idioms red 2 of 8. Of the documents the old idioms missed, 14 assert a hard-coded verdict or
survey tally (the 6 chapter reports plus 8 others) under the definition above. The register's
figure was 13, measured at an earlier ref and against a looser definition that also counted
corpus-structure mentions; at today's ref that looser definition gives 20, of which 4 documents
postdate the register. Both numbers are reported rather than one being tuned to match -- the
checkable acceptance criterion is the 8 of 8 and the reproduced 2 of 8.

Proved on real files, both directions

Eight mutations, one per idiom shape, appended to a real document in this repo -- each confirmed to
have landed by digest, then reverted byte-exact:

  RED  landed=True  SLASH_RUN     tight tuple                      RED  landed=True  LABELLED  abbreviated columns
  RED  landed=True  SLASH_RUN     spaced tuple                     RED  landed=True  AGAINST_TOTAL prose
  RED  landed=True  LABELLED      slash-separated                  RED  landed=True  TABLE_ROW markdown table
  RED  landed=True  LABELLED      backticked (the 1-in-8 miss)     RED  landed=True  ARITHMETIC closes to total

  GREEN HL7 field notation   GREEN status codes   GREEN corpus structure
  GREEN the method doc template ("N of 345")      GREEN a sum one off the total
  FAILURES: 0

Every one of those shapes is now a permanent test, positive and negative. 53 tests across the two
modules.

Forward-only, and the ratchet

Existing tallies are recorded, not edited, in scripts/docs/asvs_tally_baseline.txt -- 18 claims,
23 occurrences, across 4 files in this repo (docs/adr/0019-*, docs/Secure_Build_Scorecard_MEFOR.md,
docs/BACKLOG.md, docs/archive/backlog/BACKLOG-CLOSED.md; the design document calls these the
public-repo leaks). A sweep was refused on the record: ~850 net lines, its own repair commit, and the
defect regenerated inside four days.

The baseline stores an occurrence count per claim, so it cannot idle:

• a tally not in the baseline -> fail;
• an extra copy of a grandfathered tally -> fail (count exceeded);
• a tally removed without lowering the entry -> fail (baseline over-counts the tree).

The key is path + idiom + the numbers, never the line number and never the surrounding prose --
a baseline invalidated by an edit above it is a list that has to be re-typed, which is the
decaying-budget failure this programme is unpicking elsewhere.

Trigger: the gate must see a docs-only PR

The defect is written into documents, so the PR carrying it is usually docs-only -- and a
docs-only PR sets code=false and skips the entire pytest suite. A guard reachable only through
pytest would not run on the shape it exists to police, which is exactly why two guards in ci.yml
are deliberately ungated. Measured with the live workflow globs and the live noncode regex:

NEW gate  OLD: pytest ran?  shape
FIRES     NO (skipped)      docs-only: a new tally written into a document
FIRES     NO (skipped)      docs-only: an ADR edit
FIRES     yes               the lint itself
FIRES     yes               the frozen baseline
FIRES     yes               THE GATE'S OWN FILE
skip      yes               code-only, no documents
skip      yes               a path that cannot exist (negative control)
MISMATCHES vs expectation: 0

files matching 'tally' on origin/main: 0   <- there was also no lint to run

So a new workflow, not a pytest test alone. It is not a required check (paths-filtered; a
required check that cannot report wedges the PR forever) and it is not in
.github/required-contexts.txt.

Why not ci.yml's ungated doc-guards step, which is the natural home: ci.yml's changes job is
under open PR #299 and a second edit to those lines would conflict. The workflow header records the
follow-up -- fold this into the doc-guards list once #299 lands, so one place decides which guards run
on a docs-only PR rather than two.

Where the lint can actually run

Stated plainly, because it is the same shape as claim 1 above. The engine can only lint the
documents it holds. The bulk of the affected corpus is in the assessment repo and can only be
linted there. The tool is stdlib-only for exactly that reason -- it is mirrorable and runs on a bare
interpreter with no install, the same contract scripts/asvs/scorecard.py is held to -- but wiring it
into a workflow on that side is a change in that repo, not this one.

Part 1's guard was generalised rather than copied for this: the stdlib-only rule is now stated
once over a list of mirrored tools, and the tally lint is on that list before it acquires a
dependency rather than after someone discovers the mirror will not run. The fix that does not
generalise is the one that comes back.

Retiring the existing 44 documents is explicitly not in scope. That is a sweep; it was costed and
refused; it is the owner's decision.

One safety check worth recording

scorecard.py::_python_sources scans messagefoundry, messagefoundry_webconsole, harness and
scripts for *.py, and every absence claim greps that corpus -- so a new file under scripts/
can flip an absence claim to FALSE. tests/test_docs_db_grants.py records this trap in its own
header. Run before committing each of the two new files under scripts/: 0 of 276 absence
patterns match, with a control pattern proving the check was live and an impossible pattern proving
it was not matching everything.

---

Part 3 -- #1205: a forward-only lint refusing a NEW file:line citation in scorecard prose

Half the record is prose nothing checks. Measured at the pinned record: 2,052 citations inside
residual across 247 cells, 1,086 of them bare basenames. A sample measured 44.9% stale.

The demonstration case, verified against public engine code

Cell 6.3.3's gated evidence anchor for the exposure check points at
messagefoundry/__main__.py:1125:

:1125  instance_exposed = not settings.api.is_loopback or settings.api.tls_terminated_upstream
:1917  "warning: the browser console is exposed on a PHI instance with "
'instance_exposed = not settings...' actually occurs at line(s): [1125]

The prose in the same cell cites __main__.py:1917 -- the middle of a warning string. The gated
half is right and the read half is wrong, about one control, in one cell. A reviewer reads the
prose.

A round number made finer

On "1,064 bare basenames that cannot resolve even in principle", resolved against this worktree
(2,950 files, 2,630 distinct basenames): of 1,086 bare basenames, 129 match no file at all, 276
match two or more, and 677 match exactly one. So unresolvable in practice is 405; the other 677
resolve only by luck of there being a single candidate, which the citation format does not guarantee.
Both readings are reported rather than one being quoted.

The key is the design

A citation is identified by cell id + field + FILE, never by its line number:

• the population of (cell, field, file) pairs cannot grow;
• the frozen baseline carries an occurrence count, so an existing pair cannot grow either -- the
  total can only shrink;
• repairing a stale line is FREE and needs no baseline edit; adding a citation is refused;
  deleting one costs an edit.

That ordering is chosen. The rejected bulk-promotion approach made delete the citation the cheapest
compliant act on ~1,000 basenames; this makes correction the cheapest, which is what you actually
want from someone who has just noticed a citation is wrong. Bulk promotion is not attempted --
~1,000 hand-authored tokens and it doubles the gated surface.

It refuses to succeed on an empty scan

This tool runs where the data is, which is not this repo, so the dangerous failure is the silent one:
pointed at a renamed field or a document with no cells, "no new citations" and "nothing was
examined" would otherwise share an exit code. Both exit 2, and the scan inventory prints before
the verdict.

Proved at scale against the real record (locally; nothing committed)

BASELINE GENERATED: 1233 claims, 2052 occurrences
CONTROL rc=0   345 cells | 1,105,070 chars of prose | 2052 citations / 247 cells | 1086 bare

MUST RED (rc=1)
  rc=1 landed=True  a NEW citation to a file that cell never cited
  rc=1 landed=True  a NEW bare basename
THE HEADLINE PROPERTY (rc=0)
  rc=0 landed=True  repair 6.3.3's stale __main__.py line, no baseline edit
REMOVAL MUST RED UNTIL THE BASELINE COMES DOWN (rc=1)
  rc=1 landed=True  delete the citation, baseline untouched
EMPTY-SCAN REFUSALS (rc=2, never 0)
  rc=2  a field name that no longer resolves
  rc=2  a baseline path that does not exist
FAILURES: 0

43 fixture tests, both directions on every shape (including settings.py with no line, 06:17,
127.0.0.1:8765, a version string, and an unknown extension).

Where it can run

Unlike the tally lint, this one has no engine-side subject at all -- the record exists only in the
assessment repo. What ships here is the tool (stdlib-only, mirrorable on the ADR 0156 section 7
footing, now on the mirrored-tools list) plus fixture tests. The baseline and the wiring belong beside
the data. No engine workflow is added, deliberately: a gate pointed at nothing is precisely the
failure the tool refuses to commit itself.

A hypothesis I held, and measurement refuted

I expected a cell-id-keyed baseline to leak verdicts, on the reasoning that residuals are a property
of unresolved cells. Measured: cited residuals occur across every verdict class, including a large
majority of the passing ones, and cited-residual cells split close to evenly between pass and
not-pass. The verdict-leak argument does not hold and is withdrawn. The baseline still belongs
beside the data, on the stronger ground that a frozen list is unverifiable from a repo that cannot see
what it grandfathers.

Also measured, not actioned

Two other fields carry file:line: absence[].mutation (32 citations across 24 cells) and one in
evidence[].expect. The field set is a parameter (--field, supporting the absence[].mutation
list-of-tables form, with a test), but the default stays residual -- widening the default would
silently change what an existing baseline means.

---

Carried forward, not actioned here

Both routed to the session holding #299; ci.yml is untouched by this PR:

• ci.yml:836's allowlist comment still documents .gitignore/.gitattributes as docs-only.
  .gitignore was removed from the regex at line 847 by #327 and the comment was never updated, so
  it states the opposite of the code -- the third instance of that pattern in this file.
  .gitattributes is still allowlisted, and fix(ci): an executable file is CODE wherever it lives, including under docs/ (#1200) #299 does not change that (its alwayscode rule keys on
  file extension, and .gitattributes has none).
• An emoji sits in a ci.yml comment near line 170, against CLAUDE.md section 11.

Verification

ruff check . and ruff format --check . clean over 1,063 files. 162 tests across the three new /
extended modules plus the ASVS scorecard and backlog guards pass. ledger_check.py --ci rc=0 against
freshly fetched origin/main. Backlog banner invariant OK at 474 items. The new workflow passes
actionlint and its YAML anchor expands to the same four globs on both triggers. BACKLOG #1202, #1204
and #1205 allocated with scripts/coord/alloc.ps1, never by grep. The #1204 and #1205 backlog entries
were themselves run through the #1204 lint -- neither trips it.

docs/BACKLOG.md conflicted once against main (#1201). Both sides were pure appends -- verified
rather than assumed (+103 -0 mine, +53 -0 theirs, each side's first 8,715 lines the base
verbatim). The resolution was constructed as base + theirs + mine rather than hand-edited, then
checked: every line of both sides present in order, each heading occurring exactly once, and a
neither-side sentinel absent.

Two pre-existing local failures reproduce with these changes removed and are unrelated: mypy
reports 21 errors in 4 files (optional-extra typing, e.g. transports/dicom.py).