feat(asvs): promote the scorecard writer to scripts/asvs/apply.py, with tests - #298
Open
wshallwshall wants to merge 2 commits into
Open
feat(asvs): promote the scorecard writer to scripts/asvs/apply.py, with tests#298wshallwshall wants to merge 2 commits into
wshallwshall wants to merge 2 commits into
Conversation
…h tests P0 of the ASVS tracking rework. The only tool that WRITES the record of record lived at docs/security/asvs-apply-cells.py in the vault: hardcoded absolute path, no argparse, zero tests, zero references, and outside the CI path filter. THE OLD LOCATION WAS WORSE THAN UNTESTED. ci.yml's docs-only detector treats ^docs/ as non-code, so a PR touching ONLY the writer set code=false and skipped install, lint, type-check and the entire pytest suite. The tool that can silently un-close an owner-closed cell was classified as documentation. Verified by running the workflow's own regex against all three paths. The hardcoded path pointed at the SHARED vault checkout, which several sessions edit at once, so running it from a worktree rewrote a record the operator was not looking at -- I hit that this session and worked around it with a patched copy. --scorecard is now REQUIRED with no default: the one thing a writer must never guess is which record it is rewriting. BOTH SHIPPED INVARIANTS KEPT VERBATIM, as directed, and now proved rather than asserted: - The non-allowlist. render() enumerates only what it ORDERS; every other key on the live cell survives by default. Test: a payload omitting decision_closed and decision_closed_by leaves both intact. That is the 7818991d incident, where an ALLOWLIST silently un-closed two owner-closed cells while every gate stayed green, because an absent decision_closed is a valid False. - The set(was) - set(now) backstop, MUTATION-PROVED: render is replaced with one that drops the decision_* keys, reproducing the historical defect in the one function that could reintroduce it, and the write must be refused. The test asserts the refusal names those keys, because a non-zero exit is not evidence -- several guards return 1, and a mutation proof that trips an unrelated one proves nothing about the invariant it claims to test. Also proved to fire: anchor_repair byte-identity on prose and verdict, the owner-closed rescore refusal, the glyph fail-closed check (which fired for real on 13.3.4 this session), unknown-cell refusal, and dry-run-by-default. Two mypy errors were sitting in this file and are fixed here -- it had never been type-checked, because of the path filter above. Not in scope: deleting the vault copy. That is the two-repo consolidation and it needs the mirror settled first.
![github-advanced-security[bot]](https://avatars.githubusercontent.com/in/57789?s=60&v=4){kind=small}
| #: writing one into a security record where a later reader would copy the vocabulary forward. | ||
| _BANNED = re.compile( | ||
| "[" | ||
| "\u26a0\u26d4\u2705\u2b50\u274c\u2714\u2716\u2717\u2718" # warning, no-entry, check, star, crosses |
The one refusal in this writer against a WELL-FORMED payload. Every other guard rejects malformed input; this rejects input that is valid and means more than its author intended -- a verdict moving during a pass whose stated purpose was mechanical (an anchor repair, a re-render, a bulk transform). That is this writer's whole failure mode, so the safe thing is now the default and the dangerous thing is explicit: --allow-verdict-change. The refusal names the cell and BOTH verdicts, per review. A refusal that says only 'verdict changed' leaves the operator's actual next question -- which cell, and to what -- unanswered, and an unanswerable refusal gets re-run with the override reflexively, which converts the guard into a speed bump. BOTH HALVES TESTED, because the first commit of this guard passed all 11 existing tests while nothing exercised it. Adding a guard no test drives is how a guard that cannot fire ships looking green. So: the move is refused and the message names 1.1.1 and 'partial' -> 'pass'; AND the flag actually lifts it. Without the second, the flag could be misspelled, unwired or shadowed and the refusal test would still pass.
wshallwshall
enabled auto-merge (squash)
August 9, 2026 08:39
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
P0 of the ASVS tracking rework, per the design on the parallel session's branch.
The only tool that writes the record of record lived at
docs/security/asvs-apply-cells.pyin the vault: hardcoded absolute path, no argparse, zero tests, zero references, outside the CI path filter.The old location was worse than untested
ci.yml's docs-only detector treats^docs/as non-code. So a PR touching only the writer setcode=falseand skipped install, lint, type-check and the entire pytest suite. The tool that can silently un-close an owner-closed cell was classified as documentation. Verified by running the workflow's own regex:docs/security/asvs-apply-cells.pyscripts/asvs/apply.pyThe hardcoded path pointed at the shared vault checkout that several sessions edit at once, so running it from a worktree rewrote a record the operator wasn't looking at. I hit that this session and worked around it with a patched copy.
--scorecardis now required with no default — the one thing a writer must never guess is which record it is rewriting.Both shipped invariants kept verbatim, and now proved
render()enumerates only what it orders; every other key survives by default. Test: a payload omittingdecision_closed/decision_closed_byleaves both intact. That's the7818991dincident, where an allowlist silently un-closed two owner-closed cells while every gate stayed green — an absentdecision_closedis a validFalse.set(was) - set(now)backstop, mutation-proved.renderis replaced with one that drops thedecision_*keys, reproducing the historical defect in the one function that could reintroduce it. The write must be refused, and the test asserts the refusal names those keys — a non-zero exit isn't evidence on its own, since several guards return 1 and a proof that trips an unrelated one proves nothing.Also proved to fire:
anchor_repairbyte-identity on prose and verdict, the owner-closed rescore refusal, the glyph fail-closed check (which fired for real on 13.3.4 this session), unknown-cell refusal, dry-run-by-default.Two mypy errors were sitting in this file and are fixed — it had never been type-checked, for the reason above.
Not in scope: deleting the vault copy. That's the two-repo consolidation and it needs the mirror (#297) settled first.