fix(ci+test): the DAST closure guard matched inside INCOMPLETE, and the doc-guard list omitted it - #326
Merged
Merged
Conversation
…nd reddened main
origin/main is RED on
test_dast_claims.py::test_no_file_claims_dast_closes_the_independent_gap.
The trigger is prose I wrote in the #1008 amendment:
docs/BACKLOG.md:3739
'independent of the gate:** the original WIP was **INCOMPLETE'
That sentence asserts the OPPOSITE of a closure. The guard read it as a
claim that the independent-verification gap is closed.
MECHANISM, measured rather than inferred. The pattern was
(pen ?test|penetration test|independen\w*).{0,60}(complete|satisfied|closed|covered|discharg)
with NO word boundary before the verb, so 'complete' matches INSIDE
'INCOMPLETE'. The same hole accepts 'uncovered', 'unsatisfied' and
'undisclosed' -- every one a word whose meaning is the NEGATION of the
verb being hunted. Adding \b loses no true positive, because a real
closure claim spells the verb as its own word. Verified both directions
against the live sentence before the change landed.
I FIXED THE GUARD RATHER THAN THE PROSE, DELIBERATELY. Rewording my
sentence would have turned main green while leaving a guard that fires on
the next person who writes 'independent ... incomplete' -- and it would
have destroyed the only live specimen of the fault. The sentence is
correct English and correct on the facts; the pattern was wrong.
THE PRE-EXISTING NEGATIVE CONTROL COULD NOT HAVE CAUGHT THIS. The file
already asserted that 'the independent engagement has not been performed'
does not match -- but that passes because 'performed' is not in the verb
list at all, not because the pattern understands negation. A guard needs
a negative control PER FAILURE MODE, not one per test. The new test pins
the negation mode with the REAL failing sentence kept verbatim, because a
fixture invented to match the fix is not evidence that the fix covers the
fault.
VERIFIED, and the control had to be run twice because the first attempt
was vacuous. Arming it through a bash heredoc collapsed the backslash in
the word-boundary token, so the mutation never applied and the tests
'passed' over unmutated code -- indistinguishable from a surviving
mutation. My occurrence assert refused the no-op rather than reporting a
pass, which is exactly why that assert exists. Re-armed from a written
file:
boundary REMOVED -> test_the_closure_pattern_does_not_read_a_negation_as_a_claim FAILS
boundary RESTORED -> 18 passed
This branch carries main's docs/BACKLOG.md unchanged, so 18/18 green here
IS the proof that main goes green. Test-only change; no product code
touched.
…ext to a term of art Belt-and-braces, NOT the load-bearing fix. The guard fix in the previous commit already turns main green on its own: 29 passed across test_dast_claims.py and test_link_resolution.py with this wording still in place, and the fixed pattern does not match the original sentence while still matching a real closure claim. Both measured just now. The reword stands on a DIFFERENT reason, and it is the coordinator's and it is fair: in a document where 'independent verification' is a TERM OF ART, writing 'independent of the gate' to mean 'separately from the gate question' is ambiguous to a human reader, not merely to a regex. A reader who knows the term will parse it the wrong way for a beat. So two independent defects, two independent fixes: - the guard read a NEGATION as an AFFIRMATION -> fixed in the pattern - the prose collided with a term of art in context -> fixed here Doing only the reword would have left a guard that fires on the next person who writes 'independent ... incomplete'. Doing only the guard fix would have left a sentence that reads wrong to a careful human. Neither alone is the whole answer, which is why the earlier framing -- 'fix the guard, not the prose' -- was half right rather than right. Verified locally on this branch BEFORE handover, which is the practice this incident exists to establish: test_dast_claims.py and test_link_resolution.py both green, ledger gates clean, 484 items each declaring exactly one status.
…itten twice The docs-only blind mode recurred on 2026-08-11. THE STEP WAS ALREADY THERE; its list was simply incomplete. tests/test_dast_claims.py scans documents for prose reading as though the independent-verification gap were closed. It was never in the list, so a docs-only PR (#322) merged green and RED MAIN on the push afterwards. THE PROOF IS UNPLANTED, WHICH IS THE BEST KIND. This branch is cut from pristine origin/main, which still carries the real defect. Running the new list here reproduces it: FAILED tests/test_dast_claims.py::test_no_file_claims_dast_closes_the_independent_gap 1 failed, 209 passed, 89 skipped So had this module been listed, #322 would have failed PRE-MERGE instead of reddening main. I did not have to invent a fixture; the fault was sitting on main and the step catches it. (The 89 skips are the documented structural vault-absence class, not a gap.) SECOND DEFECT, found while fixing the first: THE LIST WAS WRITTEN TWICE -- once for the printf and once for pytest -- so the step's own 'print what you scanned' defence COULD LIE. The two copies could drift and then it prints a module it does not run, or runs one it does not print. A defence implemented by duplication defeats itself. Now one variable used twice, plus an existence check that fails loudly on a path typo and a printed module count. WHY THE BLIND MODE IS WORSE THAN A PLAIN GAP, recorded in the step: the failure surfaces on the next PR that touches code, so it is MISATTRIBUTED to whoever opens it. That happened -- the next code PR inherited a red its author had no part in. AND THE LESSON IS ABOUT THE LIST, NOT THE GATING. A curated allowlist silently omits, and nothing in a green run says 'a doc guard exists that I did not run.' The step now says: when you add a doc-scanning module anywhere in tests/, add it HERE in the same commit, and confirm it needs no extras -- a module that cannot run on [dev] alone would red every docs-only PR, the failure the minimal-install note exists to avoid. I checked test_dast_claims against that rule before adding it: its only imports are re, pathlib, pytest and scripts.security.dast_auth_sweep, whose own top-level imports are httpx and messagefoundry.auth -- all base install, no extras. SCOPE STATED HONESTLY: a loose scan suggests other doc-reading modules may also be missing, but that scan matches any test mentioning 'docs' and over-counts badly (it flags test_push_guard and test_required_contexts, which are not doc guards). I am NOT claiming the list is now complete -- only that the one module with a demonstrated live failure is in it, and that the next addition has a stated rule to follow. Triaging the rest needs reading, not a grep. Carries the guard fix from w3-fix-dast-guard so this branch is green in either landing order. Verified: 29 passed across test_dast_claims.py and test_link_resolution.py; ci.yml parses and the doc-guard list now appears exactly once.
U+21D2 RIGHTWARDS DOUBLE ARROW raises UnicodeEncodeError on a stock Windows cp1252 console, which is the stated reason in CLAUDE.md section 11. It is a CI comment people read from terminals. Replaced with '=>'. MEASURED BEFORE CHANGING ANYTHING ELSE, because the report also named the em dashes and the section sign and those are a different case: U+2014 EM DASH encodes FINE in cp1252 U+00A7 SECTION SIGN encodes FINE in cp1252 U+21D2 ARROW raises UnicodeEncodeError <- the only real one And on origin/main this file ALREADY carries 134 non-ASCII characters, 127 of them em dashes. Stripping 7 more from my added lines while leaving 127 in place would change nothing a terminal can observe and would make the file internally inconsistent. Section 11's subject is GLYPHS AND EMOJI -- the banner alphabet and pictographs -- and its own text uses 'section' markers throughout; an em dash is punctuation, not a glyph. So: the arrow goes, the em dashes stay, and the reason for each is the same single test rather than a blanket rule applied by eye. Added lines now carry ZERO cp1252-unsafe characters, verified by encoding every non-ASCII character in the diff's added lines rather than by reading them. SEPARATELY, AND NOT MINE TO FIX HERE: origin/main's ci.yml already carries THREE U+2192 arrows, which are cp1252-unsafe by the same test that condemned mine. Pre-existing, unrelated to this change, and reported rather than silently swept in -- a drive-by edit to unrelated lines would make this diff harder to review for no gain. Verified after the change: yaml parses, 29 passed across test_dast_claims.py and test_link_resolution.py.
wshallwshall
enabled auto-merge
August 11, 2026 15:40
wshallwshall
added a commit
that referenced
this pull request
Aug 11, 2026
…l merge Session A's resolve and Session A's measurements. Committed by Session B only because of an ownership constraint, stated here so the next reader does not wonder why the sole writer did not write it: BACKLOG numbers 1202, 1204 and 1205 are allocated to THIS worktree (C:/Users/Scott/Code/MessageFoundry-asvs-ci-hygiene), and ledger_check's owns() compares the claim's `worktree` field casefolded against the repo path and never consults the branch -- so the pre-commit gate refuses this commit from any other path, including from A's own worktree. Nobody used --no-verify. WHY THIS IS NOT A LINE-LEVEL MERGE, which is the thing a future reader most needs so they do not "simplify" it back. Measured on this branch against origin/main 48f8712: live archive namespace in BOTH files origin/main 248 236 484 -- this worktree (42050cd) 281 195 476 0 MECHANICAL keep-both-sides 289 236 525 38 <-- the trap this commit 251 236 487 0 41 items differ between the branch and main. THIRTY-EIGHT OF THEM ARE ITEMS MAIN HAS SINCE ARCHIVED. A keep-both-sides merge resurrects every one into the live ledger while leaving it in the archive: duplicated across both files, the namespace inflated 484 -> 525, and each of those 38 reading OPEN again after being closed. GIT RAISES NO CONFLICT MARKER, because both sides added lines near the tail. Only THREE of the 41 are genuinely new: 1202, 1204, 1205. So both files are taken from main WHOLESALE and exactly those three are spliced back, before #1208 (item headings ascend even though the file as a whole does not). A's script asserts its own result: +3 headings exactly, none lost, no duplicates. THE ARCHIVE HALF IS DELIBERATE AND EXPLICIT, not incidental. The branch had touched BACKLOG-CLOSED.md zero times (main-only 41, branch-only 0), so taking main's copy loses nothing and is what a merge would do anyway -- but doing it as a stated step is what makes the branch self-consistent BEFORE it lands. Without it the tree reads 251/195/446 and a pre-merge namespace check fails against a resolve that is actually correct: a false red aimed at the right answer, which is the kind that gets a correct fix reverted. Re-derived from origin/main at 48f8712 (post-#326) rather than replaying a patch cut against ea75c37, because main moved underneath it. One number corrected in passing: the mechanical figure was first circulated as 286/522. It is 289/525 -- the smaller version credited the mechanical merge with telling the 3 good items from the 38 bad ones, which is precisely the one thing "mechanical" means it cannot do, and it erred in the direction that understates the damage. Verified here independently rather than quoted. Expect test_link_resolution to red branch-locally: main's ledger cites a test file main has and this older branch does not. Not in the spliced items; A simulated the merge and the file is in the merged tree, so it resolves on landing. Taking a newer ledger onto a stale tree necessarily imports references the tree cannot satisfy yet. Commit only -- deliberately not pushed. This worktree's HEAD is two commits ahead of PR #302's head and those two are on no remote, so a push from here would put them on the PR. That is the Coordinator's call, not a side effect of this commit. They are anchored at refs/rescue/unpushed/asvs-ci-hygiene-42050cd4 and 8391699's content is already on origin/main.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
mainis currently red. This is the fix, plus the reason it will not recur. Three defects, each withits own fix -- none alone is the whole answer.
1. The guard read a negation as an affirmation
The closure-claim pattern had no word boundary before the verb group:
completetherefore matches insideINCOMPLETE. A ledger sentence asserting the exact oppositeof a closure tripped a guard whose purpose is to catch closure claims. The same hole accepts
uncovered,unsatisfiedandundisclosed-- each a word whose meaning is the negation of the verbbeing hunted. Adding
\bloses no true positive, because a real closure claim spells the verb as itsown word.
A regression test pins the negation mode using the real failing sentence verbatim -- a fixture
invented to match the fix is not evidence the fix covers the fault.
2. The prose collided with a term of art
Separate from the regex: in a document where independent verification is a term of art, "independent
of the gate" is ambiguous to a human reader, not merely to a matcher. Reworded to "separately from
the gate question".
Doing only the reword leaves a guard that fires on the next person who writes "independent ...
incomplete". Doing only the guard fix leaves a sentence that reads wrong to a careful reader.
The reword is belt-and-braces, not load-bearing, and that is measured so nobody later concludes the
guard fix was unnecessary: the fixed guard does not match the original sentence, still does
match a real closure claim, and a full run with the original wording gives 29 passed.
3. How it reached
main-- the list, not the gatingThe first diagnosis of this was wrong, and it would have produced the wrong fix. It was recorded as a
missed local step. It was not.
The pre-merge doc-guard step already existed at
ci.yml:188, already ungated on docs-only PRs, withits own minimal
[dev]install. It simply did not nametests/test_dast_claims.py.That is also the entire explanation for an asymmetry that otherwise looks like luck:
test_link_resolutionis in the list, so PR #324's link failure was caught pre-merge; the DAST guard was not, so it
reached
main. Same blind mode, two outcomes, decided purely by list membership.The blind mode is worse than a plain gap because of who pays: the failure surfaces on the next PR
that touches code, so it is misattributed to whoever opens it. That happened -- #325 inherited a red its
author had no part in.
A curated allowlist silently omits, and nothing in a green run says "a doc guard exists that I did not
run." The step now states the rule in place: when adding a doc-scanning module anywhere in
tests/,add it here in the same commit, and confirm it runs on
[dev]with no extras.Two supporting changes, both about the step being able to lie:
printf, once forpytest-- so the step's own "printwhat you scanned" defence could drift and print a module it did not run, or run one it did not print.
A defence implemented by duplication defeats itself. One
DOC_GUARDSvariable now feeds both.future
-k/--ignoreform -- silently scan nothing and read as a pass.The list is not claimed complete. A grep for doc-ish test modules over-counts (it flags
test_push_guardandtest_required_contexts, which are not doc guards), and separating the rest needsreading. An honest partial plus a stated rule beats a number nobody trusts.
Scope and verification
Touches
tests/test_dast_claims.py,.github/workflows/ci.yml, and onedocs/BACKLOG.mdsentence. Noproduct code.
The proof is unplanted. The new list was run against pristine
main, which still carries the realdefect: 1 failed, 209 passed. The guard caught the live fault rather than an invented fixture.
Mutation control run twice. The first attempt armed the mutation through a heredoc, which collapsed the
backslash in the boundary token; the replace matched nothing and the tests "passed" over unmutated code
-- indistinguishable from a surviving mutation. An occurrence assertion refused the no-op rather than
reporting a pass. Re-armed from a written file: boundary removed, the regression test fails; boundary
restored, 18 passed.
test_dast_claimsandtest_link_resolutionboth pass locally: 29 passed. YAMLparses and the doc-guard step is intact with the list appearing exactly once.