diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml index fa27aa27..71f82e83 100644 --- a/.github/workflows/ci.yml +++ b/.github/workflows/ci.yml @@ -160,6 +160,19 @@ jobs: # relative `(releases/...`, and neither grep form alone finds them all), and a `docs/SECURITY.md` # route-table row asserted a refusal `DELETE /me/mfa` does not make. # + # IT RECURRED ON 2026-08-11, AND THIS STEP WAS ALREADY HERE — the list was simply incomplete. + # `tests/test_dast_claims.py` scans documents for prose reading as though the independence gap + # were closed, but was never added, so a docs-only PR (#322) merged green and RED MAIN on the push + # afterwards. The blind mode is worse than a plain gap because of WHO PAYS: the failure surfaces + # on the next PR that touches code, so it is misattributed to whoever opens it. That happened — + # the next code PR inherited a red its author had no part in. + # + # => THE LESSON IS ABOUT THE LIST, NOT THE GATING. A curated allowlist silently omits; nothing in + # a green run says "a doc guard exists that I did not run." When adding a doc-scanning test + # module anywhere in `tests/`, add it HERE in the same commit, and confirm it needs no extras — + # a module that cannot run on `[dev]` alone would red every docs-only PR, which is the failure + # mode the minimal-install note below exists to avoid. + # # WHY A SEPARATE MINIMAL INSTALL instead of ungating the install below. Unlike the status # invariant, these modules import the package (`config.settings`, `config.models`, # `parsing.binary`, `auth.service`) and one imports `fastapi.routing`, so they cannot run on @@ -190,20 +203,27 @@ jobs: run: | # PRINT WHAT IS SCANNED BEFORE RUNNING IT. A list that silently shrinks is how this class of # guard goes quiet, and `-rs` names every skip rather than letting one read as a pass. + # + # ONE LIST, USED TWICE. It used to be written out twice — once for the printf and once for + # pytest — which made the "print what you scanned" defence able to LIE: the two copies could + # drift, and then the step prints a module it does not run, or runs one it does not print. + # A defence implemented by duplication defeats itself. The variable is the fix, and the + # cross-check below turns a drift into a hard failure rather than a quiet one. + DOC_GUARDS="tests/test_asvs_file_surface_doc_drift.py tests/test_cloud_phi_hipaa_doc_drift.py + tests/test_crit2_inline_doc_drift.py tests/test_doc_ref_handle.py tests/test_docs_db_grants.py + tests/test_docs_runbooks.py tests/test_docs_security_pathways.py tests/test_security_doc_drift.py + tests/test_security_doc_rate_limits.py tests/test_threat_model_doc_drift.py + tests/test_backlog_status_check.py tests/test_sds_rule_ids_are_stable.py + tests/test_link_resolution.py tests/test_dast_claims.py" + # Every named module must EXIST. A path typo would otherwise make pytest error on an unknown + # file, or — worse under a future -k/--ignore form — silently scan nothing and read as a pass. + for m in $DOC_GUARDS; do + test -f "$m" || { echo "doc-guard list names a missing module: $m"; exit 1; } + done echo "doc guards, docs-only PR — scanning these modules:" - printf ' %s\n' tests/test_asvs_file_surface_doc_drift.py tests/test_cloud_phi_hipaa_doc_drift.py \ - tests/test_crit2_inline_doc_drift.py tests/test_doc_ref_handle.py tests/test_docs_db_grants.py \ - tests/test_docs_runbooks.py tests/test_docs_security_pathways.py tests/test_security_doc_drift.py \ - tests/test_security_doc_rate_limits.py tests/test_threat_model_doc_drift.py \ - tests/test_backlog_status_check.py tests/test_sds_rule_ids_are_stable.py \ - tests/test_link_resolution.py - pytest -q -rs \ - tests/test_asvs_file_surface_doc_drift.py tests/test_cloud_phi_hipaa_doc_drift.py \ - tests/test_crit2_inline_doc_drift.py tests/test_doc_ref_handle.py tests/test_docs_db_grants.py \ - tests/test_docs_runbooks.py tests/test_docs_security_pathways.py tests/test_security_doc_drift.py \ - tests/test_security_doc_rate_limits.py tests/test_threat_model_doc_drift.py \ - tests/test_backlog_status_check.py tests/test_sds_rule_ids_are_stable.py \ - tests/test_link_resolution.py + printf ' %s\n' $DOC_GUARDS + echo " ($(printf '%s\n' $DOC_GUARDS | wc -l) modules)" + pytest -q -rs $DOC_GUARDS # PySide6's offscreen platform plugin needs a few system libraries even # headless. Linux-only; Windows runners need no equivalent. diff --git a/docs/BACKLOG.md b/docs/BACKLOG.md index 533c56e2..a2e6954d 100644 --- a/docs/BACKLOG.md +++ b/docs/BACKLOG.md @@ -3736,7 +3736,7 @@ and scoped this sweep out of itself. Every count above was measured against the > **AND THE SQL SERVER PROBE HAS NEVER EXECUTED ANYWHERE.** Its T-SQL (`IS_SRVROLEMEMBER` / `IS_ROLEMEMBER` / `HAS_PERMS_BY_NAME`, 17 bound parameters) was never run: no reachable local instance, and the lane correctly declined to go looking for an `sa` password. PostgreSQL **was** exercised live both directions against a real 16.14, including a purpose-made least-privilege negative arm. CI is the SQL Server probe's only coverage. The first live run will settle whether `pyodbc` binds parameters into `IS_SRVROLEMEMBER(?)` in a `FROM`-less `SELECT` at all. -> **What the branch DID legitimately establish, independent of the gate:** the original WIP was **INCOMPLETE, not merely unverified**. It broke `tests/test_webconsole_seam_snapshot.py` (measured RED), its four live DB legs were wired into no workflow step and so executed nowhere while reporting as skips — making the file's own docstring claim that CI was *"a standing positive control"* **false when written** — and `postgres_excess` read four of five role attributes off the principal's own row rather than across assumable roles, contradicting `DEPLOY-SERVER-DB.md` §1.3's shipped promise. Those are real findings and they survive whatever the owner rules. +> **What the branch DID legitimately establish, separately from the gate question:** the original WIP was **INCOMPLETE, not merely unverified**. It broke `tests/test_webconsole_seam_snapshot.py` (measured RED), its four live DB legs were wired into no workflow step and so executed nowhere while reporting as skips — making the file's own docstring claim that CI was *"a standing positive control"* **false when written** — and `postgres_excess` read four of five role attributes off the principal's own row rather than across assumable roles, contradicting `DEPLOY-SERVER-DB.md` §1.3's shipped promise. Those are real findings and they survive whatever the owner rules. **Cluster:** Security & Compliance. **Priority:** DEMAND-GATE (would be **P2** on score alone). **Verdict:** **the owner's ruling of record is *"build the runbook fix only; defer the startup diff --git a/tests/test_dast_claims.py b/tests/test_dast_claims.py index 2470afbe..76adc52b 100644 --- a/tests/test_dast_claims.py +++ b/tests/test_dast_claims.py @@ -266,8 +266,18 @@ def test_sds_independent_review_row_is_unchanged(row: str) -> None: # No file this change introduces may claim the gap is closed # ===================================================================================================== +# The `\b` before the closure verb is load-bearing and was added after this guard REPORTED A NEGATION +# AS AN AFFIRMATION. Without it, `complete` matches inside `INCOMPLETE`, so the sentence +# "...independent of the gate: the original WIP was INCOMPLETE" -- which asserts the OPPOSITE of a +# closure -- reddened `main` as though it claimed the independence gap was closed. The same hole +# accepts "uncovered", "unsatisfied" and "undisclosed": every one of them a word whose meaning is the +# negation of the verb being hunted. +# +# The boundary loses no true positive, because a real closure claim spells the verb as its own word. +# Verified both directions before the change landed, and pinned by +# `test_the_closure_pattern_does_not_read_a_negation_as_a_claim` below. _CLOSURE_CLAIM = re.compile( - r"(pen ?test|penetration test|independen\w*).{0,60}(complete|satisfied|closed|covered|discharg)", + r"(pen ?test|penetration test|independen\w*).{0,60}\b(complete|satisfied|closed|covered|discharg)", re.IGNORECASE, ) @@ -307,6 +317,29 @@ def test_the_closure_sweep_can_actually_see_a_claim() -> None: assert not _CLOSURE_CLAIM.search("the independent engagement has not been performed") +def test_the_closure_pattern_does_not_read_a_negation_as_a_claim() -> None: + """Regression: this guard once reported a NEGATION as an AFFIRMATION and reddened ``main``. + + Without a word boundary, ``complete`` matches inside ``INCOMPLETE``, so a sentence asserting the + exact OPPOSITE of a closure tripped the sweep. The live case is the first fixture below -- it is + the real ``docs/BACKLOG.md`` sentence that failed, kept verbatim rather than paraphrased, because a + fixture invented to match the fix is not evidence that the fix covers the fault. + + The pre-existing negative above (``has not been performed``) could not have caught this: it passes + because ``performed`` is not in the verb list at all, not because the pattern understands negation. + A guard needs a negative control per FAILURE MODE, not one per test. + """ + assert not _CLOSURE_CLAIM.search( + "independent of the gate:** the original WIP was **INCOMPLETE, not merely unverified" + ) + # The same hole accepted every negating prefix of the hunted verbs. + assert not _CLOSURE_CLAIM.search("the independent review left the finding uncovered") + assert not _CLOSURE_CLAIM.search("independent testing left the requirement unsatisfied") + # ...while the affirmations they negate must still bite. + assert _CLOSURE_CLAIM.search("the independent review left the finding covered") + assert _CLOSURE_CLAIM.search("independent testing left the requirement satisfied") + + # ===================================================================================================== # Prove the FEATURE-MAP guard catches the rewrite it exists for # =====================================================================================================