Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension


Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
46 changes: 33 additions & 13 deletions .github/workflows/ci.yml
Original file line number Diff line number Diff line change
Expand Up @@ -160,6 +160,19 @@ jobs:
# relative `(releases/...`, and neither grep form alone finds them all), and a `docs/SECURITY.md`
# route-table row asserted a refusal `DELETE /me/mfa` does not make.
#
# IT RECURRED ON 2026-08-11, AND THIS STEP WAS ALREADY HERE — the list was simply incomplete.
# `tests/test_dast_claims.py` scans documents for prose reading as though the independence gap
# were closed, but was never added, so a docs-only PR (#322) merged green and RED MAIN on the push
# afterwards. The blind mode is worse than a plain gap because of WHO PAYS: the failure surfaces
# on the next PR that touches code, so it is misattributed to whoever opens it. That happened —
# the next code PR inherited a red its author had no part in.
#
# => THE LESSON IS ABOUT THE LIST, NOT THE GATING. A curated allowlist silently omits; nothing in
# a green run says "a doc guard exists that I did not run." When adding a doc-scanning test
# module anywhere in `tests/`, add it HERE in the same commit, and confirm it needs no extras —
# a module that cannot run on `[dev]` alone would red every docs-only PR, which is the failure
# mode the minimal-install note below exists to avoid.
#
# WHY A SEPARATE MINIMAL INSTALL instead of ungating the install below. Unlike the status
# invariant, these modules import the package (`config.settings`, `config.models`,
# `parsing.binary`, `auth.service`) and one imports `fastapi.routing`, so they cannot run on
Expand Down Expand Up @@ -190,20 +203,27 @@ jobs:
run: |
# PRINT WHAT IS SCANNED BEFORE RUNNING IT. A list that silently shrinks is how this class of
# guard goes quiet, and `-rs` names every skip rather than letting one read as a pass.
#
# ONE LIST, USED TWICE. It used to be written out twice — once for the printf and once for
# pytest — which made the "print what you scanned" defence able to LIE: the two copies could
# drift, and then the step prints a module it does not run, or runs one it does not print.
# A defence implemented by duplication defeats itself. The variable is the fix, and the
# cross-check below turns a drift into a hard failure rather than a quiet one.
DOC_GUARDS="tests/test_asvs_file_surface_doc_drift.py tests/test_cloud_phi_hipaa_doc_drift.py
tests/test_crit2_inline_doc_drift.py tests/test_doc_ref_handle.py tests/test_docs_db_grants.py
tests/test_docs_runbooks.py tests/test_docs_security_pathways.py tests/test_security_doc_drift.py
tests/test_security_doc_rate_limits.py tests/test_threat_model_doc_drift.py
tests/test_backlog_status_check.py tests/test_sds_rule_ids_are_stable.py
tests/test_link_resolution.py tests/test_dast_claims.py"
# Every named module must EXIST. A path typo would otherwise make pytest error on an unknown
# file, or — worse under a future -k/--ignore form — silently scan nothing and read as a pass.
for m in $DOC_GUARDS; do
test -f "$m" || { echo "doc-guard list names a missing module: $m"; exit 1; }
done
echo "doc guards, docs-only PR — scanning these modules:"
printf ' %s\n' tests/test_asvs_file_surface_doc_drift.py tests/test_cloud_phi_hipaa_doc_drift.py \
tests/test_crit2_inline_doc_drift.py tests/test_doc_ref_handle.py tests/test_docs_db_grants.py \
tests/test_docs_runbooks.py tests/test_docs_security_pathways.py tests/test_security_doc_drift.py \
tests/test_security_doc_rate_limits.py tests/test_threat_model_doc_drift.py \
tests/test_backlog_status_check.py tests/test_sds_rule_ids_are_stable.py \
tests/test_link_resolution.py
pytest -q -rs \
tests/test_asvs_file_surface_doc_drift.py tests/test_cloud_phi_hipaa_doc_drift.py \
tests/test_crit2_inline_doc_drift.py tests/test_doc_ref_handle.py tests/test_docs_db_grants.py \
tests/test_docs_runbooks.py tests/test_docs_security_pathways.py tests/test_security_doc_drift.py \
tests/test_security_doc_rate_limits.py tests/test_threat_model_doc_drift.py \
tests/test_backlog_status_check.py tests/test_sds_rule_ids_are_stable.py \
tests/test_link_resolution.py
printf ' %s\n' $DOC_GUARDS
echo " ($(printf '%s\n' $DOC_GUARDS | wc -l) modules)"
pytest -q -rs $DOC_GUARDS

# PySide6's offscreen platform plugin needs a few system libraries even
# headless. Linux-only; Windows runners need no equivalent.
Expand Down
2 changes: 1 addition & 1 deletion docs/BACKLOG.md
Original file line number Diff line number Diff line change
Expand Up @@ -3736,7 +3736,7 @@ and scoped this sweep out of itself. Every count above was measured against the

> **AND THE SQL SERVER PROBE HAS NEVER EXECUTED ANYWHERE.** Its T-SQL (`IS_SRVROLEMEMBER` / `IS_ROLEMEMBER` / `HAS_PERMS_BY_NAME`, 17 bound parameters) was never run: no reachable local instance, and the lane correctly declined to go looking for an `sa` password. PostgreSQL **was** exercised live both directions against a real 16.14, including a purpose-made least-privilege negative arm. CI is the SQL Server probe's only coverage. The first live run will settle whether `pyodbc` binds parameters into `IS_SRVROLEMEMBER(?)` in a `FROM`-less `SELECT` at all.

> **What the branch DID legitimately establish, independent of the gate:** the original WIP was **INCOMPLETE, not merely unverified**. It broke `tests/test_webconsole_seam_snapshot.py` (measured RED), its four live DB legs were wired into no workflow step and so executed nowhere while reporting as skips — making the file's own docstring claim that CI was *"a standing positive control"* **false when written** — and `postgres_excess` read four of five role attributes off the principal's own row rather than across assumable roles, contradicting `DEPLOY-SERVER-DB.md` §1.3's shipped promise. Those are real findings and they survive whatever the owner rules.
> **What the branch DID legitimately establish, separately from the gate question:** the original WIP was **INCOMPLETE, not merely unverified**. It broke `tests/test_webconsole_seam_snapshot.py` (measured RED), its four live DB legs were wired into no workflow step and so executed nowhere while reporting as skips — making the file's own docstring claim that CI was *"a standing positive control"* **false when written** — and `postgres_excess` read four of five role attributes off the principal's own row rather than across assumable roles, contradicting `DEPLOY-SERVER-DB.md` §1.3's shipped promise. Those are real findings and they survive whatever the owner rules.

**Cluster:** Security & Compliance. **Priority:** DEMAND-GATE (would be **P2** on score alone).
**Verdict:** **the owner's ruling of record is *"build the runbook fix only; defer the startup
Expand Down
35 changes: 34 additions & 1 deletion tests/test_dast_claims.py
Original file line number Diff line number Diff line change
Expand Up @@ -266,8 +266,18 @@ def test_sds_independent_review_row_is_unchanged(row: str) -> None:
# No file this change introduces may claim the gap is closed
# =====================================================================================================

# The `\b` before the closure verb is load-bearing and was added after this guard REPORTED A NEGATION
# AS AN AFFIRMATION. Without it, `complete` matches inside `INCOMPLETE`, so the sentence
# "...independent of the gate: the original WIP was INCOMPLETE" -- which asserts the OPPOSITE of a
# closure -- reddened `main` as though it claimed the independence gap was closed. The same hole
# accepts "uncovered", "unsatisfied" and "undisclosed": every one of them a word whose meaning is the
# negation of the verb being hunted.
#
# The boundary loses no true positive, because a real closure claim spells the verb as its own word.
# Verified both directions before the change landed, and pinned by
# `test_the_closure_pattern_does_not_read_a_negation_as_a_claim` below.
_CLOSURE_CLAIM = re.compile(
r"(pen ?test|penetration test|independen\w*).{0,60}(complete|satisfied|closed|covered|discharg)",
r"(pen ?test|penetration test|independen\w*).{0,60}\b(complete|satisfied|closed|covered|discharg)",
re.IGNORECASE,
)

Expand Down Expand Up @@ -307,6 +317,29 @@ def test_the_closure_sweep_can_actually_see_a_claim() -> None:
assert not _CLOSURE_CLAIM.search("the independent engagement has not been performed")


def test_the_closure_pattern_does_not_read_a_negation_as_a_claim() -> None:
"""Regression: this guard once reported a NEGATION as an AFFIRMATION and reddened ``main``.

Without a word boundary, ``complete`` matches inside ``INCOMPLETE``, so a sentence asserting the
exact OPPOSITE of a closure tripped the sweep. The live case is the first fixture below -- it is
the real ``docs/BACKLOG.md`` sentence that failed, kept verbatim rather than paraphrased, because a
fixture invented to match the fix is not evidence that the fix covers the fault.

The pre-existing negative above (``has not been performed``) could not have caught this: it passes
because ``performed`` is not in the verb list at all, not because the pattern understands negation.
A guard needs a negative control per FAILURE MODE, not one per test.
"""
assert not _CLOSURE_CLAIM.search(
"independent of the gate:** the original WIP was **INCOMPLETE, not merely unverified"
)
# The same hole accepted every negating prefix of the hunted verbs.
assert not _CLOSURE_CLAIM.search("the independent review left the finding uncovered")
assert not _CLOSURE_CLAIM.search("independent testing left the requirement unsatisfied")
# ...while the affirmations they negate must still bite.
assert _CLOSURE_CLAIM.search("the independent review left the finding covered")
assert _CLOSURE_CLAIM.search("independent testing left the requirement satisfied")


# =====================================================================================================
# Prove the FEATURE-MAP guard catches the rewrite it exists for
# =====================================================================================================
Expand Down
Loading