Skip to content

test: re-run the CodeQL suppression experiment with documented placement - #82

Merged
sumitake merged 3 commits into
mainfrom
dev/claude/codeql-suppression-correct-placement
Jul 30, 2026
Merged

test: re-run the CodeQL suppression experiment with documented placement#82
sumitake merged 3 commits into
mainfrom
dev/claude/codeql-suppression-correct-placement

Conversation

@sumitake

Copy link
Copy Markdown
Owner

Experiment, not a conclusion

PR #80 concluded that inline CodeQL suppressions are not honoured in this repository. That conclusion is unsupported. The run at 0017b3f placed the annotation as a trailing same-line comment:

os.chmod(path, 0o620)  # codeql[py/overly-permissive-file] -- deliberate fixture

CodeQL Python suppression syntax requires a standalone comment line before the alert. One placement was tested and the result was generalised to all placements — the same error shape as concluding a gate is unsatisfiable after missing three of its surfaces (#78).

Caught by an independent cross-family review of the learning-ledger entry that recorded the bad conclusion.

What this tests

# codeql[py/overly-permissive-file]
os.chmod(path, 0o620)

The CodeQL check on this PR is the entire result.

Do not merge on the test suite. 46 tests pass either way; they are not what is being measured.

Verification

  • python3 -m unittest tests.test_claude_transcript_model — 46 tests, OK
  • check_release_consistency.py — OK at 4.6.0 (pre-commit hook)
  • Behaviour unchanged: both fixtures still apply the permissive mode and still assert refusal/skip.

Boundary declaration

  • No provider executor source, raw provider command, credential, private path, downloader, or post-install hook.
  • No native-runtime change.
  • No host-specific preset or provider-specific plugin.

Generated and release surfaces

  • Skill specs and generated SKILL.md unchanged.
  • Marketplaces/manifests unchanged.
  • changelog.d fragment — deliberately absent until the result is known; the user-visible statement depends on the outcome.
  • Version metadata unchanged; test-only.

Review and post-condition

Tier 2. Test-only, no production code, no behaviour change.

Post-condition: the CodeQL check conclusion on this PR, which decides whether #80 needs a correcting follow-up.

author: claude
standing_directives: verify a specification before concluding a mechanism is unavailable; test one variable at a time; do not generalise from a single placement; no production scanning coverage reduced
tier: 2
cross_check: PROCEED-WITH-MODIFICATIONS (codex, openai family, direct route, operator-authorized broker bypass, no broker attestation) - round 2 of the ledger review established that the original experiment tested trailing placement while the documented Python syntax requires a standalone line before the alert; this PR is the corrected counterfactual it asked for
post_condition: CodeQL check conclusion on this PR is the load-bearing result; green means #80 needs a correcting follow-up, red means #80 stands with evidence it previously lacked
mcp_coverage_gap: NONE
contributor_rights: OWNER-AUTHORED
operator_reserved: no

EXPERIMENT, not a conclusion. PR #80 concluded that inline CodeQL suppressions
are not honoured in this repository. That conclusion is unsupported: the run at
0017b3f placed the annotation as a TRAILING same-line comment, while CodeQL's
documented Python suppression syntax requires a standalone comment line before
the alert. One placement was tested and the result generalised to all
placements.

This tests the documented form. If CodeQL is green, the mechanism works, #80's
call-site documentation and changelog are wrong and need correcting, and the
Path.chmod call-shape dependency can be dropped for the honest os.chmod plus a
stated suppression. If CodeQL is red, the negative result finally has the
evidence #80 claimed, and this reverts.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Comment thread tests/test_claude_transcript_model.py Fixed
Comment thread tests/test_claude_transcript_model.py Fixed

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: 27b697aa1d

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread tests/test_claude_transcript_model.py Outdated
sumitake and others added 2 commits July 29, 2026 22:07
PR #80 concluded that inline suppressions are not honoured here, but the run
behind it (0017b3f) used a TRAILING same-line comment, while CodeQL's documented
Python syntax is a standalone comment line before the alert. One placement was
tested and generalised to all -- caught by cross-family review of the learning
entry that recorded the conclusion.

This branch ran the documented placement (27b697a). It also fails: alerts fire
on the annotated lines 173 and 208 with the standalone comment correctly on 172
and 207. So #80's conclusion stands and now has evidence across two placements
rather than one, and no correcting follow-up to #80 is needed.

Keep the Path.chmod workaround and name both runs at the call site, so the next
reader does not spend a third cycle on a placement variant.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Cross-check: the comment called 0017b3f and 27b697a 'runs', but they are commit
SHAs; the CodeQL check runs are 90773472337 and 90784725147. A reader following
the citation would have landed on a commit rather than the evidence.

Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
@sumitake
sumitake merged commit 47445a5 into main Jul 30, 2026
15 checks passed
@sumitake
sumitake deleted the dev/claude/codeql-suppression-correct-placement branch July 30, 2026 05:36
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants