docs: require tree_sitter_accuracy_audit.py in the PR checklist - #1762
Merged
squid-protocol merged 1 commit intoAug 16, 2026
Merged
Conversation
Two same-day PRs (#1760, #1761) both ran crucible_check.py and the full test suite, checked off the PR template honestly, and still shipped regressions the Differential Scan's 80-repo corpus diff didn't cover -- caught only by manually running tree_sitter_accuracy_audit.py, which wasn't mentioned anywhere in CONTRIBUTING.md or the PR template. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Contributor
squid-protocol
deleted the
docs/require-tree-sitter-accuracy-audit-in-pr-checklist
branch
August 16, 2026 15:55
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
CONTRIBUTING.md's Differential Scan section and the PR template's checklist only mentionedcrucible_check.py(the ~80-repo golden-master diff) for parsing/engine changes --tree_sitter_accuracy_audit.py(per-language recall/precision vs. tree-sitter ground truth) wasn't mentioned anywhere, even though it's a separate, independent gate that CI runs.crucible_check.pyand the full test suite, honestly checked off the existing PR template, and still shipped regressions the Differential Scan corpus didn't cover: fix(go): extract bodyless function declarations (#1756) #1760 dropped functions with directional-channel return types in Go, fix(prism): stop treating C++ digit separators as char-literal openers #1761 regressed C++found_functions/args_exact_matchand lost classes in a corpus file the PR's own target repo didn't touch. Both regressions were only caught by manually runningtree_sitter_accuracy_audit.py --all --ci, which nothing told either contributor to do.Changes
.github/pull_request_template.md: added a second "Touches parsing/engine logic" checklist item fortree_sitter_accuracy_audit.py --all --ci, explicit that it's a different gate fromcrucible_check.pyand a passing Differential Scan doesn't imply this also passes.CONTRIBUTING.md: added both commands side-by-side in the Differential Scan section with an explanation of why both are required, and updated the "Pass the CI Pipeline Locally" submission step to reference both.Type of change
🤖 Generated with Claude Code