From 15dea1591a2de987be80b78ce392737d30809dc6 Mon Sep 17 00:00:00 2001 From: Joe Esquibel Date: Sun, 16 Aug 2026 11:48:48 -0400 Subject: [PATCH] docs: require tree_sitter_accuracy_audit.py in the PR checklist Two same-day PRs (#1760, #1761) both ran crucible_check.py and the full test suite, checked off the PR template honestly, and still shipped regressions the Differential Scan's 80-repo corpus diff didn't cover -- caught only by manually running tree_sitter_accuracy_audit.py, which wasn't mentioned anywhere in CONTRIBUTING.md or the PR template. Co-Authored-By: Claude Sonnet 5 --- .github/pull_request_template.md | 1 + CONTRIBUTING.md | 11 ++++++++++- 2 files changed, 11 insertions(+), 1 deletion(-) diff --git a/.github/pull_request_template.md b/.github/pull_request_template.md index 27ef82309..c7b6a0fd6 100644 --- a/.github/pull_request_template.md +++ b/.github/pull_request_template.md @@ -17,6 +17,7 @@ Check only what applies to your change — CI enforces the rest, but running it - [ ] `python tests/tools/audit_check.py` passes (bundles the ruff/mypy/dead-key baseline-gated audits + `ruff format --check`) - [ ] `python -m pytest tests/` passes, or the specific test file(s) for this change (list them below) - [ ] **Touches parsing/engine logic (see above)?** `python tests/tools/crucible_check.py` passes — this is the Differential Scan against the ~80-repo calibrated baseline (see `CONTRIBUTING.md`) +- [ ] **Touches parsing/engine logic (see above)?** `python tests/tools/tree_sitter_accuracy_audit.py --all --ci` passes — a *different* gate from the Differential Scan above: it checks per-language recall/precision against tree-sitter ground truth, and a corpus-wide golden-master diff passing does not mean this also passes (it catches regressions in files the ~80-repo baseline doesn't cover) - [ ] **Changes the CLI's output JSON schema?** Verified on [GitGalaxy.io](https://gitgalaxy.io) or the local Airgap Observatory that 3D rendering still works ## Differential Scan target diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md index 72767d953..f12963091 100644 --- a/CONTRIBUTING.md +++ b/CONTRIBUTING.md @@ -35,6 +35,15 @@ When you submit a Pull Request that alters the blAST engine, your candidate chan * **The Metrics:** The output is assessed strictly on four vectors: **Accuracy, Speed, Utility, and Ethos.** * Only changes that are mathematically and practically *measurably better* across the broader ecosystem will be incorporated into the `main` branch. +**Run both gates locally before opening the PR — they catch different things:** + +``` +python tests/tools/crucible_check.py # the ~80-repo Differential Scan above +python tests/tools/tree_sitter_accuracy_audit.py --all --ci # per-language recall/precision vs. tree-sitter ground truth +``` + +A passing Differential Scan does **not** mean the tree-sitter accuracy audit also passes — it's a separate corpus and a separate comparison (measured recall/precision against real parser output, not a before/after diff), so a change can pass one and quietly regress the other in a file neither the ~80-repo baseline nor your target repo happens to cover. Both are required for any change to `gitgalaxy/core/detector.py`, `prism.py`, or `language_standards.py`. + --- ## 🛠️ Submitting Pull Requests @@ -45,7 +54,7 @@ To ensure your contribution integrates smoothly into the Zero-Trust ecosystem: 2. **Maintain the Contract:** Ensure that any backend changes to the CLI do not break the universal JSON schema contract expected by the frontend Observatory. 3. **Test Visually:** Run your newly generated `_galaxy.json` through either **[GitGalaxy.io](https://gitgalaxy.io)** or your local **Airgap Observatory** to verify that 3D WebGPU rendering and visual physics constraints remain intact. 4. **Document:** If you are adding new language phenotypes or altering the parsing logic, please update the [Language Lens](docs/wiki/02-05-language-lens.md) documentation in the wiki. -5. **Pass the CI Pipeline Locally:** Before pushing, always run `ruff check .` to check for linter errors (like the strict `flake8-bandit` rules which require you to append `# noqa: S101` to all `assert` statements in tests). You should also verify that all `pytest` checks pass. +5. **Pass the CI Pipeline Locally:** Before pushing, always run `ruff check .` to check for linter errors (like the strict `flake8-bandit` rules which require you to append `# noqa: S101` to all `assert` statements in tests). You should also verify that all `pytest` checks pass. If your change touches parsing/engine logic, also run `crucible_check.py` **and** `tree_sitter_accuracy_audit.py --all --ci` (see the Differential Scan section above) — they check different things and both are required. 6. **Submit:** Open a PR with a clear title. Explain the *why* behind your structural changes, not just the *what*. Be sure to include the link to the target repository so we can include it in the Differential Scan. ---