Skip to content

feat(ci): enforce that dispatch gates can say why they suppressed work (#9003) - #9583

Merged
JSONbored merged 1 commit into
mainfrom
feat/dispatch-gate-reason-check-9003
Jul 28, 2026
Merged

feat(ci): enforce that dispatch gates can say why they suppressed work (#9003)#9583
JSONbored merged 1 commit into
mainfrom
feat/dispatch-gate-reason-check-9003

Conversation

@JSONbored

Copy link
Copy Markdown
Owner

Closes #9003

Summary

The #9003 invariant: every decision — and every non-decision — records a reason. "Nothing happened and nothing says why" must be structurally impossible. It came out of the 2026-07-26 restart investigation, where a forceAiReview: true pass completed with no fresh review and zero audit events explaining why (#9000), costing hours of elimination — while the disposition lane, which names every hold via agentHoldAuditDetail, answered the same class of question in minutes.

Most of #9003 was already built — this PR adds the piece that keeps it true

Verified before writing anything:

What was missing is enforcement. The issue is explicit about the form: "a bare-boolean gate on a dispatch path should fail a lint/unit convention test, the same way no-direct-octokit enforces the write chokepoint."

scripts/check-dispatch-gate-reasons.ts is that test. It scans the dispatch-path modules for exported gate-shaped functions returning bare boolean and requires each to either return its reason with the decision (the evaluateVisualVisionGate shape) or be allowlisted naming its paired resolver (the shouldRequirePublicAiReviewForAdvisory shape). Both existing conventions pass as-is. What fails is exactly the shape that produced #9000.

It found the real one immediately

First honest run flagged shouldStartAiReviewForAdvisorythe exact gate from the #9000 incident. Rather than silence it, I verified end-to-end that both of its false paths are now named by the caller (the hard gate via resolvePublicAiReviewGateSkipReason; the reputation skip via maybeAddReputationSkipHold), and its allowlist entry states that mechanism. An allowlist entry here is a checkable claim, not an escape hatch.

Two of my own bugs, caught by the fixture tests while writing this

Recorded in the file because they're instructive:

  1. The return-type regex was anchored to a line start, so it only matched multi-line signatures — the whole check was passing vacuously, including against the real tree. A checker that can't fail is worse than none; the fixture tests are what caught it.
  2. A fixed-width signature slice read past a gate's own body into its neighbour's ): boolean {, wrongly flagging a reason-returning gate. Now bounded by the function's own opening brace, paren-depth-aware so an inline object-type parameter doesn't end the signature early.

Also deliberately rejected: "a resolve*SkipReason anywhere in the module exempts it" — one resolver would exempt every future gate beside it, the same neighbour false negative check-regate-sort-key.ts had to kill with brace-bounding. Pairing must be stated per gate.

Scope

Deliberately the named dispatch-path modules, not all of src/** — the rule is about gates that suppress a review or an action, and applying it to every boolean helper would bury the few that matter (the same scoping argument as inert-config.ts). Deliverables 3 and 4 of #9003 (the .catch(() => undefined) completeness pass; per-gate forced-suppression audit tests) are follow-on work the issue lists separately; this PR lands the enforceable convention that was its acceptance criterion — "CI enforces the typed-reason convention on the dispatch paths."

Testing

10 direct tests: both formatting shapes (the #9541 lesson — a family written two ways is a family a grep half-sees), exact-name allowlisting, the neighbour non-exemption, absent modules, stable output ordering, and the real tree staying green. Wired into test:ci and the validate-code job.

#9003)

The #9003 invariant: every decision -- and every NON-decision -- records a reason.
"Nothing happened and nothing says why" must be structurally impossible. It came out
of the 2026-07-26 restart investigation, where a forceAiReview pass completed with no
fresh review and zero audit events explaining why (#9000), costing hours of
elimination -- while the disposition lane, which names every hold, answered the same
class of question in minutes.

Much of the issue is already built: #9000's fix landed resolvePublicAiReviewGateSkipReason
and the audit on the suppress branch, the vision gates return typed
{ run, reason } results, and the reputation skip pushes a named finding. What was
missing is the piece that keeps it true: the issue asks that "a bare-boolean gate on a
dispatch path should fail a lint/unit convention test, the same way no-direct-octokit
enforces the write chokepoint."

scripts/check-dispatch-gate-reasons.ts is that test. It scans the dispatch-path
modules for exported gate-shaped functions (should*/evaluate*Gate*/resolve*Gate*)
returning bare boolean, and requires each to either return its reason with the
decision, or be allowlisted naming its paired resolver. Both existing conventions
pass as-is; what fails is the shape that produced #9000.

The checker found one real instance immediately: shouldStartAiReviewForAdvisory --
the exact gate from the #9000 incident. Verified end-to-end that BOTH of its false
paths are now named by the caller (the hard gate via resolvePublicAiReviewGateSkipReason,
the reputation skip via maybeAddReputationSkipHold's ai_review_inconclusive finding),
so its allowlist entry states that mechanism rather than dodging it.

Two of my own bugs were caught by the fixture tests while writing this, both worth
recording in the file:
- the return-type regex was anchored to a line start, so it only matched multi-line
  signatures -- the whole check was passing VACUOUSLY, including against the real tree
- a fixed-width signature slice read past a gate's own body into its neighbour's
  `): boolean {`, wrongly flagging a reason-returning gate. Now bounded by the
  function's own opening brace, paren-depth-aware for inline object-type params.

Also deliberately rejected: "a resolve*SkipReason anywhere in the module exempts the
module" -- one resolver would exempt every future gate beside it, the same neighbour
false negative check-regate-sort-key.ts had to kill with brace-bounding. Pairing must
be stated per gate.

10 direct tests, including both formatting shapes, exact-name allowlisting, the
neighbour non-exemption, and the real tree staying green. Wired into test:ci and the
validate-code job.
@JSONbored JSONbored self-assigned this Jul 28, 2026
@loopover-orb

loopover-orb Bot commented Jul 28, 2026

Copy link
Copy Markdown
Contributor

Warning

⏸️ LoopOver review result - manual review recommended

Review updated: 2026-07-28 18:59:36 UTC

4 files · 1 AI reviewer · no blockers · CI green · clean

⏸️ Suggested Action - Manual Review

  • Touches a guarded path — held for manual review: This PR changes guardrail-protected path(s): .github/workflows/ci.yml (matched .github/workflows/**).

Review summary
This adds `scripts/check-dispatch-gate-reasons.ts`, a static-analysis lint that scans three named dispatch-path modules for exported gate-shaped functions (`should*`, `evaluate*Gate*`, `resolve*Gate*`) returning a bare `boolean`/`Promise<boolean>` with no paired reason, and fails CI unless the gate either returns a reason-carrying object or is named in `ALLOWED_BARE_BOOLEAN` with a stated justification. It's wired into `package.json` and `.github/workflows/ci.yml`, and the accompanying test file exercises the regex/brace-matching edge cases (multi-line signatures, async, allowlist-by-exact-name, no-resolver-elsewhere-exempts) plus a real run against the actual dispatch modules, which is a genuine (not fabricated) integration check. The PR closes the issue it references (#9003), is narrowly scoped to tooling/CI/tests with no unrelated changes, and CI is green on this commit.

Nits — 6 non-blocking
  • scripts/check-dispatch-gate-reasons.ts:112 — the `signatureOf` paren-depth loop nests to depth 5, per the external review note; a small early-return or helper extraction would flatten it without changing behavior.
  • The `DISPATCH_PATH_MODULES` list (scripts/check-dispatch-gate-reasons.ts) is a hardcoded 3-file allowlist that must be manually updated whenever a new dispatch-path module is introduced — worth a comment pointing at where such additions should be considered, though the existing header already partially covers this.
  • `GATE_DECLARATION` requires the function name immediately followed by `(`, so a generic gate like `function evaluateFooGate<T>(...)` would silently escape the check — confirm no current or near-term dispatch gate uses type parameters.
  • Consider a follow-up test asserting that adding an entry to `ALLOWED_BARE_BOOLEAN` for a name that doesn't exist in any scanned module doesn't silently pass (i.e., the allowlist can't accumulate dead entries unnoticed) — minor robustness, not required for merge.
  • If new dispatch-relevant modules are anticipated soon, consider documenting in CONTRIBUTING or the script header the process for adding to `DISPATCH_PATH_MODULES`, so the allowlist doesn't drift out of sync the way `resolve*SkipReason`-anywhere heuristic once did.
  • Touches a guarded path — held for manual review — A maintainer must review and merge this change.

Decision drivers

  • ✅ Code review — No blockers (1 reviewer)
  • ⚠️ Gate result — Not blocking (Advisory; not blocking this PR.)
Context & advisory signals — never blocks the verdict
Signal Result Evidence
Linked issue ✅ Linked #9003
Related work ✅ No active overlap found No same-issue or scoped active PR overlap found.
Change scope ✅ 20/20 Low review scope from cached public metadata (1 linked issue).
Validation posture ✅ 25/25 PR body includes validation/test evidence.
Contributor workload ✅ 10/10 Author activity: 14 registered-repo PR(s), 13 merged, 300 issue(s).
Contributor context ✅ Confirmed Gittensor contributor JSONbored; Gittensor profile; 14 PR(s), 300 issue(s).
Improvement ✅ Minor risk: clean · value: minor
Linked issue satisfaction

Partially addressed
The PR adds a real, tested CI lint (deliverable 1's convention-test ask) that catches bare-boolean gates on a small named set of dispatch modules, and argues the other deliverables were already landed elsewhere — but it does not itself add the error-path/.catch() instrumentation completeness pass (deliverable 3) or the audit-coverage test that forces each gate to suppress and asserts a reason even

Review context
  • Author: JSONbored
  • Role context: owner (maintainer lane)
  • Public audience mode: oss maintainer
  • Lane context: Repository is registered but has no active allocation in the current snapshot.
  • Public profile languages: not available
  • Official Gittensor activity: 14 PR(s), 300 issue(s).
  • PR-specific overlap: none found.
Contributor next steps
  • Start here: Treat this as maintainer-lane context rather than normal contributor-lane activity.
  • Then work through the remaining 1 step in the Signals table above.
Signal definitions
  • Related work = same linked issue, overlapping active PRs, or title/path similarity.
  • Change scope = cached public metadata such as size labels, draft state, and review-burden hints.
  • Validation posture = whether the PR provides enough public validation/test evidence for maintainer review.
  • Contributor workload = public contributor activity and cleanup pressure, not a repo-wide quality failure.
  • Contributor context = public GitHub/Gittensor identity context; non-Gittensor status is not a blocker.
🧪 Chat with LoopOver

Ask LoopOver a question about this PR directly in a comment — grounded only in the same cached, public-safe facts shown above, never a new claim.

  • @loopover ask &lt;question&gt; answers contribution-quality Q&A with source citations and freshness.
  • @loopover chat &lt;question&gt; answers in natural prose from cached decision-pack facts via local inference (maintainer/collaborator; read-only).
  • A plain-language @loopover mention with a real question is routed to the closest matching read-only command automatically — no exact syntax required.

Full command reference: https://loopover.ai/docs/loopover-commands

🧪 Experimental — new and may change.

Decision record
  • action: hold · clause: guardrail_hold
  • config: 1530a87269a2f194bf39427bf7c72d42e5e323fb831691bebb668e9574a534ab · pack: oss-anti-slop · ci: passed
  • record: 66d7bdb3c1ff7a3efe8f807c82fdd496ed2e6d9b210b08fe16ca40c5ab719d12 (schema v5, head ecb5060)

🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed


💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →.

Checked by LoopOver, a quiet PR intelligence layer for OSS maintainers.

  • Re-run LoopOver review

@superagent-security

Copy link
Copy Markdown
Contributor

Superagent didn't find any vulnerabilities or security issues in this PR.

@cloudflare-workers-and-pages

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Preview URL Updated (UTC)
✅ Deployment successful!
View logs
loopover-ui ecb5060 Commit Preview URL

Branch Preview URL
Jul 28 2026, 06:39 PM

@codecov

codecov Bot commented Jul 28, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 89.84%. Comparing base (d30c6c2) to head (ecb5060).
⚠️ Report is 2 commits behind head on main.

Additional details and impacted files
@@           Coverage Diff           @@
##             main    #9583   +/-   ##
=======================================
  Coverage   89.84%   89.84%           
=======================================
  Files         875      875           
  Lines      110977   110977           
  Branches    26403    26403           
=======================================
  Hits        99702    99702           
  Misses       9993     9993           
  Partials     1282     1282           
Flag Coverage Δ
backend 95.62% <ø> (ø)
control-plane 99.86% <ø> (ø)
rees 89.62% <ø> (ø)

Flags with carried forward coverage won't be shown. Click here to find out more.

@codecov

codecov Bot commented Jul 28, 2026

Copy link
Copy Markdown

Bundle Report

Bundle size has no change ✅

@loopover-orb loopover-orb Bot added the manual-review Gittensor contributor context label Jul 28, 2026
@JSONbored
JSONbored merged commit 1ee2293 into main Jul 28, 2026
9 checks passed
@JSONbored
JSONbored deleted the feat/dispatch-gate-reason-check-9003 branch July 28, 2026 18:59
loopover-orb Bot pushed a commit that referenced this pull request Jul 28, 2026
…erasing them (#9000) (#9584)

The lost-click half of #9000, completing the issue (the skip-auditing half landed
earlier; its enforcement went in as #9583).

THE DEFECT. Three checkbox states looked identical to a maintainer: processed (panel
republished, box reset), deferred for CI (box stays ticked, honored later via the
#7626 pending marker), and DELIVERY LOST (box stays ticked, nothing recorded, nothing
will ever happen -- three such losses observed live on #8972, 2026-07-26). Only the
third is broken, and it is unrecoverable from our own state alone: the tick exists
only in the live comment body, because the webhook that would have told us about it
never became a job. Worse, the next panel republish -- whose renderer emits its
checkboxes unticked unconditionally -- would OVERWRITE the ticked box, erasing the
only evidence of the click on the exact pass best placed to honor it.

THE FIX, at zero additional API cost. createOrUpdateIssueCommentWithMarker already
fetches the existing comment for its marker search; it now returns that pre-overwrite
body. Every panel republish (webhook pass, re-gate sweep, backlog convergence) doubles
as the detector: a ticked rerun box in the body being replaced, with no recent
processing and no pending marker, IS a lost click. Recovery then:

  - records github_app.pr_panel_retrigger_recovered (the named reason #9003 demands,
    and the loop guard against re-detection),
  - persists the #7626 pending marker, so the next review pass consumes it as
    forceAiReview exactly like a deferred click, and
  - enqueues an agent-regate-pr job immediately (with the #9499 prCreatedAt sort key)
    -- a PR with a lost click is precisely the PR that cannot count on a natural pass.

No new sweep, no new persistence: the issue's "within one sweep interval" acceptance
is bounded by the republish cadence that already exists.

FALSE-POSITIVE PROOFING, the part that took the design care:

  - A pass that IS the retrigger (forceAiReview) skips detection -- overwriting its
    own tick is the receipt acknowledgement, not a loss.
  - A recently-processed retrigger (actor-agnostic audit lookup, new
    countAuditEventsForTargetSince -- hasAuditEventForDelivery is deliberately
    actor+delivery-keyed, the wrong shape when the delivery never existed) means the
    delivery raced the pass rather than being lost.
  - A DEFERRED click is guarded by ordering, not luck: the retrigger handler now marks
    the pending marker BEFORE its readiness check (consuming it right back on the
    immediate path), so the CI-wait placeholder inside that very pass sees the marker
    and stands down. A crash between mark and consume degrades to one extra forced
    review -- the safe direction for an explicit user click.

The #7626 sibling test's "never creates a pending marker" assertion is updated to the
invariant it documents ("no pending INTENT survives the immediate path") rather than
its old mechanism (key absence), since consumption can leave the one-shot consumed
sentinel behind on adapters without a real delete.

TESTS. Eight in the new suite: the #8972 regression (natural pass recovers the tick:
event + marker + job with the sort key), full end-to-end (the recovery job's pass
consumes the marker, spends exactly one fresh AI review, and does not re-diagnose its
own republish), the raced-delivery guard, the deferred-click guard (CI genuinely
pending, marker untouched afterward), the no-head ghost PR (marker guard skipped, job
omits the optional sort key), unticked and first-publish no-ops, and the empty
event-type-list guard. Plus the three github-comments assertions extended to pin
previousBody, including that it is the POSTED body, not the re-render, on the
byte-identical skip path. Changed lines: 0 uncovered statements, 0 uncovered branches.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

manual-review Gittensor contributor context

Projects

None yet

Development

Successfully merging this pull request may close these issues.

orb(observability): invariant — every decision or non-decision records a named reason; no silent gates on dispatch paths

1 participant