Skip to content

fix(review): evict a manual-review-held sibling from the merge-train gate - #9217

Merged
JSONbored merged 1 commit into
mainfrom
fix/9039-merge-train-hol-stall
Jul 27, 2026
Merged

fix(review): evict a manual-review-held sibling from the merge-train gate#9217
JSONbored merged 1 commit into
mainfrom
fix/9039-merge-train-hol-stall

Conversation

@JSONbored

Copy link
Copy Markdown
Owner

Summary

  • The merge-train FIFO gate (src/review/merge-train.ts, shouldWaitForOlderSiblings) already has
    overlap-scoping (a sibling only blocks when it shares a linked issue or a meaningful changed file)
    and a 24h staleness cap. Neither is the actual gap the confirmed incident (fix(test): add required default field to queue-3 pause commandAuthorization fixture #8735: 57 denials, 5 PRs
    blocked, 4 hours) exposed — overlap-scoping worked exactly as designed (those 5 PRs genuinely
    overlapped fix(test): add required default field to queue-3 pause commandAuthorization fixture #8735), and fix(test): add required default field to queue-3 pause commandAuthorization fixture #8735 was created only ~4h before a human intervened, nowhere near the 24h
    cap. The real gap: the gate has no concept of "this sibling is held for a reason that will never
    self-resolve without a human" — it only ever asked "is this sibling older, overlapping, and not git-
    conflicted," and a PR wearing the manual-review label passed all three, so the gate waited for it
    exactly like an ordinary in-review PR that's still making forward progress.
  • Fix: MergeTrainSibling gained a heldForManualReview field, resolved by the caller
    (agent-action-executor.ts) the SAME way the existing live approve/merge guard already resolves
    "does this PR carry the manual-review label" (ctx.manualReviewLabelnull disables it,
    absent/undefined falls back to AGENT_LABEL_NEEDS_REVIEW). A manual-review-held sibling is now
    evicted from the train exactly like a git-conflicted (dirty) one, instead of counting as "still
    viable to eventually clear on its own."
  • Wait-bound question: no separate/shorter cap added. Once a manual-review hold is evicted outright,
    the only siblings left for the 24h cap to bound are ones genuinely still in CI/AI-review/human-review
    — i.e. actively making progress toward a natural resolution, not administratively frozen. That's the
    same case the 24h cap was already designed for. A shorter cap on top would mostly just race an
    actively-progressing review, trading a hypothetical extra-fast escape hatch for spurious out-of-order
    merges on PRs that were about to clear on their own. Happy to add one if you disagree, but I don't see
    a concrete failure mode it would close that the eviction fix + existing cap don't already cover.
  • Wedge alert: wired a best-effort, flag-gated PagerDuty page when N (5) consecutive merge-train
    denials land against the SAME blocking sibling within an hour — keyed to the blocking PR number
    (owner/repo#merge-train-blocked-by-<N>), not the waiting PR, since the incident denied 5 different
    PR numbers behind the one stuck sibling. Reuses the existing isPagerDutyEnabled /
    triggerPagerDutyIncident helpers verbatim (same min-severity floor + cooldown-via-dedup_key
    ops-wire.ts already relies on for its own "count recent audit rows, page past a threshold" alerting),
    rather than inventing a second alerting mechanism. No-op unless LOOPOVER_ENABLE_PAGERDUTY is set.

Judgment calls (flagging explicitly, not asking mid-task)

Scope

Validation

  • git diff --check
  • npm run actionlint (not run — no workflow files touched)
  • npm run typecheck (npx tsc --noEmit -p tsconfig.json --incremental false, clean)
  • npm run test:coverage locally, scoped: test/unit/merge-train.test.ts (31/31) +
    test/unit/agent-action-executor.test.ts (211/211) = 242/242 passing; v8 coverage on the two
    touched src/** files is 100% branch/line on merge-train.ts and every new line/branch in
    agent-action-executor.ts is covered (the file's only uncovered lines are pre-existing, in an
    unrelated function this PR does not touch).
  • npm run test:workers (not run — no Worker-specific surface touched)
  • npm run build:mcp / npm run test:mcp-pack (not run — no MCP surface touched)
  • npm run ui:openapi:check (not run — no route/schema change)
  • npm run ui:lint / npm run ui:typecheck / npm run ui:build (not run — no UI change)
  • npm audit --audit-level=moderate was not re-run in this pass (no dependency changes)
  • New behavior has unit/integration tests for new branches, fallback paths (see below)

If any required check was skipped, explain why:

  • Full local gate (npm run test:ci) intentionally not run per this task's own instructions — scoped
    to typecheck + the touched test files instead. This PR is backend-only (src/review, src/services,
    test/unit), so the UI/MCP/worker/OpenAPI checks above are not applicable to this diff.

Safety

  • No secrets, wallet details, hotkeys, coldkeys, user PATs, private keys, raw trust scores, private
    rankings, or private maintainer evidence are exposed.
  • Public GitHub text stays sanitized, low-noise, and does not imply compensation guarantees or
    optimization tactics.
  • No auth/cookie/CORS/GitHub App/Cloudflare/session changes.
  • No API/OpenAPI/MCP behavior changed.
  • No UI changes.
  • No public docs/changelog changes (correctly not touching CHANGELOG.md).

UI Evidence

N/A — backend-only change, no visible UI/frontend/docs surface.

Notes

Closes #9039

…gate (#9039)

The merge-train FIFO gate already scopes blocking to overlapping siblings
and caps the wait at 24h, so a head-of-line PR held for manual review was
still treated as "still viable to eventually clear on its own" -- exactly
what let #8735 stall 5 overlapping PRs for 4 hours (57 denials) until a
human merged it by hand. A manual-review hold does not self-clear on any
timer, unlike an actively-reviewing PR, so shouldWaitForOlderSiblings now
evicts a sibling carrying the repo's configured manual-review label the
same way it already evicts a git-conflicted one, instead of waiting for it.

Also wires a bounded, best-effort wedge alert: N consecutive merge-train
denials against the SAME blocking sibling (regardless of mode) now page
via the existing PagerDuty helper once a repo opts in, mirroring how
ops-wire.ts already alerts on a sustained condition from audit-event
counts -- so a wedged train is visible immediately instead of only via
audit rows a human happens to notice.
@superagent-security

Copy link
Copy Markdown
Contributor

Superagent didn't find any vulnerabilities or security issues in this PR.

@JSONbored JSONbored self-assigned this Jul 27, 2026
@JSONbored
JSONbored merged commit b7d7941 into main Jul 27, 2026
3 of 4 checks passed
@JSONbored
JSONbored deleted the fix/9039-merge-train-hol-stall branch July 27, 2026 07:21
@loopover-orb loopover-orb Bot added the gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier. label Jul 27, 2026
@codecov

codecov Bot commented Jul 27, 2026

Copy link
Copy Markdown

❌ 4 Tests Failed:

Tests completed Failed Passed Skipped
22159 4 22155 21
View the top 3 failed test(s) by shortest run time
test/integration/orb-relay.test.ts > POST /v1/orb/relay/pull > REGRESSION (#4995): a DB error inside validateOrbRelayEnrollment's own lookup ALSO returns a clean 503 broker_error, not a framework 500 (the earlier of the two DB-touching calls in this handler)
Stack Traces | 0.0122s run time
AssertionError: expected 500 to be 503 // Object.is equality

- Expected
+ Received

- 503
+ 500

 ❯ test/integration/orb-relay.test.ts:1160:24
test/integration/orb-onboarding.test.ts > Central Orb installation registry routes (/v1/internal/orb/installations) > tolerates a list query that omits results (rows.results ?? [])
Stack Traces | 0.0238s run time
SyntaxError: Unexpected token 'I', "Internal S"... is not valid JSON
 ❯ test/integration/orb-onboarding.test.ts:57:14
test/integration/orb-ingest.test.ts > Orb instance registry routes (/v1/internal/orb/instances) > tolerates a list query that omits results (rows.results ?? [])
Stack Traces | 0.0263s run time
SyntaxError: Unexpected token 'I', "Internal S"... is not valid JSON
 ❯ test/integration/orb-ingest.test.ts:427:14
test/integration/orb-relay.test.ts > POST /v1/orb/relay/register > REGRESSION (#4995): a DB error inside validateOrbRelayEnrollment's own lookup ALSO returns a clean 503 broker_error, not a framework 500 (the earlier of the two DB-touching calls in this handler)
Stack Traces | 0.092s run time
AssertionError: expected 500 to be 503 // Object.is equality

- Expected
+ Received

- 503
+ 500

 ❯ test/integration/orb-relay.test.ts:155:24

To view more test analytics, go to the Test Analytics Dashboard
📋 Got 3 mins? Take this short survey to help us improve Test Analytics.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

orb(merge-train): a blocked head-of-line PR stalls every PR behind it — 5 PRs blocked 4h behind one manual-review label

1 participant