fix(gate): an unstable merge state must not summon a human - #10123
Merged
Conversation
GitHub reports `mergeable_state: "unstable"` both for a failing non-required check and for checks that are still running. The gate treated both as a manual-review hold, so every PR whose CI had not finished got an enforcement label: the executor denies merge and approve while it is present, and merge-train.ts evicts a held sibling from the train entirely. Section 1b's release condition also tested the same state, which made the label a latch -- applied while CI was red, then not liftable because CI was red. Observed on #10098, stuck until a human removed it by hand. Neither escalation path could clear it: `releasedHolds` contains only `guardrailHit`, and `guardrailEscalationCleared` requires green CI. An unstable state no longer holds. It does not become mergeable either: `wouldApprove` gains its own explicit `!unstableHolds` term (previously implied by the hold), keyed on `unstableHolds` rather than the raw state so #9810's ignored-check dismissal stays approvable, and `wouldMerge` already required `clean`. Both label sites stop emitting -- the merge-autonomy fallback is removed and the disposition ternary returns no label rather than falsely claiming `ready-to-merge`. Closes #10116
Contributor
|
Important 🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨🟨 ⏳ LoopOver is waiting…LoopOver has seen this pull request and is waiting on CI checks to finish before reviewing it. This comment will update once the review runs. 🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed · 🟨 Waiting |
Contributor
|
Superagent didn't find any vulnerabilities or security issues in this PR. |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #10123 +/- ##
==========================================
- Coverage 91.95% 91.08% -0.88%
==========================================
Files 931 931
Lines 113921 113912 -9
Branches 27504 27498 -6
==========================================
- Hits 104757 103754 -1003
- Misses 7863 9052 +1189
+ Partials 1301 1106 -195
Flags with carried forward coverage won't be shown. Click here to find out more.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Closes #10116
GitHub reports
mergeable_state: "unstable"for two different things — a non-required check is failing, and checks are still running. The gate treated both as a manual-review hold, which on a repo meant to run autonomously is wrong in both cases: a pending PR is simply not ready yet, and a failing one has already been answered by CI. Neither wants a maintainer.Observed on #10098 and JSONbored/metagraphed#8838. Neither was a guardrail hit; both sat labelled
manual-review, and #10098 stayed open with finished (red) CI until a human stripped the label by hand.Why it stuck
heldForManualReviewwasheldBy.length > 0 || unstableHolds. That OR is doing three things at once, none of them advisory:merge-train.ts:152filters!sibling.heldForManualReview, so the PR is evicted from the train entirely — which is why these get skipped with no merge-train comment while other PRs merge past them;noManualReviewHoldWanted) also tested!mergeableStateUnstable.That last one is the latch: applied while CI is red, then not liftable because CI is red.
Two things that look like escapes are not.
releasedHoldscontains onlyguardrailHit, so routing guardrail-path PRs to a stronger model withonCleanReview: proceedreleases the guardrail term and nothing else — anunstableHoldsPR stays held however the escalated review comes back. AndguardrailEscalationClearedrequiresreviewGood, which requires green CI, so on a red-CI PR it cannot fire at all.The change
An unstable state stops being a hold. It does not become mergeable.
heldForManualReviewtruefalseheldForUnstableMergeStatetruetrue(unchanged)wouldApprovefalse(via the hold)false(via its own term)wouldMergefalsefalse(unchanged)manual-reviewheldForManualReview = heldBy.length > 0— the declaredMERGE_HOLD_INPUTStable becomes the whole definition, soheldByandheldForManualReviewfinally agree. They did not before, which is how these holds reached the ledger with no cause underreason_code: "success".wouldApprovegains an explicit&& !unstableHolds. Load-bearing: it used to ride on the hold term, and dropping that without restating it here would let approve fire while merge self-suppressed — exactly fix(review): keep a fetched file distinguishable from an omitted one under thin budget #8711. Keyed onunstableHolds, notmergeable === "unstable", so fix(ops): scope the PagerDuty cooldown to rows that actually paged #9810's ignored-check dismissal stays approvable. (My first attempt used the blanket test and its own test caught it.)null— neithermanual-reviewnorready-to-merge, since promising "ready" on a PR the merge self-suppresses on is the other half of fix(review): keep a fetched file distinguishable from an omitted one under thin budget #8711. Removing only one would have fixed nothing in production:review_state_labelis not in the Orb's autonomy map, so 1f is the site that actually fired.!mergeableStateUnstablecomes out ofnoManualReviewHoldWanted, releasing the latch on labels the planner applied.mergeUnstableHoldReason/mergeUnstableHoldCommentare deleted — with no hold there is no action to hang them on (there is no comment-onlyAgentActionClass).The contributor still learns the state: the unified comment refuses "safe to merge" via
mergeStateHeld, and the Checks tab names the culprits — always current, unlike a bot comment posted once per pass on every in-flight PR.Verification
check-*sweep green (dead-exports,dead-source-files,import-specifiers,checkers-wired,typecheck).|| unstableHoldsonheldForManualReview&& !unstableHoldsfromwouldApproveready-to-merge!mergeableStateUnstablein the release condition#8758tests are rewritten rather than deleted, one per label site, so each removal is pinned independently.Follow-up
#10117 —
deriveUnifiedStatusstill renders this state's comment as "Manual Review". That is text, blocks nothing, and the same"held"status servesdirty/behindplus five unrelated paths where a human genuinely is required, so retitling it needs a status-vocabulary decision rather than a one-liner here.Confirmed against the production ledger
Not inferred from the code — read out of
decision_recordson the Orb.#10098(contributor PR, files undersrc/selfhost/**, matching no guardrail glob):Eleven holds in 33 minutes, one per check update.
reason_code: successwithfindings_count: 0means the gate passed andreviewGoodwas true — which excludes themanualHoldReasonfallback (it requires!reviewGood) and the guardrail path (guardrail_holdcarries its own reason code).reviewGood && unstableis section 1f exactly. JSONbored/metagraphed#8838 shows the identical row.The reason code is
successbecauseheldBywas empty whileheldForManualReviewwas true — the disagreement this PR removes. Over the last week on the Orb:success|holdis the largest hold bucket on the box, ahead of every named cause.