Skip to content

Commit 35fceed

Browse files
authored
fix(gate): lift the bot's own manual-review hold, and stop drafts wedging the merge train (#9942)
Two independent reasons an autonomous queue stalls with no human in it, both found on live PRs. 1) The manual-review label was a ONE-WAY LATCH. The sibling-label cleanup deliberately refuses to touch it, because the same label string is also a maintainer's manual freeze and the planner had no way to tell them apart -- and silently undoing a human's freeze is far worse than leaving a stale hold. That caution was right; the missing piece was provenance. Seen on #9935: the label went on when a finding fired against a stale head, a rebase removed the cause, a fresh escalated review returned ZERO findings, and the PR was mergeable, green and clean -- and still would not merge: agent.action.merge | denied | manual-review label "manual-review" is present on the live PR -- merge not executed It took a human removing the label by hand, in the mode that is supposed to have no human in it. The planner now records the head SHA and reason when IT applies the label, and may lift it once manualHoldReason is null (no reason wants a hold this pass, so a label applied for reason A can never be lifted by reason B clearing). A label with NO provenance -- a human's, or one predating this -- is never touched. That asymmetry is the whole safety property. 2) A draft sibling wedged the merge train. The train already evicts conflicted and manual-review-held siblings on the principle that waiting for a PR which is not trying to reach merge buys nothing the 24h cap does not already bound far worse. A DRAFT is the strongest form of that signal: GitHub itself refuses to merge one, so it is not one merge away from landing, it is one author action away from being eligible at all. The production shape: a maintainer PR with red CI cannot be auto-closed (maintainers are exempt on purpose, so they can iterate), so it stays open and overlapping for as long as the fix takes -- holding every newer overlapping PR behind it, green and approved, for up to the full 24 hours. Marking it draft is the author saying "skip me", and the train now understands that. Eviction removes ONE PR from the queue; it does not disable the gate. A real older sibling behind the draft still holds the line.
1 parent bae7b2f commit 35fceed

12 files changed

Lines changed: 334 additions & 1 deletion
Lines changed: 19 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,19 @@
1+
-- #9939: who applied the manual-review label, so the bot can lift what the bot applied.
2+
--
3+
-- The label is OVERLOADED: it is both the planner's own "held for a human" disposition and a maintainer's
4+
-- manual safety freeze. agent-actions.ts therefore refuses to auto-remove it while it clears every sibling
5+
-- disposition label, because it has no way to tell the two apart -- and auto-removing a human's deliberate
6+
-- freeze would be far worse than leaving a stale one. That caution is correct and stays.
7+
--
8+
-- The cost of having no provenance is a one-way latch. Observed on #9935: applied when a finding fired
9+
-- against a stale head, then never lifted after a rebase removed the cause and a fresh escalated review
10+
-- returned zero findings. The PR was mergeable, green and clean, and still refused to merge until a human
11+
-- removed the label by hand -- in the mode that is meant to have no human in it.
12+
--
13+
-- Recording the head SHA the PLANNER applied it at (and the reason it applied it FOR) makes the distinction
14+
-- decidable: bot-applied labels become liftable once that specific reason clears, while a label with no
15+
-- provenance row -- a human's, or one applied before this column existed -- keeps exactly today's behaviour
16+
-- and is never touched automatically. Nullable with no backfill for precisely that reason: absence means
17+
-- "not ours to lift", which is the safe reading for every pre-existing label.
18+
ALTER TABLE pull_requests ADD COLUMN manual_review_label_applied_sha TEXT;
19+
ALTER TABLE pull_requests ADD COLUMN manual_review_label_applied_reason TEXT;

src/db/repositories.ts

Lines changed: 32 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -4731,6 +4731,36 @@ export async function markPullRequestVisualCaptureUnobtainable(env: Env, fullNam
47314731
.where(and(eq(pullRequests.repoFullName, fullName), eq(pullRequests.number, number), eq(pullRequests.headSha, headSha)));
47324732
}
47334733

4734+
/** #9939: record that the PLANNER applied the manual-review label at `headSha`, for `reason`.
4735+
*
4736+
* This is the provenance that makes the label liftable. The same label string is ALSO how a maintainer
4737+
* freezes a PR by hand, and agent-actions.ts rightly refuses to auto-remove it without knowing which it is
4738+
* looking at -- silently undoing a human's freeze is far worse than leaving a stale hold. Written only on
4739+
* the planner's own add, so a human-applied label never acquires provenance and is never auto-removed.
4740+
*
4741+
* Scoped to headSha like its visual-capture siblings: a new commit is a new decision, and a hold recorded
4742+
* against an older head should not license removing a label on a head nobody has re-evaluated. */
4743+
export async function markPullRequestManualReviewLabelApplied(env: Env, fullName: string, number: number, headSha: string, reason: string): Promise<void> {
4744+
const db = getDb(env.DB);
4745+
await db
4746+
.update(pullRequests)
4747+
.set({ manualReviewLabelAppliedSha: headSha, manualReviewLabelAppliedReason: boundedString(reason, 300), updatedAt: nowIso() })
4748+
.where(and(eq(pullRequests.repoFullName, fullName), eq(pullRequests.number, number), eq(pullRequests.headSha, headSha)));
4749+
}
4750+
4751+
/** #9939: clear the provenance once the label is gone (removed by the planner, or by a maintainer).
4752+
*
4753+
* Leaving it behind would let a LATER human-applied label inherit the bot's provenance and become
4754+
* auto-removable -- exactly the override this whole mechanism exists to prevent. Not scoped to headSha: the
4755+
* label is gone regardless of which head it was applied at. */
4756+
export async function clearPullRequestManualReviewLabelProvenance(env: Env, fullName: string, number: number): Promise<void> {
4757+
const db = getDb(env.DB);
4758+
await db
4759+
.update(pullRequests)
4760+
.set({ manualReviewLabelAppliedSha: null, manualReviewLabelAppliedReason: null, updatedAt: nowIso() })
4761+
.where(and(eq(pullRequests.repoFullName, fullName), eq(pullRequests.number, number)));
4762+
}
4763+
47344764
/** Release the retry latch: the retry chain that justified it has ended without a successful capture, so the
47354765
* screenshotTableGate must stop deferring and evaluate the evidence actually present.
47364766
*
@@ -7441,6 +7471,8 @@ function toPullRequestRecordFromRow(row: typeof pullRequests.$inferSelect): Pull
74417471
visualCaptureSatisfiedSha: row.visualCaptureSatisfiedSha,
74427472
visualCaptureRetryPendingSha: row.visualCaptureRetryPendingSha,
74437473
visualCaptureRetryPendingAt: row.visualCaptureRetryPendingAt,
7474+
manualReviewLabelAppliedSha: row.manualReviewLabelAppliedSha,
7475+
manualReviewLabelAppliedReason: row.manualReviewLabelAppliedReason,
74447476
screenshotTablePresenceSatisfied: parseJson<{ headSha: string; evidenceFingerprint: string } | null>(row.screenshotTablePresenceSatisfiedJson, null),
74457477
};
74467478
}

src/db/schema.ts

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -494,6 +494,12 @@ export const pullRequests = sqliteTable(
494494
// a sha with no timestamp reads as EXPIRED, which is both honest (the row predates the column) and what
495495
// releases the PRs already stuck behind one. loopover-computed, written with the sha, cleared with it.
496496
visualCaptureRetryPendingAt: text("visual_capture_retry_pending_at"),
497+
// #9939: provenance for the manual-review label -- the head SHA the PLANNER applied it at, and the hold
498+
// reason it applied it FOR. The label is overloaded (bot disposition AND maintainer freeze), so without
499+
// this the planner cannot lift its own stale hold without risking overriding a human's. NULL means "not
500+
// ours" -- a human-applied label, or one predating this column -- and is never auto-removed.
501+
manualReviewLabelAppliedSha: text("manual_review_label_applied_sha"),
502+
manualReviewLabelAppliedReason: text("manual_review_label_applied_reason"),
497503
// #9881: the head SHA at which the bot PROVED visual capture is structurally unobtainable for this repo --
498504
// the deployments read succeeded, found none at all, and the poll budget is spent. The screenshot-table
499505
// gate degrades its CLOSE to advisory for that head rather than closing a PR over evidence no

src/queue/processors.ts

Lines changed: 3 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -2877,6 +2877,9 @@ function buildAgentMaintenancePlanInput(args: {
28772877
guardrailEscalationModel: settings.guardrailEscalationModel ?? null,
28782878
guardrailEscalationProvider: settings.guardrailEscalationProvider ?? null,
28792879
manualReviewLabel: settings.manualReviewLabel,
2880+
// #9939: provenance for the label above -- non-null only when the PLANNER applied it, which is what
2881+
// lets a later pass lift its own stale hold without ever touching a maintainer's manual freeze.
2882+
manualReviewLabelAppliedSha: pr.manualReviewLabelAppliedSha,
28802883
readyToMergeLabel: settings.readyToMergeLabel,
28812884
changesRequestedLabel: settings.changesRequestedLabel,
28822885
migrationCollisionLabel: settings.migrationCollisionLabel,

src/review/merge-train.ts

Lines changed: 16 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -49,6 +49,18 @@ export type MergeTrainSibling = {
4949
* processors.ts) rather than inventing a second, possibly-drifting definition here. Absent/undefined ⇒ not
5050
* held (unchanged from before this field existed). */
5151
heldForManualReview?: boolean | undefined;
52+
/** True when this sibling is a DRAFT (#9939). The strongest possible "not trying to merge" signal there is:
53+
* GitHub itself refuses to merge a draft, so a draft sibling is not one merge away from landing, it is one
54+
* AUTHOR ACTION away from even being eligible. Same reasoning that evicted the manual-review hold above,
55+
* only more so -- a manual-review hold at least describes a PR someone might unblock, while a draft is the
56+
* author's own declaration that it is not ready.
57+
*
58+
* This is the head-of-line case that hurt in production: a maintainer PR with red CI cannot be auto-closed
59+
* (maintainers are exempt, deliberately, so they can iterate), so it stays open and overlapping for as long
60+
* as the fix takes -- holding every newer overlapping PR behind it for up to the full 24h cap even when
61+
* those are green and approved. Marking it draft now says "skip me" in a way the train understands.
62+
* Absent/undefined ⇒ not a draft (unchanged from before this field existed). */
63+
isDraft?: boolean | undefined;
5264
};
5365

5466
/** How long an older, OVERLAPPING sibling can hold up a newer one before it's excluded from blocking (24
@@ -126,6 +138,10 @@ export function shouldWaitForOlderSiblings(input: ShouldWaitForOlderSiblingsInpu
126138
.filter((sibling) => sibling.number !== thisPrNumber)
127139
.filter((sibling) => sibling.mergeableState !== "dirty")
128140
.filter((sibling) => !sibling.heldForManualReview)
141+
// #9939: a draft is not trying to reach merge -- GitHub will not merge one at all. Evicted outright
142+
// rather than merely capped, exactly like the manual-review hold above: waiting on a PR that cannot
143+
// merge buys nothing, and the 24h cap is far too long to be the only bound on it.
144+
.filter((sibling) => !sibling.isDraft && sibling.mergeableState !== "draft")
129145
.filter((sibling) => isOlder(sibling))
130146
.filter((sibling) => overlaps(thisPrLinkedIssues, thisPrChangedFiles, sibling))
131147
.filter((sibling) => {

src/services/agent-action-executor.ts

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -32,6 +32,7 @@ import { isActingAutonomyLevel, resolveAutonomy } from "../settings/autonomy";
3232
import { boundStructuredCloseReasonsForPersistence, buildAgentActionAudit, formatAgentPermissionDenial, isGlobalAgentPause, resolveAgentActionMode, resolveAgentPermissionReadiness, type AgentActionMode } from "../settings/agent-execution";
3333
import { AGENT_LABEL_NEEDS_REVIEW, type PlannedAgentAction } from "../settings/agent-actions";
3434
import type { AgentActionClass, AgentPendingActionParams, AutonomyLevel, AutonomyPolicy } from "../types";
35+
import { clearPullRequestManualReviewLabelProvenance, markPullRequestManualReviewLabelApplied } from "../db/repositories";
3536
import { errorMessage } from "../utils/json";
3637
import {
3738
MODERATION_VIOLATION_EVENT_TYPE,
@@ -697,6 +698,10 @@ export async function executeAgentMaintenanceActions(env: Env, ctx: AgentActionE
697698
linkedIssues: sibling.linkedIssues,
698699
changedFiles: pathsByPullNumber.get(sibling.number),
699700
heldForManualReview: manualReviewLabel !== null && sibling.labels.some((label) => label.toLowerCase() === manualReviewLabel.toLowerCase()),
701+
// #9939: a draft sibling never blocks -- GitHub will not merge it, so it is not "about to land".
702+
// `?? false` rather than passing the nullable straight through: the field is optional on the record
703+
// and the gate treats absent as "not a draft", so normalising here keeps the two readings identical.
704+
isDraft: sibling.isDraft ?? false,
700705
})),
701706
nowMs: Date.now(),
702707
});
@@ -1287,8 +1292,21 @@ async function performAction(env: Env, ctx: AgentActionExecutionContext, action:
12871292
// optional comment (the Pass-1 flag warning, or the resolved note) posted alongside the label mutation.
12881293
if (action.labelOp === "remove") {
12891294
await removePullRequestLabel(env, ctx.installationId, ctx.repoFullName, ctx.pullNumber, action.label ?? "");
1295+
// #9939: the label is gone, so its provenance must go with it. Leaving a stale marker behind would let
1296+
// a LATER human-applied label inherit the bot's provenance and become auto-removable -- exactly the
1297+
// override the provenance exists to prevent. Best-effort: a failed clear only means the next pass
1298+
// re-evaluates with a marker for a label that is not there, which the `hasLabel` guard already ignores.
1299+
if (ctx.manualReviewLabel && action.label === ctx.manualReviewLabel) {
1300+
await clearPullRequestManualReviewLabelProvenance(env, ctx.repoFullName, ctx.pullNumber).catch(() => {});
1301+
}
12901302
} else {
12911303
await ensurePullRequestLabel(env, ctx.installationId, ctx.repoFullName, ctx.pullNumber, action.label ?? "", { createMissingLabel: true });
1304+
// #9939: record that the PLANNER (not a maintainer) applied this hold, so a later pass may lift it
1305+
// once nothing wants it. Written only here, on the bot's own add, which is what keeps a
1306+
// human-applied label provenance-free and therefore untouchable.
1307+
if (ctx.manualReviewLabel && action.label === ctx.manualReviewLabel && ctx.headSha) {
1308+
await markPullRequestManualReviewLabelApplied(env, ctx.repoFullName, ctx.pullNumber, ctx.headSha, action.reason).catch(() => {});
1309+
}
12921310
}
12931311
if (action.comment) await createIssueComment(env, ctx.installationId, ctx.repoFullName, ctx.pullNumber, action.comment);
12941312
return;

src/settings/agent-actions.ts

Lines changed: 39 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -286,6 +286,11 @@ export type AgentActionPlanInput = {
286286
// Configured manual-review hold label. Undefined uses the default "manual-review"; null disables only the
287287
// label, not the guardrail hold. Separate from review_state_label so operators can avoid ready/changes labels.
288288
manualReviewLabel?: string | null | undefined;
289+
/** #9939: the head SHA at which the PLANNER previously applied `manualReviewLabel`, or null/absent when the
290+
* bot did not apply it. This is the provenance that makes the label liftable: the same label string is
291+
* also how a maintainer freezes a PR by hand, so without it the planner cannot remove its own stale hold
292+
* without risking silently undoing a human's. Absent ⇒ never auto-removed. */
293+
manualReviewLabelAppliedSha?: string | null | undefined;
289294
// Optional disposition label overrides. Undefined uses generic defaults; null disables that specific label.
290295
readyToMergeLabel?: string | null | undefined;
291296
changesRequestedLabel?: string | null | undefined;
@@ -1271,6 +1276,40 @@ export function planAgentMaintenanceActions(input: AgentActionPlanInput): Planne
12711276
});
12721277
}
12731278

1279+
// 1b) #9939: LIFT a manual-review hold the planner itself applied, once nothing wants it any more.
1280+
//
1281+
// The sibling-label cleanup further down deliberately refuses to touch this label, because the same string
1282+
// is also a maintainer's manual freeze and that cleanup has no way to tell them apart. Correct -- but it
1283+
// made the label a ONE-WAY LATCH: applied on a pass where a hold was real, and never lifted once the hold
1284+
// cleared. Observed on #9935, which was mergeable, green, and re-reviewed to zero findings, and still
1285+
// refused to merge until a human removed the label by hand.
1286+
//
1287+
// `manualReviewLabelAppliedSha` is the missing provenance. Non-null means the PLANNER applied it, so the
1288+
// planner may take it back; null means a human applied it (or it predates the column) and it is left
1289+
// strictly alone. Guarded on `manualHoldReason === null`, i.e. NO reason wants a hold this pass -- which is
1290+
// why this cannot lift a label applied for reason A just because reason B cleared. Deliberately not scoped
1291+
// to the recorded head: a rebase that resolves the cause is the single most common way a hold goes stale,
1292+
// and refusing to lift it there would leave the exact #9935 case unfixed.
1293+
if (
1294+
manualHoldReason === null &&
1295+
labels.manualReview !== null &&
1296+
input.manualReviewLabelAppliedSha != null &&
1297+
hasLabel(input.pr.labels, labels.manualReview)
1298+
// No "is an add already planned this pass?" guard: every site that ADDS this label requires a live hold
1299+
// (a guardrail hit, the migration-collision fallback, the owner/automation fallback), and all of those
1300+
// set manualHoldReason -- so the null check above already excludes every case where an add could be in
1301+
// flight. A guard here would be an arm no input can reach.
1302+
) {
1303+
actions.push({
1304+
actionClass: "label",
1305+
autonomyClass: "merge",
1306+
requiresApproval: approval("merge"),
1307+
reason: `manual-review hold resolved — clearing the "${labels.manualReview}" label the bot applied`,
1308+
label: labels.manualReview,
1309+
labelOp: "remove",
1310+
});
1311+
}
1312+
12741313
// 1c) migration-collision manual-review fallback (#manual-review-coverage) — the migration-collision LABEL
12751314
// itself stays gated on review_state_label below (section 2, unchanged) so an operator's dedicated filter on
12761315
// that specific label is undisturbed. But when review_state_label is OFF (a one-shot repo that only configures

src/types.ts

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -777,6 +777,13 @@ export type PullRequestRecord = {
777777
* review/visual/visual-capture-retry-latch.ts for why a sha with no timestamp reads as expired rather than
778778
* live. Publish-written alongside the sha; read straight from the row. */
779779
visualCaptureRetryPendingAt?: string | null | undefined;
780+
/** #9939: the head SHA at which the PLANNER applied the manual-review label, and the hold reason it applied
781+
* it for. Together they are the provenance that lets the planner lift its OWN stale hold without ever
782+
* touching a maintainer's manual freeze -- the label is the same string for both. Absent ⇒ not the bot's
783+
* to remove (a human applied it, or it predates this), which is the safe default for every existing label.
784+
* Planner-written; read straight from the row. */
785+
manualReviewLabelAppliedSha?: string | null | undefined;
786+
manualReviewLabelAppliedReason?: string | null | undefined;
780787
/** Screenshot-table PRESENCE-mode staleness correlation (#stale-screenshot-table-fix): the (headSha,
781788
* evidenceFingerprint) checkpoint the screenshotTableGate's presence-mode check last satisfied for this PR
782789
* (see evaluateScreenshotTableGate's staleness comment). `null`/absent = presence mode has never satisfied

0 commit comments

Comments
 (0)