feat(review): same-rule repeat alarm — detection + alert, no autonomous action (#8092)

Closes #7983.

The existing self-correction system only detects a systematically-
wrong rule via precision-over-time (auto-tune.ts), which needs a
real, DECIDED sample (>= AUTOTUNE_MIN_DECIDED) accumulated over
however long that takes -- too slow for a bug that can mis-close 4
PRs within hours, as the 2026-07-21/22 metagraphed incident did. A
much cheaper, ground-truth-free signal already exists: the SAME
deterministic rule/blocker code rejecting several DIFFERENT PRs in a
short window in the same repo is itself a strong "something's
broken" signal, independent of whether any of those rejections is
ever confirmed or reversed by a human.

New packages/loopover-engine/src/calibration/signal-tracking.ts:
evaluateRuleRepeatAlarm(ruleId, fired, threshold) — pure, no ground
truth needed, mirrors src/orb/analytics.ts's gamingPatternFlags
precedent ("Detection only — never an automatic action").

New src/review/rule-repeat-alarm-wire.ts wires this into ORB for
real: every gate block now records a #7982 rule-fired signal per
blocker code (nothing called the ORB adapter until now), scoped
per-(repo, code) so an unrelated repo or code never contributes to
another's count, and checks the repeat alarm inline, immediately
after each block — not on a later cron tick, matching the "hours,
not days" urgency the incident exposed. A triggered alarm logs a
structured console.error (forwarded to Sentry, the same "detected an
anomaly" channel src/review/ops-wire.ts's own runOpsAlerts already
uses) and writes a cooldown marker so an ongoing incident doesn't
re-alert on every subsequent PR.

Note on the issue's own cited alert channel: notify-discord.ts/
notify-slack turned out to be the wrong fit on inspection — that's a
per-REPO, community-facing channel for PR action notifications, not
an operator-facing "an ORB rule may be systematically broken" signal
that can span any repo the instance reviews. Sentry (via the
existing structured-log forwarder) is the channel actually already
used for this class of alert.

Validated against a replay of the exact #7469/#7589/#7591/#7594
incident shape: triggers on the 3rd distinct PR, matching the
issue's own "should have alerted after the 2nd or 3rd occurrence"
bar.

Author: JSONbored

packages/loopover-engine/src/calibration/signal-tracking.ts

@@ -102,12 +102,46 @@ export function computeRulePrecision(ruleId: string, fired: readonly RuleFiredEv
 }
 
 /**
- * Count how many times `ruleId` fired against the exact same `targetKey` within `fired` -- the #7983
- * "same-rule repeat alarm" primitive (a rule re-firing against a target it already fired against once is a
- * stronger signal than a bare one-off fire, independent of whether either fire has been overridden yet).
- * Pure counting, no time-windowing here -- a caller windows `fired` itself before calling this (e.g. via
- * `queryRuleHistory`'s own `sinceMs`), matching how this whole module leaves all storage/scoping to the host.
+ * Count how many times `ruleId` fired against the exact same `targetKey` within `fired` (a rule re-firing
+ * against a target it already fired against once -- e.g. an unresolved contributor PR re-triggering the same
+ * blocker on every push -- is a different signal than a fresh one-off fire, independent of whether either fire
+ * has been overridden yet). Pure counting, no time-windowing here -- a caller windows `fired` itself before
+ * calling this (e.g. via `queryRuleHistory`'s own `sinceMs`), matching how this whole module leaves all
+ * storage/scoping to the host. See {@link evaluateRuleRepeatAlarm} for the DIFFERENT #7983 signal: the same
+ * rule firing against several DIFFERENT targets, not the same one repeatedly.
  */
 export function computeRuleRepeatCount(ruleId: string, targetKey: string, fired: readonly RuleFiredEvent[]): number {
   return fired.reduce((count, event) => (event.ruleId === ruleId && event.targetKey === targetKey ? count + 1 : count), 0);
 }
+
+/** A same-rule repeat-alarm verdict (#7983): whether `ruleId` has fired against enough DISTINCT targets within
+ *  the caller's already-windowed `fired` list to be a "something is systematically broken" signal --
+ *  independent of whether any of those firings has been confirmed or reversed by a human yet (unlike
+ *  {@link computeRulePrecision}, this needs no ground truth at all, which is exactly why it can fire fast: the
+ *  2026-07-21/22 metagraphed incident mis-closed 4 DISTINCT PRs within ~3 hours on the same rule, far faster
+ *  than a precision-over-time breaker's `AUTOTUNE_MIN_DECIDED` sample could ever accumulate real outcomes). */
+export type RuleRepeatAlarmVerdict = {
+  ruleId: string;
+  /** Every distinct targetKey `ruleId` fired against, in first-seen order. */
+  affectedTargets: string[];
+  threshold: number;
+  triggered: boolean;
+};
+
+/**
+ * Evaluate the #7983 same-rule repeat alarm for `ruleId` over an already-windowed `fired` list: `triggered` is
+ * true once the rule has fired against at least `threshold` DISTINCT targets. Deliberately returns a
+ * detection-only verdict -- no action, no severity beyond the boolean -- mirroring `src/orb/analytics.ts`'s
+ * `gamingPatternFlags` precedent ("Detection only — never an automatic action") and this module's own
+ * "no autonomous behavior" boundary; the host decides how (or whether) to surface a triggered verdict.
+ */
+export function evaluateRuleRepeatAlarm(ruleId: string, fired: readonly RuleFiredEvent[], threshold: number): RuleRepeatAlarmVerdict {
+  const affectedTargets: string[] = [];
+  const seen = new Set<string>();
+  for (const event of fired) {
+    if (event.ruleId !== ruleId || seen.has(event.targetKey)) continue;
+    seen.add(event.targetKey);
+    affectedTargets.push(event.targetKey);
+  }
+  return { ruleId, affectedTargets, threshold, triggered: affectedTargets.length >= threshold };
+}

packages/loopover-engine/test/signal-tracking.test.ts

@@ -1,7 +1,13 @@
 import assert from "node:assert/strict";
 import { test } from "node:test";
 
-import { computeRulePrecision, computeRuleRepeatCount, type HumanOverrideEvent, type RuleFiredEvent } from "../dist/index.js";
+import {
+  computeRulePrecision,
+  computeRuleRepeatCount,
+  evaluateRuleRepeatAlarm,
+  type HumanOverrideEvent,
+  type RuleFiredEvent,
+} from "../dist/index.js";
 
 function fired(ruleId: string, targetKey: string, overrides: Partial<RuleFiredEvent> = {}): RuleFiredEvent {
   return { ruleId, targetKey, outcome: "block", occurredAt: "2026-07-22T00:00:00.000Z", ...overrides };
@@ -90,3 +96,61 @@ test("computeRuleRepeatCount: counts only fires matching BOTH ruleId and targetK
 test("computeRuleRepeatCount: zero fired events yields 0, not an error", () => {
   assert.equal(computeRuleRepeatCount("rule_a", "a#1", []), 0);
 });
+
+test("barrel: the public entrypoint re-exports evaluateRuleRepeatAlarm (#7983)", () => {
+  assert.equal(typeof evaluateRuleRepeatAlarm, "function");
+});
+
+test("evaluateRuleRepeatAlarm: not triggered below the threshold", () => {
+  const verdict = evaluateRuleRepeatAlarm("rule_a", [fired("rule_a", "a#1"), fired("rule_a", "a#2")], 3);
+  assert.equal(verdict.triggered, false);
+  assert.deepEqual(verdict.affectedTargets, ["a#1", "a#2"]);
+  assert.equal(verdict.threshold, 3);
+});
+
+test("evaluateRuleRepeatAlarm: triggers once distinct targets reach the threshold", () => {
+  const verdict = evaluateRuleRepeatAlarm("rule_a", [fired("rule_a", "a#1"), fired("rule_a", "a#2"), fired("rule_a", "a#3")], 3);
+  assert.equal(verdict.triggered, true);
+  assert.deepEqual(verdict.affectedTargets, ["a#1", "a#2", "a#3"]);
+});
+
+test("evaluateRuleRepeatAlarm: replays the #7469/#7589/#7591/#7594 incident shape -- triggers on the 3rd distinct PR", () => {
+  const incidentEvents = [
+    fired("rule_a", "metagraphed/metagraphed#7469"),
+    fired("rule_a", "metagraphed/metagraphed#7589"),
+  ];
+  // Should NOT have triggered yet after only 2 distinct PRs (threshold 3).
+  assert.equal(evaluateRuleRepeatAlarm("rule_a", incidentEvents, 3).triggered, false);
+  incidentEvents.push(fired("rule_a", "metagraphed/metagraphed#7591"));
+  // The 3rd distinct PR crosses the threshold -- exactly the "should have alerted after the 2nd or 3rd
+  // occurrence" bar #7983 itself sets.
+  const thirdVerdict = evaluateRuleRepeatAlarm("rule_a", incidentEvents, 3);
+  assert.equal(thirdVerdict.triggered, true);
+  assert.deepEqual(thirdVerdict.affectedTargets, [
+    "metagraphed/metagraphed#7469",
+    "metagraphed/metagraphed#7589",
+    "metagraphed/metagraphed#7591",
+  ]);
+});
+
+test("evaluateRuleRepeatAlarm: the SAME target firing repeatedly counts once, not once per fire -- only a DISTINCT target grows the count", () => {
+  const verdict = evaluateRuleRepeatAlarm(
+    "rule_a",
+    [fired("rule_a", "a#1"), fired("rule_a", "a#1"), fired("rule_a", "a#1")],
+    2,
+  );
+  assert.equal(verdict.affectedTargets.length, 1);
+  assert.equal(verdict.triggered, false);
+});
+
+test("evaluateRuleRepeatAlarm: ignores fired events for a DIFFERENT ruleId entirely", () => {
+  const verdict = evaluateRuleRepeatAlarm("rule_a", [fired("rule_a", "a#1"), fired("rule_b", "a#2"), fired("rule_b", "a#3")], 2);
+  assert.equal(verdict.affectedTargets.length, 1);
+  assert.equal(verdict.triggered, false);
+});
+
+test("evaluateRuleRepeatAlarm: zero fired events never triggers", () => {
+  const verdict = evaluateRuleRepeatAlarm("rule_a", [], 1);
+  assert.equal(verdict.triggered, false);
+  assert.deepEqual(verdict.affectedTargets, []);
+});

src/queue/processors.ts

@@ -636,6 +636,7 @@ import {
 } from "../review/outcomes-wire";
 import { neutralHoldReasonCode, nativeGateActionFromConclusion, recordNativeGateDecision } from "../review/parity-wire";
 import { recordContributorGateDecision } from "../review/contributor-calibration";
+import { recordGateBlockersAndCheckRepeatAlarm } from "../review/rule-repeat-alarm-wire";
 import { recordPredictedGateCalibration } from "../review/predicted-gate-calibration-ledger";
 import type { SubmissionOutcome } from "../review/submitter-reputation";
 import type {
@@ -10326,6 +10327,13 @@ async function maybePublishPrPublicSurface(
         outcome: "completed",
         metadata: { blockerCodes },
       });
+      // #7983: same-rule repeat alarm — detection + alert only, never adjusts the gate. See
+      // rule-repeat-alarm-wire.ts's own header comment.
+      await recordGateBlockersAndCheckRepeatAlarm(env, {
+        repoFullName,
+        pullNumber: pr.number,
+        blockerCodes,
+      }).catch(() => undefined);
     }
     // #preconv-parity (convergence prep): SHADOW-record the gittensory-native gate decision (source=
     // 'gittensory-native') into review_audit so the pre-cutover parity harness has data to read. RECORD-ONLY,

src/review/rule-repeat-alarm-wire.ts

@@ -0,0 +1,115 @@
+// ORB wiring for #7983's same-rule repeat alarm. Records a #7982 rule-fired signal for each gate blocker code
+// on every gate block, then checks whether that SAME (repo, blocker code) pair has now fired against enough
+// DISTINCT PRs within a short window to be a "something is systematically broken" signal — independent of
+// whether any of those blocks has since been confirmed or reversed by a human (unlike the precision-over-time
+// circuit breaker in auto-tune.ts, which needs a real, DECIDED sample of >= AUTOTUNE_MIN_DECIDED and however
+// long that takes to accumulate). This is exactly the gap the 2026-07-21/22 metagraphed incident exposed: 4
+// distinct PRs mis-closed by the same rule within ~3 hours, far faster than any ground-truth-based breaker
+// could ever react.
+//
+// DETECTION + ALERT ONLY (#7983's own stated boundary, mirroring src/orb/analytics.ts's gamingPatternFlags
+// precedent: "Detection only — never an automatic action"). This module never holds, closes, or otherwise
+// changes any gate/disposition decision — it only records a signal and, once, surfaces a structured alert.
+//
+// Alert channel: NOT notify-discord.ts/notify-slack (that's a per-REPO, community-facing channel for PR
+// action notifications — the wrong audience for "an ORB rule may be systematically broken," which is an
+// OPERATOR concern that can span any repo the instance reviews). Uses the same console.error(JSON.stringify(
+// {level:"error",...})) idiom src/review/ops-wire.ts's runOpsAlerts already uses for its own operator-anomaly
+// detection — forwarded to Sentry by selfhost/sentry.ts's forwardStructuredLogToSentry, the actually-live
+// operator-facing channel for exactly this class of "detected an anomaly, not a caught exception" alert.
+
+import { evaluateRuleRepeatAlarm, type SignalStore } from "@loopover/engine";
+
+import { hasRecentAuditEvent, recordAuditEvent } from "../db/repositories";
+import { nowIso } from "../utils/json";
+import { createSignalStore } from "./signal-tracking-wire";
+
+/** How far back to look for repeat fires. Matches #7983's own "e.g. 1-24h, tunable" proposal — chosen at the
+ *  wide end so a slow-burn (not just a fast-burst) repeat still gets caught. */
+export const RULE_REPEAT_ALARM_WINDOW_MS = 24 * 60 * 60 * 1000;
+/** Distinct-target count that trips the alarm. #7983's own proposal ("e.g. >= 3 within 24h") and its own
+ *  validation bar ("should have alerted after the 2nd or 3rd occurrence" against the real incident replay). */
+export const RULE_REPEAT_ALARM_THRESHOLD = 3;
+/** Once triggered, don't re-alert for the SAME (repo, code) pair more often than this — an already-known,
+ *  ongoing incident re-alerting on every subsequent PR would be noise, not new information. Shorter than
+ *  {@link RULE_REPEAT_ALARM_WINDOW_MS} so a genuinely NEW burst (a different day, a fix that regressed again)
+ *  still re-alerts well before the detection window itself would naturally reset. */
+const RULE_REPEAT_ALARM_ALERT_COOLDOWN_MS = 6 * 60 * 60 * 1000;
+
+/** Repo-scoped rule id (#7983 wants the alarm keyed by "(deployment/repo-or-cohort, rule code)", not a bare
+ *  code across the whole fleet — a code that's simply common everywhere must not look like one repo's rule
+ *  going haywire). Reuses the same signal-tracking `ruleId` seam #7982 already defined; the repo scope is
+ *  folded directly into the id rather than needing a second dimension on {@link SignalStore}. */
+function repeatAlarmRuleId(repoFullName: string, blockerCode: string): string {
+  return `${repoFullName}:${blockerCode}`;
+}
+
+function alertAuditEventType(ruleId: string): string {
+  return `rule_repeat_alarm:${ruleId}`;
+}
+
+async function checkAndAlertRuleRepeat(
+  env: Env,
+  store: SignalStore,
+  ruleId: string,
+  blockerCode: string,
+  repoFullName: string,
+): Promise<void> {
+  const history = await store.queryRuleHistory(ruleId, Date.now() - RULE_REPEAT_ALARM_WINDOW_MS);
+  const verdict = evaluateRuleRepeatAlarm(ruleId, history.fired, RULE_REPEAT_ALARM_THRESHOLD);
+  if (!verdict.triggered) return;
+  const alertEventType = alertAuditEventType(ruleId);
+  const alreadyAlerted = await hasRecentAuditEvent(
+    env,
+    "loopover",
+    alertEventType,
+    new Date(Date.now() - RULE_REPEAT_ALARM_ALERT_COOLDOWN_MS).toISOString(),
+  );
+  if (alreadyAlerted) return;
+  console.error(
+    JSON.stringify({
+      level: "error",
+      event: "same_rule_repeat_alarm",
+      ev: ruleId,
+      repo: repoFullName,
+      blockerCode,
+      distinctTargetCount: verdict.affectedTargets.length,
+      threshold: verdict.threshold,
+      affectedTargets: verdict.affectedTargets,
+      at: nowIso(),
+    }),
+  );
+  await recordAuditEvent(env, {
+    eventType: alertEventType,
+    actor: "loopover",
+    targetKey: ruleId,
+    outcome: "completed",
+    detail: `same-rule repeat alarm: ${blockerCode} fired against ${verdict.affectedTargets.length} distinct PR(s) in ${repoFullName} within ${RULE_REPEAT_ALARM_WINDOW_MS / (60 * 60 * 1000)}h`,
+    metadata: { repoFullName, blockerCode, affectedTargets: verdict.affectedTargets },
+  }).catch(() => undefined);
+}
+
+/**
+ * Records a #7982 rule-fired signal for every blocker code on a gate block, then runs the #7983 repeat-alarm
+ * check for each. Best-effort throughout (a failure anywhere in this path is swallowed) — this is a pure
+ * measurement/alerting side channel and must never affect, delay, or fail the gate decision that produced the
+ * blocker codes it's recording.
+ */
+export async function recordGateBlockersAndCheckRepeatAlarm(
+  env: Env,
+  args: { repoFullName: string; pullNumber: number; blockerCodes: readonly string[]; occurredAt?: string },
+): Promise<void> {
+  if (args.blockerCodes.length === 0) return;
+  // createSignalStore is pure object construction (no I/O), so it never throws — no try/catch needed here.
+  // recordRuleFired below already swallows its own write failures internally (signal-tracking-wire.ts), so it
+  // never rejects either; only queryRuleHistory (inside checkAndAlertRuleRepeat) can genuinely reject, which
+  // this loop's own .catch below covers.
+  const store = createSignalStore(env);
+  const occurredAt = args.occurredAt ?? nowIso();
+  const targetKey = `${args.repoFullName}#${args.pullNumber}`;
+  for (const blockerCode of new Set(args.blockerCodes)) {
+    const ruleId = repeatAlarmRuleId(args.repoFullName, blockerCode);
+    await store.recordRuleFired({ ruleId, targetKey, outcome: "block", occurredAt });
+    await checkAndAlertRuleRepeat(env, store, ruleId, blockerCode, args.repoFullName).catch(() => undefined);
+  }
+}