[Epic] Drift detection v1: introduction-time + time-emergent

[julianken]

WHY

Drift audit 2026-05-04 produced this remediation plan via REPORT.md § "Recommended drift-detection automation" (lines 204-264). The next move per the detecting-drift-in-ai-repos skill is to decompose findings into composable signals so future audits run continuously.

This epic is filed but not gating. Per FINAL-PLAN.md line 7, the drift:* taxonomy is not a precondition for any #226…#247 issue. Per the drift skill's cold-start protocol, audit findings the team chooses NOT to fix immediately get a wontfix issue (existing label) so day-one of the future nightly doesn't refile them — that ceremony is now #248e-1.

WHAT (epic checklist)

Tracking-only epic. Each child is a separate PR delivered in shadow mode for 5-10 firings before being verdict-capable. Per REPORT.md line 278: cap narrative drift at IMPORTANT during shadow rollout.

PRE-REQ chain: #248a → all others. No downstream child fires until the label taxonomy lands.

• #248a — drift:* label taxonomy. Labels: drift:spec, drift:plan, drift:dependency, drift:infra, drift:docs, drift:label, drift:test, drift:automated, drift:aging, drift:escalated, drift:acknowledged. Reuse existing wontfix instead of drift:wont-fix. PRE-REQ for every child below.

• #248b-1 — Bot rubric: canary-phrase grep on narrative-doc diffs. Path: ~/.claude/skills/reviewing-as-julianken-bot/SKILL.md. Scans README/CLAUDE.md/AGENTS.md/docs/* for: vercel, Vercel Blob, BLOB_READ_WRITE_TOKEN, Transaction Pooler, npm run, Phase 1 (current), All rights reserved. IMPORTANT cap, full shadow. PRE-REQ: #248a.

• #248b-2 — Bot rubric: CONTENT_MODEL.md cross-check on src/collections/ diffs. Flags field-list mismatches. IMPORTANT cap, shadow. PRE-REQ: #248a.

• #248b-3 — Bot rubric: auto-label agentic-design-patterns on PRs touching src/(data|app)/agentic-design-patterns/. Zero FP risk; ships verdict-capable on day one. PRE-REQ: #248a.

• #248c — Workflow: check-required-contexts. Compares .mergify.yml check-success + branch-protection contexts to emitted name: fields in .github/workflows/*.yml. Fails CI if drifted. Prevents chore(ci): add missing E2E shard contexts to .mergify.yml #225-class regressions. PRE-REQ: #248a.

• #248d — Workflow: canary-phrase-guard. ESLint-style grep over tracked narrative files; fails CI on canary matches. Canary list lives in workflow YAML. PRE-REQ: #248a.

• #248e-1 — Cold-start ceremony. File wontfix issues for SUGGESTION-tier deferrals from this audit so day-one of #248e-2 doesn't refile them. (Replaces former [Audit log] Drift audit 2026-05-04 — deferred items #249 placeholder.) PRE-REQ: #248a.

• #248e-2 — Nightly extractor: shell script + signals + dedup. Deterministic only, no LLM. Signals: Dependabot PRs >7d open; issues in active epic without epic's labels; migrations without paired .json snapshots; doc files with passed Next Audit: dates; components without paired tests; literal vercel in tracked files (excluding ADP example bodies). Includes exclusion list for the tool's own artifacts. Ships and bakes before #248e-3. PRE-REQ: #248a, #248e-1.

• #248e-3 — LLM dispatch synthesizer. Gated on signal severity from #248e-2. Single issue per cycle. Synthesizer failure doesn't poison signal data. PRE-REQ: #248e-2.

DONE WHEN

All eight children land and complete at least one shadow-mode firing cycle without producing >3 false-positive issues per week. Plus: kill-review reminder issue filed (see below).

Kill review reminder

Per drift skill § 5, the kill metric requires both a quantified threshold AND a calendar trigger.

Metric: closed-as-fixed / closed-total < 40% over a calendar month → kill LLM-driven rules (#248e-3, #248b-*), keep deterministic checks (#248c, #248d, #248e-2).

Trigger: within 24h of #248e-2 first nightly fire, file a follow-up issue with label 60-day-followup and target date = merge_date(#248e-2) + 60d. Template:

gh issue create --repo julianken/detached-node \
  --title "Drift Detection v1 — kill-metric review (60d)" \
  --label "60-day-followup" \
  --body "Review window: $(date -v+60d +%Y-%m-%d). Compute closed-as-fixed/closed-total over the prior calendar month for issues labeled drift:*. If <40%, kill #248b-* and #248e-3; keep #248c/#248d/#248e-2."

Falsifiability statement

Pay-for-itself: 22 BLOCKERs from REPORT.md remediated within 30 days; future pivots produce <5 net-new BLOCKER prose-drift findings 30 days post-pivot; Dependabot ai-sdk PR closure rate >80%.

Waste: kill-metric trips; team disables nightly within 3 months; BLOCKERs recur on same artifacts after one full fix cycle.

OUT OF SCOPE

• Filing children as standalone issues now (do that only when ready to ship each).
• Promoting any child to ship-now (deferred until docs(deploy): rewrite deployment guide for Cloud Run #226…chore(repo): one-shot debris sweep #247 land).
• #248f (untested-components backlog) was moved out of Epic E to standalone issue Test coverage backfill: untested root + ADP components (Phase-3) #250 (Phase-3 quality gate, not drift-detection work; not gated by drift-automation).

PRE-REQS

None for the epic itself. Internal chain: #248a precedes all other children.

[julianken-bot]

Verdict: NEEDS-REVISION. Holistic review of this tracking epic against REPORT.md § "Recommended drift-detection automation", FINAL-PLAN.md Epic E, and the detecting-drift-in-ai-repos skill's rollout discipline. Cap 800w.

---

What this epic gets right

• Filed-but-not-gating posture is correct (skill: cold-start without gating prevents the noise → disable cycle). Body line 3 is explicit.
• Falsifiability statement is present with both pay-for-itself and waste evidence — meets skill's section "5. Kill-threshold and falsifiability." Three pay metrics, three waste metrics. Good.
• Cold-start ceremony correctly punted to a separate [Audit log] Drift audit 2026-05-04 — deferred items #249 issue — body line 3 names it. This is the right shape.
• drift:wont-fix deduplication: body correctly reuses existing wontfix instead of creating drift:wont-fix (per critique-of-draft-1 § 3 and FINAL-PLAN line 70). Carry-through is clean.
• Bot rubric path ~/.claude/skills/reviewing-as-julianken-bot/SKILL.md (#248b) matches the canonical path I just verified at /Users/j/.claude/skills/reviewing-as-julianken-bot/SKILL.md. Critique-of-draft-1 § 4 found this defect in plan-draft-1; epic body fixed it. Good.
• Shadow-mode discipline: body line 5 ("each child is a separate PR delivered in shadow mode for at least 5-10 firing instances before being verdict-capable") and IMPORTANT, never BLOCKER cap — matches skill § "Rollout Discipline" point 2. Good.

---

Defects requiring revision

D1 (BLOCKER for the epic itself, not for shipping): kill-metric is missing a calendar reminder. Skill § 5 mandates both a quantified metric AND a calendar reminder. The falsifiability section names the metrics but no concrete date / mechanism (e.g., "open follow-up issue with @mergifyio snooze for 60 days post-#248e merge"). Without the calendar trigger this section is decorative. Fix: add a "DONE WHEN" sub-bullet "issue filed with 60-day-followup label and target date set to merge_date(#248e) + 60d" — or whatever Julian's snooze convention is.

D2 (IMPORTANT): #248e bundles a workflow + LLM dispatcher + 8 distinct signals into one child. That's three load-bearing decisions in one PR (bash signal extractor, prompt-fragment template, dedup logic). Skill § "Three Composable Mechanism Families" C calls these out as separate concerns. The shadow-mode logging discipline alone deserves its own commit. Fix: split #248e into #248e1 — nightly extractor scaffold + signal harness (deterministic, no LLM) and #248e2 — LLM synthesizer + issue dedup (judgment layer). The first child can ship and bake before LLM dispatch lands; failure in the synthesizer doesn't poison the signal data.

D3 (IMPORTANT): #248b is three rules in one child, repeating the plan-draft-1 mistake. Critique-of-draft-1 row 32 already flagged this: "this issue conflates 5 distinct rules… Consider splitting into N rules per child issue." The epic absorbed only the path-fix from that critique, not the splitting recommendation. Three rules with three different drift surfaces (canary phrases / collections-vs-CONTENT_MODEL / auto-labeling) and three different FP profiles. Shadow-mode promotion gates need to fire per-rule, not aggregated. Fix: make #248b an umbrella with sub-children #248b1 (canary), #248b2 (collections cross-check), #248b3 (auto-label). The auto-label rule has effectively zero FP risk and could ship verdict-capable on day one; the canary rule needs full shadow.

D4 (IMPORTANT): precondition ordering for #248a is implicit. Body says "use existing wontfix instead of creating drift:wont-fix" but drift:* taxonomy creation is itself the precondition for everything downstream. Body never states "#248a MUST land before #248b/c/d/e fires." Fresh subagent dispatched on #248c could ship a workflow that emits drift:infra labels that don't exist. Fix: add explicit "PRE-REQS: #248a" line on each of #248b–#248e in the body.

D5 (SUGGESTION): #248f promotion vs deferral. The 10 + 13 untested-component backlog is a quality gate, not drift detection. It doesn't share triggering, dispatch vocabulary, or rollout discipline with the other Epic E children. Putting it under this epic dilutes the epic's coherence. Fix: spin out #248f as its own non-Epic-E ship-now issue under enhancement (Phase-3 follow-up); keep Epic E focused on detection-and-dispatch primitives only. This also matches the "Epic E is filed but not gating" framing — #248f doesn't need to wait.

---

Skill-discipline checklist

Check	Status
Two regimes (introduction-time + time-emergent)	YES — #248b is intro-time, #248e is time-emergent
drift:* taxonomy precedes detectors	IMPLIED via #248a but not stated as PRE-REQ on downstream (D4)
Cold-start ceremony scoped	YES — #249 punt is correct
Shadow mode for LLM rules	YES — body line 5 + cap rule
Cap narrative drift at IMPORTANT	YES — body line 5 cites it
Kill metric quantified	YES
Calendar reminder	NO (D1)
Falsifiability statement	YES
Exclusion list for tool's own artifacts	NOT MENTIONED — recommend adding to #248e WHAT block
One PR per child	NO for #248b and #248e (D2, D3)

---

Verdict: NEEDS-REVISION. Address D1, D2, D3, D4 before the first child gets filed as a standalone issue. D5 is optional but recommended.

[julianken]

Revised cycle-2: added kill-review reminder (60-day calendar trigger + gh template); split #248b into #248b-1/2/3 (canary / collections cross-check / auto-label); split #248e into #248e-1/2/3 (cold-start / deterministic extractor / LLM synthesizer); explicit PRE-REQ chain on #248a; #248f promoted out to standalone #250 (Phase-3, not gated by drift work).

[julianken-bot]

Verdict: APPROVED. Cycle-2 holistic review of revised Epic E. All five cycle-1 findings (D1-D5) addressed. Cap 600w.

---

Cycle-1 → cycle-2 carry-through

Finding	Cycle-1 severity	Cycle-2 status
D1 — calendar reminder missing	BLOCKER	RESOLVED
D2 — #248e bundled 3 concerns	IMPORTANT	RESOLVED
D3 — #248b bundled 3 rules	IMPORTANT	RESOLVED
D4 — implicit PRE-REQ ordering	IMPORTANT	RESOLVED
D5 — #248f scope dilution	SUGGESTION	RESOLVED

D1 verification. "Kill review reminder" section now present. Metric closed-as-fixed / closed-total < 40% over a calendar month is quantified. Trigger is concrete: 24h post-#248e-2 first-fire, file 60-day-followup issue with target merge_date(#248e-2) + 60d. Paste-ready gh issue create template included with the kill set spelled out (kill #248b-* and #248e-3; keep #248c/#248d/#248e-2). This satisfies skill § 5 — both quantified threshold AND calendar mechanism are present. Choosing "60d post first-fire" (not post-merge) is the correct anchor because shadow mode delays meaningful signal accumulation; merge-date alone could trigger the review before the system has fired even once.

D2 verification. #248e split into three independently-failing children: #248e-1 (cold-start ceremony, replaces former #249 placeholder), #248e-2 (deterministic shell-script extractor with explicit "no LLM"), #248e-3 (LLM synthesizer, compound PRE-REQ on #248a + #248e-2). Critically, the body adds "Synthesizer failure doesn't poison signal data" — that's the right failure-isolation framing. The exclusion list for the tool's own artifacts (cycle-1 sidebar concern) is now called out explicitly in #248e-2's WHAT.

D3 verification. #248b split into #248b-1 (canary grep, IMPORTANT cap, full shadow), #248b-2 (CONTENT_MODEL.md cross-check, IMPORTANT cap, shadow), #248b-3 (auto-label, "Zero FP risk; ships verdict-capable on day one"). The per-rule promotion-gate framing is now structurally enforceable. Auto-label-on-day-one matches my cycle-1 recommendation verbatim.

D4 verification. Body preamble states "PRE-REQ chain: #248a → all others. No downstream child fires until the label taxonomy lands." Every child has explicit PRE-REQ: #248a trailer. Compound chain (PRE-REQ: #248a, #248e-1) on #248e-2 and (PRE-REQ: #248e-2) on #248e-3 is correct.

D5 verification. #248f promoted out to standalone #250 (verified: title "Test coverage backfill: untested root + ADP components (Phase-3)", labels enhancement, body explicitly states "Not gated by Epic E"). OUT OF SCOPE block in the epic body documents the promotion with rationale. Epic E coherence preserved.

---

Mandatory-find second pass (R8)

Second pass with explicit prior "this contains at least one improvement." One minor finding:

S1 (SUGGESTION) — count drift in DONE WHEN. Body says "All eight children land" but the WHAT checklist now enumerates nine children: #248a, #248b-1, #248b-2, #248b-3, #248c, #248d, #248e-1, #248e-2, #248e-3. The split-out of cycle-2 changed the count. Fix: "All nine children land" (or generalize: "every child in the WHAT checklist"). Cosmetic; not blocking.

---

Skill-discipline checklist (delta from cycle-1)

Check	Cycle-1	Cycle-2
Calendar reminder	NO	YES
One PR per child	NO (#248b, #248e bundled)	YES
Explicit PRE-REQ on detectors	IMPLIED	EXPLICIT
Exclusion list for tool artifacts	UNMENTIONED	CALLED OUT in #248e-2
Epic coherence (no Phase-3 dilution)	NO (#248f mixed in)	YES (#248f→#250)

All previously-failing rows now pass.

---

Verdict: APPROVED. S1 is optional cleanup — fix opportunistically when the next edit touches the body. Ready to start filing children, beginning with #248a.