Add staleness check before code agent works on aged issues

[fullsend-ai-retro]

What happened

Issue #644 was labeled ready-to-code on 2026-05-05. The code agent didn't successfully produce a PR until 2026-05-11 (6 days later, due to a failed first run). During that window, the installation guide (docs/guides/admin/installation.md) was being reworked by other PRs, making the original problem obsolete. The code agent produced a correct fix (PR #800) that received two approvals but was closed without merge because the fix was no longer needed.

This wasted approximately: 2 code agent runs, 1 review agent run, and one human reviewer's time (ralphbean spent effort writing 3 thoughtful inline comments on a PR that would never merge).

What could go better

When a ready-to-code issue has been sitting for multiple days, the code agent (or a pre-flight check in the dispatch layer) should verify the issue is still relevant before investing compute. Possible signals:

• File-level change detection: If the files mentioned in the triage summary have been modified on main since the triage ran, flag potential staleness.
• Issue comment recency: If no human has interacted with the issue recently and other PRs have landed touching the same area, prompt for re-triage.
• Triage expiry: Treat triage summaries as having a TTL (e.g., 7 days). If the code agent is dispatched after the TTL, re-run triage first to confirm the problem still exists.

Confidence: Medium. This specific case is clear-cut (6-day delay + install process overhaul), but designing a general staleness heuristic that doesn't add false-positive friction is hard. The file-change-detection signal is the most concrete and least likely to produce false positives.

This pattern may recur: the repo has active development on installation/setup flows, and issues in that area can become obsolete quickly.

Proposed change

Add a pre-flight staleness check to the code dispatch pipeline (likely in post-triage.sh or the code dispatch workflow). Before dispatching the code agent:

1. Check if the files referenced in the triage summary have been modified on main since the triage timestamp.
2. If yes, post a comment on the issue noting that referenced files have changed and suggest re-triaging (/triage) before coding.
3. Optionally, skip the code dispatch and wait for human confirmation.

This could also be implemented as a TTL on triage summaries: if the triage is older than N days (suggest 5), auto-re-triage before dispatching the code agent.

Related issues: #281 (triage agent lacks 'already-fixed' action), #576 (triage agent lacks 'superseded' close reason).

Validation criteria

Over the next 20 code agent dispatches on issues where the triage summary is >3 days old, measure: (1) how many would have been flagged by the staleness check, and (2) of those flagged, how many resulted in PRs that were ultimately merged vs. closed. Success = at least 50% of flagged cases would have been correctly identified as stale, with a false-positive rate under 20%.

---

Generated by retro agent from #800

[github-actions]

fullsend triage is working on this — view logs

[fullsend-ai-triage]

Triage Summary

Severity: Medium | Category: Enhancement

This issue proposes adding a pre-flight staleness check before the code agent is dispatched on aged triage summaries. The problem is well-illustrated by the incident on issue #644, where a 6-day delay between triage and code dispatch resulted in a correct-but-obsolete PR that wasted agent compute and human review time.

Problem

The code dispatch pipeline currently has no freshness validation. It dispatches the code agent on any ready-to-code issue regardless of triage age or whether referenced files have changed on main since triage.

Recommended Approach

The staleness check should live in the dispatch layer (e.g., as a pre-flight step in reusable-code.yml or a pre-script for the code stage) rather than in post-triage.sh, since staleness is a function of dispatch time, not triage time. The most actionable signal is file-change detection: compare files referenced in the triage summary against commits on main since the triage timestamp. A TTL fallback (e.g., 5 days) handles cases where no specific files are referenced.

Related open issues #281 (already-fixed action) and #576 (superseded close reason) are complementary but not prerequisites. Issue #948 (convert post-triage.sh to Python) may affect implementation choices but is also not blocking.

Proposed Test Cases

1. Referenced files unchanged since triage → dispatch proceeds normally
2. Referenced file modified on main after triage → staleness flagged, dispatch gated
3. Triage summary older than TTL threshold → staleness flagged regardless of file changes
4. No specific files referenced in triage summary → TTL-only check
5. Edge case: file modified only in whitespace/comments → still flag (conservative)

Validation

The issue includes well-defined validation criteria: over 20 aged dispatches, at least 50% of flagged cases should be correctly stale, with a false-positive rate under 20%.