Skip to content

docs: agent failure modes — validated catalogue + structural mitigations#131

Merged
Jammy2211 merged 1 commit into
mainfrom
feature/agent-failure-modes
Jul 16, 2026
Merged

docs: agent failure modes — validated catalogue + structural mitigations#131
Jammy2211 merged 1 commit into
mainfrom
feature/agent-failure-modes

Conversation

@Jammy2211

Copy link
Copy Markdown
Contributor

Summary

Design deliverable for #130 (build-chain campaign PyAutoBuild#155, Phase 5 — deliberately last): docs/agent_failure_modes.md. Docs only; each proposed mechanism lands as its own PR behind its own plan.

What it contains, beyond the brief:

  • The original A–E catalogue validated by measurement (git 2.34.1 ✓; repos.yaml now 33; smoke_tests.txt 11 ✓) and enlarged out-of-sample with this campaign's own live errors (F1–F5): the shared-index sweep repeated ×3 in one day with a same-morning memory note about it, three piped-exit/silent-flag instrument errors, one wrong checkpoint claim published then caught by a deliberate falsifying probe, two stale-claim refusal catches, one API-gate false positive.
  • The taxonomy compressed to two root causes: verification aimed at the believed path, and state consulted from memory instead of the world.
  • The hypothesis tested and sharpened: refusals still catch ~100% of what they guard; informs still catch ~0% — including the agent's own live self-awareness (F3 was committed mid-campaign about that exact error shape). The dividing line is whether anything interrupts the action. Refusals now carry a measured false-positive cost column (F5).
  • Six ranked mitigations with per-entry catch mapping, cost, and failure mode — led by set -o pipefail as the agent-shell default (deletes F2's class outright, one line) and a Mind commit guard (the E1/F1 four-incident class, converted from the memory note that failed).
  • An honest verdict on memory (works consult-on-purpose, fails as tripwire — convert guard-shaped notes into refusals) and a first estimate of the unnoticed-error set (~7% caught-late rate among boundary-crossing claims → expect 1–3 undetected false statements live in the record).

Parallel PR from main (docs/ only; no file overlap with any active claim). Shipped under the human-authorized corrective-PR exception for Heart RED (standing this session). PR-open only; merge stays human.

API Changes

None — documentation only.

Test Plan

  • PyAutoBrain suite in the worktree (gate leg).
  • Every §1 validation number carries its command; §2 incidents cite the campaign issue trail.

Generated by the PyAutoLabs agent workflow.

…ons (#130)

Design deliverable for the build-chain campaign (PyAutoBuild#155 Phase 5):
the 2026-07-15 A-E catalogue validated by measurement and enlarged
out-of-sample with this campaign's own errors (F1-F5, incl. the shared-index
sweep repeated x3 with a same-morning memory note about it); taxonomy
compressed to two root causes; the refusals-vs-informs hypothesis tested and
sharpened (informs include the agent's own live self-awareness) with a
measured refusal false-positive cost; six ranked mitigations led by
pipefail-as-default and a Mind commit guard; memory verdict
(consult-on-purpose yes, tripwire no); first estimate of the unnoticed-error
set. Docs only; each mechanism is its own future PR.

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant