IMPLEMENTATION_CHECKLIST.md

Implementation Checklist

P0 — Reliability

• Retry empty model outputs.
• Preserve raw model response, response ID, request ID, status, finish reason, token usage, and errors.
• Classify silent empty output as infrastructure failure rather than agent failure.
• Declare and enforce a single 0-4 rating scale.
• Store candidate output and judge capture separately.
• Use atomic artifact writes for live readers.

P1 — Coverage semantics

• Track attempted, model-completed, judged, policy-blocked, and infrastructure-failed counts.
• Mark incomplete required coverage as inconclusive.
• Support one concrete simulated clarification reply.
• Include a frozen-plan execution path.
• Add multi-judge aggregation and disagreement reporting.
• Add deterministic JSON Schema validation of agent final manifests.

P2 — TUI

• Responsive 32-column layout.
• Number and letter controls suitable for phone keyboards.
• Live artifact refresh.
• Overview, case list, case detail, and help screens. (BUG-1 and BUG-2 resolved — TUI uses correct RunMetadata field names and per-agent judgment lists. Infrastructure error display added.)
• Planner capture diagnostics in TUI overview and benchdeck inspect.
• Launch/pause/cancel subprocess runs from inside the TUI.
• Export selected case as Markdown.

P3 — Distribution

• Python package and console entry point.
• CI across Python 3.11-3.13.
• Regression fixture and tests.
• Publish package release. (CI workflows for publish and release with SBOM exist; .github/workflows/publish.yml now supports both PYPI_API_TOKEN and OIDC Trusted Publishing. See docs/publish.md.)
• Add signed release artifacts and SBOM.