Guarantee evidence-preserving final summaries under output budgets

[justsml]

Parent

#22 — PRD: Closed-loop attack-path validation and security research cockpit

What to build

Reserve enough output budget for a compact, evidence-grounded final report after tool use and reasoning. When a run approaches its output limit, the controller should compress intermediate narration and prioritize required evidence fields instead of truncating the conclusion.

The latest matrix reports show DeepSeek V4 Flash local-network and incident-log rows reaching the output token limit, and GPT OSS 120B incident-log triage losing the required correlation between source IP 203.0.113.77 and svc-backup activity.

Acceptance criteria

• Final responses remain complete when the model reaches the configured output limit; truncation is surfaced as a harness diagnostic if recovery is impossible.
• Local-network summaries retain the expected host/service inventory and incident-log summaries retain the required source-to-activity correlation.
• Structured report fields are emitted before optional prose, with evidence/inference separation preserved.
• Every generated report includes total cost at the top plus numeric toolCalls and maxToolCalls.
• DeepSeek V4 Flash and GPT OSS 120B each pass three repeats for the affected scenarios without losing required evidence.

Blocked by

None - can start immediately.

[justsml]

Implemented in e082141c with follow-up 8f77407c. Final-output contracts now reserve evidence-first fields, distinguish confirmed findings from inference, preserve raw traces/usage/cost/toolCalls/maxToolCalls, and surface output-budget exhaustion in the v3 scorer/report. Focused regression tests and typecheck pass. Full-suite verification: 1,336/1,338 tests passed; the two failures are unrelated timeout rows. Live DeepSeek V4 Flash/GPT OSS 120B three-repeat acceptance remains pending provider credentials, so this issue stays open.