Targeted: 0.5.0 (reliability program). Baseline: release/0.3.0.
Four measurement defects hide real failures:
- Non-zero exits count as success. The exec result payload hardcodes
"ok": True (processes.py:280), so failed tests/builds/timeouts are invisible to the dashboard. Keep ok = "the tool call worked", add outcome: exited_0 | exited_nonzero | timeout | signal | spawn_error, and bucket tool_summary/session_end by it.
consecutive_failures is a single global slot cleared by any success (telemetry.py:400-407). A 17-failure patch streak interleaved with successful reads never exceeds 5. Track per (tool, error_code) (bounded, LRU ~16), cleared only by a success of the same tool.
- Summaries only flush at
finish() — a long-lived HTTP process effectively never reports them (documented in 0.3.0). Flush incrementally every N calls / T minutes.
- One shared output schema for all tools (
tool_output_schema()): command_id, next_action, truncated, status, exit_code are undeclared. Give each tool its real output schema — also a prerequisite for the structured-edit tool.
Privacy red lines stay as in 0.3.0: closed schema, no paths or content, sanitized clientInfo only.
Targeted: 0.5.0 (reliability program). Baseline:
release/0.3.0.Four measurement defects hide real failures:
"ok": True(processes.py:280), so failed tests/builds/timeouts are invisible to the dashboard. Keepok= "the tool call worked", addoutcome:exited_0 | exited_nonzero | timeout | signal | spawn_error, and buckettool_summary/session_endby it.consecutive_failuresis a single global slot cleared by any success (telemetry.py:400-407). A 17-failure patch streak interleaved with successful reads never exceeds 5. Track per(tool, error_code)(bounded, LRU ~16), cleared only by a success of the same tool.finish()— a long-lived HTTP process effectively never reports them (documented in 0.3.0). Flush incrementally every N calls / T minutes.tool_output_schema()):command_id,next_action,truncated,status,exit_codeare undeclared. Give each tool its real output schema — also a prerequisite for the structured-edit tool.Privacy red lines stay as in 0.3.0: closed schema, no paths or content, sanitized clientInfo only.