Phase F: skill-eval description optimizer + HTML viewer (+ telegram isolation, applied map-plan description)

[azalio]

Phase F — Skill-Eval Description Optimizer + HTML Viewer (+ fixes)

Builds the optimization/reporting layer on top of the shipped F.1 eval engine, plus two fixes surfaced by a real end-to-end run.

What's in it

Phase F (9 subtasks, each Monitor-approved):

• eval_schema.py: OptimizeIterationRecord / OptimizeResult / ProposerFn (round-trip to_dict/from_dict).
• proposer.py: claude -p description proposer (argv-list, MAP_INVOKED_BY, None on all failures).
• description_optimizer.py: anti-overfit optimizer — deterministic hashlib 60/40 split, selection by held-out TEST pass-rate (overfit candidates flagged and never selected), per-iteration resume=False jsonl isolation, candidate .claude/ re-seed cleaned in finally. Clock-free (caller supplies run_ts).
• viewer.py: jinja2 HTML report — autoescape=True (XSS-safe), difflib per-iteration diff, overfit rows red.
• apply_patcher.py: fail-loud block-scalar frontmatter --apply patcher → re-render to byte-identical trees, scoped git add, path-safe.
• optimize / view CLI subcommands (exit 2/0/1, dry-run budget, --open best-effort, JSON+HTML artifacts).
• 3 optimizer eval-set fixtures (≥8 entries) + authoring note; no-anthropic guard over all 4 new modules.

Fix 1 — eval/telegram isolation (f46e78d): the eval claude -p subprocess inherited the user's telegram-bridge SessionStart hook (tg listen), which could block on the Telegram long-poll until the dispatch timeout (a triggered-skill cell then mis-records as a non-trigger). The dispatcher now sets TG_STATE_DIR to a config-less path so any tg/tg listen the agent runs exits immediately instead of blocking. (Best-effort: the SessionStart injection still appears via the plugin hook env, so a perfectly-obedient agent can still spend a few turns on it; the hard block is removed.)

Fix 2 — apply optimized map-plan description + correct the description cap (b1c515f): a real optimize run on map-plan selected a candidate that doubled the held-out TEST pass-rate (25%→50%, no overfit); applied it. The winner is 622 chars, which the old self-imposed 250-char test rejected — but 250 was a transient Claude Code v2.1.86 cap (raised to 1536 in v2.1.105, replaced by a usage-ranked listing budget in v2.1.129+). The Agent Skills spec maximum is 1024 chars and the model loads the full description for triggering, so the test now enforces 1024 and the proposer caps candidates at 1024 (prompt instruction + hard rejection of over-limit proposals). Refs: anthropics/claude-code #40121, #47627; code.claude.com/docs/en/skills.

Real run evidence (2 skills, not mocked)

• map-plan: winner = iter 1 — baseline 20%/25% (train/test) → candidate 20%/50%, selected on held-out test, not overfit.
• map-debug: no improvement (full tie → baseline retained), 25%.

Verification

• make check (lint + pyright 0/0/0 + full pytest) and make check-render green on a clean run (2247→2256 tests as the suite grew across subtasks).
• Final Verifier (Ralph loop): PASS.

Note for reviewers: on the author's machine, two timing-sensitive hook tests flake under heavy local load + an unusually large active-session transcript (test_hook_inventory_smoke::test_every_configured_hook_execs_via_shebang — resolve_session_id scanning a multi-MB transcript; test_safety_guardrails::test_execution_under_100ms — 116ms vs 100ms). Both are unrelated to this diff (it touches no hooks/memory) and pass in clean CI.

🤖 Generated with Claude Code