test(parsers): a new parser could land with no test and no CI objection

[pitimon]

Summary

Closes the harness half of #104.

Coverage here is inverted relative to risk. CLAUDE.md:169-172 records the historical failure modes — dedup failures, cached-input semantics, snapshot-vs-cumulative confusion — as 1.6× to 7× magnitude errors in the numbers the whole product exists to report. Four parsers (hermes, codebuddy, craft, kimi) had no test at all, and README:116 invites contributors to add more.

The ratchet

test/parser-conformance.test.js enumerates the parsers from source, not from a hand-written list — 24 found. Each must have either a conformance fixture or an allowlist.json entry saying why not. A parser added without either fails; it cannot be silently skipped.

The allowlist fails if it grows past max_size, and fails if it is shorter — telling you to lower max_size in the same commit so the ratchet records the ground gained. An allowlist that can grow is a TODO list.

Per fixture: the column invariant total = input + output + cache_creation + cache_read + reasoning, non-negative counts, a model present, a 30-minute UTC bucket, and that re-parsing unchanged input adds nothing — the "run sync twice" lesson, which is what catches dedup-key instability.

Fixtures for three of the four named parsers: craft, kimi, codebuddy. hermes is on the allowlist because it reads a SQLite database across profile directories, which needs a real schema rather than a JSONL line.

Two things found while building it

Both are recorded in the code rather than smoothed over.

1. The column invariant does not catch cache-read double-counting. A parser that folds cache reads into input_tokens and inflates total_tokens to match is internally consistent — the sum balances while the user's number is ~4× too big. That is precisely the class this issue is about, and this harness does not catch it. There is now a test pinning that blind spot so nobody reads the harness as covering more than it does. Only a real sample log with known-correct expected values catches it, which is what the per-tool tests are for.

2. The kimi fixture initially produced ZERO rows and would have passed as a green test proving nothing. The harness asserts a fixture yields rows before checking them, and that guard caught it. Cause: kimi's timestamp is epoch seconds and the fixture omitted it entirely.

Why the codebuddy fixture is the interesting one

CodeBuddy's prompt_tokens is OpenAI-style and includes cached tokens, so input_tokens = prompt_tokens - cache_read. The fixture feeds 3000 prompt tokens of which 2400 are cache reads, and the parser emits input_tokens: 640 — the subtraction, held still.

Test plan

• ci:local exit 0 — 918 root tests (+9), 256 dashboard
• The conformance checker is tested against four known-bad rows plus a valid control, so it is not a checker nothing has ever failed
• All three fixtures produce rows and pass the invariants; verified the actual emitted rows by hand, not just the exit code
• CONTRIBUTING.md step 4 documents the fixture requirement and, explicitly, what it does not prove

Still open on #104

A hermes fixture, and fixtures for the 21 allowlisted parsers. The ratchet is what makes those get written as anyone touches them.