Skip to content

test(parsers): the hermes fixture the allowlist was standing in for - #120

Merged
pitimon merged 2 commits into
mainfrom
test/104-hermes-fixture
Jul 25, 2026
Merged

test(parsers): the hermes fixture the allowlist was standing in for#120
pitimon merged 2 commits into
mainfrom
test/104-hermes-fixture

Conversation

@pitimon

@pitimon pitimon commented Jul 25, 2026

Copy link
Copy Markdown
Owner

Summary

Closes the last DoD item on #104 — the hermes fixture the allowlist was standing in for.

The allowlist entry said a fixture "needs a real schema, not a JSONL line". It does, so this builds one. node:sqlite ships with Node 22+, already the floor here, and readHermesSessions falls back to it when the sqlite3 CLI is absent.

Allowlist 21 → 20, with max_size lowered in the same commit as the ratchet requires.

What the fixture actually covers

Schema taken from the parser's own query (rollout.js:3216), not guessed. started_at/ended_at are epoch seconds, and the bucket comes from ended_at ?? started_at.

Three sessions across two databases, because the profile walk is what is unique to this parser:

  • hermes/state.db — a finished session with every column filled, plus one with cache and reasoning at zero, which still has to satisfy the column sum rather than be excused from it
  • hermes/profiles/work/state.db — a second database that must be picked up
  • hermes/profiles/empty/ — a profile directory with no database, which must be skipped rather than throw

Verified by reading the emitted rows, not the exit code: 3 sessions → 2 buckets grouped by model, 4290+820+15000+300+120 = 20530 and 500+60+0+0+40 = 600, and claude-opus-5 present proves the profile database was read.

A flake this run surfaced, fixed in the second commit

I introduced it in #117 and had not seen it until now.

Giving the notify handler an isolated HOME was the right fix — it stopped the child resolving the developer's real ~/.tokentracker. But the handler deliberately spawns a detached background sync, and that sync now writes inside the test's own temp directory. The write can land after execFile's callback fires, so the fs.rm in finally races it and fails with ENOTEMPTY.

Cleanup retries rather than waiting on a process the handler detaches on purpose. Verified by running the file 5 times: 19 pass, 0 fail, every run — a single green run proves nothing about a race.

Process error worth recording

The hermes commit landed on a red gate. I ran ci:local and chained the commit after echo "exit=$?", which always succeeds, so the && never gated on the failing suite. Fixed by capturing the exit code into a variable and testing it. The failure was the flake above, now fixed, and the branch is green — but the guard failed open and that is worth naming.

Test plan

  • ci:local exit 0 — 933 root tests, 262 dashboard
  • test/init-uninstall.test.js run 5× consecutively: 19 pass, 0 fail each time
  • Conformance harness: 10 pass, hermes included
  • Emitted rows inspected by hand for both databases and the column sums

Remaining on #104

20 parsers still on the allowlist, each with a recorded reason. The ratchet is what makes those get written as anyone touches them.

itarun.p added 2 commits July 26, 2026 06:07
Closes the last DoD item on issue 104. The allowlist entry said a fixture
"needs a real schema, not a JSONL line" — so this builds one.

node:sqlite ships with Node 22+, already the floor here, and readHermesSessions
falls back to it when the sqlite3 CLI is absent. If neither is available the
harness reports the fixture produced no rows rather than passing quietly — the
same guard that caught the empty kimi fixture.

Schema taken from the parser's own query (rollout.js:3216), not guessed, and
started_at/ended_at are epoch SECONDS with the bucket coming from
`ended_at ?? started_at`.

Three sessions across two databases, because the profile walk is what is unique
to this parser: a default state.db plus profiles/work/state.db, and a
profiles/empty/ with no database that must be skipped rather than throw. Also a
session with cache and reasoning at zero, which still has to satisfy the column
sum rather than be excused from it.

Verified by reading the emitted rows, not just the exit code: 3 sessions ->
2 buckets grouped by model, 4290+820+15000+300+120 = 20530 and 500+60+0+0+40 =
600, and claude-opus-5 present proves the profile database was read.

Allowlist 21 -> 20, max_size lowered in the same commit as the ratchet requires.

ci:local exit 0: 933 root tests, 262 dashboard.
A flake I introduced in #117 and did not see until the hermes fixture run.

Giving the notify handler an isolated HOME was the right fix — it stopped the
child resolving the developer's real ~/.tokentracker. But the handler
deliberately spawns a DETACHED background sync, and that sync now writes inside
the test's own temp directory. The write can land after execFile's callback has
fired, so the `fs.rm` in `finally` races it and fails with ENOTEMPTY.

Cleanup retries instead of waiting on a process the handler detaches on purpose.
16 call sites, one helper.

Verified by running the file 5 times: 19 pass, 0 fail, every run. A single green
run proves nothing about a race.

Also recording the process error that let this reach a commit: I ran `ci:local`
and chained the commit after `echo "exit=$?"`, which always succeeds, so the
`&&` did not gate on the red suite. The commit landed on a failing gate. Fixed
by capturing the exit code into a variable and testing it.

ci:local exit 0: 933 root tests, 262 dashboard.
@pitimon
pitimon merged commit 8b976c2 into main Jul 25, 2026
1 check passed
@pitimon
pitimon deleted the test/104-hermes-fixture branch July 25, 2026 23:10
pitimon added a commit that referenced this pull request Jul 26, 2026
15 commits since v0.39.43. Tracker: #125.

User-facing:
- The Projects panel honours the date range. It read no query parameter at
  all, so picking "24h" narrowed every other card while Projects kept showing
  all-time totals with nothing on screen saying so (#118).
- Per-repo cost, with rows written before the change named as unpriced rather
  than shown as a confident $0 (#121).
- Sources with no per-repo attribution are named, instead of their absence
  reading as "that tool cost nothing here" (#118).
- A plan-value card answering what README:32 already promised, labelled as
  list-price-equivalent rather than a saving (#122).
- The Codex quota chip no longer vanishes on a 401 (#119).
- `sync --compact`, which reclaimed 34,924 lines to 5,636 on a real install
  (#117).

Behind the scenes: outbound-privacy validator hardening (#111, #114), avatar
proxy per-hop revalidation, port-aware allowlisting and a real download cap
(#109, #112), a parser conformance ratchet (#116, #120), scripts/graph out of
the product gate (#115), and a usage-limits fixture capture tool (#124).

Version bumped in all four lockstep locations by the `version` hook:
package.json, package-lock.json, both TokenTrackerBar targets, and the Windows
csproj. validate:version-lockstep passes.

WATCH AFTER SHIPPING: #121 migrates cursors.json on every user's first sync
after upgrading, re-keying project buckets from project|source|hour to
project|source|model|hour. Without that the old bucket strands and its usage is
counted twice. Verified read-only against a real 2,015-bucket state — every key
migrated, total_tokens preserved exactly at 6,281,653,062, and all 2,015
queuedKey markers survived, the loss of which would re-append every row. That
is one machine; a differently-shaped state is the residual risk.

ci:local exit 0: 972 root tests, 302 dashboard.

Co-authored-by: itarun.p <itarun.p@somapait.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant