test(parsers): the hermes fixture the allowlist was standing in for - #120
Merged
Conversation
added 2 commits
July 26, 2026 06:07
Closes the last DoD item on issue 104. The allowlist entry said a fixture "needs a real schema, not a JSONL line" — so this builds one. node:sqlite ships with Node 22+, already the floor here, and readHermesSessions falls back to it when the sqlite3 CLI is absent. If neither is available the harness reports the fixture produced no rows rather than passing quietly — the same guard that caught the empty kimi fixture. Schema taken from the parser's own query (rollout.js:3216), not guessed, and started_at/ended_at are epoch SECONDS with the bucket coming from `ended_at ?? started_at`. Three sessions across two databases, because the profile walk is what is unique to this parser: a default state.db plus profiles/work/state.db, and a profiles/empty/ with no database that must be skipped rather than throw. Also a session with cache and reasoning at zero, which still has to satisfy the column sum rather than be excused from it. Verified by reading the emitted rows, not just the exit code: 3 sessions -> 2 buckets grouped by model, 4290+820+15000+300+120 = 20530 and 500+60+0+0+40 = 600, and claude-opus-5 present proves the profile database was read. Allowlist 21 -> 20, max_size lowered in the same commit as the ratchet requires. ci:local exit 0: 933 root tests, 262 dashboard.
A flake I introduced in #117 and did not see until the hermes fixture run. Giving the notify handler an isolated HOME was the right fix — it stopped the child resolving the developer's real ~/.tokentracker. But the handler deliberately spawns a DETACHED background sync, and that sync now writes inside the test's own temp directory. The write can land after execFile's callback has fired, so the `fs.rm` in `finally` races it and fails with ENOTEMPTY. Cleanup retries instead of waiting on a process the handler detaches on purpose. 16 call sites, one helper. Verified by running the file 5 times: 19 pass, 0 fail, every run. A single green run proves nothing about a race. Also recording the process error that let this reach a commit: I ran `ci:local` and chained the commit after `echo "exit=$?"`, which always succeeds, so the `&&` did not gate on the red suite. The commit landed on a failing gate. Fixed by capturing the exit code into a variable and testing it. ci:local exit 0: 933 root tests, 262 dashboard.
This was referenced Jul 25, 2026
Merged
pitimon
added a commit
that referenced
this pull request
Jul 26, 2026
15 commits since v0.39.43. Tracker: #125. User-facing: - The Projects panel honours the date range. It read no query parameter at all, so picking "24h" narrowed every other card while Projects kept showing all-time totals with nothing on screen saying so (#118). - Per-repo cost, with rows written before the change named as unpriced rather than shown as a confident $0 (#121). - Sources with no per-repo attribution are named, instead of their absence reading as "that tool cost nothing here" (#118). - A plan-value card answering what README:32 already promised, labelled as list-price-equivalent rather than a saving (#122). - The Codex quota chip no longer vanishes on a 401 (#119). - `sync --compact`, which reclaimed 34,924 lines to 5,636 on a real install (#117). Behind the scenes: outbound-privacy validator hardening (#111, #114), avatar proxy per-hop revalidation, port-aware allowlisting and a real download cap (#109, #112), a parser conformance ratchet (#116, #120), scripts/graph out of the product gate (#115), and a usage-limits fixture capture tool (#124). Version bumped in all four lockstep locations by the `version` hook: package.json, package-lock.json, both TokenTrackerBar targets, and the Windows csproj. validate:version-lockstep passes. WATCH AFTER SHIPPING: #121 migrates cursors.json on every user's first sync after upgrading, re-keying project buckets from project|source|hour to project|source|model|hour. Without that the old bucket strands and its usage is counted twice. Verified read-only against a real 2,015-bucket state — every key migrated, total_tokens preserved exactly at 6,281,653,062, and all 2,015 queuedKey markers survived, the loss of which would re-append every row. That is one machine; a differently-shaped state is the residual risk. ci:local exit 0: 972 root tests, 302 dashboard. Co-authored-by: itarun.p <itarun.p@somapait.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Closes the last DoD item on #104 — the
hermesfixture the allowlist was standing in for.The allowlist entry said a fixture "needs a real schema, not a JSONL line". It does, so this builds one.
node:sqliteships with Node 22+, already the floor here, andreadHermesSessionsfalls back to it when thesqlite3CLI is absent.Allowlist 21 → 20, with
max_sizelowered in the same commit as the ratchet requires.What the fixture actually covers
Schema taken from the parser's own query (
rollout.js:3216), not guessed.started_at/ended_atare epoch seconds, and the bucket comes fromended_at ?? started_at.Three sessions across two databases, because the profile walk is what is unique to this parser:
hermes/state.db— a finished session with every column filled, plus one with cache and reasoning at zero, which still has to satisfy the column sum rather than be excused from ithermes/profiles/work/state.db— a second database that must be picked uphermes/profiles/empty/— a profile directory with no database, which must be skipped rather than throwVerified by reading the emitted rows, not the exit code: 3 sessions → 2 buckets grouped by model,
4290+820+15000+300+120 = 20530and500+60+0+0+40 = 600, andclaude-opus-5present proves the profile database was read.A flake this run surfaced, fixed in the second commit
I introduced it in #117 and had not seen it until now.
Giving the notify handler an isolated
HOMEwas the right fix — it stopped the child resolving the developer's real~/.tokentracker. But the handler deliberately spawns a detached background sync, and that sync now writes inside the test's own temp directory. The write can land afterexecFile's callback fires, so thefs.rminfinallyraces it and fails withENOTEMPTY.Cleanup retries rather than waiting on a process the handler detaches on purpose. Verified by running the file 5 times: 19 pass, 0 fail, every run — a single green run proves nothing about a race.
Process error worth recording
The hermes commit landed on a red gate. I ran
ci:localand chained the commit afterecho "exit=$?", which always succeeds, so the&&never gated on the failing suite. Fixed by capturing the exit code into a variable and testing it. The failure was the flake above, now fixed, and the branch is green — but the guard failed open and that is worth naming.Test plan
ci:localexit 0 — 933 root tests, 262 dashboardtest/init-uninstall.test.jsrun 5× consecutively: 19 pass, 0 fail each timeRemaining on #104
20 parsers still on the allowlist, each with a recorded reason. The ratchet is what makes those get written as anyone touches them.