Skip to content

fix(codex): exclude subagent fork replay history from token accounting (#75)#77

Merged
pitimon merged 1 commit into
mainfrom
fix/75-codex-fork-history-overcount
Jul 21, 2026
Merged

fix(codex): exclude subagent fork replay history from token accounting (#75)#77
pitimon merged 1 commit into
mainfrom
fix/75-codex-fork-history-overcount

Conversation

@pitimon

@pitimon pitimon commented Jul 21, 2026

Copy link
Copy Markdown
Owner

What & why

Closes the Codex subagent fork-history overcount (#75). A spawned Codex subagent rollout (session_meta.payload.source.subagent.thread_spawn) replays the parent thread's cumulative token_count history before a deterministic inter_agent_communication_metadata boundary, then the genuine child turns. The parser counted every replayed parent delta as the child's consumption — inflating Cache Read by up to ~120x on multi-agent workflows (the replay can be ~99% of the file).

Root cause (verified against real local rollouts)

The cumulative total_token_usage counter is continuous across the boundary — the last replayed cumulative and the first child cumulative differ only by the child's genuine first turn. Example from a real fork file: last replay total=701736 → first child total=726896 (Δ 25160 = the child's real first turn). So the correct fix is baseline-tracking, not a reset.

The fix

  • In replay mode: advance the delta baseline (totals = totalUsage) but attribute nothing.
  • At the boundary: flip accounting on. The first child delta = firstChildCumulative − lastReplayCumulative — robust even if last_token_usage is absent.
  • src/lib/rollout.js (parseRolloutFile, the incremental path behind dashboard Cache Read): the fork lifecycle (null/replay/child) is persisted in the per-file cursor as codexFork and threaded through parseRolloutIncremental. A partial fork with no boundary yet fails closed — cursor held at its start offset, incoming baseline returned unchanged (never poisoned) — so the completed file is re-read next pass.
  • src/lib/codex-rollout-parser.js (parseCodexRolloutFile, context-breakdown drill-down): same baseline-tracking; replay also skips collecting pending tool/exec calls. Fork-control records are handled before the date/timestamp gate so a query window can never drop the boundary and leak replay back in.
  • The non-fork path keeps its original guard order (the replay-only advance is scoped to forks), so no other provider's accounting changes — locked by a regression test against a cumulative dip.
  • Stuck/partial forks are surfaced under TOKENTRACKER_DEBUG; the record-type string literals are marked as a Codex format contract.

Local-only guarantee

Parsing and tests read only local files; nothing is sent anywhere (per #75's privacy requirement).

Empirical result (measured on the primary path)

Ran the fixed parseRolloutIncremental over 153 real fork rollouts in ~/.codex with an empty cursor, versus the pre-fix main parser over the same files:

Cache Read (cached_input) Total
Before (main) 8,497,618,432 8,734,717,609
After (this PR) 204,393,984 217,757,279

~41.6x Cache Read inflation removed across the local fork corpus (the issue's 120x was one 6-file workflow; this is the aggregate over all local forks). The drill-down parser drops the same file's total_tokens from 1,922,762 → 1,221,026 (the ~700K replay excluded).

Note: this PR fixes go-forward parsing. Already-ingested aggregates keep their inflated values (those files sit at offset = EOF and are not re-read) until the follow-up migration recomputes them. #75 stays open after this merges.

Test plan

test/codex-fork-history.test.js — 9 regression tests (issue's 5 fixtures + extras):

  • normal (non-fork) rollout unaffected
  • non-fork cumulative dip does not re-baseline (no over-count on recovery)
  • complete fork counts only post-boundary child turns
  • partial fork (no boundary) ingests nothing and does not advance the cursor
  • fork resumes correctly once the boundary is written
  • child turns appended in a later incremental read counted exactly once
  • multiple subagents forked from the same parent are isolated
  • drill-down parser child-only (plain + day-scoped window)
  • npm run ci:local green (root suite 760/760, validators, dashboard build)

Follow-up (separate PR, per review)

Idempotent one-time rebuild of already-inflated Codex queue + project aggregates (reset codex cursors + recompute) with a no-network acceptance test. Staged separately because it mutates persisted local data.

Refs #75

Codex subagent forks (session_meta.source.subagent.thread_spawn) begin by
replaying the parent thread's cumulative token_count history before a
deterministic inter_agent_communication_metadata boundary, then the genuine
child turns. The rollout parser counted every replayed parent delta as the
child's consumption, inflating Cache Read by up to ~120x on multi-agent
workflows where the replay is ~99% of the file (issue #75).

The cumulative total_token_usage counter is continuous across the boundary
(verified against real local rollouts), so the fix is baseline-tracking, not
reset: in replay mode the delta baseline still advances but nothing is
attributed; accounting flips on at the boundary, so the first child delta is
measured from the last replayed cumulative. This is robust even when
last_token_usage is absent.

- src/lib/rollout.js: parseRolloutFile (incremental path behind the dashboard
  Cache Read totals). The fork lifecycle (null/replay/child) is persisted in
  the per-file cursor as codexFork and threaded through parseRolloutIncremental.
  A partially-written fork with no boundary yet fails closed: the cursor is held
  at its start offset and the incoming baseline is returned unchanged (never
  poisoned), so the completed file is re-read next pass.
- src/lib/codex-rollout-parser.js: parseCodexRolloutFile (context-breakdown
  drill-down). Same baseline-tracking; replay also skips collecting pending
  tool/exec calls. Fork-control records (session_meta, boundary) are handled
  before the date/timestamp gate so a query window can never drop the boundary
  and leak replayed history back in.
- The non-fork path keeps its original guard order untouched (the replay-only
  baseline advance is scoped to forks), so no other provider's accounting
  changes. A regression test locks this against a cumulative dip.
- Stuck/partial forks are surfaced under TOKENTRACKER_DEBUG; the record-type
  string literals are marked as a Codex format contract.

test/codex-fork-history.test.js adds 9 regression tests: normal unaffected,
non-fork dip no-rebaseline, complete fork child-only, partial fork fail-closed,
resume once the boundary lands, incremental child growth counted once,
multi-subagent isolation, and the drill-down parser (plain + day-scoped).

Refs #75

Co-authored-by: itarun.p <itarun.p@somapait.com>
@pitimon
pitimon merged commit 23fc2d1 into main Jul 21, 2026
1 check passed
@pitimon
pitimon deleted the fix/75-codex-fork-history-overcount branch July 21, 2026 23:58
pitimon added a commit that referenced this pull request Jul 22, 2026
…78)

The parser fix (PR #77) only corrects go-forward parsing. Files ingested
before it sit at offset = EOF and are never re-read, so their fork-inflated
contribution (up to ~40x Cache Read on the local corpus) stays baked into the
persisted aggregates and the dashboard. This adds the one-time rebuild that
retracts that inflated history and forces a clean re-parse.

migrateCodexForkOvercountRebuild (run-once via a migrations sentinel, modeled
on migrateRolloutCumulativeDeltaBuckets, scoped to source=codex/every-code so
no other provider is touched):

- Drops the per-file cursors so the same-run parseRolloutIncremental re-parses
  from offset 0 with the fixed accounting (and clears the obsolete codexFork
  field).
- Retracts the inflated hourly buckets: deletes them from the accumulator and
  appends zero rows to queue.jsonl. The dashboard keep-lasts per
  source|model|hour_start, so the re-parse's corrected row supersedes the zero,
  or the zero stands for a file no longer on disk.
- Resets the project accumulator buckets WITHOUT writing zero rows. Files still
  on disk rebuild and their corrected project row wins via keep-last; a bucket
  whose file was deleted or whose repo moved keeps its prior queue value rather
  than being zeroed. Unlike the hourly path (whose orphans were already
  retracted by the earlier delta migration), project aggregates were never
  retracted before, so a zero here would be a brand-new, unrecoverable loss.

Local-only: no network. The acceptance test runs cmdSync with a rejecting
fetch stub and asserts zero network calls, that the seeded 8.2M-token inflated
codex bucket becomes child-only (cached 80 / total 130) on the queue.jsonl
read path, and that a non-codex row is byte-identical. Additional tests cover
the direct migration (codex reset, other providers untouched), orphaned
project buckets (queue row preserved, no loss), and sentinel idempotency.

Refs #75

Co-authored-by: itarun.p <itarun.p@somapait.com>
pitimon added a commit that referenced this pull request Jul 22, 2026
Ships the issue #75 Codex subagent fork-history fix that is already on main:
- #77 parser fix (exclude replayed fork parent history from token accounting)
- #78 one-time rebuild migration for already-inflated Codex aggregates

The version bump is what lets npm-publish.yml actually publish main (the job
skips while the package.json version already exists on npm). Required so the
dashboard LaunchAgent and `npx @ipv9/tokentracker-cli sync` pick up the fixed
parser + migration instead of the still-inflated 0.39.38.

Co-authored-by: itarun.p <itarun.p@somapait.com>
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant