feat(copilot): read per-request input/cache from session-store.db

CHANGELOG.md

@@ -11,6 +11,9 @@
 - **Codex throughput tracking**: per-model Tok/s in the dashboard and report, active time excludes tool wait. (#805, thanks @ihearttokyo)
 - `codeburn sync push --attribution` (opt-in): sends git attribution spans — the session→commit correlation from `codeburn yield` (`codeburn.session.attribution` and `codeburn.commit` span types with normalized repo remote, commit SHAs, merged/reverted state, and PR links). Nothing new is sent without the flag; local-only repos and Windows filesystem paths are never emitted as repo identities, and sessions whose project path no longer resolves never inherit the push-time working directory's repo. See docs/sync/README.md "Git attribution".
 
+### Added (CLI)
+- **Copilot input/cache tokens are read per request from `~/.copilot/session-store.db`.** The Copilot CLI and the GitHub Copilot desktop app both write this SQLite store unconditionally, one `assistant_usage_events` row per API request — where the `session.shutdown` rollup in `events.jsonl` appears only on clean shutdown (a crash silently lost the whole session's input/cache accounting), lumps each session leg into one per-model total, and resets its counters at in-session compaction (a clean 107-request session's sole rollup was observed covering exactly its five post-compaction requests), so even cleanly-closed long sessions were underreported. Sessions the store covers now take their input, cache-read, cache-write and reasoning tokens from the DB rows, per request and with real timestamps; their now-redundant shutdown rollups are dropped at serve time — the serve set is one coherent snapshot, so nothing double-counts no matter how or when the two representations reached the cache; sessions from CLI builds predating the table (or a missing store) keep the rollup path unchanged, and a transiently locked store defers only its own re-read while previously read rows keep serving (and keep taking precedence). Per-turn output stays with the `events.jsonl` per-turn calls. Copilot reasoning tokens are no longer double-billed: they are a subset of the output tokens the per-turn calls already price (per the store's own `token_details_json`), but the query-time cost recompute charged them again at the output rate on the supplementary input/cache calls. The copilot session cache takes a parse-version bump and the daily cache bumps from v17 to v18 so already-finalized days re-derive with per-request day attribution and the corrected reasoning pricing. Billing-grade cost from `total_nano_aiu` and throughput from the store's latency columns are follow-ups (#890).
+
 ### Fixed (CLI)
 - **Copilot CLI sessions report their input and cache tokens.** The Copilot CLI writes the same `producer: 'copilot-agent'` in its `session.start` events that VS Code transcripts carry, so content-based detection classified every CLI session as a transcript and skipped its `session.shutdown` rollup — the only place the CLI records input, cache-read and cache-write tokens — leaving cache hit rate at 0.0% and dramatically underreporting cost. Whether a file is a transcript is now decided by where discovery found it, never by its contents. Resumed sessions, whose legs each append a cumulative rollup, are billed as per-leg deltas so a growing session never double-counts or goes stale; the GitHub Copilot desktop app writes the same session store, so its usage is covered by the same fix. The copilot session cache takes a parse-version bump and the daily cache bumps from v16 to v17 for the one-time re-parse that heals already-recorded days whose logs still exist. (#944)
 - **Copilot CLI subagent runs are attributed to their agent.** Newer CLIs announce delegation with `subagent.started`/`subagent.completed` rather than `subagent.selected`, so delegated turns lost their agent label; the label now also clears when the subagent completes instead of bleeding onto the parent's later turns. Rides the #944 re-parse, so already-cached sessions gain the attribution. (#944)

src/daily-cache.ts

@@ -5,7 +5,15 @@ import { homedir } from 'os'
 import { join } from 'path'
 import type { DateRange, ProjectSummary } from './types.js'
 
-// Bumped to 17: copilot CLI sessions were misclassified as VS Code transcripts
+// Bumped to 18: copilot input/cache tokens for sessions covered by the CLI's
+// session-store.db moved from one shutdown-rollup lump (stamped at session
+// end) to per-request DB rows with real timestamps. Lifetime totals barely
+// move, but per-day attribution does — a session straddling midnight now
+// lands its input/cache on the days the requests actually happened — so days
+// finalized at v17 would disagree with the live parse. Re-derivation rides
+// the v14 carry-forward semantics; sourceless days carry forward as-is.
+//
+// v17: copilot CLI sessions were misclassified as VS Code transcripts
 // (#944), so days finalized at v16 or earlier carry output-only copilot costs —
 // the session.shutdown rollup's input/cache tokens were dropped. Raising
 // MIN_SUPPORTED_VERSION forces the one-time re-derivation under the
@@ -73,8 +81,8 @@ import type { DateRange, ProjectSummary } from './types.js'
 // that older binaries skipped. v8 added local-model savings to the daily
 // rollup; the `savingsConfigHash` field is invalidated separately when the
 // user changes their `localModelSavings` mapping.
-export const DAILY_CACHE_VERSION = 17
-const MIN_SUPPORTED_VERSION = 17
+export const DAILY_CACHE_VERSION = 18
+const MIN_SUPPORTED_VERSION = 18
 // Version-suffixed so different binaries each own a distinct file and never
 // clobber an incompatible schema. Bumping the version mints a fresh filename;
 // adoptOlderDailyCaches then unions days out of every previous file (including

src/parser.ts

@@ -2524,7 +2524,14 @@ function providerCallsToCachedTurns(calls: ParsedProviderCall[]): CachedTurn[] {
 
 function cachedCallToApiCall(call: CachedCall): ParsedApiCall {
   const u = call.usage
-  const outputForCost = call.provider === 'claude'
+  // Claude thinking and Copilot reasoning tokens are already INSIDE
+  // outputTokens (Copilot's own per-request token_details_json prices
+  // input/cache/output and nothing else, and its reasoning counts are a
+  // subset of the output count), so adding them here would bill them twice —
+  // for copilot literally so: its session-store/shutdown supplementary calls
+  // carry reasoningTokens with outputTokens 0 while the per-turn call bills
+  // the full output. Other providers report reasoning separately from output.
+  const outputForCost = call.provider === 'claude' || call.provider === 'copilot'
     ? u.outputTokens
     : u.outputTokens + u.reasoningTokens
   const costUSD = calculateCost(
@@ -3039,11 +3046,18 @@ async function parseProviderSources(
     }
   }
 
-  // 90-day age-out for durable providers: remove entries whose newest call is
-  // older than 90 days so the cache doesn't grow unboundedly over time.
+  // 90-day age-out for durable providers: remove ORPHANED entries whose
+  // newest call is older than 90 days so the cache doesn't grow unboundedly.
+  // Entries whose path is still discovered are exempt — their source can be
+  // re-read at will, so there is no unbounded-orphan growth to cap, and
+  // pruning them zeroes data the source still holds: an idle machine whose
+  // session-store rows are all >90d old would otherwise serve NOTHING for
+  // those sessions while their events.jsonl rollups stay suppressed by the
+  // store's coverage (the store must be servable wherever it suppresses).
   if (!readOnly && provider.durableSources) {
     const cutoffMs = Date.now() - 90 * 24 * 60 * 60 * 1000
     for (const [cachedPath, cachedFile] of Object.entries(section.files)) {
+      if (allDiscoveredFiles.has(cachedPath)) continue
       const newestTs = cachedFile.turns
         .flatMap(t => t.calls)
         .map(c => new Date(c.timestamp).getTime())
@@ -3056,6 +3070,42 @@ async function parseProviderSources(
     }
   }
 
+  // Copilot rollup-vs-store precedence, enforced at SERVE time — the sole
+  // suppression mechanism. Parsers cache both representations of a covered
+  // session unconditionally (per-request store rows and the shutdown
+  // rollup); the serve set is the one coherent snapshot, so dropping rollup
+  // calls here whenever it holds store calls for that session cannot be
+  // raced by writers between a probe and a parse, and heals any path into
+  // the cache (stale coverage epochs, restored files, historical
+  // double-caching). Only a still-DISCOVERED store suppresses: rows cached
+  // from a store since deleted or reset must not shadow fresh rollups that
+  // are now the only live record — during such an absence epoch the overlap
+  // legs may double-count for up to 90 days until the orphaned store entry
+  // ages out (accepted; the inverse choice would zero live sessions
+  // forever). Gates on the discovered sources, not allDiscoveredFiles,
+  // because read-only runs add cached orphans to the latter — an absent
+  // store must not suppress in read-only when it wouldn't in a refresh.
+  let copilotStoreSessions: Set<string> | null = null
+  if (providerName === 'copilot') {
+    const discoveredPaths = new Set(sources.map(s => s.path))
+    for (const [cachedPath, cachedFile] of Object.entries(section.files)) {
+      if (!discoveredPaths.has(cachedPath)) continue
+      for (const turn of cachedFile.turns) {
+        if (turn.calls.some(c => c.deduplicationKey.startsWith('copilot-store:'))) {
+          ;(copilotStoreSessions ??= new Set()).add(turn.sessionId)
+        }
+      }
+    }
+  }
+  const dropSuppressedRollups = (turn: CachedTurn): CachedTurn | null => {
+    if (!copilotStoreSessions?.has(turn.sessionId)) return turn
+    const kept = turn.calls.filter(
+      c => !c.deduplicationKey.startsWith(`copilot:${turn.sessionId}:shutdown:`)
+    )
+    if (kept.length === turn.calls.length) return turn
+    return kept.length > 0 ? { ...turn, calls: kept } : null
+  }
+
   // Query-time: derive SessionSummary from all cached turns.
   // Uses seenKeys (shared across providers) for cross-provider dedup.
   const sessionMap = new Map<string, { project: string; projectPath?: string; workingDirectory?: string; turns: ClassifiedTurn[]; prLinks?: Set<string>; title?: string }>()
@@ -3064,7 +3114,9 @@ async function parseProviderSources(
     const cachedFile = section.files[source.path]
     if (!cachedFile) continue
 
-    for (const turn of cachedFile.turns) {
+    for (const rawTurn of cachedFile.turns) {
+      const turn = dropSuppressedRollups(rawTurn)
+      if (!turn) continue
       const hasDup = turn.calls.some(c => seenKeys.has(c.deduplicationKey))
       if (hasDup) continue
 
@@ -3121,7 +3173,9 @@ async function parseProviderSources(
     for (const [cachedPath, cachedFile] of Object.entries(section.files)) {
       if (allDiscoveredFiles.has(cachedPath)) continue  // already counted above
 
-      for (const turn of cachedFile.turns) {
+      for (const rawTurn of cachedFile.turns) {
+        const turn = dropSuppressedRollups(rawTurn)
+        if (!turn) continue
         const hasDup = turn.calls.some(c => seenKeys.has(c.deduplicationKey))
         if (hasDup) continue