feat(usage): add canonical token accounting v2

[seakee]

Summary

Adapt CPA token accounting v2 to the current dev architecture and make its canonical token_breakdown the source of truth for persisted usage, analytics, and cost estimation.
Legacy usage remains compatible, while ambiguous or malformed accounting is preserved as unclassified and excluded from unsafe pricing.

Scope

• Frontend panel
• Manager Server
• CPA panel mode
• Full Docker mode
• Native packages / release
• Docs / Wiki
• CI / build / tooling

Changes

• Add canonical, non-overlapping input, cache-read, cache-write, non-reasoning output, reasoning, and unclassified token buckets with strict validation and conservative legacy fallback.
• Add resumable SQLite backfill and derived-data rebuild state for accounting v2, including saturating token aggregation and fail-closed schema/version handling.
• Propagate accounting quality through usage events, projections, monitoring, hourly/price rollups, dashboard summaries, JSONL import/export, and cost calculation.
• Present canonical token composition and incomplete-estimate warnings in realtime monitoring, usage analytics, dashboard token mix, and isolated demo fixtures.

User Impact

Users get consistent token totals and cost estimates across realtime events, monitoring, dashboard, and analytics.
Reasoning and cache buckets no longer overlap, and requests with ambiguous token semantics remain visible as unclassified instead of being guessed or incorrectly billed.

Compatibility / Runtime Notes

• CPA panel mode: Supports CPA accounting-v2 payloads while retaining provider-aware fallback for older CPA versions and legacy usage payloads.
• Manager Server mode: Adds nullable canonical columns, runs a resumable background backfill, and rebuilds affected derived projections/rollups from usage_events.
• Full Docker / native packages: No configuration change. Existing databases migrate automatically; migration progress is exposed through the existing system status surface.

Data / Security Notes

SQLite schema and derived rollup state are updated. usage_events remains the source of truth, original legacy fields and raw provenance are retained, and malformed canonical claims fail closed into unclassified totals.
No new credentials, secrets, external telemetry, or raw failure-body exposure are introduced.

Risk / Rollback

Risk level: Medium

Rollback notes:

Stop the Manager Server and restore a pre-upgrade SQLite backup before returning to an older binary. The migration is additive and preserves source usage events, but pricing and monitoring derived-state versions advance to accounting-v2 schemas and should be rebuilt by the matching application version.

Verification

• Type check
• Lint
• Tests
• Build
• Manual UI check
• Docs/link check
• Not applicable, docs-only

Commands / evidence:

GOCACHE=/tmp/cpamp-usage-accounting-go-cache go test ./...
GOCACHE=/tmp/cpamp-usage-accounting-go-cache go test -race ./...
GOCACHE=/tmp/cpamp-usage-accounting-go-cache go vet ./...
npm run type-check
npm run lint
npm run test
npm run build
npm run check:demo-isolation
git diff --check

Go full suite: passed, including migration, SQLite, monitoring, pricing, rollup, service, and worker packages.
Go race suite: passed.
Frontend: 189 test files / 2321 tests passed.
Lint: passed with one pre-existing Fast Refresh warning in AccountHealthBadge.tsx.
Production single-file build and post-build demo-isolation check: passed.

Screenshots / Recordings

N/A — this extends token data shown inside existing dashboard, analytics, and realtime layouts; component and presentation tests cover the changed states.

Docs

• README / README_CN updated for user-visible capabilities
• Matching docs manual and navigation updated
• Demo fixtures, screenshots, and deep links reviewed
• Release notes needed
• Not needed — explanation included below

Docs decision:

No new setup or operator workflow is introduced. The next release notes should call out the automatic SQLite backfill, canonical token composition, and incomplete-cost semantics.

Related

N/A