Add token usage metrics to agentsview

[tpn]

Summary

Add a first-pass Tokens metric to agentsview so token usage is visible at the message, session, and aggregate analytics levels.

Today agentsview does a good job tracking things like messages, sessions, activity, and duration, but it is still hard to answer basic questions like:

• How many tokens were involved in this message?
• How many tokens did this session use overall?
• Which projects / agents / time windows consumed the most tokens?

Current state

This is not completely greenfield:

• agentsview already stores per-message token data where parsers provide it
• sessions already store total_output_tokens and peak_context_tokens
• the session header already shows a ctx / out badge for the active session

So the main gap is not basic persistence. The harder problem is defining and surfacing a coherent Tokens metric across heterogeneous agents and models.

Recommended v1

Treat tokens as raw provider-reported usage, not as a normalized cross-model estimate.

That means:

• show token counts when the underlying agent/session format explicitly reports them
• allow blanks / partial values when a source does not provide token usage
• avoid inventing a fake cross-model equivalent token number in v1
• keep the metric honest and explainable even if coverage is uneven at first

This seems like the best first pass because it is the easiest to trust and the easiest to explain.

Why not normalize in v1?

Tokenizers differ between models and vendors, so a single normalized token metric would be tempting but misleading.

A more aggressive approach could try one of these:

1. raw reported tokens plus a separate estimated normalized metric
2. fully estimated / normalized tokens across all agents and models

Both are harder to defend, harder to maintain, and much easier to misinterpret. v1 should stay grounded in source-reported usage.

Scope ideas for a first implementation

Possible first-pass scope:

• expose token usage consistently on message records when present
• add clearer session-level token summaries beyond the current ctx / out badge
• add aggregate token analytics alongside existing message/session analytics
• support token-based ranking / summaries where it makes sense
• make absence of token data explicit rather than silently implying zero usage

Important semantics / UX notes

A few details will matter:

• 0 and unknown / not reported should not be conflated
• metrics may be partial depending on parser coverage and provider metadata
• cross-agent comparisons should be labeled as provider-reported token usage, not normalized usage
• any aggregates should handle mixed coverage carefully so totals do not overstate completeness

Open questions

• What exact token dimensions should we surface first: context/input, output, total, peak context, cache read/write, etc.?
• Which analytics views should gain token metrics in v1?
• Should sessions/messages expose token coverage metadata so the UI can distinguish missing from zero?
• Should subagent token usage roll up into parent summaries, or remain per-session only?

Goal

Ship an honest, useful first-pass token metric that improves visibility without pretending token counts are directly comparable across every model and provider.