Skip to content

feat: per-activity and per-workflow token use tracking and cost estimation (#232)#233

Open
m2ux wants to merge 4 commits into
mainfrom
feat/232-token-use-tracking-and-cost-estimation
Open

feat: per-activity and per-workflow token use tracking and cost estimation (#232)#233
m2ux wants to merge 4 commits into
mainfrom
feat/232-token-use-tracking-and-cost-estimation

Conversation

@m2ux

@m2ux m2ux commented Jul 14, 2026

Copy link
Copy Markdown
Owner

Summary

Adds per-activity and per-workflow token use tracking and cost estimation to the workflow-server, persisting rolled-up usage in session.json with optional harness-relayed cost estimates from a versioned price table.

🐛 Issue 📐 Engineering 📋 Test plan


Motivation

Workflow sessions provide no visibility into the token consumption or monetary cost of executing a workflow. Token use cannot be attributed to individual activities, so operators cannot identify expensive activities, compare workflow efficiency across runs, or account for what a completed work package cost to execute.

Without a durable record, that information is lost the moment a session ends. Tracking usage per activity, aggregating it per workflow, and logging the results to the planning artifact gives every work package a permanent usage and cost record.


Changes

  • Server — history event — Added usage_recorded to the shared history-event vocabulary for per-activity audit events.
  • Server — session schema — Added optional usage field to SessionFile (per-activity record + child-inclusive workflowTotal), absent-until-first-use.
  • Server — serialization — Registered usage in TOP_LEVEL_KEY_PRIORITY for canonical session ordering.
  • Server — cost estimation — Added versioned config price table and estimateCost helper with derived cache multipliers; unknown model yields cost_usd null.
  • Server — roll-up — Added sumUsageTree for completion-time child-inclusive workflow totals via in-tree traversal.
  • Server — capture seam — Wired optional usage param on next_activity; record at activity exit, roll up incrementally, fold children at completion.
  • Corpus — populate instruction — Direct the orchestrator to relay harness-reported completion usage on each next_activity call (graceful degradation when unavailable) — corpus tasks 7–8 tracked separately in workflows worktree.
  • Corpus — completion renderer — Technique to render NN-token-usage.md + README summary/link from rolled-up session state on completion — corpus tasks 7–8 tracked separately.

Tests — 17 cases across unit, integration, and e2e scopes mapping to SC-1..SC-6: schema round-trip, cost estimation, child roll-up, next_activity capture, graceful omission, and completion artifact rendering.

Key decisions: Declared usage param + config price table (not OTEL ingest or server-side re-tokenization); server-first build sequence; cost labelled as estimate with subscription caveat; server owns structured state, corpus owns human-readable artifacts.


🤖 AI Assistance

  • Assistant / Model: Cursor / composer-2.5-fast
  • Context scope: mixed
  • Prompt classes: code-generation, test-writing, refactoring, docs
  • Provenance log: not recorded — task rows were not appended during implement

📌 Submission Checklist

  • Changes are backward-compatible (or flagged if breaking)
  • Pull request description explains why the change is needed
  • Self-reviewed the diff
  • I have included a change file, or skipped for this reason: repo has no changes/ folder convention
  • If the changes introduce a new feature, I have bumped the node minor version — N/A (Node MCP server, not node runtime)
  • Update documentation (if relevant)
  • No new todos introduced

🔱 Fork Strategy

  • Node Runtime Update
  • Node Client Update
  • Other
  • N/A

🗹 TODO before merging

  • Ready for review

Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
@m2ux m2ux self-assigned this Jul 14, 2026
m2ux and others added 3 commits July 15, 2026 12:08
Signed-off-by: Mike Clay <mike.clay@shielded.io>
Co-Authored-By: Claude Fable 5 <noreply@anthropic.com>
Persist harness-relayed usage at the next_activity transition seam into
session.json with optional cost estimation from a versioned price table.
Includes schema, handler wiring, usage helpers, and unit/integration tests.

Signed-off-by: Mike Clay <mike.clay@shielded.io>
@m2ux m2ux marked this pull request as ready for review July 15, 2026 11:11
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant