Harden end-to-end workflow-run cost capture and attribution

[manishiitg]

Problem

The product can appear to capture only builder/Pulse costs even though workflow execution cost support exists. The current implementation writes builder-style phases to costs/phase and normal workflow runs to costs/execution, and /api/workflow/costs merges both. Real workspace ledgers confirm that execution entries are produced.

However, complete workflow-run capture is not guaranteed end to end:

• ContextAwareEventBridge persists each token_usage event asynchronously.
• WaitForTokenPersistence(...) exists but currently has no production caller, so workflow finalization/process shutdown can race outstanding cost writes.
• The current real workflow cost test proves provider event -> bridge -> mock persister bucketing, but not a complete scheduled/full workflow -> on-disk execution ledger -> /api/workflow/costs -> UI path.
• Missing pricing for a newly introduced/effective CLI model can produce token usage without a dollar total, which is easy to misread as “cost not captured.”
• Builder/Pulse, execution, and evaluation are separate ledgers, so the UI/report must clearly label and total all three without double counting.

Evidence

Relevant implementation:

• agent_go/pkg/orchestrator/context_aware_bridge.go: intercepts token_usage and calls asynchronous persistence; defines WaitForTokenPersistence.
• agent_go/pkg/orchestrator/base_orchestrator_tokens.go: writes workflow runs to costs/execution/<group>/<date>.json and phase agents to costs/phase.
• agent_go/cmd/server/cost_storage.go and workflow_review_data.go: merge phase, execution, and evaluation ledgers for /api/workflow/costs.
• frontend/src/components/workflow/CostsPopup.tsx: has separate builder and automation-run sections.
• agent_go/pkg/orchestrator/workflow_cost_e2e_real_test.go: tests bridge bucketing with a mock persister, not a complete workflow run.

Observed local data also contains recent costs/execution/.../*.json files with per-run, per-step, and per-model usage, so this is a reliability/coverage/visibility gap rather than a total absence of workflow execution capture.

Proposed fix

1. Drain pending token persistence before every normal, scheduled, Pulse, and evaluation workflow finalization path. Persistence failure should be logged and surfaced as telemetry degradation without changing an otherwise successful workflow result.
2. Add a deterministic end-to-end test that runs a small multi-step workflow with a fake token-emitting provider and verifies:
   • every step is present in costs/execution;
   • the run folder and step IDs are correct;
   • /api/workflow/costs returns execution, phase, and evaluation buckets correctly;
   • totals do not double count cumulative token events.
3. Add a scheduled-run regression test, because scheduled/Pulse execution uses different orchestration entry points.
4. Make missing/unpriced usage explicit in API/UI/reporting: show captured tokens/calls plus “pricing unavailable” instead of silently showing zero/blank cost.
5. Add a per-run reconciliation signal: emitted token events vs persisted calls/steps, so Pulse can report incomplete telemetry.
6. Ensure the cost UI has a clear combined total and separate subtotals for:
   • workflow execution;
   • evaluation;
   • builder/Pulse/maintenance overhead.

Acceptance criteria

• A completed workflow cannot finalize while its token persistence goroutines are still pending.
• Normal chat-triggered, full-workflow, scheduled, Pulse, and evaluation runs are covered by tests.
• Each workflow run is visible in the execution-cost section with step/model attribution.
• Unpriced models are reported as unpriced usage, not as zero-cost or missing usage.
• Builder/Pulse overhead remains separately attributable and combined totals do not double count.
• Persistence failures are visible in logs and cost/report UI as incomplete telemetry.