Cost Attribution & Budget Tracking per Pipeline Run

[github-actions]

Strategic Improvement

Current cost tracking is aggregate only. Add per-pipeline cost attribution with stage-level breakdown, model usage distribution, and budget forecasting. Enable users to see "this pipeline cost $8.40" with drill-down into which stages and models consumed the budget. This makes cost optimization actionable and helps hit the <$5 per pipeline target. Complements #580's cost optimizer by providing the data foundation.

Acceptance Criteria

• Per-pipeline cost logged to .claude/pipeline-artifacts/cost.json after each run
• Cost breakdown by stage (intake: $0.12, build: $6.20, review: $1.80, etc.)
• Model usage distribution (haiku: 45%, sonnet: 40%, opus: 15%)
• shipwright cost show --pipeline <issue> displays cost breakdown
• Dashboard shows cost trends: per-pipeline average over time, by template
• Budget forecasting: "at current rate, $X remaining budget covers Y more pipelines"
• Intelligence layer input: cost per stage informs adaptive timeouts and model routing

Context

• Priority: P3
• Complexity: standard
• Generated by: Strategic Intelligence Agent
• Strategy alignment: P3: Cost Efficiency

[github-actions]

⚓ Shipwright Triage — Issue Declined

This issue was reviewed by the automated triage system and does not appear to align with Shipwright's project goals.

Reason: Could not parse triage response — rejecting for safety

If you believe this is incorrect, a maintainer can override by:

1. Running the workflow manually with "Skip triage" enabled
2. Adding context to the issue and re-applying the shipwright label

For questions or support, please use Discussions instead.

[github-actions]

⚓ Shipwright Pipeline Starting

Field	Value
Template	standard
Complexity	standard
Max Iterations	20
Triage	Extends existing cost system with per-pipeline attribution, stage/model breakdown, and forecasting—aligned with Shipwright's cost intelligence mission, well-scoped acceptance criteria, spans cost tracking/CLI/dashboard/intelligence layers.
Runner	ubuntu-latest
Risk Level	unknown
Workflow	27018031055

The autonomous pipeline is processing this issue. Updates will be posted as stages complete.

[github-actions]

[github-actions]

[github-actions]

📋 Implementation Plan

Click to expand full plan (261 lines)

Implementation Plan: Cost Attribution & Budget Tracking per Pipeline Run

Issue: #613 · Template: standard · Complexity: standard · Priority: P3

Summary

Today cost tracking is aggregate: the pipeline accumulates TOTAL_INPUT_TOKENS /
TOTAL_OUTPUT_TOKENS across every stage and writes a single
cost_record … "pipeline" <issue> entry at the end
(scripts/lib/pipeline-commands.sh:1258). There is no per-stage or per-model
breakdown, and no per-run artifact.

This change adds per-stage cost attribution by snapshotting the global token
counters around each stage in the generic stage runner
(scripts/lib/pipeline-execution.sh), attributing the delta to that stage and the
model it used (stage_model_used, already computed at line 784). Aggregated
results are written atomically to .claude/pipeline-artifacts/cost.json, surfaced
via shipwright cost show --pipeline <issue>, charted on the dashboard, and
exposed to the intelligence layer for adaptive timeouts / model routing.

Minimum viable change

Snapshot tokens per stage → write cost.json → read it back in
cost show --pipeline. Everything else (dashboard trends, forecasting,
intelligence input) builds on that one artifact and degrades gracefully when it
is absent.

---

Design Decisions (ADR)

• Context: Tokens are already accumulated globally; all stages run through one
  generic loop. Decision: Compute per-stage deltas in that loop rather than
  threading new counters through every stage_* function.
  Alternatives: (a) per-stage cost_record calls inside each stage —
  rejected: ~14 edit sites, easy to miss, duplicates the existing accumulator;
  (b) reconstruct from the events log post-hoc — rejected: pipeline.cost events
  are aggregate, not per-stage. Consequence: one choke point, but Claude calls
  made via helpers other than parse_claude_tokens (e.g. haiku
  detection/classification in pipeline-detection.sh) are attributed to the
  enclosing stage, or to a synthetic other bucket — acceptable, and the stage
  sum still equals the total.
• Storage: Reuse cost_calculate + COST_MODEL_RATES from sw-cost.sh for
  the dollar math so pricing stays single-sourced. cost.json is a derived
  artifact, not the system of record — costs.json / SQLite remain
  authoritative for budgets. Small blast radius; rollback = delete the file.
• Forecasting: Pure arithmetic over historical per-pipeline averages
  (remaining_budget / avg_pipeline_cost). No new persistence; reads existing
  costs.json stage=="pipeline" entries + budget.json /
  cost_remaining_budget.
• Dependency direction: the new lib lives in scripts/lib/ and is sourced

✅ Task Checklist

• Task 1: Create scripts/lib/pipeline-cost-attribution.sh with reset/record/finalize (TSV accumulator, bash 3.2 safe). (blocks 2,3,4,7)
• Task 2: Implement cost_attr_finalize atomic cost.json writer with fail-open + other residual bucket. (needs 1)
• Task 3: Source the lib in pipeline bootstrap and call cost_attr_reset at start. (needs 1)
• Task 4: Snapshot + attribute per-stage deltas in pipeline-execution.sh generic runner. (needs 1,3)
• Task 5: Call cost_attr_finalize from pipeline-commands.sh after aggregate cost_record. (needs 2)
• Task 6: Attribute build-stage delta correctly (largest cost; verify it is captured by the generic hook, add explicit snapshot if build bypasses the loop). (needs 4)
• Task 7: Add cost_show_pipeline (+--json, issue validation) to sw-cost.sh. (needs 2)
• Task 8: Add cost_forecast budget-runway function to sw-cost.sh.
• Task 9: Wire show --pipeline and forecast into dispatcher + show_help. (needs 7,8)
• Task 10: Dashboard /api/costs/pipelines endpoint + trend card in dashboard/server.ts.
• Task 11: Feed byStage cost into adaptive timeouts / model routing (read-only, optional).
• Task 12: Write sw-cost-attribution-test.sh; extend sw-cost-test.sh; register in package.json.
• Task 13: Update .claude/CLAUDE.md, run docs sync, sync VERSION headers.
• .claude/pipeline-artifacts/cost.json written after each run; stage costs
• shipwright cost show --pipeline <issue> shows per-stage and per-model
• Model usage distribution (% by model) present and sums to ~100%.
• shipwright cost forecast reports remaining-budget runway; handles
• Dashboard shows per-pipeline cost trend over time and by template.
• Intelligence layer can read per-stage cost (adaptive/model-router consume
• New sw-cost-attribution-test.sh passes; sw-cost-test.sh extended; both

---

Generated by shipwright pipeline at 2026-06-05T13:44:13Z

[github-actions]

[github-actions]

📐 Architecture Decision Record

Click to expand ADR (164 lines)

Design: Cost Attribution & Budget Tracking per Pipeline Run

Context

Cost tracking today is aggregate-only. The pipeline accumulates TOTAL_INPUT_TOKENS / TOTAL_OUTPUT_TOKENS across all stages and emits a single cost_record … "pipeline" <issue> entry at the end (scripts/lib/pipeline-commands.sh:1258). There is no per-stage or per-model breakdown and no per-run artifact, which blocks three downstream needs: (1) users diagnosing where a run's spend went, (2) the dashboard charting cost trends by template, and (3) the intelligence layer biasing adaptive timeouts / model routing on real per-stage cost.

Constraints from the codebase:

• Bash 3.2 — no declare -A, readarray, ${var,,}/${var^^}. Forces a TSV-on-disk accumulator instead of an associative array.
• Single-source pricing — dollar math must reuse cost_calculate + COST_MODEL_RATES from sw-cost.sh. Two pricing tables would drift.
• System of record — costs.json / SQLite are authoritative for budgets. The new artifact must be derived, disposable, and non-authoritative.
• Dependency direction — scripts/lib/* is sourced by pipeline stages, never invoked as a script; sw-cost.sh may read the artifact but must not depend on pipeline internals.
• Fail-open mandate — cost reporting must never fail an otherwise-successful pipeline (echoes knowledge.json failures around mktemp/cleanup fragility).
• Worktree isolation — parallel --worktree runs each own their .claude/pipeline-artifacts/, so the artifact is naturally per-run.

Decision

Attribute cost at one choke point — the generic stage runner in scripts/lib/pipeline-execution.sh (~770–850) — by snapshotting the global token counters around each run_stage_with_retry call and attributing the delta to that stage and its already-computed stage_model_used (line 784).

Data flow:

pipeline start
   └─ cost_attr_reset            (truncate .cost-stages.tsv)
generic stage loop (per stage):
   ├─ _in0=$TOTAL_INPUT_TOKENS  _out0=$TOTAL_OUTPUT_TOKENS
   ├─ run_stage_with_retry "$id"
   └─ cost_attr_record_stage "$id" "$stage_model_used" $((TOTAL_INPUT_TOKENS-_in0)) $((TOTAL_OUTPUT_TOKENS-_out0))
        → append "stage<TAB>model<TAB>din<TAB>dout" to .cost-stages.tsv  (skip if din+dout==0)
pipeline end:
   ├─ cost_record … "pipeline" <issue>      (existing, authoritative aggregate)
   └─ cost_attr_finalize                    (derived artifact)
        → for each TSV row: cost_calculate din dout model
        → reconcile recorded-vs-final TOTAL_* → synthetic "other" row
        → jq build byStage/byModel + pctOfTotal (awk)
        → mktemp → jq empty validate → mv     (else rm -f; return 0)

Error handling — fail-open everywhere: cost_attr_finalize builds into a mktemp temp file, validates with jq empty, and mvs only on success; any failure does rm -f "$tmp"; return 0. Every call site is guarded: type cost_attr_finalize >/dev/null 2>&1 && cost_attr_finalize || true. A corrupt or absent cost.json degrades every consumer to "no cost data" — never an error.

Residual reconciliation (the key correctness invariant): Tokens spent outside parse_claude_tokens (haiku detection/classification in pipeline-detection.sh, commit-quality calls) may not land inside any stage's delta window. cost_attr_finalize compares sum(recorded) against final TOTAL_* and books the difference to a synthetic other stage. This guarantees sum(byStage.costUsd) == totalCostUsd (±0.01) — nothing is silently lost, and the artifact stays reconcilable against the authoritative aggregate.

Consumers read-only and guarded: sw-cost.sh gains cost show --pipeline <issue> (+--json) and cost forecast; dashboard/server.ts gains /api/costs/pipelines; adaptive/model-router optionally read byStage. All treat a missing artifact as empty, not fatal.

Alternatives Considered

1. Per-stage cost_record calls inside each stage_* function — Pros: explicit, co-located with the work. Cons: ~14 edit sites, easy to miss a stage, duplicates the existing global accumulator, and every new stage must remember to opt in. Rejected — violates single-responsibility (each stage would own cost bookkeeping) and is high-maintenance.

2. Reconstruct per-stage cost post-hoc from events.jsonl — Pros: zero changes to the hot path; pure read-side feature. Cons: pipeline.cost events are aggregate, not per-stage — the data simply isn't there to reconstruct from. Rejected — infeasible without first emitting per-stage events, which is the same instrumentation problem.

3. New SQLite table pipeline_stage_costs as system of record — Pros: queryable, indexed, joins with DORA metrics. Cons: schema migration + rollback risk, couples cost attribution to the DB layer, larger blast radius, contradicts the "derived artifact, disposable" constraint. Rejected for v1 — the JSON artifact is sufficient and trivially rollback-able (rm cost.json). SQLite remains a future enhancement once the attribution data proves valuable.

4. In-memory associative array, flush once at finalize — Pros: no temp file churn during the run. Cons: Bash 3.2 has no declare -A; emulating it with eval/dynamic var names is exactly the fragile state-management anti-pattern flagged in failures.json. Rejected — append-only TSV is simpler and crash-safe.

Implementation Plan

Files to create:

• /home/runner/work/shipwright/shipwright/scripts/lib/pipeline-cost-attribution.sh — sourced lib: cost_attr_reset, cost_attr_record_stage, cost_attr_finalize. set -euo pipefail, bash 3.2, VERSION= synced to package.json.
• /home/runner/work/shipwright/shipwright/scripts/sw-cost-attribution-test.sh — unit tests.

Files to modify:

• /home/runner/work/shipwright/shipwright/scripts/lib/pipeline-execution.sh — snapshot + record around run_stage_with_retry in the generic runner.

---

Generated by shipwright pipeline design stage at 2026-06-05T13:47:00Z

[github-actions]

[github-actions]