Fix Google streaming usage billing

[pablopunk]

Summary

Fixes Google/Gemini streaming usage billing in the Nevermind gateway.

Production showed successful Google streaming requests being logged with:

• input_tokens=0
• output_tokens=0
• cost_credits=0

Root cause: the Google SSE usage parser assumed each chunk contained complete data: lines. In production, usageMetadata can arrive split across stream chunks, so the parser silently missed the final token usage payload.

Changes

• Buffer pending stream text for usage parsers and flush the final partial line at stream end.
• Make Google streaming usage parsing resilient to split SSE lines.
• Count Google thoughtsTokenCount as output tokens alongside candidatesTokenCount.
• Add a backend contract test that reproduces split Google streaming usage and verifies non-zero recorded billing.
• Document two process learnings:
  • bug fixes should include repro tests and red/green verification when practical
  • vercel logs is live-only when Axiom has no runtime rows

Verification

• Confirmed the new test fails against the old parser and reproduces the prod symptom (0 tokens / 0 credits).
• Confirmed the same test passes with this fix (inputTokens=12, outputTokens=6, costCredits=1).
• mise exec -- pnpm -C backend test
• mise exec -- pnpm test

Roadmap handoff

The remaining self-serve gateway roadmap has been moved into focused GitHub issues and should be treated as the source of truth:

• Launch readiness: production envs, compatibility, and gateway smoke #6 Launch readiness
• Stripe self-serve billing: checkout, portal, webhooks, credits #7 Stripe billing
• Desktop connection polish: profile connect UI, nvm:// deep link, BYO fallback #8 Desktop connect/deep link
• Production observability and ops hardening #9 Ops/observability
• Multi-upstream routing and provider failover #10 Multi-upstream/failover
• Public polish, docs, legal, and account deletion #11 Public/docs/legal
• Cross-cutting backend hardening, idempotency, tests, and staging #12 Cross-cutting hardening/staging

Summary by CodeRabbit

• Bug Fixes

  • Fixed Google model token accounting to properly include candidate and thoughts token counts in output calculations
  • Fixed streaming usage accounting when usage metadata is fragmented across multiple stream chunks

• Documentation

  • Updated Vercel logs debugging guidance with improved troubleshooting instructions

• Tests

  • Added comprehensive test coverage for Google model streaming with fragmented usage metadata

[vercel]

The latest updates on your projects. Learn more about Vercel for GitHub.

Project	Deployment	Actions	Updated (UTC)
nvm	Ready	Preview, Comment	Jun 6, 2026 12:17am

[coderabbitai]

Looking for one thing? Review this PR in Change Stack to search files, summaries, diffs, and code without losing your place.

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: a1f3b708-b377-444c-8861-051fc74091cf

📥 Commits

Reviewing files that changed from the base of the PR and between 5fb8cc7 and c083996.

📒 Files selected for processing (5)

• .agents/skills/production-debugger/vercel.md
• AGENTS.md
• backend/src/lib/proxy.ts
• backend/src/pages/api/contract-routes.test.ts
• backend/src/pages/api/v1/models/[...path].ts

---

Walkthrough

This PR addresses correct handling of Google API streaming usage metadata that arrives split across multiple HTTP response chunks. It extends StreamUsageAccumulator with a pendingText buffer field, adds helper functions to reassemble streaming data: JSON lines and compute output tokens from both candidatesTokenCount and thoughtsTokenCount, and reworks the streaming parser to use line reassembly while buffering partial lines. The streaming flush logic now sends a final newline to finalize any buffered text. Comprehensive test coverage validates correct reconstruction and token accounting when upstream metadata splits across chunk boundaries. Documentation updates clarify Vercel logging workflow and add verification guidelines for streaming parser chunk-boundary test cases.

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name	Status	Explanation	Resolution
Docstring Coverage	⚠️ Warning	Docstring coverage is 0.00% which is insufficient. The required threshold is 80.00%.	Write docstrings for the functions missing them to satisfy the coverage threshold.

✅ Passed checks (4 passed)

Check name	Status	Explanation
Title check	✅ Passed	The title 'Fix Google streaming usage billing' is concise, clear, and directly describes the primary fix in the changeset.
Description check	✅ Passed	The PR description covers all required sections: it includes a clear summary of the root cause, detailed changes, verification steps, and backend API compatibility confirmation with test updates.
Linked Issues check	✅ Passed	Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check	✅ Passed	Check skipped because no linked issues were found for this pull request.

_{✏️ Tip: You can configure your own custom pre-merge checks in the settings.}

---

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

• X
• Mastodon
• Reddit
• LinkedIn

_{Comment @coderabbitai help to get the list of available commands and usage tips.}