From 769f6de030b2cc7d56bf8cf88e6f5e5d7ef9a057 Mon Sep 17 00:00:00 2001 From: "itarun.p" Date: Sun, 26 Jul 2026 06:54:16 +0700 Subject: [PATCH] feat(limits): the capture tool that unblocks issue 105 Part 2 MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit I reported Part 2 as blocked on data I could not produce. That was wrong. The credentials are on the maintainer's machine and getUsageLimits already takes an injectable fetchImpl, so the seam to capture through existed the whole time. scripts/capture-limits-fixtures.cjs runs one real request per provider, sanitizes the responses, and writes them where a fixture test can read them. --dry-run prints and writes nothing. WHY CAPTURE RATHER THAN HAND-WRITE. A fixture written from reading our own normalizer encodes our reading of it and is green whether or not the provider ever sends those fields. ~/.claude/lessons has the blunt version: "a mock payload proves render logic, never data-source presence." SANITISATION IS ALLOWLIST-SHAPED. Anything not provably a quota shape is replaced, because a blocklist leaks the first field nobody thought of and these payloads are headed for a public repository. - Keys are classified SEGMENT-WISE across both naming conventions. A lone identifying word redacts (`accountId`); one qualified by a quota word does not (`tokenType`). - `id` and `name` are read WITH THEIR PARENT: `currentTier.id` is a tier label the normalizer reads, `user.id` is an account identifier. Both are the commonest keys in these payloads and mean opposite things. - Numbers under an unrecognised key are bucketed. An account id is an integer, so "numbers are safe" would publish it. - Timestamps keep their shape but lose SUB-MINUTE PRECISION, which fingerprints an account's billing cycle even though no field there is an identifier. THE WORST BUG IT HAD, found by running it: it captured `/bin/ps -ax` — the whole machine's process list, with paths and project names — because that command runs during Antigravity discovery. It survived only because an email address happened to appear in the output and took the whole string with it. A capture tool for a privacy-first product must not be one lucky regex away from committing a process list. Only genuine quota queries are captured now; `ps`, `lsof` and `which` are not. TWO MORE FOUND BY TESTING AGAINST RESEARCHED REAL SHAPES rather than invented ones. The key regex was snake_case-only, so camelCase `remainingFraction` did not match `remaining` and a real 0.94 quota figure was bucketed to 0; and `tokenType` matched the credential word `token` and was redacted although it is a category label. Both would have produced fixtures that looked fine and proved nothing. Shapes for all nine providers are documented in the fixture README with public sources — CodexBar for Claude/Codex/Kiro/Antigravity, google-gemini/gemini-cli for Gemini, dmwyatt/cursor-usage for a committed real Cursor capture, and others. None is inferred from our own code, which would have been circular. That research is also what surfaced two facts the sanitizer needed: Gemini returns a GCP project id in `cloudaicompanionProject`, and Antigravity's GetUserStatus is the only source of `accountEmail`. 16 tests, written from the attacker's side: for each thing that must not survive, assert it appears nowhere in the serialised output. Every hostile value is assembled at runtime — the repo's own secret scanner refused this file when the JWT was spelled out, correctly, since it cannot tell a sample from a leak. No fixtures are committed. Only the maintainer can produce them, and the script prints what it redacted so the output can be read before `git add`. ci:local exit 0: 972 root tests (+16), 302 dashboard. --- scripts/capture-limits-fixtures.cjs | 370 ++++++++++++++++++++++++++ test/capture-limits-sanitizer.test.js | 300 +++++++++++++++++++++ test/fixtures/usage-limits/README.md | 68 +++++ 3 files changed, 738 insertions(+) create mode 100644 scripts/capture-limits-fixtures.cjs create mode 100644 test/capture-limits-sanitizer.test.js create mode 100644 test/fixtures/usage-limits/README.md diff --git a/scripts/capture-limits-fixtures.cjs b/scripts/capture-limits-fixtures.cjs new file mode 100644 index 00000000..1d9c7f09 --- /dev/null +++ b/scripts/capture-limits-fixtures.cjs @@ -0,0 +1,370 @@ +#!/usr/bin/env node +"use strict"; + +// Captures ONE REAL RESPONSE per usage-limits provider, sanitizes it, and writes +// it where a fixture test can read it. +// +// Why this exists, rather than a set of hand-written fixtures: usage-limits.js +// talks to nine providers' undocumented private endpoints. A fixture written +// from reading our own normalizer only encodes our reading of it — it is green +// whether or not the provider ever sends those fields. The lesson that named +// this is blunt about it: "a mock payload proves render logic, never +// data-source presence." +// +// Only the person with the credentials can produce the real thing, so this is +// the tool that lets them, in one command, without hand-editing JSON. +// +// node scripts/capture-limits-fixtures.cjs # capture + sanitize + write +// node scripts/capture-limits-fixtures.cjs --dry-run # print, write nothing +// +// SANITISATION IS ALLOWLIST-SHAPED, NOT BLOCKLIST-SHAPED. Anything that is not +// provably safe to keep is replaced. A blocklist would leak the first field +// nobody thought of, and these payloads are about to enter a public repository. +// +// It does not commit anything. It prints what it wrote and what it redacted so +// the output can be read before `git add` — the review is the point. + +const fs = require("node:fs"); +const os = require("node:os"); +const path = require("node:path"); + +const { getUsageLimits } = require("../src/lib/usage-limits"); + +const OUT_DIR = path.resolve(__dirname, "..", "test", "fixtures", "usage-limits"); +const DRY_RUN = process.argv.includes("--dry-run"); + +// --------------------------------------------------------------------------- +// Sanitisation +// --------------------------------------------------------------------------- + +// Keys whose value is kept verbatim regardless of type. Everything here is a +// quota shape — counts, limits, window boundaries, resource labels. None of it +// identifies a person. +// Key matching is SEGMENT-WISE, and the segments come from both naming +// conventions. These APIs mix them: Gemini and Kimi are camelCase +// (`remainingFraction`, `resetTime`), the ChatGPT one is snake_case +// (`limit_window_seconds`). +// +// A snake_case-only regex quietly got this wrong: `remainingFraction` did not +// match `remaining`, so a real 0.94 quota figure was bucketed to 0, and +// `tokenType` matched the credential word `token` and was redacted although it +// is a category label. Both were found by running the researched real shapes +// through it before pointing it at an account. +function keySegments(key) { + return String(key) + .replace(/([a-z0-9])([A-Z])/g, "$1_$2") + .toLowerCase() + .split(/[^a-z0-9]+/) + .filter(Boolean); +} + +// Words that make a key a quota shape — counts, limits, window boundaries, +// resource labels. None of them identifies a person. +const SAFE_WORDS = new Set([ + "used", "usage", "usages", "utilization", "utilisation", "limit", "limits", + "remaining", "quota", "percent", + "percentage", "total", "totals", "count", "max", "min", "window", "windows", + "reset", "resets", "expires", "expiry", "start", "starts", "end", "ends", + "seconds", "minutes", "hours", "days", "time", "unit", "units", "type", "kind", + "status", "state", "enabled", "active", "allowed", "exceeded", "overage", + "tier", "level", "interval", "period", "granted", "consumed", "balance", + "credits", "requests", "tokens", "messages", "fraction", "ratio", "amount", + "entitlement", "entitlements", "snapshot", "snapshots", "group", "groups", + "bucket", "buckets", "detail", "details", "duration", "model", "models", "plan", +]); + +// Words that make a key identifying, whatever it contains. +const IDENTIFYING_WORDS = new Set([ + "token", "secret", "key", "password", "credential", "credentials", "auth", + "cookie", "session", "bearer", "jwt", "email", "mail", "phone", "address", + "user", "users", "account", "customer", "owner", "member", "membership", + "org", "organisation", "organization", "team", "workspace", "project", + "tenant", "subscription", "invoice", "billing", "payment", "card", "login", + "handle", "nickname", "avatar", "url", "uri", "endpoint", "host", "ip", "id", + // `name` is here and `tier`/`plan` are in SAFE_WORDS, so the qualification + // rule below keeps `currentTier.name` (a tier label) while redacting + // `display_name` (a person). Dropping `name` from this list to save the first + // one let the second one through — caught by the hostile-payload test. + "name", +]); + +// `tokenType` is a category; `accessToken` is a credential. The difference is +// whether a SAFE word qualifies the identifying one, so a key counts as safe +// when any segment is safe and no segment is identifying-without-qualification. +// `parentKey` matters for the bare `id` and `name` cases, which are the two most +// common keys in these payloads and mean opposite things depending on what they +// hang off. `currentTier.id` is the tier label gemini-cli reads; `user.id` is an +// account identifier. Without the parent, keeping both leaks and redacting both +// throws away the field the normalizer actually uses. +function classifyKey(key, parentKey = "") { + const segments = keySegments(key); + if (segments.length === 1 && (segments[0] === "id" || segments[0] === "name")) { + const parent = keySegments(parentKey); + if (parent.some((w) => SAFE_WORDS.has(w)) && !parent.some((w) => IDENTIFYING_WORDS.has(w))) { + return "safe"; + } + return "identifying"; + } + const identifying = segments.filter((w) => IDENTIFYING_WORDS.has(w)); + const safe = segments.filter((w) => SAFE_WORDS.has(w)); + if (identifying.length === 0) return safe.length > 0 ? "safe" : "unknown"; + // A single identifying word next to a safe one is a label (`tokenType`, + // `usageLimit`). Two or more, or one on its own, is an identifier + // (`accountId`, `userEmail`, `token`). + if (identifying.length === 1 && safe.length > 0) return "safe"; + return "identifying"; +} + +// Keys that are identifying by name, whatever they contain. + +const EMAIL = /[^\s@]+@[^\s@]+\.[^\s@]+/; +const JWT = /^ey[A-Za-z0-9_-]{8,}\./; +const UUID = /^[0-9a-f]{8}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{4}-[0-9a-f]{12}$/i; +const LONG_OPAQUE = /^[A-Za-z0-9_\-+/=]{24,}$/; +const ISO_DATE = /^\d{4}-\d{2}-\d{2}([T ]\d{2}:\d{2}|$)/; + +// A reset time is quota shape and has to survive. Its SUB-MINUTE PRECISION does +// not: `2026-04-11T07:00:00.528743+00:00` is unique to one account's billing +// cycle, so a set of these correlates a fixture back to whoever produced it even +// though no field is an identifier. Truncating to the minute keeps everything a +// normalizer parses and drops the fingerprint. +function roundTimestamp(value, keyPath) { + const rounded = value.replace( + /^(\d{4}-\d{2}-\d{2}[T ]\d{2}:\d{2}):[\d.]+(.*)$/, + (_m, head, tail) => `${head}:00${tail ? "Z" : ""}`, + ); + if (rounded !== value) note(keyPath, "timestamp truncated to the minute"); + return rounded; +} + +const redactions = []; + +function note(pathStr, why) { + redactions.push({ path: pathStr, why }); +} + +// A string is kept only when it is recognisably NOT an identifier: a timestamp, +// or a short lowercase enum-ish token like "pro", "five_hour", "active". +function sanitizeString(value, keyPath, keyName, parentKey) { + if (classifyKey(keyName, parentKey) === "identifying") { + note(keyPath, `key name looks identifying (${keyName})`); + return ""; + } + if (EMAIL.test(value)) { + note(keyPath, "value looks like an email address"); + return ""; + } + if (JWT.test(value)) { + note(keyPath, "value looks like a JWT"); + return ""; + } + if (UUID.test(value)) { + note(keyPath, "value looks like a UUID"); + return ""; + } + if (ISO_DATE.test(value)) return roundTimestamp(value, keyPath); + if (LONG_OPAQUE.test(value)) { + note(keyPath, `value is a long opaque string (${value.length} chars)`); + return ""; + } + if (value.length > 64) { + note(keyPath, `value is long free text (${value.length} chars)`); + return ""; + } + return value; +} + +// Numbers are kept only under a quota-shaped key. A bare number under an +// identifying or unrecognised key can be an account id, so it is bucketed to a +// magnitude rather than kept or dropped — the shape survives, the value does not. +function sanitizeNumber(value, keyPath, keyName, parentKey) { + const kind = classifyKey(keyName, parentKey); + if (kind === "safe") return value; + if (kind === "identifying") { + note(keyPath, `numeric value under an identifying key (${keyName})`); + return 0; + } + note(keyPath, `numeric value under an unrecognised key (${keyName}) — bucketed`); + return Number.isInteger(value) ? Math.sign(value) * String(Math.abs(value)).length : 0; +} + +function sanitize(value, keyPath = "$", keyName = "", parentKey = "") { + if (value === null || typeof value === "boolean") return value; + if (typeof value === "number") return sanitizeNumber(value, keyPath, keyName, parentKey); + if (typeof value === "string") return sanitizeString(value, keyPath, keyName, parentKey); + if (Array.isArray(value)) { + // Two elements are enough to show it is a list and that the elements share a + // shape. Keeping all of them multiplies the disclosure surface for nothing. + return value.slice(0, 2).map((item, i) => sanitize(item, `${keyPath}[${i}]`, keyName, parentKey)); + } + if (typeof value === "object") { + const out = {}; + for (const [key, inner] of Object.entries(value)) { + out[key] = sanitize(inner, `${keyPath}.${key}`, key, keyName); + } + return out; + } + return null; +} + +// --------------------------------------------------------------------------- +// Capture +// --------------------------------------------------------------------------- + +const captured = []; + +function recordingFetch(realFetch) { + return async function capturingFetch(input, init) { + const url = typeof input === "string" ? input : String(input?.url || input); + const response = await realFetch(input, init); + // Read the body from a CLONE so the caller still gets an unconsumed stream. + let body = null; + let parseError = null; + try { + body = await response.clone().json(); + } catch (e) { + parseError = e?.message || String(e); + } + captured.push({ url, status: response.status, body, parseError }); + return response; + }; +} + +// Only commands that ARE a provider quota query. Discovery commands are run too +// — `ps -ax` to find the Antigravity process, `lsof` to find its port, `which` to +// test for a binary — and their output is the machine's process list and open +// files: absolute paths, project names, every other tool the user is running. +// +// The first version of this captured all of them. The ps output survived only +// because an email address happened to appear in it and took the whole string +// with it. A capture tool for a privacy-first product must not be one lucky +// regex away from committing a process list. +const CAPTURABLE_COMMAND = /(^|\/)kiro(-cli)?$/; + +function recordingCommandRunner(realRunner) { + return function capturingRunner(command, args, options) { + const result = realRunner(command, args, options); + if (CAPTURABLE_COMMAND.test(String(command))) { + captured.push({ + url: `command:${command} ${(args || []).join(" ")}`, + status: result?.status ?? null, + text: typeof result?.stdout === "string" ? result.stdout : null, + }); + } + return result; + }; +} + +// A URL identifies its provider by host, which is the only mapping that does not +// have to be kept in step with the internals of getUsageLimits. +const PROVIDER_BY_HOST = [ + [/(^|\.)anthropic\.com$/, "claude"], + [/(^|\.)chatgpt\.com$/, "codex"], + [/(^|\.)cursor\.com$/, "cursor"], + [/(^|\.)kimi\.com$/, "kimi"], + [/(^|\.)z\.ai$/, "zai"], + [/(^|\.)googleapis\.com$/, "gemini"], + [/(^|\.)github\.com$/, "copilot"], +]; + +function providerFor(entry) { + if (entry.url.startsWith("command:")) { + if (entry.url.includes("kiro")) return "kiro"; + return "command"; + } + let host = ""; + try { + host = new URL(entry.url).hostname; + } catch { + return "unknown"; + } + for (const [pattern, provider] of PROVIDER_BY_HOST) { + if (pattern.test(host)) return provider; + } + return host.replace(/[^a-z0-9]+/gi, "-"); +} + +async function main() { + const cp = require("node:child_process"); + const realFetch = globalThis.fetch; + + process.stdout.write("Capturing one real response per provider…\n\n"); + + await getUsageLimits({ + home: os.homedir(), + env: process.env, + fetchImpl: recordingFetch(realFetch), + commandRunner: recordingCommandRunner(cp.spawnSync), + providerTimeoutMs: 15000, + }); + + if (captured.length === 0) { + process.stdout.write( + "Nothing was captured. Every provider was skipped for want of a credential,\n" + + "so there is nothing to sanitize — sign in to at least one and re-run.\n", + ); + return; + } + + const byProvider = new Map(); + for (const entry of captured) { + const provider = providerFor(entry); + // First response per provider is enough; later ones are refreshes. + if (!byProvider.has(provider)) byProvider.set(provider, entry); + } + + if (!DRY_RUN) fs.mkdirSync(OUT_DIR, { recursive: true }); + + for (const [provider, entry] of [...byProvider].sort()) { + redactions.length = 0; + const fixture = { + _README: + "Captured from a live account and sanitized by scripts/capture-limits-fixtures.cjs. " + + "Values are redacted allowlist-style: anything not provably a quota shape was replaced. " + + "This proves the normalizer's reading of a REAL payload, not upstream drift — the provider " + + "can change this shape without notice and nothing here will notice.", + _captured_at: new Date().toISOString().slice(0, 10), + _source: entry.url.startsWith("command:") ? entry.url : new URL(entry.url).origin + new URL(entry.url).pathname, + _status: entry.status, + response: + entry.text != null + ? { _kind: "text", text: sanitizeString(entry.text, "$.text", "text") } + : entry.body != null + ? sanitize(entry.body) + : { _kind: "unparseable", error: entry.parseError }, + }; + + const file = path.join(OUT_DIR, `${provider}.json`); + const json = JSON.stringify(fixture, null, 2) + "\n"; + + process.stdout.write(`── ${provider} (${entry.status ?? "no status"})\n`); + process.stdout.write(` ${fixture._source}\n`); + process.stdout.write(` ${redactions.length} value(s) redacted\n`); + for (const r of redactions.slice(0, 8)) { + process.stdout.write(` ${r.path} — ${r.why}\n`); + } + if (redactions.length > 8) { + process.stdout.write(` … and ${redactions.length - 8} more\n`); + } + if (DRY_RUN) { + process.stdout.write(`${json}\n`); + } else { + fs.writeFileSync(file, json); + process.stdout.write(` wrote ${path.relative(process.cwd(), file)}\n`); + } + process.stdout.write("\n"); + } + + process.stdout.write( + DRY_RUN + ? "Dry run — nothing written.\n" + : "READ THESE FILES BEFORE COMMITTING THEM. The sanitizer is deliberately\n" + + "aggressive, but it cannot know which field your provider decided to put an\n" + + "account name in this week. Nothing has been staged.\n", + ); +} + +main().catch((error) => { + process.stderr.write(`capture failed: ${error?.stack || error}\n`); + process.exitCode = 1; +}); diff --git a/test/capture-limits-sanitizer.test.js b/test/capture-limits-sanitizer.test.js new file mode 100644 index 00000000..2d8b256b --- /dev/null +++ b/test/capture-limits-sanitizer.test.js @@ -0,0 +1,300 @@ +"use strict"; + +// The sanitizer in scripts/capture-limits-fixtures.cjs is about to be pointed at +// live credentials, and its output is about to enter a public repository. So it +// gets tested against a payload deliberately stuffed with every shape of +// identifier BEFORE it is pointed at anything real. +// +// The test is written from the attacker's side: for each thing that must not +// survive, assert it does not appear anywhere in the serialised output. A +// field-by-field check would pass while an identifier survived one level deeper +// than it looked. +// +// Every hostile value below is ASSEMBLED AT RUNTIME rather than written as a +// literal. The repo's own secret scanner refused this file when the JWT was +// spelled out — correctly, since it cannot tell a synthetic sample from a real +// leak. Assembling them keeps the guardrail useful for everyone else. + +const assert = require("node:assert/strict"); +const { execFileSync } = require("node:child_process"); +const fs = require("node:fs"); +const os = require("node:os"); +const path = require("node:path"); +const { test } = require("node:test"); + +const SCRIPT = path.join(__dirname, "..", "scripts", "capture-limits-fixtures.cjs"); + +// The script is a command with no exports. Rather than reshape a tool to suit +// its test, load its pure half into a sandbox module. If sanitize() is renamed +// this fails loudly instead of quietly testing nothing. +function loadSanitizer() { + const source = fs.readFileSync(SCRIPT, "utf8"); + assert.match(source, /function sanitize\(/, "sanitize() is gone — this test is stale"); + const marker = source.indexOf("// Capture\n"); + assert.ok(marker > 0, "the Capture section marker moved — this test is stale"); + const pure = source.slice(0, source.lastIndexOf("// ---", marker)); + const dir = fs.mkdtempSync(path.join(os.tmpdir(), "tt-sanitizer-")); + const stub = path.join(dir, "s.cjs"); + fs.writeFileSync( + stub, + pure.replace('require("../src/lib/usage-limits")', "{}") + + "\nmodule.exports = { sanitize };\n", + ); + return require(stub); +} + +const { sanitize } = loadSanitizer(); + +const b64 = (text) => Buffer.from(text).toString("base64").replace(/=+$/, ""); +const FAKE_JWT = [b64('{"alg":"HS256"}'), b64('{"sub":"1234"}'), "c2lnbmF0dXJl"].join("."); +const FAKE_OAUTH = ["sk", "ant", "oat01", "AAAABBBBCCCCDDDDEEEEFFFFGGGGHHHH"].join("-"); +const FAKE_API_KEY = ["sk", "live", "DEADBEEFDEADBEEFDEADBEEF"].join("-"); +const FAKE_UUID = "8f14e45f-ceea-467a-9e2c-1a3b4c5d6e7f"; +const FAKE_EMAIL = ["someone", "example.com"].join("@"); + +const HOSTILE = { + // Must not survive. + access_token: FAKE_OAUTH, + id_token: FAKE_JWT, + account_id: FAKE_UUID, + email: FAKE_EMAIL, + user: { login: "a-real-person", display_name: "A Real Person" }, + organization: "acme-corporation", + subscription_id: 998877665544, + session_cookie: "WorkosCursorSessionToken=abc", + note: "This account belongs to a named individual at a named company and should never be committed.", + + // Must survive, or the fixture proves nothing. + plan_type: "pro", + five_hour: { used: 4213, limit: 20000, percent_used: 21, resets_at: "2026-05-14T09:00:00Z" }, + seven_day: { used: 120345, limit: 500000, remaining: 379655 }, + windows: [ + { window: "5h", used: 10, limit: 100 }, + { window: "7d", used: 20, limit: 200 }, + { window: "30d", used: 30, limit: 300 }, + ], + enabled: true, + overage_allowed: false, + quota_reset_seconds: 3600, +}; + +const serialised = JSON.stringify(sanitize(HOSTILE)); + +test("no credential, id or personal string survives anywhere in the output", () => { + for (const secret of [ + FAKE_OAUTH, + FAKE_JWT.slice(0, 20), + FAKE_UUID, + FAKE_EMAIL, + "a-real-person", + "A Real Person", + "acme-corporation", + "WorkosCursorSessionToken", + "named individual", + ]) { + assert.ok( + !serialised.includes(secret), + `${secret.slice(0, 24)} survived sanitisation — it would have been committed`, + ); + } +}); + +test("a numeric identifier is not kept just for being a number", () => { + // The trap a type-based rule falls into: an account id is an integer, and + // "numbers are safe" would publish it. + assert.ok(!serialised.includes("998877665544"), "a numeric account id survived"); +}); + +test("the quota shape survives, or the fixture proves nothing", () => { + const out = sanitize(HOSTILE); + assert.equal(out.five_hour.used, 4213); + assert.equal(out.five_hour.limit, 20000); + assert.equal(out.five_hour.percent_used, 21); + assert.equal(out.five_hour.resets_at, "2026-05-14T09:00:00Z", "timestamps must be kept"); + assert.equal(out.seven_day.remaining, 379655); + assert.equal(out.quota_reset_seconds, 3600); + assert.equal(out.enabled, true); + assert.equal(out.overage_allowed, false, "false must survive, not become null"); + assert.equal(out.plan_type, "pro", "a short enum-ish label is shape, not identity"); +}); + +test("nesting is preserved so the normalizer can still walk it", () => { + const out = sanitize(HOSTILE); + assert.equal(typeof out.five_hour, "object"); + assert.equal(typeof out.user, "object", "an object keeps its shape; only its values are redacted"); + assert.ok(Array.isArray(out.windows)); + assert.equal(out.windows[0].window, "5h"); +}); + +test("arrays are truncated, so a long list is not a long disclosure", () => { + assert.equal(sanitize(HOSTILE).windows.length, 2, "three elements in, two out"); +}); + +test("an unrecognised key holding a number is bucketed, not published", () => { + // The default has to be safe: a field nobody has classified yet must not pass + // through intact just because it is numeric. + const out = sanitize({ some_future_field: 123456789 }); + assert.notEqual(out.some_future_field, 123456789); + assert.equal(typeof out.some_future_field, "number", "the shape stays a number"); +}); + +test("null and boolean pass through untouched", () => { + assert.deepEqual(sanitize({ a: null, b: true, c: false }), { a: null, b: true, c: false }); +}); + +test("a deeply buried secret is still caught", () => { + // Recursion is the whole risk: a rule applied only at the top level looks + // correct on a flat fixture and leaks on a real one. + const out = sanitize({ data: { attributes: { meta: { api_key: FAKE_API_KEY } } } }); + assert.ok(!JSON.stringify(out).includes("DEADBEEF"), "a nested key survived"); +}); + +test("the script writes nothing in --dry-run", () => { + // The property that matters most before this is pointed at live credentials: + // a first run can be inspected without producing files. Run against an EMPTY + // home so it reaches no network and needs no credential. + const emptyHome = fs.mkdtempSync(path.join(os.tmpdir(), "tt-empty-home-")); + const out = execFileSync(process.execPath, [SCRIPT, "--dry-run"], { + encoding: "utf8", + env: { ...process.env, HOME: emptyHome }, + timeout: 60000, + }); + assert.match(out, /Dry run — nothing written\.|Nothing was captured/); + assert.deepEqual( + fs.readdirSync(emptyHome), + [], + "a dry run must not create anything, not even in the home it was pointed at", + ); +}); + +// --- Against the real shapes, from public sources ----------------------------- +// These are not invented. Each is quoted from an open-source project that calls +// the same endpoint, found by research and cited in the fixture README. Running +// the sanitizer against them BEFORE pointing it at an account is what caught two +// defects in it: +// +// - the key regex was snake_case-only, so camelCase `remainingFraction` did not +// match `remaining` and a real 0.94 quota figure was bucketed to 0; +// - `tokenType` matched the credential word `token` and was redacted, although +// it is a category label. + +test("Gemini: the GCP project id goes, the tier label and quota stay", () => { + // Shape from google-gemini/gemini-cli packages/core/src/code_assist/types.ts. + // cloudaicompanionProject is a GCP project identifier — the clearest PII in + // any of these payloads. + const out = sanitize({ + cloudaicompanionProject: "my-company-prod-4471", + currentTier: { id: "free-tier", name: "Free" }, + buckets: [ + { + remainingAmount: "940", + remainingFraction: 0.94, + resetTime: "2026-07-26T00:00:00Z", + tokenType: "INPUT", + modelId: "gemini-2.5-pro", + }, + ], + }); + assert.equal(out.cloudaicompanionProject, ""); + assert.equal(out.currentTier.id, "free-tier", "the tier id is what the normalizer reads"); + assert.equal(out.buckets[0].remainingFraction, 0.94, "a camelCase quota number must survive"); + assert.equal(out.buckets[0].tokenType, "INPUT", "a category label is not a credential"); + assert.equal(out.buckets[0].modelId, "gemini-2.5-pro"); +}); + +test("Antigravity: accountEmail goes, remainingFraction stays", () => { + // Shape from steipete/CodexBar docs/antigravity.md — GetUserStatus is + // documented there as the only source of accountEmail and planName. + const out = sanitize({ + userStatus: { + accountEmail: ["real.person", "company.com"].join("@"), + planName: "Pro", + cascadeModelConfigData: { + clientModelConfigs: [ + { quotaInfo: { remainingFraction: 0.42, resetTime: "2026-07-26T00:00:00Z" } }, + ], + }, + }, + }); + assert.ok(!JSON.stringify(out).includes("real.person"), "an email survived"); + assert.equal( + out.userStatus.cascadeModelConfigData.clientModelConfigs[0].quotaInfo.remainingFraction, + 0.42, + ); +}); + +test("Copilot: the quota snapshot survives intact", () => { + // Shape from the reverse-engineered api.github.com/copilot_internal/user + // documented in Noisemaker111/openusage-opencode docs/providers/copilot.md. + const out = sanitize({ + copilot_plan: "pro", + quota_reset_date: "2026-08-01T00:00:00Z", + quota_snapshots: { + premium_interactions: { + percent_remaining: 80, + entitlement: 300, + remaining: 240, + quota_id: "premium", + }, + }, + }); + assert.equal(out.quota_snapshots.premium_interactions.entitlement, 300); + assert.equal(out.quota_snapshots.premium_interactions.percent_remaining, 80); + assert.equal(out.quota_snapshots.premium_interactions.remaining, 240); + assert.equal(out.copilot_plan, "pro"); +}); + +test("Kimi and Z.AI: string-typed counters are not mistaken for opaque blobs", () => { + // Kimi shape from luisleineweber/usagebar plugins/kimi/plugin.js; Z.AI from + // guyinwonder168/opencode-glm-quota src/index.ts. Both report counts as + // STRINGS, which a "numbers are data, strings are suspect" rule would destroy. + const kimi = sanitize({ + usage: { limit: "100", remaining: "74", resetTime: "2026-02-11T17:32:50.757941Z" }, + user: { membership: { level: "LEVEL_INTERMEDIATE" } }, + }); + assert.equal(kimi.usage.limit, "100"); + assert.equal(kimi.usage.remaining, "74"); + // Truncated to the minute — see the timestamp test below. + assert.equal(kimi.usage.resetTime, "2026-02-11T17:32:00Z"); + + const zai = sanitize({ + data: { level: "pro", limits: [{ type: "TOKENS_LIMIT", percentage: 37 }] }, + }); + assert.equal(zai.data.level, "pro"); + assert.equal(zai.data.limits[0].type, "TOKENS_LIMIT"); + assert.equal(zai.data.limits[0].percentage, 37); +}); + +test("a user object's id and name go; a tier object's do not", () => { + // The pair that motivated parent-aware classification. `id` and `name` are the + // two commonest keys in these payloads and mean opposite things depending on + // what they hang off — keeping both leaks, redacting both throws away the + // field the normalizer uses. + assert.equal(sanitize({ currentTier: { id: "free-tier" } }).currentTier.id, "free-tier"); + assert.equal(sanitize({ user: { id: "12345" } }).user.id, ""); + assert.equal(sanitize({ user: { name: "A Person" } }).user.name, ""); +}); + + +test("a reset time keeps its shape but loses its sub-minute fingerprint", () => { + // Claude's oauth/usage returns microsecond precision — e.g. + // "2026-04-11T07:00:00.528743+00:00". No field there is an identifier, but a + // set of those timestamps correlates a fixture back to one account's billing + // cycle. The research that found the shape said so explicitly: round them + // before committing. + // + // The value still has to parse as a timestamp afterwards, or the fixture stops + // exercising the normalizer's date handling. + const out = sanitize({ + five_hour: { utilization: 33.0, resets_at: "2026-04-11T07:00:00.528743+00:00" }, + }); + assert.equal(out.five_hour.resets_at, "2026-04-11T07:00:00Z"); + assert.ok(Number.isFinite(Date.parse(out.five_hour.resets_at)), "must still be a parseable date"); + assert.equal(out.five_hour.utilization, 33, "the headline quota number must survive intact"); +}); + +test("a date with no sub-minute part is left exactly as it is", () => { + const out = sanitize({ resets_at: "2026-04-17T00:00:00Z", billing_start: "2026-04-02" }); + assert.equal(out.resets_at, "2026-04-17T00:00:00Z"); + assert.equal(out.billing_start, "2026-04-02"); +}); diff --git a/test/fixtures/usage-limits/README.md b/test/fixtures/usage-limits/README.md new file mode 100644 index 00000000..67ada2fc --- /dev/null +++ b/test/fixtures/usage-limits/README.md @@ -0,0 +1,68 @@ +# Usage-limits response fixtures + +Produced by `node scripts/capture-limits-fixtures.cjs`, which captures **one real +response per provider** from the machine it runs on, sanitizes it, and writes it +here. Run `--dry-run` first and read the output. + +## Why capture rather than hand-write + +`src/lib/usage-limits.js` talks to nine providers' undocumented private +endpoints. A fixture written from reading our own normalizer only encodes our +reading of it — it is green whether or not the provider ever sends those fields. +The lesson that named this is blunt: *a mock payload proves render logic, never +data-source presence.* + +## What these fixtures do and do not prove + +**They do:** lock the normalizer's behaviour against a payload the provider +really sent, so our own refactors cannot silently change how it is parsed. + +**They do not:** detect upstream drift. Nothing short of live credentials in CI +does, and that is explicitly rejected — nine providers' secrets in repository +secrets, with the rotation burden and disclosure surface that implies, to catch +something a user hitting the Limits page catches anyway. + +## Sanitisation + +Allowlist-shaped, not blocklist-shaped: anything not provably a quota shape is +replaced. Specifically — + +- keys are classified **segment-wise** across both naming conventions, because + these APIs mix them (`remainingFraction` next to `limit_window_seconds`); +- a lone identifying word redacts (`token`, `accountId`); one qualified by a + quota word does not (`tokenType`, `usageLimit`); +- `id` and `name` are read **with their parent**: `currentTier.id` is a tier + label the normalizer reads, `user.id` is an account identifier; +- numbers under an unrecognised key are bucketed to a magnitude — an account id + is an integer, so "numbers are safe" would publish it; +- timestamps keep their shape but lose **sub-minute precision**, which is a + per-account fingerprint even though no field there is an identifier; +- arrays are truncated to two elements. + +`scripts/capture-limits-fixtures.cjs` never captures discovery commands. `ps -ax` +and `lsof` are run to find the Antigravity process, and their output is the +machine's process list and open files. The first version captured them; it +survived only because an email address happened to appear in the `ps` output and +took the whole string with it. + +## Response shapes, from public sources + +These were used to test the sanitizer **before** it was pointed at an account, +and finding them is what caught two defects in it. Each is quoted from a project +that calls the same endpoint — none is inferred from our own code, which would +have been circular. + +| Provider | Public source | Shape highlights | +|---|---|---| +| Claude — `api.anthropic.com/api/oauth/usage` | [steipete/CodexBar](https://github.com/steipete/CodexBar) `ClaudeOAuthUsageFetcher.swift` + tests; independently corroborated in [Claude-Code-Usage-Monitor#202](https://github.com/Maciek-roboblog/Claude-Code-Usage-Monitor/issues/202) | `five_hour`/`seven_day`/`seven_day_opus` each `{utilization, resets_at}`; newer `limits[]` with `kind`, `group`, `percent`, `scope.model` | +| Codex — `chatgpt.com/backend-api/wham/usage` | [steipete/CodexBar](https://github.com/steipete/CodexBar) `CodexOAuthUsageFetcher.swift` + tests | `plan_type`, `rate_limit.{primary,secondary}_window.{used_percent, reset_at, limit_window_seconds}`, `additional_rate_limits[]`, `credits` | +| Cursor — `cursor.com/api/usage-summary` | [dmwyatt/cursor-usage](https://github.com/dmwyatt/cursor-usage) `testdata/summary_response.json` (a committed real capture) + CodexBar | `individualUsage.plan.{used,limit,remaining,totalPercentUsed}`, `teamUsage.onDemand`, cents. `www.cursor.com` 308-redirects and strips the auth cookie | +| Kimi — `api.kimi.com/coding/v1/usages` | [luisleineweber/usagebar](https://github.com/luisleineweber/usagebar) `plugins/kimi/plugin.js` | `usage.{limit,remaining,resetTime}` as **strings**; `limits[].window.{duration,timeUnit}` | +| Z.AI — `api.z.ai/api/monitor/usage/quota/limit` | [guyinwonder168/opencode-glm-quota](https://github.com/guyinwonder168/opencode-glm-quota) `src/index.ts` | `data.limits[].{type,percentage,nextResetTime}`. Auth header takes **no** `Bearer` prefix | +| Gemini — `cloudcode-pa.googleapis.com` | [google-gemini/gemini-cli](https://github.com/google-gemini/gemini-cli) `code_assist/types.ts` (Google's own OSS) | `buckets[].{remainingAmount,remainingFraction,resetTime,tokenType,modelId}`; `cloudaicompanionProject` is a **GCP project id** | +| Copilot | [Official docs](https://docs.github.com/en/rest/billing/usage) for the billing endpoint; [Noisemaker111/openusage-opencode](https://github.com/Noisemaker111/openusage-opencode) for the reverse-engineered `copilot_internal/user` | `quota_snapshots.premium_interactions.{percent_remaining,entitlement,remaining,quota_id}` | +| Kiro — `kiro-cli chat --no-interactive /usage` | [steipete/CodexBar](https://github.com/steipete/CodexBar) `docs/kiro.md` | **Not JSON.** ANSI-decorated box drawing; percentage from the bar, `resets on MM/DD`. [kirodotdev/Kiro#5423](https://github.com/kirodotdev/Kiro/issues/5423) confirms no JSON mode exists | +| Antigravity — local HTTPS RPC | [steipete/CodexBar](https://github.com/steipete/CodexBar) `docs/antigravity.md`, corroborated by [Antigravity-Context-Window-Monitor](https://github.com/AGI-is-going-to-arrive/Antigravity-Context-Window-Monitor) | `RetrieveUserQuotaSummary` → `groups[].buckets[].remaining.remainingFraction`. `GetUserStatus` is the only source of **`accountEmail`** | + +Sources were current when checked on 2026-07-25; several of these projects are +actively maintained and the shapes can change without notice.