Skip to content

[qa][infra] rc3 VM sweep quota-aborted (HTTP 429 session limit) — invalid as a measurement #845

Description

@100yenadmin

Not a product result — recording for honesty + traceability.

The rc3 VM sweep (@a245a2c) hit the claude account session limit (HTTP 429) mid-run:

No scores.db row written (would be junk). Results archived on the VM at results.rc3-QUOTA-ABORTED-*.

Root cause (operational): a clean 5-persona Opus release sweep needs ~5 cold-opens + ~200 beats of Opus generation in one account session-quota window — and that window is shared with the AI's own heavy agent/workflow usage. The 4× parallel cold-opens burst-burned the remainder and tripped the limit.

Fix landed: PR #844 — the sweep now (a) runs personas sequentially so the first 429 aborts before the rest spend, (b) detects the 429 and emits an explicit ABORTED status instead of a number, (c) attributes infra-abort vs product-failure in the rollup. So a future quota-storm can never masquerade as an RRI again.

Re-run: the hardened sequential sweep at current main (not a245a2c — main has moved to #834+) after the quota resets / on a quota-isolated window. Tracks toward the first clean non-partial v1.0.4 RRI.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions