Retry on 429 with Retry-After / exp backoff; omit_temperature for Opus 4.7+

[to-ha]

Summary

Two cloud-resilience knobs in bench/client.py so a bench.py run against a rate-limited hosted endpoint no longer aborts the moment a TPM cap kicks in, and so Claude Opus 4.7+ (which deprecated temperature) can be used at all.

Both changes are backwards-compatible — existing local-server configs are unaffected. New omit_temperature defaults to false; the 429 retry loop only activates on 429.

Changes

1. HTTP 429 retry with Retry-After / exponential backoff (bench/client.py)

• On 429, sleep and retry up to MAX_RETRIES_ON_429 (3) times.
• Honors the standard Retry-After header (integer seconds) when present.
• Falls back to exponential backoff anchored at 60s when the header is missing.
• Other 4xx/5xx errors are not retried (raise immediately, same as before).
• Bounded retry: structurally-too-large requests still fail fast rather than looping forever.

2. omit_temperature flag (bench/client.py, bench/config.py)

• New optional bool in ClientConfig. When true, the temperature field is not sent in the request payload.
• Required for Claude Opus 4.7+, which deprecated the parameter and rejects requests that include it.
• Default false; existing configs are unaffected.

Tested against

• gpt-5.1 jquery (16 functions, OpenAI Tier 1 / 500k TPM): 13 successful retries, run completes — without the patch fail-fast aborted after 2 errors.
• claude-sonnet-4-6 http_server (Anthropic Tier 1 / 30k ITPM): 11/11 through repeated 60s sleeps with Retry-After honored.
• claude-opus-4-7 http_server with omit_temperature=true: 11/11 — the same config without the flag was rejected with temperature is deprecated for this model.
• Local existing configs (qwen3.5/3.6 family, no 429 path): unchanged behavior. Existing smoke_test.py and 6 local model runs pass.

New test file: client_smoke_test.py

Eight assertion cases covering the new client behavior, no live server. Uses unittest.mock to patch httpx.Client and time.sleep, runs in milliseconds. Style mirrors smoke_test.py (assert-based, exits non-zero on failure).

$ uv run python client_smoke_test.py
✓ happy path: single 200, returns content, sends temperature
✓ 429 with Retry-After: sleeps the indicated duration, retries once
✓ 429 without Retry-After: exponential backoff (60, 120)
✓ bounded retry: after 3 retries, raises RuntimeError
✓ non-429 error (500): raises immediately, no retry
✓ non-429 error (400): raises immediately, no retry
✓ omit_temperature=true: temperature absent from payload
✓ default (omit_temperature=false): temperature present in payload
✅ all client smoke checks passed

Configs added

• configs/models/claude-opus-4-7.toml — uses omit_temperature=true (Opus 4.7 deprecated the field).
• configs/models/gpt-5-1.toml — GPT-5.1 chat-completions, mainstream OpenAI frontier.
• configs/models/gpt-5-2.toml — GPT-5.2 reasoning-on-by-default; reasoning_effort="none" skips CoT.

configs/models/claude-sonnet-4-6.toml tightened: max_tokens 8000 → 1500 (the 20-line answer needs much less than 8000), and relax_indent=true added for consistency with the local-MLX configs that ship the same flag.

Docs

• README.md: new "Hosted endpoints with rate limits" section with example output, plus client_smoke_test.py added to the module map.
• configs/CONFIG_README.md: omit_temperature in the model-config field table and the hosted-model gotcha matrix.