fix(modelinfo): decode the LiteLLM window map per entry, not as one typed map

[OsherElhadad]

Closes #39

internal/modelinfo's LiteLLM fetch has never resolved a single context window in
production, so every fraction-based Trigger threshold (min_request_frac,
min_output_frac, huge_output_frac) has silently no-opped since it landed.

Root cause

fetch decoded the whole document into one typed map[string]struct{…}. The upstream
map is community-maintained and not schema-clean: it carries a sample_spec documentation
entry whose numeric fields hold prose ("deprecation_date": "date when the model becomes deprecated in the format YYYY-MM-DD"), and 7 models spell an integer window as a float.
encoding/json aborts the entire map on the first type error, so it parsed everything and
then threw it away:

whole-map decode: entries=2988 err=json: cannot unmarshal string into Go struct field .max_input_tokens of type int

fetch returned (nil, err), l.byKey stayed nil, and Window returned 0, false for
every model forever.

Fix

Decode per entry: map[string]json.RawMessage, then unmarshal each value and continue
past the ones that fail. sample_spec is skipped by name so the document's own schema notes
never even count as a failure. Window fields are read as float64 and truncated, so a
float-spelled integer resolves.

And stop degrading silently — which is how this hid for the life of the package. An empty
map was indistinguishable from "no model has a window" at every call site, because every
lookup deliberately fails open. Now an empty result warns that fraction triggers will
not fire, and skipped entries are logged with examples.

Evidence

Against a checked-in gzipped snapshot of the real upstream document (2,988 entries,
internal/modelinfo/testdata/litellm_prices.json.gz):

	keys resolved	aws/claude-sonnet-5	aws/claude-haiku-4-5
before (main)	0	window=0 ok=false	window=0 ok=false
after	3,664	window=1000000 ok=true	window=200000 ok=true

Blast radius

This changes compaction behaviour, which is why it is its own PR rather than riding
along in #38: components/trigger.go resolves fraction thresholds against Ctx.CtxWindow,
which is always 0 today, so fractions are ignored and only absolute floors apply. After
this, a fraction-configured deployment starts compacting differently on the first request.

Practical risk today is low — no shipped preset uses a fraction (presetConfigs use
absolute min_tokens/min_request_tokens), and deploy/harbor/swebench.py:100 records
that the benchmarked codesmart config deliberately switched to absolutes. So the published
SWE-bench and Terminal-Bench numbers do not depend on this path. But that was luck, and it
deserves to be bisectable on its own.

Tests

• TestLiteLLMSkipsMalformedEntries — minimal fixture with a sample_spec-shaped
  prose entry plus a float-spelled window; asserts a malformed sibling does not poison
  the map.
• TestLiteLLMDecodesTheRealDocument — the checked-in real snapshot; asserts >2,900
  keys decode and a non-zero window for aws/claude-sonnet-5 and aws/claude-haiku-4-5.
  This test fails on main with 0 keys.

go build -tags cg_skeleton ./..., go test -race ./... (20/20), make lint, gofmt -l
all clean.

[OsherElhadad]

Verified end-to-end against the live document

I reproduced the original bug and then confirmed this fix resolves it, going past the fixture-based tests to the real network path.

Before (on main prior to this change) — the whole-map typed decode aborts on LiteLLM's prose-filled sample_spec entry and discards everything:

whole-map decode err: json: cannot unmarshal string into Go struct field .max_input_tokens of type int
entries decoded: 2988      <- all 2,988 successfully decoded, then thrown away

After (this branch), resolving from the live upstream document:

model	window before	window after
aws/claude-sonnet-5	0	1,000,000
aws/claude-haiku-4-5	0	200,000

All five tests pass, including TestLiteLLMDecodesTheRealDocument against a checked-in gzipped fixture — the right call, since a network-dependent assertion would be flaky in CI while still leaving the live path unverified. I covered the live path here manually instead.

One thing that cost me time and is worth writing down

My first three attempts all showed window=0 ok=false after the fix, which looked like the bug surviving. It wasn't. refreshIfStale deliberately launches a detached background fetch and returns immediately (modelinfo.go:96-99), so the very first Window call always misses by design — it returned in 0.004 s, having attempted nothing. Once warmed, it resolved in 750 ms.

That is correct behaviour for a hot-path lookup (never block a request on an upstream fetch), but it means a short-lived process may never see a resolved window at all — the fetch outlives the request that triggered it, and if the process exits first the work is wasted. Not a blocker for the proxy, which is long-lived. Worth a sentence in the doc comment so the next person doesn't spend the time I did concluding the fetch is broken.

On the split

Correct call to separate this from #38. Three files, one concern, independently bisectable — and it changes compaction behaviour via fraction triggers, which has no business riding along in a dashboard PR. Confirmed the diff is exactly modelinfo.go + its test + the fixture, with nothing else swept in.

Merging.