feat(bench): fold typed registry capability fields into _model_caps

[thinmintdev]

Item 3 of #1823.

Why

_model_caps matched only the freeform capabilities/tags lists. Those are unmaintained in practice — across 49 installed models on a live box, only a handful carry a semantic label. That set stamps identity.model.caps onto every benchmark record, which is where the public leaderboard's capability pills come from, so a model whose capabilities live in the typed fields showed no pill at all.

The typed fields are the surface an operator actually edits (PATCH /api/models accepts defaults and capability_flags), so a typed edit should reach the roster without also hand-editing a parallel string list.

What

Folded in following the existing defaults.mtp precedent:

pill	derived from
vision	carrying an mmproj, unless defaults.vision is explicitly False
tool-calling	capability_flags.tool_calling is True
reasoning	defaults.enable_thinking is True

vision keys off the projector, not the flag, because registry/model.py documents defaults.vision=None as AUTO — "the mmproj sidecar loads whenever the model carries one" — and True as "an explicit no-op affirmation". So mmproj presence is the real signal and an explicit False (force-suppress) is the only thing that revokes it.

Only unambiguous positives fold in. Every flag is tri-state and None means "unset / decided elsewhere", never "lacks the capability" — so None and False both add nothing rather than asserting an absence.

Safety

Widening this set is safe for benchmark selection: suites match with caps_any, so extra labels can only make a model match more selectors, never fewer. The shipped suites (roster.toml, smoke.toml) select on ["chat", "coder"], which none of these labels touch. There's a test pinning the narrow ⊆ wide property.

Honest scope

This is plumbing, not a fix for the coverage gap itself. On today's data it recovers exactly one model, because the typed fields are nearly as empty as the freeform ones — capability_flags.tool_calling is set on 1 of 49 models, defaults.enable_thinking on 1, and the 6 with an mmproj already carry a freeform vision label. What it buys is that labelling from here on lands in one place and propagates automatically.

Also worth knowing before anyone measures a labelling pass: caps are captured into records at run time, so the leaderboard only reflects new labels once the affected models are re-benchmarked.

Verification

• pytest tests/bench tests/cli — 1113 passed
• ruff check + ruff format --check clean
• 11 new tests. Red-checked against the unmodified planner: the 4 behavioural ones fail without this change, the other 7 are guards (tri-state handling, no-projector, union-safety) that pass in both and must not regress.

🤖 Generated with Claude Code

[chatgpt-codex-connector]

You have reached your Codex usage limits for code reviews. You can see your limits in the Codex usage dashboard.
To continue using code reviews, you can upgrade your account or add credits to your account and enable them for code reviews in your settings.