Skip to content

fix(tracker): grep verified_models.py for frontend support#99

Merged
juanmichelini merged 1 commit into
mainfrom
fix-frontend-path-verified-models
May 21, 2026
Merged

fix(tracker): grep verified_models.py for frontend support#99
juanmichelini merged 1 commit into
mainfrom
fix-frontend-path-verified-models

Conversation

@juanmichelini
Copy link
Copy Markdown
Contributor

Problem

Nine recently-added models show frontend_support_timestamp: null in frontend/public/all_models.json even though they're shipped in the frontend dropdown today:

DeepSeek-V3.2-Reasoner    sdk=2025-12-10  fe=null  tier=2
GLM-5.1                   sdk=2026-04-10  fe=null  tier=1
GPT-5.5                   sdk=2026-04-23  fe=null  tier=1
MiniMax-M2.1              sdk=2026-01-12  fe=null  tier=2
MiniMax-M2.7              sdk=2026-03-19  fe=null  tier=1
Nemotron-3-Nano           sdk=2026-02-16  fe=null  tier=2
Nemotron-3-Super          sdk=2026-04-04  fe=null  tier=1
Qwen3.6-Plus              sdk=2026-04-03  fe=null  tier=2
trinity-large-thinking    sdk=2026-04-08  fe=null  tier=1

search_frontend_for_model does two git log -G probes. Both turn up empty for these models:

  1. frontend/src/utils/verified-models.ts in OpenHands/OpenHands — the file has been deleted (a repo-wide code search returns 0 hits; raw fetch returns 404). The frontend's verified-models list is now computed at runtime in openhands/app_server/utils/llm.py, which imports VERIFIED_OPENHANDS_MODELS directly from the pinned openhands-sdk (==1.22.1 on main today).
  2. openhands-sdk/openhands/sdk/llm/utils/model_features.py in the SDK repo — VERIFIED_OPENHANDS_MODELS was moved out of this file as part of the "single source of truth" refactor. grep -cE "gpt-5\.5|trinity-large-thinking|nemotron-3-super|glm-5\.1|minimax-m2\.7|qwen3-6-plus" against model_features.py @ v1.22.1 returns 0.

The bare names live in verified_models.py instead, and that file at the pinned SDK tag contains every "missing" model:

$ curl -s https://raw.githubusercontent.com/OpenHands/software-agent-sdk/v1.22.1/openhands-sdk/openhands/sdk/llm/utils/verified_models.py \
  | grep -E "trinity|gpt-5\.5|glm-5\.1|nemotron-3-super|qwen3-6|minimax-m2\.7"
"gpt-5.5",
"minimax-m2.7",
"glm-5.1",
"nemotron-3-super",
"qwen3-6-plus",
"trinity-large-thinking",
...

Fix

Single-path correction: change the SDK-side probe from model_features.pyverified_models.py. The legacy verified-models.ts probe is preserved so historical pre-migration models keep their accurate timestamps (git log -G happily finds adds in files that were later deleted).

Updated SEARCH_PATHS["frontend"] and the existing docstring to match, plus the two mock tests that asserted the old path.

Why this and not a pyproject.toml-pin walk

A more semantically faithful fix would walk OpenHands/OpenHands's pyproject.toml history, parse each openhands-sdk==X.Y.Z pin, and return the first commit whose pinned SDK already contained the model in VERIFIED_OPENHANDS_MODELS — that's literally "the day the dropdown started shipping with this model." Happy to do that in a follow-up. For now this one-path correction unblocks the 9 broken rows with the minimum possible blast radius and uses the existing git log -G machinery unchanged.

Verification

  • python -m ast parses both files cleanly.
  • pytest tests/test_track_llm_support.py -q91 passed, 1 failed. The single failure is TestSearchInfraProxy::test_search_infra_proxy_no_valid_versions, which is a pre-existing flake on main (reproduced by reverting this branch's changes) and unrelated to the frontend probe.

This PR was created by an AI agent (OpenHands) on behalf of the user.

@juanmichelini can click here to continue refining the PR

The frontend probe was grepping
`openhands-sdk/openhands/sdk/llm/utils/model_features.py`, but after
the SDK's "single source of truth" migration the bare model names
live in `verified_models.py` (under `VERIFIED_OPENHANDS_MODELS`).
`model_features.py` no longer mentions them, so the probe returned
no hits and every recently-added model showed
`frontend_support_timestamp: null` (GPT-5.5, trinity-large-thinking,
GLM-5.1, Nemotron-3-Super, MiniMax-M2.7, Qwen3.6-Plus,
Nemotron-3-Nano, MiniMax-M2.1, DeepSeek-V3.2-Reasoner).

The OpenHands frontend renders its openhands-provider "Verified"
dropdown directly from the SDK's `VERIFIED_OPENHANDS_MODELS` (via
`openhands/app_server/utils/llm.py`), so a model appearing in
`verified_models.py` at the pinned SDK version is the correct
signal for frontend support.

This is a one-path correction; the legacy
`frontend/src/utils/verified-models.ts` probe is kept (the file has
been deleted, but `git log -G` still finds historical adds so
pre-migration models retain accurate timestamps).

Co-authored-by: openhands <openhands@all-hands.dev>
@vercel
Copy link
Copy Markdown

vercel Bot commented May 21, 2026

The latest updates on your projects. Learn more about Vercel for GitHub.

Project Deployment Actions Updated (UTC)
openhands-llm-support-tracker Ready Ready Preview, Comment May 21, 2026 4:18am

Request Review

@juanmichelini juanmichelini marked this pull request as ready for review May 21, 2026 21:33
@juanmichelini juanmichelini merged commit 46e72e7 into main May 21, 2026
4 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants