fix(tracker): grep verified_models.py for frontend support#99
Merged
Conversation
The frontend probe was grepping `openhands-sdk/openhands/sdk/llm/utils/model_features.py`, but after the SDK's "single source of truth" migration the bare model names live in `verified_models.py` (under `VERIFIED_OPENHANDS_MODELS`). `model_features.py` no longer mentions them, so the probe returned no hits and every recently-added model showed `frontend_support_timestamp: null` (GPT-5.5, trinity-large-thinking, GLM-5.1, Nemotron-3-Super, MiniMax-M2.7, Qwen3.6-Plus, Nemotron-3-Nano, MiniMax-M2.1, DeepSeek-V3.2-Reasoner). The OpenHands frontend renders its openhands-provider "Verified" dropdown directly from the SDK's `VERIFIED_OPENHANDS_MODELS` (via `openhands/app_server/utils/llm.py`), so a model appearing in `verified_models.py` at the pinned SDK version is the correct signal for frontend support. This is a one-path correction; the legacy `frontend/src/utils/verified-models.ts` probe is kept (the file has been deleted, but `git log -G` still finds historical adds so pre-migration models retain accurate timestamps). Co-authored-by: openhands <openhands@all-hands.dev>
|
The latest updates on your projects. Learn more about Vercel for GitHub.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Problem
Nine recently-added models show
frontend_support_timestamp: nullinfrontend/public/all_models.jsoneven though they're shipped in the frontend dropdown today:search_frontend_for_modeldoes twogit log -Gprobes. Both turn up empty for these models:frontend/src/utils/verified-models.tsinOpenHands/OpenHands— the file has been deleted (a repo-wide code search returns 0 hits; raw fetch returns 404). The frontend's verified-models list is now computed at runtime inopenhands/app_server/utils/llm.py, which importsVERIFIED_OPENHANDS_MODELSdirectly from the pinnedopenhands-sdk(==1.22.1onmaintoday).openhands-sdk/openhands/sdk/llm/utils/model_features.pyin the SDK repo —VERIFIED_OPENHANDS_MODELSwas moved out of this file as part of the "single source of truth" refactor.grep -cE "gpt-5\.5|trinity-large-thinking|nemotron-3-super|glm-5\.1|minimax-m2\.7|qwen3-6-plus"againstmodel_features.py @ v1.22.1returns0.The bare names live in
verified_models.pyinstead, and that file at the pinned SDK tag contains every "missing" model:Fix
Single-path correction: change the SDK-side probe from
model_features.py→verified_models.py. The legacyverified-models.tsprobe is preserved so historical pre-migration models keep their accurate timestamps (git log -Ghappily finds adds in files that were later deleted).Updated
SEARCH_PATHS["frontend"]and the existing docstring to match, plus the two mock tests that asserted the old path.Why this and not a
pyproject.toml-pin walkA more semantically faithful fix would walk
OpenHands/OpenHands'spyproject.tomlhistory, parse eachopenhands-sdk==X.Y.Zpin, and return the first commit whose pinned SDK already contained the model inVERIFIED_OPENHANDS_MODELS— that's literally "the day the dropdown started shipping with this model." Happy to do that in a follow-up. For now this one-path correction unblocks the 9 broken rows with the minimum possible blast radius and uses the existinggit log -Gmachinery unchanged.Verification
python -m astparses both files cleanly.pytest tests/test_track_llm_support.py -q→ 91 passed, 1 failed. The single failure isTestSearchInfraProxy::test_search_infra_proxy_no_valid_versions, which is a pre-existing flake onmain(reproduced by reverting this branch's changes) and unrelated to the frontend probe.This PR was created by an AI agent (OpenHands) on behalf of the user.
@juanmichelini can click here to continue refining the PR