Problem
The model table in coding-agents/opencode/settings/juspay.nix hand-maintains each
model's context (max input) and output (max output) token limits. These drift
from what the gateway actually serves, and nothing catches the drift.
Concrete example (found while working on #81): glm-latest (GLM-5.2) is configured as
context = 202752; output = 32000, but the gateway advertises and enforces a much
larger window.
The gateway already exposes the source of truth
grid.ai.juspay.net is a LiteLLM proxy. Two authenticated endpoints return live
model metadata (the JUSPAY_API_KEY works for reads — VPN only needed to create a key):
GET /v1/models — the list of served model ids
GET /v1/model/info — per-model litellm_params.model (the upstream model, e.g.
zai-org/GLM-5.2-dev) and model_info.{max_input_tokens, max_output_tokens, max_tokens, supports_reasoning, …}
Live values today:
| key |
upstream |
gateway max_input |
gateway max_output |
config today |
glm-latest |
zai-org/GLM-5.2-dev |
1000000 |
1000000 |
202752 / 32000 |
glm-flash-experimental |
GLM-4.7-Flash |
128000 |
128000 |
262144 / 32000 |
Both differ from the committed config.
Proposal
Add a way to regenerate the limits from the gateway, e.g.:
- a
just recipe / script (just refresh-models) that curls /v1/model/info with
JUSPAY_API_KEY and updates the context/output values, and/or
- a CI check (alongside the daily
flake.lock bump) that flags drift.
Caveats to design around
- Auth in CI —
/v1/model/info needs JUSPAY_API_KEY as a secret.
- Placeholder values —
glm-latest reports 1000000 for all three of
max_tokens/input/output: it's a single shared 1M window, not a 1M output cap.
max_tokens = 1000000 actually 400s with ContextWindowExceededError once a
prompt is included, and 1000001 is rejected ("supports at most 1000000 completion
tokens"). So a naive copy is unsafe — the refresher needs a sane output cap
(e.g. min(max_output, 128000)). Note OpenCode also caps the wire max_tokens at
32000 for custom-provider models regardless of limit.output, so the refresher
should keep output at 32000 — a larger value just shrinks the usable input budget.
- Hand-curated fields — which models to expose, friendly names, the
reasoningEffort tiers, and modalities are not in model_info. This should
update limits for already-listed models, not blindly regenerate the whole table.
Status
#81 corrects the GLM-5.2 limits by hand (context = 1000000, output = 128000).
This issue tracks automating the same for the whole table so it never drifts again
— e.g. glm-flash-experimental is also off (config 262144 / 32000 vs gateway
128000 / 128000).
Problem
The model table in
coding-agents/opencode/settings/juspay.nixhand-maintains eachmodel's
context(max input) andoutput(max output) token limits. These driftfrom what the gateway actually serves, and nothing catches the drift.
Concrete example (found while working on #81):
glm-latest(GLM-5.2) is configured ascontext = 202752; output = 32000, but the gateway advertises and enforces a muchlarger window.
The gateway already exposes the source of truth
grid.ai.juspay.netis a LiteLLM proxy. Two authenticated endpoints return livemodel metadata (the
JUSPAY_API_KEYworks for reads — VPN only needed to create a key):GET /v1/models— the list of served model idsGET /v1/model/info— per-modellitellm_params.model(the upstream model, e.g.zai-org/GLM-5.2-dev) andmodel_info.{max_input_tokens, max_output_tokens, max_tokens, supports_reasoning, …}Live values today:
glm-latestzai-org/GLM-5.2-devglm-flash-experimentalGLM-4.7-FlashBoth differ from the committed config.
Proposal
Add a way to regenerate the limits from the gateway, e.g.:
justrecipe / script (just refresh-models) thatcurls/v1/model/infowithJUSPAY_API_KEYand updates thecontext/outputvalues, and/orflake.lockbump) that flags drift.Caveats to design around
/v1/model/infoneedsJUSPAY_API_KEYas a secret.glm-latestreports1000000for all three ofmax_tokens/input/output: it's a single shared 1M window, not a 1M output cap.
max_tokens = 1000000actually 400s withContextWindowExceededErroronce aprompt is included, and
1000001is rejected ("supports at most 1000000 completiontokens"). So a naive copy is unsafe — the refresher needs a sane
outputcap(e.g. min(max_output, 128000)). Note OpenCode also caps the wire
max_tokensat32000 for custom-provider models regardless of
limit.output, so the refreshershould keep
outputat 32000 — a larger value just shrinks the usable input budget.reasoningEfforttiers, and modalities are not inmodel_info. This shouldupdate limits for already-listed models, not blindly regenerate the whole table.
Status
#81 corrects the GLM-5.2 limits by hand (
context = 1000000,output = 128000).This issue tracks automating the same for the whole table so it never drifts again
— e.g.
glm-flash-experimentalis also off (config262144 / 32000vs gateway128000 / 128000).