You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
The model axis currently has two image kinds: the generic gateway flavors (models/bifrost|litellm|portkey, model selected at runtime via EVAL_MODEL) and five pinned per-model images (models/gpt-5, models/gpt-5.4, models/gpt-4.1-mini, models/claude-opus-4, models/claude-sonnet-4, all FROM core/litellm).
The per-model images create constant confusion:
Two overlapping meanings of "model": an image name (EVAL_GATEWAY_IMAGE=gpt-5.4) vs a runtime handle (EVAL_MODEL=openai/gpt-5.4). The deploy/oc tooling conflates them (--set gatewayImage=$MODEL with --model bifrost), and its handle plumbing (evalModel) isn't even read by the chart.
A parallel base image (core/litellm) and version machinery (EVAL_LITELLM_VERSION, eval.model.litellm_version labels, rego gates) that exist only for these five images — the runtime override has had no implementer since Delete dead eval-entrypoint.sh #50.
Kill the per-model image concept entirely. One shared model gateway: EVAL_MODEL=<provider>/<model> is the only way to pick a model; EVAL_GATEWAY_IMAGE picks only the proxy flavor (bifrost default; litellm/portkey). models/replay stays as the sole sanctioned pinned exception (models rule 17).
Delete the five containers/models/<model> dirs and the now-orphaned containers/core/litellm base.
Doctrine: models rule 1 forbids per-model images; deprecate rules 12–13's litellm-version machinery; gateways rules 16–17 become "one combo per flavor, never per model"; drop EVAL_LITELLM_VERSION/--litellm-version from principle 9 and src rules 10/12.
Retarget the guards: static sweep + check.rs assert every models/* image is FROM gateways/<flavor> (replay excepted); remove the unreachable litellm-version rego rules.
Fix the deploy/oc scripts so --model is the handle and --gateway the flavor.
Collapse docs/PR/issue templates to the single-path story.
Per contributing rule 2, this lands as two PRs: a doctrine half and an implementation half (the #187 pattern).
Problem
The model axis currently has two image kinds: the generic gateway flavors (
models/bifrost|litellm|portkey, model selected at runtime viaEVAL_MODEL) and five pinned per-model images (models/gpt-5,models/gpt-5.4,models/gpt-4.1-mini,models/claude-opus-4,models/claude-sonnet-4, allFROM core/litellm).The per-model images create constant confusion:
EVAL_GATEWAY_IMAGE=gpt-5.4) vs a runtime handle (EVAL_MODEL=openai/gpt-5.4). Thedeploy/octooling conflates them (--set gatewayImage=$MODELwith--model bifrost), and its handle plumbing (evalModel) isn't even read by the chart.core/litellm) and version machinery (EVAL_LITELLM_VERSION,eval.model.litellm_versionlabels, rego gates) that exist only for these five images — the runtime override has had no implementer since Delete dead eval-entrypoint.sh #50.Proposal
Kill the per-model image concept entirely. One shared model gateway:
EVAL_MODEL=<provider>/<model>is the only way to pick a model;EVAL_GATEWAY_IMAGEpicks only the proxy flavor (bifrost default; litellm/portkey).models/replaystays as the sole sanctioned pinned exception (models rule 17).containers/models/<model>dirs and the now-orphanedcontainers/core/litellmbase.EVAL_LITELLM_VERSION/--litellm-versionfrom principle 9 and src rules 10/12.check.rsassert everymodels/*image isFROM gateways/<flavor>(replay excepted); remove the unreachable litellm-version rego rules.deploy/ocscripts so--modelis the handle and--gatewaythe flavor.Per contributing rule 2, this lands as two PRs: a doctrine half and an implementation half (the #187 pattern).