Skip to content

fix(external-llm): match Ollama models carrying the default :latest tag - #2512

Open
Hoang130203 wants to merge 1 commit into
Osmantic:mainfrom
Hoang130203:fix/external-llm-ollama-latest-tag
Open

fix(external-llm): match Ollama models carrying the default :latest tag#2512
Hoang130203 wants to merge 1 commit into
Osmantic:mainfrom
Hoang130203:fix/external-llm-ollama-latest-tag

Conversation

@Hoang130203

Copy link
Copy Markdown

Summary

external_llm_model_matches() normalises both sides and requires an exact
match. The normaliser translated : to - after stripping quantization
suffixes, so whatever Ollama put in the tag position survived into the name.

Ollama's default tag is latestollama pull qwen3-30b-a3b is reported by
/api/tags as qwen3-30b-a3b:latest. Against origin/main:

  qwen3-30b-a3b:latest             -> qwen3-30b-a3b-latest
  gemma-4-e4b-it:latest            -> gemma-4-e4b-it-latest
  qwen3.5-2b:q8_0                  -> qwen3.5-2b-q8-0

  NO MATCH  qwen3-30b-a3b    <- qwen3-30b-a3b:latest
  NO MATCH  gemma-4-e4b-it   <- gemma-4-e4b-it:latest
  NO MATCH  qwen3.5-2b       <- qwen3.5-2b:q8_0
  MATCH     qwen3.5-9b       <- qwen3.5:9b

So the commonest case there is — the user already has exactly the model the
tier map wants, pulled by bare name — was declined, and the installer
downloaded a duplicate multi-GB GGUF instead of reusing it. That is the
opposite of what feat(installer): reuse validated external model services
exists to do.

The last line is why this survived: a user who happened to pull with an
explicit size tag (qwen3.5:9b) matched fine, and that is the case the
existing test covers.

Fix

Do tag handling before the quantization strip:

-e 's/:latest$//'      # Ollama's default tag is not part of the name
-e 's/:/-/'            # ...then a quant in the tag position is strippable
-e 's/[-_.](q[0-9]+...|iq[0-9]+...|fp(8|16|32)|bf16)([-_.].*)?$//'

Two properties I was careful to keep:

  • Size tags are identity and must survive. qwen3.5:9bqwen3.5-9b,
    llama3.2:3bllama3.2-3b.
  • latest is only ever a tag. Anchored to :latest$, so a model named
    my-latest-model is untouched.

After:

  MATCH     qwen3-30b-a3b    <- qwen3-30b-a3b:latest
  MATCH     gemma-4-e4b-it   <- gemma-4-e4b-it:latest
  MATCH     qwen3.5-2b       <- qwen3.5-2b:q8_0
  MATCH     qwen3.5-9b       <- qwen3.5:9b
  MATCH     qwen3.5-9b       <- Qwen3.5-9B-Q4_K_M.gguf

  no match  qwen3.5-9b       <- llama3.2:3b
  no match  qwen3.5-9b       <- qwen3.5:4b
  no match  qwen3-30b-a3b    <- qwen3-8b:latest

Test

Extends tests/test-external-services.sh with 12 assertions: every tag shape,
the tier-target-against-Ollama-listing matches, and negative cases so a
different size or a different model carrying :latest still does not match.

AI Assistance

AI assisted with the sed ordering and wording this description. I found the bug
by running the shipped normaliser against the model ids Ollama actually reports
for the models this repo's tier map selects, and checked the false-positive
cases before pushing.

Release Lane

  • Stable hotfix targeting release/2.6.x
  • Mainline change targeting main
  • Next-minor work targeting the next feature/minor release
  • Not sure; reviewer should help classify

Stable hotfix reason:

n/a

Changed Surface

  • Docs only
  • Tests only
  • Dashboard UI
  • Dashboard API / host agent
  • Installer / bootstrap / lifecycle
  • Docker Compose / service manifests
  • Model routing / Hermes / capabilities
  • Network exposure / auth / proxy
  • Dependencies / runtime wiring

(One sed pipeline in installers/lib/external-services.sh, plus assertions in
its existing test.)

Risk And Validation

  • Risk level: Low
  • Validation run:
    • git diff --check
    • Markdown/link sanity for docs
    • Focused tests listed below
    • Dashboard lint/test/build
    • Extension audit / compose validation
    • Release-grade fleet or scoped hardware validation
    • Stable-lane patch validation, if targeting release/2.6.x

Commands/results:

$ bash -n installers/lib/external-services.sh
syntax OK

$ bash tests/test-external-services.sh
Result: 29 passed, 5 failed

# the same suite with origin/main's normaliser and this PR's assertions:
[FAIL] drops Ollama's default :latest tag (expected 'qwen3-30b-a3b', got 'qwen3-30b-a3b-latest')
[FAIL] drops :latest from a hyphenated name (expected 'gemma-4-e4b-it', got 'gemma-4-e4b-it-latest')
[FAIL] strips a quantization supplied as an Ollama tag (expected 'qwen3.5-2b', got 'qwen3.5-2b-q8-0')
[FAIL] strips an IQ quantization tag (expected 'qwen3.5-9b', got 'qwen3.5-9b-iq4-xs')
[FAIL] matches a tier target against Ollama's :latest listing
[FAIL] matches a hyphenated tier target against :latest
[FAIL] matches a tier target against a quantization tag
Result: 22 passed, 12 failed

Caveat on those 5 remaining failures — they are not mine. Unmodified
origin/main reports 17 passed, 5 failed on my host; with this PR it is
29 passed, 5 failed. The same five fail either way. They are the phase-06
and URL-validation cases that shell out to python3, which on my Windows dev
box resolves to the Microsoft Store alias and refuses to run (the same issue
#2368 addresses for two other scripts). CI has a real python3. My change adds
12 passing assertions and introduces no new failures.

Operational Change Check

external_llm_normalize_model_name runs during install when ODS probes an
existing Ollama or LM Studio for a model it can reuse. The change makes the
matcher accept ids it previously rejected; it does not make it accept
different models — the negative assertions above pin that. The visible effect
is that an install which previously downloaded a duplicate GGUF now reuses the
local copy, which is the documented intent of the feature.

  • This is not an operational change.
  • This is an operational change and validation is recorded above.
  • This is an operational change and validation is intentionally deferred for:

Notes For Reviewers

Why :latest gets its own rule rather than falling out of the colon
translation.
Dropping it entirely is right — latest carries no identity —
but only in the tag position. Translating first and then trying to strip a
trailing -latest would also eat the tail of a model legitimately named
...-latest, so the rule is anchored to :latest$ before the colon is gone.
The my-latest-model assertion pins that.

Tags I deliberately did not touch. Ollama also uses tags like
:instruct, :text and :fp16. fp16 is already in the quant pattern and
now reachable through the tag position too. instruct stays part of the name
(qwen3.5:9b-instruct-q4_K_Mqwen3.5-9b-instruct), which is correct —
instruct and base are different models. If your tier targets ever name an
instruct variant, that will match; if they name the base, it will not, which is
the behaviour you want.

The external-model reuse path normalises both sides of a comparison and
then requires an exact string match. Normalisation translated `:` to `-`
only after stripping quantization suffixes, so anything Ollama put in the
tag position survived into the name.

Ollama's default tag is `latest`. `ollama pull qwen3-30b-a3b` is listed by
/api/tags as `qwen3-30b-a3b:latest`, which normalised to
`qwen3-30b-a3b-latest` and matched the tier target `qwen3-30b-a3b` at
nothing:

    qwen3-30b-a3b:latest   -> qwen3-30b-a3b-latest   NO MATCH
    gemma-4-e4b-it:latest  -> gemma-4-e4b-it-latest  NO MATCH
    qwen3.5-2b:q8_0        -> qwen3.5-2b-q8-0        NO MATCH

That is the commonest case there is: the user already has exactly the
model ODS wants, pulled by bare name. Reuse declined it and the installer
downloaded a duplicate multi-GB GGUF — the opposite of what the feature
exists to do. A user who happened to pull with an explicit size tag
(qwen3.5:9b) matched fine, which is why this survived.

Do tag handling before the quantization strip: drop a trailing `:latest`,
then translate the colon so a quantization arriving as a tag is
recognisable to the existing pattern. Size tags still survive as part of
the identity (qwen3.5:9b -> qwen3.5-9b), and `latest` is only ever
stripped as a tag, never from inside a name.

Extends tests/test-external-services.sh with 12 assertions: every tag
shape, the tier-target-to-Ollama-listing matches, and negative cases so a
different size or a different model carrying :latest still does not match.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant