fix(dashboard): report the model size the installer recorded - #2515
Open
Hoang130203 wants to merge 1 commit into
Open
fix(dashboard): report the model size the installer recorded#2515Hoang130203 wants to merge 1 commit into
Hoang130203 wants to merge 1 commit into
Conversation
get_model_info() guessed the model size from the model name using a
ladder of \b<N>b\b patterns, defaulting to 15.0 GB when nothing matched.
Three of the ten models the tier maps actually select are misreported:
tier-map LLM_MODEL declared guessed delta
qwen3-coder-next 47.4 GB 15.0 GB -32.4
qwen3.6-35b-a3b 20.6 GB 15.0 GB -5.6
qwen3.5-2b 1.2 GB 1.5 GB +0.3
Both of the large ones are MoE models whose names do not spell a
parameter count the ladder recognises, so they silently take the 15.0 GB
default. NV_ULTRA's flagship model is shown at less than a third of its
real size.
The exact number is already available. The tier maps set
LLM_MODEL_SIZE_MB for every tier, .env.example and .env.schema.json
document it, settings.py tracks it, and model activation persists it —
get_model_info was the one place that did not read it.
Prefer the declared value; keep the name ladder as the fallback for
installs that predate the key or do not carry it. A missing, non-numeric
or non-positive value falls back rather than reporting 0.0 GB.
Quantization detection moves to a helper so both paths share it instead
of the declared-size path having to duplicate the awq/gptq/gguf checks.
Adds 10 assertions to TestGetModelInfo covering both paths, the MoE
names the ladder cannot read, and every unusable declared value.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
helpers.get_model_info()guesses the model size from the model name, witha ladder of
\b<N>b\bpatterns and a 15.0 GB default when nothing matches.Checked against every model the tier maps actually select, and their declared
LLM_MODEL_SIZE_MB:The two large misses are MoE models —
qwen3-coder-nextandqwen3.6-35b-a3b— whose names do not spell a parameter count the ladderrecognises, so they take the 15.0 GB default. NV_ULTRA's flagship model is
displayed at less than a third of its real size, and SH_LARGE's is off by
5.6 GB. On a dashboard whose job includes telling you whether a model fits your
VRAM, a 32 GB understatement is the wrong direction to be wrong in.
The number was already there
LLM_MODEL_SIZE_MBis not something I am introducing:installers/lib/tier-map.shsets it (48500,21110,1221…);.env.exampleand.env.schema.jsondocument it;settings.pylists it among the tracked keys;test_model_activate.pyalready asserts activation persists it ("21110").get_model_info()was the one place that did not read it.Fix
Prefer the declared value, keep the name ladder as the fallback:
unaffected.
auto, a trailing comment) or non-positive value (0on theCLOUD tier,
-1) → ladder, not0.0 GB._detect_quantization()helper soboth paths share it rather than the new path duplicating the awq/gptq/gguf
checks.
Verified end to end against real
.envshapes:Test
Adds 10 assertions to the existing
TestGetModelInfo: the declared path, thedeclared value beating a name the ladder would have matched, both MoE names
the ladder cannot read, the no-key fallback, every unusable declared value
(parametrised over
"",auto,notanumber,0,-1), and thatquantization detection still works on the declared path.
AI Assistance
AI assisted with drafting the fallback logic, the test matrix, and this
description. I built the comparison table above by running the shipped ladder
against the tier maps' own declared sizes, which is how the misses surfaced.
Release Lane
release/2.6.xmainStable hotfix reason:
Changed Surface
(One function in
helpers.pyplus a small extracted helper, and its tests.)Risk And Validation
git diff --checkrelease/2.6.xCommands/results:
No pre-existing test changed; the 14 that were already in
TestGetModelInfostill pass unmodified, which is what pins the fallback behaviour.
Operational Change Check
get_model_info()is a read path behind the dashboard's system-status view. Thechange alters one reported number, and only towards the value the installer
recorded. Nothing is written, no schema changes, and an install without
LLM_MODEL_SIZE_MBin its.envbehaves exactly as before — which is what theuntouched existing tests assert.
Notes For Reviewers
The ladder is now the fallback, and it is still wrong for those two models.
I deliberately did not add
\b35b\band acoder-nextspecial case: patchingthe ladder treats the symptom, and any model added tomorrow reintroduces it.
The declared value is the fix; the ladder only has to cover installs that
predate the key. If you would rather delete the ladder entirely and return
Nonewhen the size is unknown — arguably more honest than a 15.0 GB guess —that is a smaller diff than this one and I am happy to send it instead.
Adjacent, not fixed here: the same
.envread also parsesCTX_SIZE/MAX_CONTEXTwith a 32768 default. That default is not obviously right forTier 0 (8192) or NV_ULTRA (131072), but it only applies when both keys are
missing or unparseable, which the installer does not produce. I left it alone
rather than widen the diff.