Commit 5476142
authored
[Misc] improve error messages for quantization mismatches and netloader config validation (vllm-project#10222)
### What this PR does / why we need it?
This PR improves error messages for three user-facing scenarios
identified during code audit, replacing bare Python exceptions with
clear guidance:
**1. Unsupported ModelSlim quantization scheme (`modelslim_config.py`)**
When a model is quantized with a scheme that vLLM Ascend recognizes but
doesn't fully implement (e.g., `quant_type=W4A16` registered for `moe`
but not for `linear`), the error previously only stated the fact without
suggesting a resolution:
```text
Before: NotImplementedError: Currently, vLLM Ascend doesn't support quant_type=W4A16 for layer_type=linear.
After: NotImplementedError: ... Please use a supported quantization format or load the model with its original float weights.
```
**2. NetLoader crashes with bare AttributeError on malformed config JSON
(`netloader.py`)**
When a user passes a non-dict JSON value (e.g., a string or array) to
`--model-loader-extra-config`, `config.get()` on a `str`/`list` raises a
bare `AttributeError`:
```text
Before: AttributeError: 'str' object has no attribute 'get'
After: RuntimeError: NetLoader requires --model-loader-extra-config to be a JSON object.
```
The fix adds an `isinstance(extra, dict)` check, following the same
pattern already used by `rfork_loader.py:43-46`.
- Fixes # (internal code audit, no GitHub issue)
### Does this PR introduce _any_ user-facing change?
Yes — error messages for three scenarios are changed:
1. When loading a ModelSlim model with an unsupported quantization
scheme (e.g., W4A16 + linear layer), the `NotImplementedError` now
includes usage guidance.
2. When `--model-loader-extra-config` contains a non-dict JSON value for
NetLoader, a `RuntimeError` with format guidance replaces the bare
`AttributeError`.
- vLLM version: v0.22.1
- vLLM main:
vllm-project/vllm@967c5c3
---------
Signed-off-by: xuchi <xuchicolson@163.com>1 parent 21e8dcf commit 5476142
2 files changed
Lines changed: 13 additions & 3 deletions
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
62 | 62 | | |
63 | 63 | | |
64 | 64 | | |
| 65 | + | |
| 66 | + | |
| 67 | + | |
| 68 | + | |
| 69 | + | |
| 70 | + | |
65 | 71 | | |
66 | 72 | | |
67 | 73 | | |
| |||
| Original file line number | Diff line number | Diff line change | |
|---|---|---|---|
| |||
393 | 393 | | |
394 | 394 | | |
395 | 395 | | |
396 | | - | |
397 | | - | |
398 | | - | |
| 396 | + | |
| 397 | + | |
| 398 | + | |
| 399 | + | |
| 400 | + | |
| 401 | + | |
| 402 | + | |
399 | 403 | | |
400 | 404 | | |
401 | 405 | | |
| |||
0 commit comments