Skip to content

Commit 5476142

Browse files
authored
[Misc] improve error messages for quantization mismatches and netloader config validation (vllm-project#10222)
### What this PR does / why we need it? This PR improves error messages for three user-facing scenarios identified during code audit, replacing bare Python exceptions with clear guidance: **1. Unsupported ModelSlim quantization scheme (`modelslim_config.py`)** When a model is quantized with a scheme that vLLM Ascend recognizes but doesn't fully implement (e.g., `quant_type=W4A16` registered for `moe` but not for `linear`), the error previously only stated the fact without suggesting a resolution: ```text Before: NotImplementedError: Currently, vLLM Ascend doesn't support quant_type=W4A16 for layer_type=linear. After: NotImplementedError: ... Please use a supported quantization format or load the model with its original float weights. ``` **2. NetLoader crashes with bare AttributeError on malformed config JSON (`netloader.py`)** When a user passes a non-dict JSON value (e.g., a string or array) to `--model-loader-extra-config`, `config.get()` on a `str`/`list` raises a bare `AttributeError`: ```text Before: AttributeError: 'str' object has no attribute 'get' After: RuntimeError: NetLoader requires --model-loader-extra-config to be a JSON object. ``` The fix adds an `isinstance(extra, dict)` check, following the same pattern already used by `rfork_loader.py:43-46`. - Fixes # (internal code audit, no GitHub issue) ### Does this PR introduce _any_ user-facing change? Yes — error messages for three scenarios are changed: 1. When loading a ModelSlim model with an unsupported quantization scheme (e.g., W4A16 + linear layer), the `NotImplementedError` now includes usage guidance. 2. When `--model-loader-extra-config` contains a non-dict JSON value for NetLoader, a `RuntimeError` with format guidance replaces the bare `AttributeError`. - vLLM version: v0.22.1 - vLLM main: vllm-project/vllm@967c5c3 --------- Signed-off-by: xuchi <xuchicolson@163.com>
1 parent 21e8dcf commit 5476142

2 files changed

Lines changed: 13 additions & 3 deletions

File tree

vllm_ascend/model_loader/netloader/netloader.py

Lines changed: 6 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -62,6 +62,12 @@ def __init__(self, load_config: LoadConfig):
6262

6363
# Try to read config file at first
6464
extra = load_config.model_loader_extra_config
65+
66+
if extra is not None and not isinstance(extra, dict):
67+
err_msg = "NetLoader requires --model-loader-extra-config to be a JSON object."
68+
logger.error(err_msg)
69+
raise RuntimeError(err_msg)
70+
6571
if extra and "CONFIG_FILE" in extra:
6672
try:
6773
logger.info("Reading configs in file %s ...", load_config.model_loader_extra_config["CONFIG_FILE"])

vllm_ascend/quantization/modelslim_config.py

Lines changed: 7 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -393,9 +393,13 @@ def create_scheme_for_layer(
393393
if scheme_cls is not None:
394394
return scheme_cls()
395395

396-
err_msg = f"Currently, vLLM Ascend doesn't support quant_type={quant_type} for layer_type={layer_type}."
397-
logger.error(err_msg)
398-
raise NotImplementedError(err_msg)
396+
err_msg = (
397+
"Currently, vLLM Ascend doesn't support quant_type=%s for layer_type=%s. "
398+
"Please use a supported quantization format "
399+
"or load the model with its original float weights."
400+
)
401+
logger.error(err_msg, quant_type, layer_type)
402+
raise NotImplementedError(err_msg % (quant_type, layer_type))
399403

400404

401405
@register_quantization_config(ASCEND_QUANTIZATION_METHOD)

0 commit comments

Comments
 (0)