Gemma 4 blocked: mlx_lm 0.31.1 lacks gemma4 architecture module

[weklund-agent]

Summary

Gemma 4 models (e.g., gemma-4-e2b-it) cannot be served because vllm-mlx 0.2.6's bundled mlx_lm (v0.31.1) does not include a gemma4 model architecture module.

Error

ModuleNotFoundError: No module named 'mlx_lm.models.gemma4'

Full traceback from mlx-stack logs:

File ".../mlx_lm/utils.py", line 176, in _get_classes
    arch = importlib.import_module(f"mlx_lm.models.{model_type}")
ModuleNotFoundError: No module named 'mlx_lm.models.gemma4'

ValueError: Model type gemma4 not supported.
ERROR:    Application startup failed. Exiting.

Catch-22 with #17

This creates a dependency deadlock:

• vllm-mlx 0.2.6 — stable, but mlx_lm 0.31.1 doesn't support gemma4
• vllm-mlx 0.2.7 — crashes on startup due to missing return in load_model_with_fallback() (vllm-mlx continuous batching disabled due to upstream bug (waybarrios/vllm-mlx#211) #17)

Gemma 4 support is blocked until vllm-mlx ships a version that both fixes the 0.2.7 regression AND bundles a newer mlx_lm with gemma4 support.

Workaround

None currently — Gemma 4 models cannot be used. Substitute with Qwen 3 8B or similar for the fast tier.

Environment

• mlx-stack 0.3.8
• vllm-mlx 0.2.6 (pinned due to vllm-mlx continuous batching disabled due to upstream bug (waybarrios/vllm-mlx#211) #17)
• mlx_lm 0.31.1
• macOS, Apple M4 Pro 64GB