Unable to get above 50/tks output Qwen 3.6 35 A3b Q4_K_M

[Kamalaja]

`version: '3.8'

services:
lucebox:
image: ghcr.io/luce-org/lucebox-hub:cuda12
restart: unless-stopped
runtime: nvidia
deploy:
resources:
reservations:
devices:
- capabilities: [gpu]
restart_policy:
condition: on-failure
environment:
# Minimal extra memory usage and enable Luce optimizations
- DFLASH27B_KV_TQ3=1
- DFLASH_SPARK=1
volumes:
- ./models:/opt/lucebox-hub/server/models:ro
ports:
- "28450:8080"
shm_size: '1gb'`

Docker compose ^

I am trying to run Qwen close to native on my 3090, however I cannot get more than 50tk/s - Could you advise on what I am doing wrong?

[davide221]

Two things that are slowing you down:

1. No draft = no speculative decode. The 3-6× speedup comes from DFlash + DDTree, which only turn on when a draft model is present. Your compose mounts no draft, so the server runs target-only autoregressive decode. Download a draft into models/draft/ and the entrypoint auto-attaches it.

https://huggingface.co/z-lab/Qwen3.6-35B-A3B-DFlash

2. Drop DFLASH_SPARK=1. Spark is for MoE that doesn't fit in VRAM. The 35B-A3B Q4_K_M fits fine on 24GB cards and Spark forces experts off-GPU into a PCIe swap ring.

[Kamalaja]

Thanks for the response!

I removed DFLASH_SPARK=1 and attempted to download the following dflashes into a models folder:
https://huggingface.co/Anbeeld/Qwen3.6-35B-A3B-DFlash-GGUF?show_file_info=qwen36-35b-a3b-dflash-Q4_K_M.gguf
https://huggingface.co/starskyzheng/Qwen3.6-35B-DFlash-GGUF

Both Q4_K_M

However, once I did so, the model enters token loops and does not appear to be any faster at that.

I tried mounting
- ./models/draft:/opt/lucebox-hub/server/models/draft:ro
separately as well as just letting my other mount handle it, but I am not noticing a difference