RTX 5060 Ti (Blackwell sm_120): draft OOM despite model sizes fitting — 3× driverInitFileInfo result=11 causes ~820 MiB of orphaned VRAM overhead

[EnricoFerrero64]

Environment

• GPU: NVIDIA GeForce RTX 5060 Ti, 16 GB GDDR7 (15,816 MiB CUDA-visible)
• Architecture: Blackwell, compute capability sm_120
• Driver: 595.71.05
• CUDA (in-container): 13.2
• Image: ghcr.io/luce-org/lucebox-hub:cuda12
• Host: Unraid 7.x, Linux kernel 6.18
• Container: docker run --runtime nvidia (also tested with --privileged, same result)

Problem

Every startup produces 3 CUDA initialization errors before any model loads:

ERROR: driverInitFileInfo 578 result=11
ERROR: init 664 result=11
ERROR: init 250 result=11

These errors appear to leave orphaned VRAM allocations. Measured overhead (via nvidia-smi after server loads target-only, no draft):

• CUDA-visible VRAM: 15,816 MiB
• nvidia-smi used (target only, no draft): 15,640 MiB
• Target model GPU footprint (13.84 GiB Q4_K_S): 14,172 MiB
• Unexplained overhead: 1,020 MiB (expected: ~200–300 MiB on non-Blackwell)

The draft model (dflash-draft-3.6-q4_k_m.gguf, 1,007 MiB) then fails to allocate:

ggml_backend_cuda_buffer_type_alloc_buffer: allocating 1007.00 MiB on device 0: cudaMalloc failed: out of memory
draft load: ggml_backend_alloc_ctx_tensors failed (draft GGUF)
[backend_factory] Qwen35Backend init failed

Root cause hypothesis

The 3× result=11 failures at CUDA init (likely Blackwell-specific subsystems: VMM file-backed
cache, NVLink manager, or compute engine init) each appear to leave ~340 MiB of VRAM
unreachable. This inflates the fixed overhead from ~200 MiB (typical on Ampere/Ada) to ~1,020 MiB.

Math:

• target (14,172) + overhead (1,020) + draft (1,007) = 16,199 MiB > 15,816 MiB → gap 383 MiB
• Even with max_ctx=0 (no KV cache), draft cannot fit

What I tried (all ineffective)

• DFLASH_KVFLASH=72: no effect (env var not forwarded to binary)
• DFLASH_MAX_CTX=8192: KV cache saves 336 MiB, gap still 494 MiB
• --chunk 32: no effect on fixed overhead
• --privileged in Docker: result=11 persist (not container permission issue)
• CUDA_CACHE_DISABLE=1 + CUDA_MODULE_LOADING=LAZY: no effect
• --draft-device cpu:0: binary rejects it ("expected backend:gpu")
• --draft-residency request-scoped + --prefill-drafter: only defers the prefill drafter
  (Qwen3-0.6B), not the spec-decode draft — which must be persistent

Expected behavior

On Ampere/Ada (e.g. RTX 3090 24 GB), the same model combination reportedly works.
The Lucebox docs mention ≥22 GB as recommended, but the gap on 16 GB is entirely
caused by the anomalous 820 MiB extra overhead on Blackwell, not by the model sizes
themselves (14,172 + 200 + 1,007 = 15,379 MiB would fit).

Request

1. Investigate the 3× driverInitFileInfo result=11 errors on Blackwell (sm_120)
2. If these are causing orphaned VRAM allocations, a fix would allow the 27B target + draft
   to coexist on 16 GB Blackwell cards — currently an impossible configuration despite the
   hardware being theoretically sufficient
3. Consider a warning at startup if these init errors are detected:
   "Blackwell CUDA init partially failed — VRAM overhead may exceed expected values"