Eval bug: CUDA error: unsupported value or parameter in cublasSgemm_v2 during large context processing

[rbenrax]

Name and Version

version: 9775 (be4a6a6)
built with GNU 13.3.0 for Linux x86_64

Operating systems

Linux

GGML backends

CUDA

Hardware

2 x rtx3060

Models

Qwen3.6-35B-A3B-MTP-GGUF/Qwen3.6-35B-A3B-UD-IQ4_XS.gguf

Problem description & steps to reproduce

Bug Description

The llama.cpp server crashes with a CUDA error when processing a prompt with a very large context. The error occurs specifically in cublasSgemm_v2 during matrix multiplication.

Error Message

First Bad Commit

Stack Trace#0 __GI___wait4

#1 ggml_print_backtrace
#2 ggml_abort
#3 ggml_cuda_error
#4 ggml_cuda_op_mul_mat_cublas
#5 ggml_cuda_op_mul_mat
#6 ggml_backend_cuda_graph_compute
#7 ggml_backend_sched_graph_compute_async
#8 llama_context::graph_compute
#9 llama_context::process_ubatch
#10 llama_context::decode
#11 llama_decode
#12 server_context_impl::decode
#13 server_context_impl::update_slots
#14 server_queue::start_loop
#15 llama_server
#16 __libc_start_call_main
#17 __libc_start_main_impl
#18 _start

Relevant log output

Details

## Context Information from Logs

• Slot ID: 1
• Task ID: 19910
• Context slot size (n_ctx_slot): 131,072 tokens
• Prompt tokens (task.n_tokens): 105,375 tokens
• Cached tokens: 105,371 (after incremental caching)
• Context checkpoint created: 32 of 32 checkpoints
• Checkpoint size: 269.744 MiB
• Position: 104,923 tokens

Environment

• OS: Linux (Ubuntu/Debian based on paths)
• llama.cpp path: /opt/llama.cpp-beta/
• CUDA device: 0
• Build type: Beta branch (llama.cpp-beta)

Steps to Reproduce

1. Start llama.cpp server with a model supporting large context (n_ctx = 131072)
2. Send a prompt with approximately 105,375 tokens
3. The server processes the prompt, creates context checkpoints, caches tokens
4. During the decode phase, the CUDA error occurs in cublasSgemm_v2

Additional Notes

• The error happens after successfully caching most of the prompt (105,371 out of 105,375 tokens)
• Context checkpoints were being created successfully (32 checkpoints)
• The error specifically mentions "an unsupported value or parameter was passed to the function" in cuBLAS
• This suggests a potential integer overflow or dimension mismatch when passing matrix dimensions to cublasSgemm_v2

Possible Related Issues

• Large matrix dimensions causing integer overflow in cuBLAS parameters
• Context size exceeding certain CUDA/cuBLAS limits
• Memory alignment issues with very large tensors

Logs

/opt/llama.cpp-beta/build/bin/llama-server
-m /mnt/disco2/models/Qwen3.6-35B-A3B-MTP-GGUF/Qwen3.6-35B-A3B-UD-IQ4_NL.gguf
-mm /mnt/disco2/models/Qwen3.6-35B-A3B-GGUF/mmproj-Qwen3.6-35B-A3B-BF16.gguf
--image-min-tokens 1024
-mg 1
-ngl 999
-sm layer
-ts 10,12
-t 8
-tb 6
--temp 0.6
--top-p 0.95
--top-k 20
--min-p 0.0
--presence-penalty 0.0
--repeat-penalty 1.0
--reasoning on
--spec-type draft-mtp
--spec-draft-n-max 3
--jinja
-b 256
-ub 256
-np 3
--cache-idle-slots
-c 131072
-fa on
-kvu
--cache-type-k q8_0
--cache-type-v q8_0
--host 0.0.0.0
--port 11434

[rbenrax]

opt/llama.cpp-beta/ggml/src/ggml-cuda/ggml-cuda.cu:104: CUDA error
38.24.611.605 E CUDA error: an unsupported value or parameter was passed to the function
38.24.611.612 E current device: 0, in function ggml_cuda_op_mul_mat_cublas at /opt/llama.cpp-beta/ggml/src/ggml-cuda/ggml-cuda.cu:1774
38.24.611.613 E cublasSgemm_v2(ctx.cublas_handle(id), CUBLAS_OP_T, CUBLAS_OP_N, row_diff, src1_ncols, ne10, &alpha, src0_ddf_i, ne00, src1_ddf1_i, ne10, &beta, dst_dd_i, ldc)
[Nuevo LWP 32923]
[Nuevo LWP 32922]
[Nuevo LWP 32921]
[Nuevo LWP 32920]
[Nuevo LWP 32919]
[Nuevo LWP 32918]
[Nuevo LWP 32917]
[Nuevo LWP 32916]
[Nuevo LWP 32915]
[Nuevo LWP 32914]
[Nuevo LWP 32913]
[Nuevo LWP 32912]
[Nuevo LWP 32885]
[Nuevo LWP 32884]
[Depuración de hilo usando libthread_db enabled]
Using host libthread_db library "/lib/x86_64-linux-gnu/libthread_db.so.1".
0x00007e0a5a510813 in __GI___wait4 (pid=34514, stat_loc=0x0, options=0, usage=0x0) at ../sysdeps/unix/sysv/linux/wait4.c:30
aviso: 30 ../sysdeps/unix/sysv/linux/wait4.c: No existe el archivo o el directorio
#0 0x00007e0a5a510813 in __GI___wait4 (pid=34514, stat_loc=0x0, options=0, usage=0x0) at ../sysdeps/unix/sysv/linux/wait4.c:30
30 in ../sysdeps/unix/sysv/linux/wait4.c
#1 0x00007e0a5aafd663 in ggml_print_backtrace () from /opt/llama.cpp-beta/build/bin/libggml-base.so.0
#2 0x00007e0a5aafd80b in ggml_abort () from /opt/llama.cpp-beta/build/bin/libggml-base.so.0
#3 0x00007e0a4d0327c7 in ggml_cuda_error(char const*, char const*, char const*, int, char const*) () from /opt/llama.cpp-beta/build/bin/libggml-cuda.so.0
#4 0x00007e0a4d03dc08 in ggml_cuda_op_mul_mat_cublas(ggml_backend_cuda_context&, ggml_tensor const*, ggml_tensor const*, ggml_tensor*, char const*, float const*, char const*, float*, long, long, long, long, CUstream_st*) () from /opt/llama.cpp-beta/build/bin/libggml-cuda.so.0
#5 0x00007e0a4d043854 in ggml_cuda_op_mul_mat(ggml_backend_cuda_context&, ggml_tensor const*, ggml_tensor const*, ggml_tensor*, void ()(ggml_backend_cuda_context&, ggml_tensor const, ggml_tensor const*, ggml_tensor*, char const*, float const*, char const*, float*, long, long, long, long, CUstream_st*), void ()(float const, int const*, void*, ggml_type, long, long, long, long, long, long, long, long, CUstream_st*)) () from /opt/llama.cpp-beta/build/bin/libggml-cuda.so.0
#6 0x00007e0a4d04a811 in ggml_backend_cuda_graph_compute(ggml_backend*, ggml_cgraph*) () from /opt/llama.cpp-beta/build/bin/libggml-cuda.so.0
#7 0x00007e0a5ab1af57 in ggml_backend_sched_graph_compute_async () from /opt/llama.cpp-beta/build/bin/libggml-base.so.0
#8 0x00007e0a59ae1fe1 in llama_context::graph_compute(ggml_cgraph*, bool) () from /opt/llama.cpp-beta/build/bin/libllama.so.0
#9 0x00007e0a59ae612a in llama_context::process_ubatch(llama_ubatch const&, llm_graph_type, llama_memory_context_i*, ggml_status&) () from /opt/llama.cpp-beta/build/bin/libllama.so.0
#10 0x00007e0a59aecbf0 in llama_context::decode(llama_batch const&) () from /opt/llama.cpp-beta/build/bin/libllama.so.0
#11 0x00007e0a59aee87f in llama_decode () from /opt/llama.cpp-beta/build/bin/libllama.so.0
#12 0x00007e0a5ad947cc in server_context_impl::decode(int&, int, llama_batch&) () from /opt/llama.cpp-beta/build/bin/libllama-server-impl.so
#13 0x00007e0a5ad96840 in server_context_impl::update_slots() () from /opt/llama.cpp-beta/build/bin/libllama-server-impl.so
#14 0x00007e0a5ad3f511 in server_queue::start_loop(long) () from /opt/llama.cpp-beta/build/bin/libllama-server-impl.so
#15 0x00007e0a5ace4fb5 in llama_server(int, char**) () from /opt/llama.cpp-beta/build/bin/libllama-server-impl.so
#16 0x00007e0a5a42a1ca in __libc_start_call_main (main=main@entry=0x5bf7e1be8270

, argc=argc@entry=62, argv=argv@entry=0x7fff38aaea68) at ../sysdeps/nptl/libc_start_call_main.h:58
aviso: 58 ../sysdeps/nptl/libc_start_call_main.h: No existe el archivo o el directorio
#17 0x00007e0a5a42a28b in __libc_start_main_impl (main=0x5bf7e1be8270 , argc=62, argv=0x7fff38aaea68, init=, fini=, rtld_fini=, stack_end=0x7fff38aaea58) at ../csu/libc-start.c:360
aviso: 360 ../csu/libc-start.c: No existe el archivo o el directorio
#18 0x00005bf7e1be82a5 in _start ()
[Inferior 1 (process 32882) detached]