Eval bug: DeepSeek V3.2 no longer reasons (<think>) when using jinja template from DeepSeek V3.2 Exp and no reasoning_content is returned by the server

[fairydreaming]

Name and Version

$ ./bin/llama-server --version
ggml_cuda_init: found 8 CUDA devices:
Device 0: NVIDIA RTX PRO 6000 Blackwell Server Edition, compute capability 12.0, VMM: yes
Device 1: NVIDIA RTX PRO 6000 Blackwell Server Edition, compute capability 12.0, VMM: yes
Device 2: NVIDIA RTX PRO 6000 Blackwell Server Edition, compute capability 12.0, VMM: yes
Device 3: NVIDIA RTX PRO 6000 Blackwell Server Edition, compute capability 12.0, VMM: yes
Device 4: NVIDIA RTX PRO 6000 Blackwell Server Edition, compute capability 12.0, VMM: yes
Device 5: NVIDIA RTX PRO 6000 Blackwell Server Edition, compute capability 12.0, VMM: yes
Device 6: NVIDIA RTX PRO 6000 Blackwell Server Edition, compute capability 12.0, VMM: yes
Device 7: NVIDIA RTX PRO 6000 Blackwell Server Edition, compute capability 12.0, VMM: yes
version: 8406 (f4049ad)
built with GNU 13.3.0 for Linux x86_64

Operating systems

Linux

GGML backends

CUDA

Hardware

AMD EPYC 9654 + 8 x NVIDIA RTX PRO 6000 Blackwell Server Edition

Models

Observed in DeepSeek V3.2 Q8_0 ran with dense attention.
GGUF: https://huggingface.co/sszymczyk/DeepSeek-V3.2-nolight-GGUF

Problem description & steps to reproduce

Few months ago I tested DeepSeek V3.2 with dense attention in lineage-bench. It had near-perfect scores up to lineage-192.

Now when I test it in lineage-128 by using recent llama.cpp build (commit f4049ad) multiple wrong answers are observed and overall accuracy is below 0.7 (I used only 16 quizzes):

$ for length in 128; do ./lineage_bench.py -s -l $length -n 4 -r 42|./run_openrouter.py --url http://127.0.0.1:6666/v1/chat/completions -t 16 -m "deepseek-ai/DeepSeek-V3.2" -r --temp 1.0 --top-p 0.95 --max-tokens 32000 -o "results/f4049ad7_$length"|tee "results/f4049ad7_$length.csv"; done|./compute_metrics.py
100%|███████████████████████████████████████████████████████████████████████████████████████| 16/16 [17:38<00:00, 66.14s/it]
Successfully generated 16 of 16 quiz solutions.
|   Nr | model_name                |   lineage |   lineage-128 |
|-----:|:--------------------------|----------:|--------------:|
|    1 | deepseek-ai/DeepSeek-V3.2 |     0.688 |         0.688 |

When I ran the same by using an old llama.cpp build (commit 68b4d51) you can see the accuracy is still perfect (1.0):

$ for length in 128; do ./lineage_bench.py -s -l $length -n 4 -r 42|./run_openrouter.py --url http://127.0.0.1:6666/v1/chat/completions -t 16 -m "deepseek-ai/DeepSeek-V3.2" -r --temp 1.0 --top-p 0.95 --max-tokens 32000 -o "results/68b4d516_$length"|tee "results/68b4d516_$length.csv"; done|./compute_metrics.py
100%|████████████████████████████████████████████████████████████████████████████████████| 16/16 [1:00:37<00:00, 227.35s/it]
Successfully generated 16 of 16 quiz solutions.
|   Nr | model_name                |   lineage |   lineage-128 |
|-----:|:--------------------------|----------:|--------------:|
|    1 | deepseek-ai/DeepSeek-V3.2 |     1.000 |         1.000 |

Used llama-server cmd:

$ ./bin/llama-server -m /workspace/.hf_home/hub/models--sszymczyk--DeepSeek-V3.2-nolight-GGUF/snapshots/08d2f45c097687064c864d9c6bb360a82245ebc1/Q8_0/DeepSeek-V3.2-nolight-Q8_0-00001-of-00031.gguf --mmap --host 127.0.0.1 --port 6666 -np 16 -c 512000 --jinja --chat-template-file ../ds32.jinja

ds32.jinja is DeepSeek V3.2-Exp jinja chat template. You can download it here.

Preparing lineage-bench to run:

git clone https://github.com/fairydreaming/lineage-bench
cd lineage-bench/
uv venv
source .venv/bin/activate
uv pip install -r requirements.txt

First Bad Commit

Currently unknown.

Relevant log output

In server logs you can clearly see the difference in the answer lengths.

Logs

For current f4049ad build:

slot print_timing: id 15 | task 49 |                                                                                        
prompt eval time =    2811.56 ms /   993 tokens (    2.83 ms per token,   353.18 tokens per second)                         
       eval time =  198745.50 ms /   926 tokens (  214.63 ms per token,     4.66 tokens per second)   
      total time =  201557.06 ms /  1919 tokens                                                                             
slot      release: id 15 | task 49 | stop processing: n_tokens = 1918, truncated = 0                                        
srv  log_server_r: done request: POST /v1/chat/completions 127.0.0.1 200                                                    
slot print_timing: id  0 | task 35 |                                                                                        
prompt eval time =    3322.11 ms /   980 tokens (    3.39 ms per token,   294.99 tokens per second)                         
       eval time =  336167.08 ms /  1477 tokens (  227.60 ms per token,     4.39 tokens per second)                         
      total time =  339489.18 ms /  2457 tokens                                                                             
slot      release: id  0 | task 35 | stop processing: n_tokens = 2465, truncated = 0                                        
srv  log_server_r: done request: POST /v1/chat/completions 127.0.0.1 200                                                    
slot print_timing: id 13 | task 48 |                                                                                        
prompt eval time =    3500.24 ms /   987 tokens (    3.55 ms per token,   281.98 tokens per second)                         
       eval time =  334157.77 ms /  1548 tokens (  215.86 ms per token,     4.63 tokens per second)                         
      total time =  337658.01 ms /  2535 tokens                                                                             
slot      release: id 13 | task 48 | stop processing: n_tokens = 2534, truncated = 0                                        
srv  log_server_r: done request: POST /v1/chat/completions 127.0.0.1 200                                                    
slot print_timing: id  8 | task 43 |                                                                                        
prompt eval time =    7130.09 ms /   989 tokens (    7.21 ms per token,   138.71 tokens per second)                         
       eval time =  415767.97 ms /  1904 tokens (  218.37 ms per token,     4.58 tokens per second)                         
      total time =  422898.05 ms /  2893 tokens                                                                             
slot      release: id  8 | task 43 | stop processing: n_tokens = 2892, truncated = 0                                        
srv  log_server_r: done request: POST /v1/chat/completions 127.0.0.1 200                                                    
slot print_timing: id  1 | task 34 |                                   
prompt eval time =    3322.80 ms /   978 tokens (    3.40 ms per token,   294.33 tokens per second)                         
       eval time =  443910.73 ms /  1961 tokens (  226.37 ms per token,     4.42 tokens per second)                         
      total time =  447233.53 ms /  2939 tokens                                                                             
slot      release: id  1 | task 34 | stop processing: n_tokens = 2947, truncated = 0                                        
srv  log_server_r: done request: POST /v1/chat/completions 127.0.0.1 200                                                    
slot print_timing: id  3 | task 37 |                                                                                        
prompt eval time =    3492.31 ms /   987 tokens (    3.54 ms per token,   282.62 tokens per second)                         
       eval time =  481360.73 ms /  2123 tokens (  226.74 ms per token,     4.41 tokens per second)                         
      total time =  484853.04 ms /  3110 tokens                                                                             
slot      release: id  3 | task 37 | stop processing: n_tokens = 3109, truncated = 0                                        
srv  log_server_r: done request: POST /v1/chat/completions 127.0.0.1 200                                                    
slot print_timing: id  7 | task 42 |                                                                                        
prompt eval time =    3538.98 ms /   987 tokens (    3.59 ms per token,   278.89 tokens per second)                         
       eval time =  490591.99 ms /  2186 tokens (  224.42 ms per token,     4.46 tokens per second)                         
      total time =  494130.97 ms /  3173 tokens                                                                             
slot      release: id  7 | task 42 | stop processing: n_tokens = 3172, truncated = 0                                        
srv  log_server_r: done request: POST /v1/chat/completions 127.0.0.1 200                                                    
slot print_timing: id 11 | task 41 |                                                                                        
prompt eval time =    3593.35 ms /     1 tokens ( 3593.35 ms per token,     0.28 tokens per second)                         
       eval time =  498018.74 ms /  2231 tokens (  223.23 ms per token,     4.48 tokens per second)                         
      total time =  501612.10 ms /  2232 tokens                                                                             
slot      release: id 11 | task 41 | stop processing: n_tokens = 3217, truncated = 0                                        
srv  log_server_r: done request: POST /v1/chat/completions 127.0.0.1 200                                                    
slot print_timing: id  5 | task 38 |                                                                                        
prompt eval time =    3815.82 ms /   991 tokens (    3.85 ms per token,   259.71 tokens per second)                         
       eval time =  554314.44 ms /  2430 tokens (  228.11 ms per token,     4.38 tokens per second)                         
      total time =  558130.27 ms /  3421 tokens                                                                             
slot      release: id  5 | task 38 | stop processing: n_tokens = 3420, truncated = 0                                        
srv  log_server_r: done request: POST /v1/chat/completions 127.0.0.1 200                                                    
slot print_timing: id 14 | task 47 |                                                                                        
prompt eval time =    6310.91 ms /   987 tokens (    6.39 ms per token,   156.40 tokens per second)                         
       eval time =  577346.75 ms /  2566 tokens (  225.00 ms per token,     4.44 tokens per second)                         
      total time =  583657.67 ms /  3553 tokens                                                                             
slot      release: id 14 | task 47 | stop processing: n_tokens = 3552, truncated = 0                                        
srv  log_server_r: done request: POST /v1/chat/completions 127.0.0.1 200                                                    
slot print_timing: id  4 | task 39 |                                                                                        
prompt eval time =    7307.28 ms /   989 tokens (    7.39 ms per token,   135.34 tokens per second)                         
       eval time =  595852.01 ms /  2593 tokens (  229.79 ms per token,     4.35 tokens per second)                         
      total time =  603159.28 ms /  3582 tokens                                                                             
slot      release: id  4 | task 39 | stop processing: n_tokens = 3581, truncated = 0                                        
srv  log_server_r: done request: POST /v1/chat/completions 127.0.0.1 200                                                    
slot print_timing: id 10 | task 45 |                                                                                        
prompt eval time =    3592.68 ms /     1 tokens ( 3592.68 ms per token,     0.28 tokens per second)                         
       eval time =  635447.41 ms /  2850 tokens (  222.96 ms per token,     4.49 tokens per second)                         
      total time =  639040.09 ms /  2851 tokens                                                                             
slot      release: id 10 | task 45 | stop processing: n_tokens = 3836, truncated = 0                                        
srv  log_server_r: done request: POST /v1/chat/completions 127.0.0.1 200                                                    
slot print_timing: id 12 | task 46 |                                                                                        
prompt eval time =    7092.62 ms /   976 tokens (    7.27 ms per token,   137.61 tokens per second)                         
       eval time =  701795.71 ms /  3272 tokens (  214.49 ms per token,     4.66 tokens per second)                         
      total time =  708888.33 ms /  4248 tokens                                                                             
slot      release: id 12 | task 46 | stop processing: n_tokens = 4256, truncated = 0                                        
srv  log_server_r: done request: POST /v1/chat/completions 127.0.0.1 200                                                    
slot print_timing: id  2 | task 36 |                                                                                        
prompt eval time =    6814.07 ms /   976 tokens (    6.98 ms per token,   143.23 tokens per second)                         
       eval time =  751702.88 ms /  3559 tokens (  211.21 ms per token,     4.73 tokens per second)                         
      total time =  758516.95 ms /  4535 tokens                                                                             
slot      release: id  2 | task 36 | stop processing: n_tokens = 4543, truncated = 0                                        
srv  log_server_r: done request: POST /v1/chat/completions 127.0.0.1 200                                                    
slot print_timing: id  6 | task 40 |                                                                                        
prompt eval time =    7353.80 ms /   987 tokens (    7.45 ms per token,   134.22 tokens per second)                         
       eval time =  864604.08 ms /  4972 tokens (  173.89 ms per token,     5.75 tokens per second)                         
      total time =  871957.88 ms /  5959 tokens                                                                             
slot      release: id  6 | task 40 | stop processing: n_tokens = 5958, truncated = 0                                        
srv  log_server_r: done request: POST /v1/chat/completions 127.0.0.1 200                                                    
slot print_timing: id  9 | task 44 |                                                                                        
prompt eval time =    3591.98 ms /   985 tokens (    3.65 ms per token,   274.22 tokens per second)                         
       eval time = 1040439.82 ms /  9472 tokens (  109.84 ms per token,     9.10 tokens per second)                         
      total time = 1044031.80 ms / 10457 tokens                                                                             
slot      release: id  9 | task 44 | stop processing: n_tokens = 10456, truncated = 0                                       
srv  update_slots: all slots are idle                                                                                       

For old 68b4d51 build:

slot print_timing: id 14 | task 2 |                                                                                         
prompt eval time =   28510.08 ms /   989 tokens (   28.83 ms per token,    34.69 tokens per second)                         
       eval time = 1423441.13 ms /  5882 tokens (  242.00 ms per token,     4.13 tokens per second)                         
      total time = 1451951.21 ms /  6871 tokens                                                                             
slot      release: id 14 | task 2 | stop processing: n_tokens = 6870, truncated = 0                                         
srv  log_server_r: request: POST /v1/chat/completions 127.0.0.1 200                                                         
srv  log_server_r: request: GET /slots 127.0.0.1 200                                                                        
slot print_timing: id  3 | task 13 |                                                                                        
prompt eval time =    3528.74 ms /   987 tokens (    3.58 ms per token,   279.70 tokens per second)                         
       eval time = 1725416.78 ms /  6851 tokens (  251.85 ms per token,     3.97 tokens per second)                         
      total time = 1728945.53 ms /  7838 tokens                                                                             
slot      release: id  3 | task 13 | stop processing: n_tokens = 7837, truncated = 0                                        
srv  log_server_r: request: POST /v1/chat/completions 127.0.0.1 200                                                         
slot print_timing: id 10 | task 6 |                                                                                         
prompt eval time =    7451.69 ms /   993 tokens (    7.50 ms per token,   133.26 tokens per second)                         
       eval time = 1909086.61 ms /  7425 tokens (  257.12 ms per token,     3.89 tokens per second)                         
      total time = 1916538.30 ms /  8418 tokens                                                                             
slot      release: id 10 | task 6 | stop processing: n_tokens = 8417, truncated = 0                                         
srv  log_server_r: request: POST /v1/chat/completions 127.0.0.1 200                                                         
srv  log_server_r: request: GET /slots 127.0.0.1 200                                                                        
slot print_timing: id 12 | task 3 |                                                                                         
prompt eval time =    3615.76 ms /   989 tokens (    3.66 ms per token,   273.53 tokens per second)                         
       eval time = 2454961.73 ms /  8802 tokens (  278.91 ms per token,     3.59 tokens per second)                         
      total time = 2458577.48 ms /  9791 tokens                                                                             
slot      release: id 12 | task 3 | stop processing: n_tokens = 9790, truncated = 0                                         
srv  log_server_r: request: POST /v1/chat/completions 127.0.0.1 200                                                         
slot print_timing: id 11 | task 5 |                                                                                         
prompt eval time =    3714.34 ms /   987 tokens (    3.76 ms per token,   265.73 tokens per second)                         
       eval time = 2588703.53 ms /  9201 tokens (  281.35 ms per token,     3.55 tokens per second)                         
      total time = 2592417.86 ms / 10188 tokens                                                                             
slot      release: id 11 | task 5 | stop processing: n_tokens = 10187, truncated = 0                                        
srv  log_server_r: request: POST /v1/chat/completions 127.0.0.1 200                                                         
slot print_timing: id  0 | task 16 |                                                                                        
prompt eval time =    3441.55 ms /   985 tokens (    3.49 ms per token,   286.21 tokens per second)                         
       eval time = 2641725.42 ms /  9307 tokens (  283.84 ms per token,     3.52 tokens per second)                         
      total time = 2645166.97 ms / 10292 tokens                                                                             
slot      release: id  0 | task 16 | stop processing: n_tokens = 10291, truncated = 0                                       
srv  log_server_r: request: POST /v1/chat/completions 127.0.0.1 200                                                         
slot print_timing: id 15 | task 1 |                                                                                         
prompt eval time =    2762.68 ms /   987 tokens (    2.80 ms per token,   357.26 tokens per second)                         
       eval time = 2636130.50 ms /  9350 tokens (  281.94 ms per token,     3.55 tokens per second)                         
      total time = 2638893.18 ms / 10337 tokens                                                                             
slot      release: id 15 | task 1 | stop processing: n_tokens = 10336, truncated = 0                                        
srv  log_server_r: request: POST /v1/chat/completions 127.0.0.1 200                                                         
slot print_timing: id 13 | task 0 |                                                                                         
prompt eval time =    3615.53 ms /   987 tokens (    3.66 ms per token,   272.99 tokens per second)                         
       eval time = 2718748.26 ms /  9562 tokens (  284.33 ms per token,     3.52 tokens per second)                         
      total time = 2722363.79 ms / 10549 tokens                                                                             
slot      release: id 13 | task 0 | stop processing: n_tokens = 10548, truncated = 0                                        
srv  log_server_r: request: POST /v1/chat/completions 127.0.0.1 200                                                         
slot print_timing: id  5 | task 11 |                                                                                        
prompt eval time =    4004.02 ms /   987 tokens (    4.06 ms per token,   246.50 tokens per second)                         
       eval time = 2730599.18 ms /  9658 tokens (  282.73 ms per token,     3.54 tokens per second)                         
      total time = 2734603.20 ms / 10645 tokens                                                                             
slot      release: id  5 | task 11 | stop processing: n_tokens = 10644, truncated = 0                                       
srv  log_server_r: request: POST /v1/chat/completions 127.0.0.1 200                                                         
slot print_timing: id  6 | task 10 |                                                                                        
prompt eval time =    7683.90 ms /   989 tokens (    7.77 ms per token,   128.71 tokens per second)                         
       eval time = 2842339.82 ms / 10126 tokens (  280.70 ms per token,     3.56 tokens per second)                         
      total time = 2850023.72 ms / 11115 tokens                                                                             
slot      release: id  6 | task 10 | stop processing: n_tokens = 11114, truncated = 0                                       
srv  log_server_r: request: POST /v1/chat/completions 127.0.0.1 200                                                         
slot print_timing: id  1 | task 15 | 
prompt eval time =    3441.30 ms /   985 tokens (    3.49 ms per token,   286.23 tokens per second)
       eval time = 3221104.52 ms / 11743 tokens (  274.30 ms per token,     3.65 tokens per second)
      total time = 3224545.81 ms / 12728 tokens
slot      release: id  1 | task 15 | stop processing: n_tokens = 12727, truncated = 0
srv  log_server_r: request: POST /v1/chat/completions 127.0.0.1 200
slot print_timing: id  7 | task 9 | 
prompt eval time =    3675.16 ms /   991 tokens (    3.71 ms per token,   269.65 tokens per second)
       eval time = 3279430.62 ms / 12100 tokens (  271.03 ms per token,     3.69 tokens per second)
      total time = 3283105.78 ms / 13091 tokens
slot      release: id  7 | task 9 | stop processing: n_tokens = 13090, truncated = 0
srv  log_server_r: request: POST /v1/chat/completions 127.0.0.1 200
slot print_timing: id  4 | task 12 | 
prompt eval time =    7536.58 ms /   987 tokens (    7.64 ms per token,   130.96 tokens per second)
       eval time = 3297549.86 ms / 12184 tokens (  270.65 ms per token,     3.69 tokens per second)
      total time = 3305086.44 ms / 13171 tokens
slot      release: id  4 | task 12 | stop processing: n_tokens = 13170, truncated = 0
srv  log_server_r: request: POST /v1/chat/completions 127.0.0.1 200
slot print_timing: id  8 | task 8 | 
prompt eval time =    7414.78 ms /   985 tokens (    7.53 ms per token,   132.84 tokens per second)
       eval time = 3393097.40 ms / 12989 tokens (  261.23 ms per token,     3.83 tokens per second)
      total time = 3400512.19 ms / 13974 tokens
slot      release: id  8 | task 8 | stop processing: n_tokens = 13973, truncated = 0
srv  log_server_r: request: POST /v1/chat/completions 127.0.0.1 200
slot print_timing: id  2 | task 14 | 
prompt eval time =    6974.60 ms /   987 tokens (    7.07 ms per token,   141.51 tokens per second)
       eval time = 3414844.27 ms / 13101 tokens (  260.66 ms per token,     3.84 tokens per second)
      total time = 3421818.86 ms / 14088 tokens
slot      release: id  2 | task 14 | stop processing: n_tokens = 14087, truncated = 0
srv  log_server_r: request: POST /v1/chat/completions 127.0.0.1 200
slot print_timing: id  9 | task 7 | 
prompt eval time =    3731.39 ms /   987 tokens (    3.78 ms per token,   264.51 tokens per second)
       eval time = 3615583.87 ms / 17668 tokens (  204.64 ms per token,     4.89 tokens per second)
      total time = 3619315.26 ms / 18655 tokens
slot      release: id  9 | task 7 | stop processing: n_tokens = 18654, truncated = 0
srv  update_slots: all slots are idle
srv  log_server_r: request: POST /v1/chat/completions 127.0.0.1 200

As you can observe with the old llama.cpp build the model answers are much longer.

Overall the model responses with the current build are coherent, but have subtly reasoning errors. Full responses of the model for both llama.cpp builds attached.

logs_ds_regression.zip

[CISC]

Maybe start with ruling out direct changes to MLA: #18986 and #19067

[fairydreaming]

Maybe start with ruling out direct changes to MLA: #18986 and #19067

I did some more tests, with d9c6ce4 all seemed OK (accuracy 1.0), then I tested 752584d and it started showing some issues (2 infinite generation loops) but they were different from f4049ad (short and invalid answers). I guess more conversion of $ to heat is needed to pinpoint this.

Update: after checking them in detail they were not infinite generation loops, I suppose 32k tokens weren't enough for DeepSeek V3.2 to answer.

[CISC]

So it would seem it is a regression outside of the model code itself, try to disable fusion (GGML_CUDA_DISABLE_FUSION envvar).

[fairydreaming]

I'm an idiot - there's no reasoning in responses. It must be template-related, will look into this tomorrow.

[CISC]

I'm an idiot - there's no reasoning in responses. It must be template-related, will look into this tomorrow.

{% if not thinking is defined %}{% set thinking = false %}{% endif %}

Are you sending the right query to set thinking to true?

[fairydreaming]

@CISC What's the right query/request? I had openrouter-style "reasoning": {"enabled": true} in requests, but I don't think llama-server uses that.

I found that adding --chat-template-kwargs '{"thinking": true}' to server params enabled back reasoning (I see <think> at the end of the request and before final answer), but reasoning_content is still missing in responses even if I explicitly set --reasoning-format deepseek.

May be related to #20008

[CISC]

See OpenAI-compatible Chat Completions API you can set chat_template_kwargs per request.

Not sure about reasoning, @pwilkin?

[fairydreaming]

So I basically wasted several days and a good deal of money to investigate subtle differences in tensor values while the model wasn't even reasoning, awesome. 😭

[ggerganov]

For similar reasons, my evaluation script dumps a detailed HTML report with all the reasoning traces from the benches.

[pwilkin]

Let me merge #20424 since that might be related and then we can diagnose further.

reasoning is not a special kwarg as of now, I intend to work on detecting the "reasoning variable" in various templates to map enable_thinking correctly for cases like this (but this also means you should be able to pass it safely).

[createthis]

@pwilkin I think this is still broken. DS 3.2 does not reason by default like 3.1-Terminus and when it is given --chat-template-kwargs '{"thinking": true}' it does not appear to be outputting reasoning as reasoning_content:

Startup commands I'm using:

Details

./build/bin/llama-server \
    --model  /data2/DeepSeek-V3.2-light-GGUF/Q4_K_M/DeepSeek-V3.2-light-Q4_K_M-00001-of-00031.gguf \
    --alias DeepSeek-V3.2-light-GGUF:671b-q4_k_m \
    --numa numactl \
    --threads 32 \
    --ctx-size 10840 \
    --n-gpu-layers 62 \
    -ot "blk\.(3|4|5|6|7|8|9|10)\.ffn_.*=CUDA0" \
    -ot exps=CPU \
    -ub 4096 -b 4096 \
    --seed 3407 \
    --temp 0.6 \
    --top-p 0.95 \
    --min-p 0.1 \
    --log-colors on \
    --flash-attn on \
    --host 0.0.0.0 \
    --prio 2 \
    --jinja \
    --verbose \
    --port 11434 \
    --chat-template-file /data2/DeepSeek-V3.2-Exp/chat_template.json \
    --chat-template-kwargs '{"thinking": true}'

I've tried with the 3.1-Terminus chat template as well with the same result.

[elvizlai]

I test it with deepseek-v3.2-exp/assets/xxx.jinja template, it not worked.