Misc. bug: Gemma 3 spacing and empty line bug in llama-cli

[arch-btw]

Name and Version

llama.cpp latest commit: b7555

I'm not completely sure if this is an issue with the model itself but ever since Gemma 3 was implemented, when I run those models in llama-cli they engage in excessive spacing and empty lines. I was wondering if anyone can assist me in how to debug this or would be so kind to look into it.

Just to be clear, it's not that the spacing itself bothers me but it makes me wonder if the inference quality might be affected.

Here's an example for the prompt "hello" using llama-cli:

Hello there! How can I help you today? 😊 
(empty line 1)
(empty line 2)
(empty line 3)
Let me know what's on your mind, or if you just want to chat!

In verbose it does mention \n\n\n\n:

Parsed message: {"role":"assistant","content":"Hello there! How can I help you today? 😊 \n\n\n\nLet me know what's on your mind, or if you just want to chat!"}

But here's where it gets confusing, in llama-server it always gets parsed normally:

Furthermore it's not just empty lines, but it's spacing too for example when writing a story:

unique adaptations that set them apart. Think of them as

Notice there are 2 spaces between the period and the word Think. However, one sentence later it's normal spacing again:

Birds don't have teeth! They have beaks

Operating systems

Linux

Which llama.cpp modules do you know to be affected?

llama-cli

Command line

llama-cli -m gemma-3-12b-it-Q4_K_S.gguf -cnv --jinja -fa on -ngl 99 --no-mmap

Problem description & steps to reproduce

1. Run gemma-3-12b-it or the qat version
2. Ask a question (might have to try a few times)
3. Notice the excessive (sometimes triple) spacing and empty lines (also triple sometimes)

First Bad Commit

I ran the gemma model shortly after release and it was already an issue back then, so it's likely this one:
#12343

Relevant log output

Click to expand verbose log

▄▄ ▄▄
██ ██
██ ██  ▀▀█▄ ███▄███▄  ▀▀█▄    ▄████ ████▄ ████▄
██ ██ ▄█▀██ ██ ██ ██ ▄█▀██    ██    ██ ██ ██ ██
██ ██ ▀█▄██ ██ ██ ██ ▀█▄██ ██ ▀████ ████▀ ████▀
                                    ██    ██
                                    ▀▀    ▀▀

build      : b7540-85c40c9b0
model      : google_gemma-3-12b-it-qat-Q4_0.gguf
modalities : text

available commands:
  /exit or Ctrl+C     stop or exit
  /regen              regenerate the last response
  /clear              clear the chat history
  /read               add a text file


> hello

res  add_waiting_: add task 0 to waiting list. current waiting = 0 (before add)
que          post: new task, id = 0, front = 0
|slot get_availabl: id  0 | task -1 | selected slot by LRU, t_last = -1
slot        reset: id  0 | task -1 |
slot launch_slot_: id  0 | task -1 | launching slot : {"id":0,"n_ctx":131072,"speculative":false,"is_processing":false}
slot launch_slot_: id  0 | task -1 | sampler chain: logits -> penalties -> dry -> top-n-sigma -> top-k -> typical -> top-p -> min-p -> xtc -> temp-ext -> dist
slot launch_slot_: id  0 | task 0 | processing task
que    start_loop: update slots
srv  update_slots: posting NEXT_RESPONSE
que          post: new task, id = 1, front = 0
slot update_slots: id  0 | task 0 | new prompt, n_ctx_slot = 131072, n_keep = 0, task.n_tokens = 10
slot update_slots: id  0 | task 0 | n_tokens = 0, memory_seq_rm [0, end)
slot update_slots: id  0 | task 0 | prompt processing progress, n_tokens = 10, batch.n_tokens = 10, progress = 1.000000
slot update_slots: id  0 | task 0 | prompt done, n_tokens = 10, batch.n_tokens = 10
srv  update_slots: decoding batch, n_tokens = 10
clear_adapter_lora: call
set_embeddings: value = 0
|res          send: sending result for task id = 0
res          send: task id = 0 pushed to result queue
slot process_toke: id  0 | task 0 | n_decoded = 1, n_remaining = -1, next token:  9259 'Hello'
srv  update_slots: run slots completed
que    start_loop: waiting for new tasks
que    start_loop: processing new tasks
que    start_loop: processing task, id = 1
que    start_loop: update slots
srv  update_slots: posting NEXT_RESPONSE
que          post: new task, id = 2, front = 0
slot update_slots: id  0 | task 0 | slot decode token, n_ctx = 131072, n_tokens = 11, truncated = 0
srv  update_slots: decoding batch, n_tokens = 1
clear_adapter_lora: call
set_embeddings: value = 0
srv  update_chat_: Parsing chat message: Hello
Parsing input with format Content-only: Hello
Hellores          send: sending result for task id = 0
 thereres          send: task id = 0 pushed to result queue
slot process_toke: id  0 | task 0 | n_decoded = 2, n_remaining = -1, next token:   993 ' there'
srv  update_slots: run slots completed
que    start_loop: waiting for new tasks
que    start_loop: processing new tasks
que    start_loop: processing task, id = 2
que    start_loop: update slots
srv  update_slots: posting NEXT_RESPONSE
que          post: new task, id = 3, front = 0
srv  update_chat_: Parsing chat message: Hello there
Parsing input with format Content-only: Hello there
slot update_slots: id  0 | task 0 | slot decode token, n_ctx = 131072, n_tokens = 12, truncated = 0
srv  update_slots: decoding batch, n_tokens = 1
clear_adapter_lora: call
set_embeddings: value = 0
res          send: sending result for task id = 0
res          send: task id = 0 pushed to result queue
slot process_toke: id  0 | task 0 | n_decoded = 3, n_remaining = -1, next token: 236888 '!'
!srv  update_slots: run slots completed
que    start_loop: waiting for new tasks
que    start_loop: processing new tasks
que    start_loop: processing task, id = 3
que    start_loop: update slots
srv  update_slots: posting NEXT_RESPONSE
que          post: new task, id = 4, front = 0
slot update_slots: id  0 | task 0 | slot decode token, n_ctx = 131072, n_tokens = 13, truncated = 0
srv  update_slots: decoding batch, n_tokens = 1
clear_adapter_lora: call
set_embeddings: value = 0
srv  update_chat_: Parsing chat message: Hello there!
Parsing input with format Content-only: Hello there!
res          send: sending result for task id = 0
res          send: task id = 0 pushed to result queue
slot process_toke: id  0 | task 0 | n_decoded = 4, n_remaining = -1, next token:  2088 ' How'
 Howsrv  update_slots: run slots completed
que    start_loop: waiting for new tasks
que    start_loop: processing new tasks
que    start_loop: processing task, id = 4
que    start_loop: update slots
srv  update_slots: posting NEXT_RESPONSE
que          post: new task, id = 5, front = 0
slot update_slots: id  0 | task 0 | slot decode token, n_ctx = 131072, n_tokens = 14, truncated = 0
srv  update_slots: decoding batch, n_tokens = 1
clear_adapter_lora: call
set_embeddings: value = 0
srv  update_chat_: Parsing chat message: Hello there! How
Parsing input with format Content-only: Hello there! How
res          send: sending result for task id = 0
res          send: task id = 0 pushed to result queue
slot process_toke: id  0 | task 0 | n_decoded = 5, n_remaining = -1, next token:   740 ' can'
srv  update_slots: run slots completed
que    start_loop: waiting for new tasks
que    start_loop: processing new tasks
que    start_loop: processing task, id = 5
que    start_loop: update slots
 cansrv  update_slots: posting NEXT_RESPONSE
que          post: new task, id = 6, front = 0
slot update_slots: id  0 | task 0 | slot decode token, n_ctx = 131072, n_tokens = 15, truncated = 0
srv  update_slots: decoding batch, n_tokens = 1
clear_adapter_lora: call
set_embeddings: value = 0
srv  update_chat_: Parsing chat message: Hello there! How can
Parsing input with format Content-only: Hello there! How can
res          send: sending result for task id = 0
res          send: task id = 0 pushed to result queue
slot process_toke: id  0 | task 0 | n_decoded = 6, n_remaining = -1, next token:   564 ' I'
 Isrv  update_slots: run slots completed
que    start_loop: waiting for new tasks
que    start_loop: processing new tasks
que    start_loop: processing task, id = 6
que    start_loop: update slots
srv  update_slots: posting NEXT_RESPONSE
que          post: new task, id = 7, front = 0
slot update_slots: id  0 | task 0 | slot decode token, n_ctx = 131072, n_tokens = 16, truncated = 0
srv  update_slots: decoding batch, n_tokens = 1
clear_adapter_lora: call
set_embeddings: value = 0
srv  update_chat_: Parsing chat message: Hello there! How can I
Parsing input with format Content-only: Hello there! How can I
res          send: sending result for task id = 0
res          send: task id = 0 pushed to result queue
slot process_toke: id  0 | task 0 | n_decoded = 7, n_remaining = -1, next token:  1601 ' help'
 helpsrv  update_slots: run slots completed
que    start_loop: waiting for new tasks
que    start_loop: processing new tasks
que    start_loop: processing task, id = 7
que    start_loop: update slots
srv  update_slots: posting NEXT_RESPONSE
que          post: new task, id = 8, front = 0
slot update_slots: id  0 | task 0 | slot decode token, n_ctx = 131072, n_tokens = 17, truncated = 0
srv  update_slots: decoding batch, n_tokens = 1
clear_adapter_lora: call
set_embeddings: value = 0
srv  update_chat_: Parsing chat message: Hello there! How can I help
Parsing input with format Content-only: Hello there! How can I help
res          send: sending result for task id = 0
res          send: task id = 0 pushed to result queue
slot process_toke: id  0 | task 0 | n_decoded = 8, n_remaining = -1, next token:   611 ' you'
 yousrv  update_slots: run slots completed
que    start_loop: waiting for new tasks
que    start_loop: processing new tasks
que    start_loop: processing task, id = 8
que    start_loop: update slots
srv  update_slots: posting NEXT_RESPONSE
que          post: new task, id = 9, front = 0
slot update_slots: id  0 | task 0 | slot decode token, n_ctx = 131072, n_tokens = 18, truncated = 0
srv  update_slots: decoding batch, n_tokens = 1
clear_adapter_lora: call
set_embeddings: value = 0
srv  update_chat_: Parsing chat message: Hello there! How can I help you
Parsing input with format Content-only: Hello there! How can I help you
res          send: sending result for task id = 0
res          send: task id = 0 pushed to result queue
slot process_toke: id  0 | task 0 | n_decoded = 9, n_remaining = -1, next token:  3124 ' today'
 todaysrv  update_slots: run slots completed
que    start_loop: waiting for new tasks
que    start_loop: processing new tasks
que    start_loop: processing task, id = 9
que    start_loop: update slots
srv  update_slots: posting NEXT_RESPONSE
que          post: new task, id = 10, front = 0
slot update_slots: id  0 | task 0 | slot decode token, n_ctx = 131072, n_tokens = 19, truncated = 0
srv  update_slots: decoding batch, n_tokens = 1
clear_adapter_lora: call
set_embeddings: value = 0
srv  update_chat_: Parsing chat message: Hello there! How can I help you today
Parsing input with format Content-only: Hello there! How can I help you today
res          send: sending result for task id = 0
res          send: task id = 0 pushed to result queue
?slot process_toke: id  0 | task 0 | n_decoded = 10, n_remaining = -1, next token: 236881 '?'
srv  update_slots: run slots completed
que    start_loop: waiting for new tasks
que    start_loop: processing new tasks
que    start_loop: processing task, id = 10
que    start_loop: update slots
srv  update_slots: posting NEXT_RESPONSE
que          post: new task, id = 11, front = 0
slot update_slots: id  0 | task 0 | slot decode token, n_ctx = 131072, n_tokens = 20, truncated = 0
srv  update_slots: decoding batch, n_tokens = 1
clear_adapter_lora: call
set_embeddings: value = 0
srv  update_chat_: Parsing chat message: Hello there! How can I help you today?
Parsing input with format Content-only: Hello there! How can I help you today?
res          send: sending result for task id = 0
 😊res          send: task id = 0 pushed to result queue
slot process_toke: id  0 | task 0 | n_decoded = 11, n_remaining = -1, next token: 103453 ' 😊'
srv  update_slots: run slots completed
que    start_loop: waiting for new tasks
que    start_loop: processing new tasks
que    start_loop: processing task, id = 11
que    start_loop: update slots
srv  update_slots: posting NEXT_RESPONSE
que          post: new task, id = 12, front = 0
slot update_slots: id  0 | task 0 | slot decode token, n_ctx = 131072, n_tokens = 21, truncated = 0
srv  update_slots: decoding batch, n_tokens = 1
clear_adapter_lora: call
set_embeddings: value = 0
srv  update_chat_: Parsing chat message: Hello there! How can I help you today? 😊
Parsing input with format Content-only: Hello there! How can I help you today? 😊
res          send: sending result for task id = 0
res          send: task id = 0 pushed to result queue
slot process_toke: id  0 | task 0 | n_decoded = 12, n_remaining = -1, next token: 236743 ' '
 srv  update_slots: run slots completed
que    start_loop: waiting for new tasks
que    start_loop: processing new tasks
que    start_loop: processing task, id = 12
que    start_loop: update slots
srv  update_slots: posting NEXT_RESPONSE
que          post: new task, id = 13, front = 0
slot update_slots: id  0 | task 0 | slot decode token, n_ctx = 131072, n_tokens = 22, truncated = 0
srv  update_slots: decoding batch, n_tokens = 1
clear_adapter_lora: call
set_embeddings: value = 0
srv  update_chat_: Parsing chat message: Hello there! How can I help you today? 😊
Parsing input with format Content-only: Hello there! How can I help you today? 😊
res          send: sending result for task id = 0
res          send: task id = 0 pushed to result queue
slot process_toke: id  0 | task 0 | n_decoded = 13, n_remaining = -1, next token:   110 '



'




srv  update_slots: run slots completed
que    start_loop: waiting for new tasks
que    start_loop: processing new tasks
que    start_loop: processing task, id = 13
que    start_loop: update slots
srv  update_slots: posting NEXT_RESPONSE
que          post: new task, id = 14, front = 0
srv  update_chat_: Parsing chat message: Hello there! How can I help you today? 😊




slot update_slots: id  0 | task 0 | slot decode token, n_ctx = 131072, n_tokens = 23, truncated = 0
srv  update_slots: decoding batch, n_tokens = 1
Parsing input with format Content-only: Hello there! How can I help you today? 😊




clear_adapter_lora: call
set_embeddings: value = 0
Letres          send: sending result for task id = 0
res          send: task id = 0 pushed to result queue
slot process_toke: id  0 | task 0 | n_decoded = 14, n_remaining = -1, next token:  6481 'Let'
srv  update_slots: run slots completed
que    start_loop: waiting for new tasks
que    start_loop: processing new tasks
que    start_loop: processing task, id = 14
que    start_loop: update slots
srv  update_slots: posting NEXT_RESPONSE
que          post: new task, id = 15, front = 0
slot update_slots: id  0 | task 0 | slot decode token, n_ctx = 131072, n_tokens = 24, truncated = 0
srv  update_slots: decoding batch, n_tokens = 1
clear_adapter_lora: call
set_embeddings: value = 0
srv  update_chat_: Parsing chat message: Hello there! How can I help you today? 😊



Let
Parsing input with format Content-only: Hello there! How can I help you today? 😊



Let
res          send: sending result for task id = 0
res          send: task id = 0 pushed to result queue
slot process_toke: id  0 | task 0 | n_decoded = 15, n_remaining = -1, next token:   786 ' me'
 mesrv  update_slots: run slots completed
que    start_loop: waiting for new tasks
que    start_loop: processing new tasks
que    start_loop: processing task, id = 15
que    start_loop: update slots
srv  update_slots: posting NEXT_RESPONSE
que          post: new task, id = 16, front = 0
slot update_slots: id  0 | task 0 | slot decode token, n_ctx = 131072, n_tokens = 25, truncated = 0
srv  update_slots: decoding batch, n_tokens = 1
clear_adapter_lora: call
set_embeddings: value = 0
srv  update_chat_: Parsing chat message: Hello there! How can I help you today? 😊



Let me
Parsing input with format Content-only: Hello there! How can I help you today? 😊



Let me
res          send: sending result for task id = 0
res          send: task id = 0 pushed to result queue
slot process_toke: id  0 | task 0 | n_decoded = 16, n_remaining = -1, next token:  1281 ' know'
 knowsrv  update_slots: run slots completed
que    start_loop: waiting for new tasks
que    start_loop: processing new tasks
que    start_loop: processing task, id = 16
que    start_loop: update slots
srv  update_slots: posting NEXT_RESPONSE
que          post: new task, id = 17, front = 0
slot update_slots: id  0 | task 0 | slot decode token, n_ctx = 131072, n_tokens = 26, truncated = 0
srv  update_slots: decoding batch, n_tokens = 1
clear_adapter_lora: call
set_embeddings: value = 0
srv  update_chat_: Parsing chat message: Hello there! How can I help you today? 😊



Let me know
Parsing input with format Content-only: Hello there! How can I help you today? 😊



Let me know
res          send: sending result for task id = 0
res          send: task id = 0 pushed to result queue
 whatslot process_toke: id  0 | task 0 | n_decoded = 17, n_remaining = -1, next token:  1144 ' what'
srv  update_slots: run slots completed
que    start_loop: waiting for new tasks
que    start_loop: processing new tasks
que    start_loop: processing task, id = 17
que    start_loop: update slots
srv  update_slots: posting NEXT_RESPONSE
que          post: new task, id = 18, front = 0
slot update_slots: id  0 | task 0 | slot decode token, n_ctx = 131072, n_tokens = 27, truncated = 0
srv  update_slots: decoding batch, n_tokens = 1
clear_adapter_lora: call
set_embeddings: value = 0
srv  update_chat_: Parsing chat message: Hello there! How can I help you today? 😊



Let me know what
Parsing input with format Content-only: Hello there! How can I help you today? 😊



Let me know what
res          send: sending result for task id = 0
res          send: task id = 0 pushed to result queue
slot process_toke: id  0 | task 0 | n_decoded = 18, n_remaining = -1, next token: 236789 '''
'srv  update_slots: run slots completed
que    start_loop: waiting for new tasks
que    start_loop: processing new tasks
que    start_loop: processing task, id = 18
que    start_loop: update slots
srv  update_slots: posting NEXT_RESPONSE
que          post: new task, id = 19, front = 0
slot update_slots: id  0 | task 0 | slot decode token, n_ctx = 131072, n_tokens = 28, truncated = 0
srv  update_slots: decoding batch, n_tokens = 1
clear_adapter_lora: call
set_embeddings: value = 0
srv  update_chat_: Parsing chat message: Hello there! How can I help you today? 😊



Let me know what'
Parsing input with format Content-only: Hello there! How can I help you today? 😊



Let me know what'
res          send: sending result for task id = 0
res          send: task id = 0 pushed to result queue
slot process_toke: id  0 | task 0 | n_decoded = 19, n_remaining = -1, next token: 236751 's'
ssrv  update_slots: run slots completed
que    start_loop: waiting for new tasks
que    start_loop: processing new tasks
que    start_loop: processing task, id = 19
que    start_loop: update slots
srv  update_slots: posting NEXT_RESPONSE
que          post: new task, id = 20, front = 0
slot update_slots: id  0 | task 0 | slot decode token, n_ctx = 131072, n_tokens = 29, truncated = 0
srv  update_slots: decoding batch, n_tokens = 1
clear_adapter_lora: call
set_embeddings: value = 0
srv  update_chat_: Parsing chat message: Hello there! How can I help you today? 😊



Let me know what's
Parsing input with format Content-only: Hello there! How can I help you today? 😊



Let me know what's
res          send: sending result for task id = 0
res          send: task id = 0 pushed to result queue
 onslot process_toke: id  0 | task 0 | n_decoded = 20, n_remaining = -1, next token:   580 ' on'
srv  update_slots: run slots completed
que    start_loop: waiting for new tasks
que    start_loop: processing new tasks
que    start_loop: processing task, id = 20
que    start_loop: update slots
srv  update_slots: posting NEXT_RESPONSE
que          post: new task, id = 21, front = 0
slot update_slots: id  0 | task 0 | slot decode token, n_ctx = 131072, n_tokens = 30, truncated = 0
srv  update_slots: decoding batch, n_tokens = 1
clear_adapter_lora: call
set_embeddings: value = 0
srv  update_chat_: Parsing chat message: Hello there! How can I help you today? 😊



Let me know what's on
Parsing input with format Content-only: Hello there! How can I help you today? 😊



Let me know what's on
 yourres          send: sending result for task id = 0
res          send: task id = 0 pushed to result queue
slot process_toke: id  0 | task 0 | n_decoded = 21, n_remaining = -1, next token:   822 ' your'
srv  update_slots: run slots completed
que    start_loop: waiting for new tasks
que    start_loop: processing new tasks
que    start_loop: processing task, id = 21
que    start_loop: update slots
srv  update_slots: posting NEXT_RESPONSE
que          post: new task, id = 22, front = 0
srv  update_chat_: Parsing chat message: Hello there! How can I help you today? 😊



Let me know what's on your
slot update_slots: id  0 | task 0 | slot decode token, n_ctx = 131072, n_tokens = 31, truncated = 0
srv  update_slots: decoding batch, n_tokens = 1
Parsing input with format Content-only: Hello there! How can I help you today? 😊



Let me know what's on your
clear_adapter_lora: call
set_embeddings: value = 0
res          send: sending result for task id = 0
res          send: task id = 0 pushed to result queue
slot process_toke: id  0 | task 0 | n_decoded = 22, n_remaining = -1, next token:  3666 ' mind'
 mindsrv  update_slots: run slots completed
que    start_loop: waiting for new tasks
que    start_loop: processing new tasks
que    start_loop: processing task, id = 22
que    start_loop: update slots
srv  update_slots: posting NEXT_RESPONSE
que          post: new task, id = 23, front = 0
slot update_slots: id  0 | task 0 | slot decode token, n_ctx = 131072, n_tokens = 32, truncated = 0
srv  update_slots: decoding batch, n_tokens = 1
clear_adapter_lora: call
set_embeddings: value = 0
srv  update_chat_: Parsing chat message: Hello there! How can I help you today? 😊



Let me know what's on your mind
Parsing input with format Content-only: Hello there! How can I help you today? 😊



Let me know what's on your mind
res          send: sending result for task id = 0
res          send: task id = 0 pushed to result queue
slot process_toke: id  0 | task 0 | n_decoded = 23, n_remaining = -1, next token: 236764 ','
,srv  update_slots: run slots completed
que    start_loop: waiting for new tasks
que    start_loop: processing new tasks
que    start_loop: processing task, id = 23
que    start_loop: update slots
srv  update_slots: posting NEXT_RESPONSE
que          post: new task, id = 24, front = 0
slot update_slots: id  0 | task 0 | slot decode token, n_ctx = 131072, n_tokens = 33, truncated = 0
srv  update_slots: decoding batch, n_tokens = 1
clear_adapter_lora: call
set_embeddings: value = 0
srv  update_chat_: Parsing chat message: Hello there! How can I help you today? 😊



Let me know what's on your mind,
Parsing input with format Content-only: Hello there! How can I help you today? 😊



Let me know what's on your mind,
res          send: sending result for task id = 0
res          send: task id = 0 pushed to result queue
slot process_toke: id  0 | task 0 | n_decoded = 24, n_remaining = -1, next token:   653 ' or'
 orsrv  update_slots: run slots completed
que    start_loop: waiting for new tasks
que    start_loop: processing new tasks
que    start_loop: processing task, id = 24
que    start_loop: update slots
srv  update_slots: posting NEXT_RESPONSE
que          post: new task, id = 25, front = 0
slot update_slots: id  0 | task 0 | slot decode token, n_ctx = 131072, n_tokens = 34, truncated = 0
srv  update_slots: decoding batch, n_tokens = 1
clear_adapter_lora: call
set_embeddings: value = 0
srv  update_chat_: Parsing chat message: Hello there! How can I help you today? 😊



Let me know what's on your mind, or
Parsing input with format Content-only: Hello there! How can I help you today? 😊



Let me know what's on your mind, or
 ifres          send: sending result for task id = 0
res          send: task id = 0 pushed to result queue
slot process_toke: id  0 | task 0 | n_decoded = 25, n_remaining = -1, next token:   768 ' if'
srv  update_slots: run slots completed
que    start_loop: waiting for new tasks
que    start_loop: processing new tasks
que    start_loop: processing task, id = 25
que    start_loop: update slots
srv  update_slots: posting NEXT_RESPONSE
que          post: new task, id = 26, front = 0
slot update_slots: id  0 | task 0 | slot decode token, n_ctx = 131072, n_tokens = 35, truncated = 0
srv  update_slots: decoding batch, n_tokens = 1
clear_adapter_lora: call
srv  update_chat_: Parsing chat message: Hello there! How can I help you today? 😊



Let me know what's on your mind, or if
Parsing input with format Content-only: Hello there! How can I help you today? 😊



Let me know what's on your mind, or if
set_embeddings: value = 0
 youres          send: sending result for task id = 0
res          send: task id = 0 pushed to result queue
slot process_toke: id  0 | task 0 | n_decoded = 26, n_remaining = -1, next token:   611 ' you'
srv  update_slots: run slots completed
que    start_loop: waiting for new tasks
que    start_loop: processing new tasks
que    start_loop: processing task, id = 26
que    start_loop: update slots
srv  update_slots: posting NEXT_RESPONSE
que          post: new task, id = 27, front = 0
slot update_slots: id  0 | task 0 | slot decode token, n_ctx = 131072, n_tokens = 36, truncated = 0
srv  update_slots: decoding batch, n_tokens = 1
clear_adapter_lora: call
set_embeddings: value = 0
srv  update_chat_: Parsing chat message: Hello there! How can I help you today? 😊



Let me know what's on your mind, or if you
Parsing input with format Content-only: Hello there! How can I help you today? 😊



Let me know what's on your mind, or if you
res          send: sending result for task id = 0
res          send: task id = 0 pushed to result queue
slot process_toke: id  0 | task 0 | n_decoded = 27, n_remaining = -1, next token:  1164 ' just'
 justsrv  update_slots: run slots completed
que    start_loop: waiting for new tasks
que    start_loop: processing new tasks
que    start_loop: processing task, id = 27
que    start_loop: update slots
srv  update_slots: posting NEXT_RESPONSE
que          post: new task, id = 28, front = 0
slot update_slots: id  0 | task 0 | slot decode token, n_ctx = 131072, n_tokens = 37, truncated = 0
srv  update_slots: decoding batch, n_tokens = 1
clear_adapter_lora: call
set_embeddings: value = 0
srv  update_chat_: Parsing chat message: Hello there! How can I help you today? 😊



Let me know what's on your mind, or if you just
Parsing input with format Content-only: Hello there! How can I help you today? 😊



Let me know what's on your mind, or if you just
res          send: sending result for task id = 0
res          send: task id = 0 pushed to result queue
slot process_toke: id  0 | task 0 | n_decoded = 28, n_remaining = -1, next token:  1461 ' want'
 wantsrv  update_slots: run slots completed
que    start_loop: waiting for new tasks
que    start_loop: processing new tasks
que    start_loop: processing task, id = 28
que    start_loop: update slots
srv  update_slots: posting NEXT_RESPONSE
que          post: new task, id = 29, front = 0
slot update_slots: id  0 | task 0 | slot decode token, n_ctx = 131072, n_tokens = 38, truncated = 0
srv  update_slots: decoding batch, n_tokens = 1
clear_adapter_lora: call
set_embeddings: value = 0
srv  update_chat_: Parsing chat message: Hello there! How can I help you today? 😊



Let me know what's on your mind, or if you just want
Parsing input with format Content-only: Hello there! How can I help you today? 😊



Let me know what's on your mind, or if you just want
res          send: sending result for task id = 0
res          send: task id = 0 pushed to result queue
 toslot process_toke: id  0 | task 0 | n_decoded = 29, n_remaining = -1, next token:   531 ' to'
srv  update_slots: run slots completed
que    start_loop: waiting for new tasks
que    start_loop: processing new tasks
que    start_loop: processing task, id = 29
que    start_loop: update slots
srv  update_slots: posting NEXT_RESPONSE
que          post: new task, id = 30, front = 0
slot update_slots: id  0 | task 0 | slot decode token, n_ctx = 131072, n_tokens = 39, truncated = 0
srv  update_slots: decoding batch, n_tokens = 1
srv  update_chat_: Parsing chat message: Hello there! How can I help you today? 😊



Let me know what's on your mind, or if you just want to
Parsing input with format Content-only: Hello there! How can I help you today? 😊



Let me know what's on your mind, or if you just want to
clear_adapter_lora: call
set_embeddings: value = 0
res          send: sending result for task id = 0
res          send: task id = 0 pushed to result queue
 chatslot process_toke: id  0 | task 0 | n_decoded = 30, n_remaining = -1, next token: 11184 ' chat'
srv  update_slots: run slots completed
que    start_loop: waiting for new tasks
que    start_loop: processing new tasks
que    start_loop: processing task, id = 30
que    start_loop: update slots
srv  update_slots: posting NEXT_RESPONSE
que          post: new task, id = 31, front = 0
slot update_slots: id  0 | task 0 | slot decode token, n_ctx = 131072, n_tokens = 40, truncated = 0
srv  update_slots: decoding batch, n_tokens = 1
clear_adapter_lora: call
set_embeddings: value = 0
srv  update_chat_: Parsing chat message: Hello there! How can I help you today? 😊



Let me know what's on your mind, or if you just want to chat
Parsing input with format Content-only: Hello there! How can I help you today? 😊



Let me know what's on your mind, or if you just want to chat
res          send: sending result for task id = 0
res          send: task id = 0 pushed to result queue
slot process_toke: id  0 | task 0 | n_decoded = 31, n_remaining = -1, next token: 236888 '!'
!srv  update_slots: run slots completed
que    start_loop: waiting for new tasks
que    start_loop: processing new tasks
que    start_loop: processing task, id = 31
que    start_loop: update slots
srv  update_slots: posting NEXT_RESPONSE
que          post: new task, id = 32, front = 0
slot update_slots: id  0 | task 0 | slot decode token, n_ctx = 131072, n_tokens = 41, truncated = 0
srv  update_slots: decoding batch, n_tokens = 1
clear_adapter_lora: call
set_embeddings: value = 0
srv  update_chat_: Parsing chat message: Hello there! How can I help you today? 😊



Let me know what's on your mind, or if you just want to chat!
Parsing input with format Content-only: Hello there! How can I help you today? 😊



Let me know what's on your mind, or if you just want to chat!
res          send: sending result for task id = 0
res          send: task id = 0 pushed to result queue
slot process_toke: id  0 | task 0 | stopped by EOS
slot process_toke: id  0 | task 0 | n_decoded = 32, n_remaining = -1, next token:   106 ''

slot print_timing: id  0 | task 0 |
prompt eval time =     127.06 ms /    10 tokens (   12.71 ms per token,    78.70 tokens per second)
       eval time =    3105.55 ms /    32 tokens (   97.05 ms per token,    10.30 tokens per second)
      total time =    3232.60 ms /    42 tokens
res          send: sending result for task id = 0
res          send: task id = 0 pushed to result queue
slot      release: id  0 | task 0 | stop processing: n_tokens = 41, truncated = 0
srv  update_slots: run slots completed
srv  update_chat_: Parsing chat message: Hello there! How can I help you today? 😊



Let me know what's on your mind, or if you just want to chat!
que    start_loop: waiting for new tasks
que    start_loop: processing new tasks
Parsing input with format Content-only: Hello there! How can I help you today? 😊



Let me know what's on your mind, or if you just want to chat!
que    start_loop: processing task, id = 32
que    start_loop: update slots
srv  update_chat_: Parsing chat message: Hello there! How can I help you today? 😊



Let me know what's on your mind, or if you just want to chat!
srv  update_slots: all slots are idle
que    start_loop: waiting for new tasks
Parsing input with format Content-only: Hello there! How can I help you today? 😊



Let me know what's on your mind, or if you just want to chat!
Parsed message: {"role":"assistant","content":"Hello there! How can I help you today? 😊 \n\n\n\nLet me know what's on your mind, or if you just want to chat!"}
res  remove_waiti: remove task 0 from waiting list. current waiting = 1 (before remove)
srv          stop: all tasks already finished, no need to cancel

[github-actions]

This issue was closed because it has been inactive for 14 days since being marked as stale.