Skip to content

Eval bug: prompt is re-processing for no reason #57

@Ezzz-dev

Description

@Ezzz-dev

Name and Version

0.3.2

Operating systems

Linux

GGML backends

Vulkan

Hardware

7900XTX 24GB

Models

Qwen 27B Q5KS

Problem description & steps to reproduce

Just did a chat with Pi agent to get some data.
I asked another thing just 10 seconds later, and boom:

The entire prompt was re-processsed for no reason.

This issue does NOT happen with latest llama.cpp, I have only experienced it with beellama.cpp

First Bad Commit

No response

Relevant log output

4.55.765.651 I slot launch_slot_: id 0 | task 705 | processing task, is_child = 0
4.55.765.664 I slot update_slots: id 0 | task 705 | Checking checkpoint with [23286, 23286] against 23319...
4.55.765.666 I slot update_slots: id 0 | task 705 | skipping context checkpoint [23286, 23286]: target memory cannot trim from 23287
4.55.765.667 I slot update_slots: id 0 | task 705 | Checking checkpoint with [22774, 22774] against 23319...
4.55.765.667 I slot update_slots: id 0 | task 705 | skipping context checkpoint [22774, 22774]: target memory cannot trim from 22775
4.55.765.667 I slot update_slots: id 0 | task 705 | Checking checkpoint with [21996, 21996] against 23319...
4.55.765.667 I slot update_slots: id 0 | task 705 | skipping context checkpoint [21996, 21996]: target memory cannot trim from 21997
4.55.765.667 W slot update_slots: id 0 | task 705 | forcing full prompt re-processing due to lack of cache data (likely due to SWA or hybrid/recurrent memory, see ggml-org#13194 (comment))
4.55.765.670 W slot update_slots: id 0 | task 705 | erased invalidated context checkpoint (pos_min = 21996, pos_max = 21996, n_tokens = 21997, n_swa = 0, pos_next = 0, size = 174.212 MiB)
4.55.765.672 W slot update_slots: id 0 | task 705 | erased invalidated context checkpoint (pos_min = 22774, pos_max = 22774, n_tokens = 22775, n_swa = 0, pos_next = 0, size = 175.082 MiB)
4.55.766.077 W slot update_slots: id 0 | task 705 | erased invalidated context checkpoint (pos_min = 23286, pos_max = 23286, n_tokens = 23287, n_swa = 0, pos_next = 0, size = 175.654 MiB)
5.00.402.320 I slot print_timing: id 0 | task 705 | prompt processing, n_tokens = 4096, progress = 0.17, t = 4.64 s / 883.40 tokens per second
5.02.854.295 I slot print_timing: id 0 | task 705 | prompt processing, n_tokens = 6144, progress = 0.26, t = 7.09 s / 866.74 tokens per second
5.05.407.058 I slot print_timing: id 0 | task 705 | prompt processing, n_tokens = 8192, progress = 0.35, t = 9.64 s / 849.67 tokens per second
5.08.067.244 I slot print_timing: id 0 | task 705 | prompt processing, n_tokens = 10240, progress = 0.43, t = 12.30 s / 832.41 tokens per second
5.10.819.217 I slot print_timing: id 0 | task 705 | prompt processing, n_tokens = 12288, progress = 0.52, t = 15.05 s / 816.29 tokens per second
5.13.671.714 I slot print_timing: id 0 | task 705 | prompt processing, n_tokens = 14336, progress = 0.61, t = 17.91 s / 800.62 tokens per second
5.16.625.691 I slot print_timing: id 0 | task 705 | prompt processing, n_tokens = 16384, progress = 0.69, t = 20.86 s / 785.43 tokens per second
5.19.677.489 I slot print_timing: id 0 | task 705 | prompt processing, n_tokens = 18432, progress = 0.78, t = 23.91 s / 770.83 tokens per second
5.22.826.053 I slot print_timing: id 0 | task 705 | prompt processing, n_tokens = 20480, progress = 0.87, t = 27.06 s / 756.83 tokens per second
5.26.088.360 I slot print_timing: id 0 | task 705 | prompt processing, n_tokens = 22528, progress = 0.95, t = 30.32 s / 742.94 tokens per second
5.27.206.762 I slot print_timing: id 0 | task 705 | prompt processing, n_tokens = 23123, progress = 0.98, t = 31.44 s / 735.44 tokens per second
5.28.025.460 I slot print_timing: id 0 | task 705 | prompt processing, n_tokens = 23625, progress = 1.00, t = 32.26 s / 732.34 tokens per second
5.28.065.073 I slot create_check: id 0 | task 705 | created context checkpoint 1 of 8 (pos_min = 23624, pos_max = 23624, n_tokens = 23625, size = 176.032 MiB)

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions