fix: ensure prompt cache is invalidated whenever KV cache is cleared

[thunderbug1]

Summary

Fixes #486

The prompt prefix cache in /v1/chat/completions was broken in multiple ways, preventing multi-turn conversations from reusing the cached KV state.

• Non-streaming path unconditionally called clear_context() and prompt_cache.reset(), making prefix caching impossible for non-streaming requests
• Checksum mismatch — update_checksum() included tool messages while can_use_cache() skipped them, causing permanent cache misses when tools were present
• Stale cache hits after errors — error paths cleared the KV cache but didn't invalidate the prompt cache, so the next request would only insert the last message into an empty context
• Stale cache hits from other handlers — /api/chat, /api/generate, /v1/completions all clear the KV cache without invalidating the prompt cache

Changes

• Remove unconditional clear_context() and prompt_cache.reset() from the non-streaming path in handle_openai_chat_completion
• Fix update_checksum() to skip tool messages consistently with can_use_cache()
• Introduce reset_context() helper on RestHandler that couples clear_context() with prompt_cache.reset()
• Replace all bare clear_context() calls across all handlers with reset_context(), except the two cache-miss paths where the cache is immediately re-initialized

Test plan

• Start flm serve with a model
• Send a multi-turn conversation via /v1/chat/completions and verify "Use cached prompt!" appears in logs for subsequent turns
• Verify prefill chunk count drops on cache hits (only the new message is prefilled)
• Test with tools present in the conversation
• Test both streaming and non-streaming requests
• Test that interleaved requests to /api/chat or /api/generate don't cause stale cache hits on the next /v1/chat/completions request

[twflm]

Summary

Fixes #486

The prompt prefix cache in /v1/chat/completions was broken in multiple ways, preventing multi-turn conversations from reusing the cached KV state.

• Non-streaming path unconditionally called clear_context() and prompt_cache.reset(), making prefix caching impossible for non-streaming requests
• Checksum mismatch — update_checksum() included tool messages while can_use_cache() skipped them, causing permanent cache misses when tools were present
• Stale cache hits after errors — error paths cleared the KV cache but didn't invalidate the prompt cache, so the next request would only insert the last message into an empty context
• Stale cache hits from other handlers — /api/chat, /api/generate, /v1/completions all clear the KV cache without invalidating the prompt cache

Changes

• Remove unconditional clear_context() and prompt_cache.reset() from the non-streaming path in handle_openai_chat_completion
• Fix update_checksum() to skip tool messages consistently with can_use_cache()
• Introduce reset_context() helper on RestHandler that couples clear_context() with prompt_cache.reset()
• Replace all bare clear_context() calls across all handlers with reset_context(), except the two cache-miss paths where the cache is immediately re-initialized

Test plan

• Start flm serve with a model
• Send a multi-turn conversation via /v1/chat/completions and verify "Use cached prompt!" appears in logs for subsequent turns
• Verify prefill chunk count drops on cache hits (only the new message is prefilled)
• Test with tools present in the conversation
• Test both streaming and non-streaming requests
• Test that interleaved requests to /api/chat or /api/generate don't cause stale cache hits on the next /v1/chat/completions request

Thank you! Currently, prefix cache is not activated for non-stream mode. Any specific app in your mind?

[thunderbug1]

I was playing around with using claude code with flm and found some issues with kv caching. The main issue in the end was actually that claude code changes the system prompt with every message but I fixed the other issues on the way.