Chat RPC injects the entire tool registry into every request, making local/CPU inference unusable

[fistilis]

Summary

The Chat RPC unconditionally injects the entire aggregated tool registry — every connected VS Code lm.tool plus every enabled MCP server's tool schemas — into every chat request, even for plain chat with no tool use. This makes local/CPU inference effectively unusable and inflates token cost on cloud providers.

Impact / repro

On a 4-core CPU-only host (RHEL VM, Ollama backend, qwen2.5:0.5b), a one-line user message ("Reply with exactly: OK") produces a ~4095-token prompt:

[DaemonState] Registered 82 VS Code tools from ws:<workspace>
...
slot update_slots: task N | new prompt, n_ctx_slot = 4096, n_keep = 4, task.n_tokens = 4095
slot print_timing:  task N | prompt processing, n_tokens = 512, progress = 0.13, t = 34.31 s / 14.9 tokens/s

That's ~4 minutes of prompt processing before the first output token (~15 tok/s on CPU), which trips client timeouts (e.g. the VS Code extension surfaces a generic "your request failed"). The 82 tools come from installed VS Code extensions via lm.tools; the daemon aggregates and injects all of them regardless of whether the request offers tool use.

Confirmed it is the tool registry, not MCP specifically: disabling the only configured MCP server (mcp_servers.*.enabled: false, log shows MCP pool: 0 connected) left the prompt at 4095 tokens.

Additional notes

• llama.cpp reports cached n_tokens = 4, i.e. the large tool preamble is not prefix-cached, so the full ~4095 tokens are re-processed on every turn.
• Also affects cloud providers: every chat pays for the full tool preamble in input tokens even when no tool will be called.

Suggested fixes

1. Only include tools when the request/mode actually offers tool use (opt-in per request), rather than injecting the whole registry unconditionally.
2. Prefix-cache the static tool preamble so repeated turns skip re-processing it.
3. Allow a consumer to scope/cap which tools it is exposed to.

Environment

• Abbenay v2026.8.3 (daemon + VS Code extension redhat.abbenay-provider@2026.8.3)
• VS Code 1.122.1, Ollama (CPU-only), models qwen2.5:0.5b / granite3.1-dense:2b
• Host: 4-core CPU-only RHEL VM