feat(agent): no-tools-called retry for local Ollama models (complements #210)

config.yaml.example

@@ -36,3 +36,16 @@ pageindex_threshold: 20          # PDF pages threshold for PageIndex
 #   extra_headers:         # extra HTTP headers some providers need (e.g. GitHub Copilot)
 #     Editor-Version: vscode/1.95.0
 #     Copilot-Integration-Id: vscode-chat
+
+# Optional: Ollama-specific tuning for local models (issue #205).
+# Only applies when model is an ollama/... or ollama_chat/... model.
+# ollama:
+#   timeout: 300              # per-request timeout (s) for Ollama models.
+#                             # Default 300. Raise for slow local hardware
+#                             # (e.g. 12B models on a single GPU may take
+#                             # 60-130s per tool-calling turn).
+#                             # Falls back to litellm.timeout, then to 300.
+#   tool_call_retries: 3      # max retries when a model hallucinates a tool
+#                             # name (e.g. calls 'get_topics' instead of
+#                             # 'read_file'). Default 3. Increase for very
+#                             # small models that need more correction.

docs/.gitignore

@@ -6,3 +6,5 @@
 *
 !.gitignore
 !golden-principles.md
+!ollama_tool_call_adapter.md
+!ollama_no_tools_retry.md

docs/ollama_no_tools_retry.md

@@ -0,0 +1,137 @@
+# No-Tools-Called Retry
+
+## Problem
+
+When using local Ollama models (12-14B) with OpenKB's `query` and `chat`
+commands, the model may return a final answer **without calling any tools**.
+The answer is typically "I cannot find relevant information" or empty —
+the model didn't attempt to read wiki files, even though tools are
+available and the `add` (ingestion) path works correctly.
+
+This is a **model capability limitation**, not a transport bug — the
+adapter from PR #210 correctly rewrites `ollama/` → `ollama_chat/` and
+handles `ModelBehaviorError`, but it cannot force a model to *initiate*
+tool calls. The model simply answers from its own knowledge (or says "not
+found") without reading the wiki.
+
+## Solution
+
+The adapter now includes a **no-tools-called retry** mechanism. After
+`Runner.run()` completes successfully, the adapter checks whether any
+`ToolCallItem` exists in `result.new_items`. If the agent has tools but
+none were called, the adapter retries with an escalating **nudge message**:
+
+### Attempt 1 (gentle)
+```
+You answered without reading any wiki files. You have tools available.
+Call read_file with path 'summaries/index.md' first to see what
+documents exist, then answer the question.
+```
+
+### Attempt 2 (firm)
+```
+You MUST use the read_file tool. Do NOT answer without reading files
+first. Start with read_file(path='summaries/index.md').
+Available tools: read_file, get_page_content.
+```
+
+### Attempt 3 (final)
+```
+IMPORTANT: Your previous answer was not grounded in the wiki.
+Call read_file now. The available tools are: read_file, get_page_content.
+Do not answer until you have called a tool.
+```
+
+If all retries are exhausted, the adapter returns the last (ungrounded)
+answer — it does not loop infinitely.
+
+## How It Works
+
+### Detection
+
+`_has_tool_calls(result)` checks `result.new_items` for any instance of
+`ToolCallItem` (from `agents.items`). If none found and the agent has
+registered tools, the no-tools retry is triggered.
+
+### Retry Flow
+
+1. `Runner.run()` / `Runner.run_streamed()` completes successfully.
+2. Adapter checks: did the model call any tools?
+3. If yes → return result (normal path, no retry).
+4. If no → append nudge message, retry.
+5. Repeat up to `tool_call_retries` times (default 3, configurable).
+6. If exhausted → return last result with a warning log.
+
+### Interaction with ModelBehaviorError retry
+
+The no-tools retry and the `ModelBehaviorError` retry share the same
+`max_retries` budget. A run that first hallucinates a tool name (triggering
+a `ModelBehaviorError` retry) and then answers without tools (triggering
+a no-tools retry) consumes two retry attempts from the same pool.
+
+### Configuration
+
+Uses the same `ollama:` config block as PR #210:
+
+```yaml
+ollama:
+  timeout: 300              # per-request timeout (s), default 300
+  tool_call_retries: 3      # max retries (hallucination + no-tools), default 3
+```
+
+## Files
+
+| File | Description |
+|---|---|
+| `openkb/agent/ollama_adapter.py` | Added `_has_tool_calls()`, `_build_nudge_message()`, `_append_nudge()`, and no-tools retry loop in `arun_with_retry()` / `run_with_retry()` |
+| `tests/test_ollama_adapter.py` | 17 new tests (73 total, all passing) |
+
+## Testing
+
+```bash
+python -m pytest tests/test_ollama_adapter.py -v  # 73 passed
+python -m pytest tests/ -q                          # 1148 passed
+```
+
+## Sanitization of Ungrounded Output
+
+When all retries are exhausted and the model never called any tools, the
+adapter sanitizes the output before returning it to the user:
+
+- **Raw JSON** (e.g. `{"name": "read_file", ...}`) → replaced with a clear
+  message: *"I could not find relevant information in the knowledge base.
+  The available tools were not used successfully. Try rephrasing your
+  question or adding more documents."*
+- **Empty output** (`""`) → same message
+- **Reasonable text** (e.g. "I cannot find relevant information") →
+  returned as-is (it's a real answer, not raw JSON)
+
+This prevents users from seeing raw tool-call JSON in the query output.
+
+## Minimum Model Requirements
+
+Based on live testing with tg-collector:
+
+| Model | Size | add (ingestion) | query (tool-loop) |
+|---|---|---|---|
+| qwen3.5 (cloud) | 397B | ✅ stable | ✅ grounded answer, stable |
+| gemma4 | 12B | ✅ | ⚠️ unstable (grounded on some runs, "could not retrieve" on others) |
+| qwen2.5-coder | 14B | ✅ | ❌ model ignores nudges; sanitize returns clear message |
+
+**Recommendation**: for `query` and `chat` tool-loop, use models with
+strong instruction-following capability (general-purpose, not code-focused).
+Minimum tested working size: 12B (gemma4, with nudge retry). For `add`
+(ingestion), 12-14B models work fine — tool-calling is not required.
+
+See also `docs/ollama_tool_call_adapter.md` for full model requirements.
+
+## Limitations
+
+- The nudge cannot **force** a model to call tools — it can only
+  encourage. Very small models (1B) may ignore the nudge entirely.
+- Models that produce empty output (`""`) may not benefit from the nudge
+  if the empty output is caused by a timeout or inference failure rather
+  than a decision not to call tools.
+- The retry budget is shared with `ModelBehaviorError` retries. If a model
+  both hallucinates tool names AND answers without tools, the retries may
+  be consumed by the hallucination loop before the no-tools check runs.

docs/ollama_tool_call_adapter.md

@@ -0,0 +1,224 @@
+# Ollama Tool-Call Adapter
+
+## Problem
+
+When using Ollama models with OpenKB's `query` and `chat` commands, the
+openai-agents SDK tool-calling loop fails in three ways (issue #205):
+
+1. **LiteLLM prompt injection fallback** — models addressed as
+   `ollama/<model>` are treated as "legacy" by LiteLLM. Tools are stripped
+   from the API call and injected into the prompt text with
+   `format: json`. The model returns tool-call JSON in `content` instead
+   of `tool_calls`, and the SDK cannot execute the tool.
+
+2. **Tool-name hallucination** — small models call non-existent tools
+   (e.g. `get_topics` instead of `read_file`), causing
+   `ModelBehaviorError: Tool X not found in agent wiki-query`.
+
+3. **Timeouts** — local models on modest hardware can take 60-130s per
+   tool-calling turn; LiteLLM's default timeout is too short.
+
+## Solution
+
+OpenKB includes an adapter (`openkb/agent/ollama_adapter.py`) with four
+mechanisms. All are **Ollama-only** — other providers keep the original
+behaviour unchanged.
+
+### 1. Model rewrite: `ollama/` → `ollama_chat/`
+
+`rewrite_ollama_model()` converts `ollama/<model>` to
+`ollama_chat/<model>` so LiteLLM uses the native Ollama Chat API endpoint,
+which supports `tools` natively (Ollama >= 0.4). This is the **primary
+fix** — without it, LiteLLM falls back to prompt injection and tool calls
+never work.
+
+Applied in `build_query_agent()`, `build_run_config_from_bundle()`, and
+all 12 model-resolution sites in `cli.py`.
+
+### 2. Retry on tool-name hallucination
+
+`arun_with_retry()` and `run_streamed_with_retry()` wrap the SDK Runner
+in try/except for `ModelBehaviorError`. On error, a corrective message is
+appended:
+
+```
+[SYSTEM CORRECTION 1] You tried to call a tool named 'get_topics',
+but that tool does not exist. The only available tools are:
+read_file, get_page_content, get_image. Please answer the original
+question using ONLY these tools. Do not invent tool names.
+```
+
+Up to `tool_call_retries` times (configurable, default 3).
+
+### 3. Configurable timeout
+
+`_ensure_ollama_settings()` injects the resolved timeout into the agent's
+`ModelSettings.extra_args` if none is already set.
+
+### 4. `drop_params` and `api_base` propagation
+
+For `ollama_chat`, LiteLLM rejects `parallel_tool_calls` and does not read
+`OPENAI_API_BASE`. The adapter sets:
+- `litellm.drop_params = True` (drop unsupported params)
+- `OLLAMA_API_BASE` from `OPENAI_API_BASE` (env var + litellm module level)
+- `api_base` in compiler LLM calls (for the `add` path)
+
+## Configuration
+
+All settings are in `config.yaml`, under the optional `ollama:` key:
+
+```yaml
+ollama:
+  timeout: 300              # per-request timeout (s), default 300
+  tool_call_retries: 3      # max retries on hallucinated tool names, default 3
+```
+
+### Timeout precedence (highest to lowest)
+
+1. `ollama.timeout` in `config.yaml`
+2. `litellm.timeout` in `config.yaml` (process-wide)
+3. Top-level `timeout:` in `config.yaml`
+4. Built-in default: **300s**
+
+### Retry precedence
+
+1. `ollama.tool_call_retries` in `config.yaml`
+2. Built-in default: **3**
+
+### Example config for slow hardware
+
+```yaml
+model: ollama_chat/gemma4:12b
+language: en
+parallel_tool_calls: false
+ollama:
+  timeout: 600              # 10 minutes per request
+  tool_call_retries: 5      # more patience for small models
+litellm:
+  drop_params: true
+```
+
+### Environment variables
+
+For Ollama, set in `.env`:
+
+```
+LLM_API_KEY=dummy
+OPENAI_API_BASE=http://your-ollama-host:11434
+```
+
+The adapter automatically propagates `OPENAI_API_BASE` → `OLLAMA_API_BASE`.
+
+## How It Works
+
+### Detection
+
+`is_ollama_backend(model)` returns `True` for any model string containing
+`ollama/` or `ollama_chat/` (with or without `litellm/` prefix).
+
+### Rewrite
+
+`rewrite_ollama_model(model)` converts:
+- `ollama/gemma4:12b` → `ollama_chat/gemma4:12b`
+- `litellm/ollama/gemma4:12b` → `ollama_chat/gemma4:12b`
+- `ollama_chat/gemma4:12b` → unchanged
+- `openai/gpt-4o` → unchanged
+
+### Retry Flow
+
+1. The agent runs normally via `Runner.run()` / `Runner.run_streamed()`.
+2. If `ModelBehaviorError` is raised, the bad tool name is extracted.
+3. A corrective user message is appended to the input.
+4. The run is retried (up to `tool_call_retries` times).
+5. If all retries are exhausted, the original error is re-raised.
+
+### Streaming
+
+`_RetryableStreamResult` wraps `RunResult.stream_events()`. If a
+`ModelBehaviorError` occurs mid-stream, the wrapper re-creates the stream
+with a corrective message and continues yielding events seamlessly.
+
+## Files
+
+| File | Description |
+|---|---|
+| `openkb/agent/ollama_adapter.py` | Adapter module (rewrite + retry + timeout + config) |
+| `openkb/agent/query.py` | Uses adapter for Ollama models in query/chat |
+| `openkb/agent/compiler.py` | Passes `api_base` from env for CLI path |
+| `openkb/cli.py` | `rewrite_ollama_model` on all model resolutions |
+| `openkb/add_coordinator.py` | Imports adapter for side effects |
+| `tests/test_ollama_adapter.py` | 56 unit tests |
+| `docs/ollama_tool_call_adapter.md` | This documentation |
+
+## Testing
+
+```bash
+# Run adapter tests
+python -m pytest tests/test_ollama_adapter.py -v  # 80 passed
+
+# Run full test suite (non-Ollama path unchanged)
+python -m pytest tests/ -q  # 1148 passed
+```
+
+## Minimum Model Requirements
+
+OpenKB uses two LLM workloads with different requirements:
+
+### 1. Ingestion (`add`) — low requirements
+
+The `add` command compiles documents into wiki pages (summary, concepts,
+entities). It uses direct LLM completion — **no tool-calling needed**.
+Models ≥12B work reliably.
+
+### 2. Query / Chat — higher requirements
+
+The `query` and `chat` commands use a multi-step tool-calling loop: the
+model must call `read_file` to read wiki pages, process the results, and
+produce a grounded answer. This requires **instruction-following** and
+**tool-use capability**, which smaller models may lack.
+
+### Tested models
+
+| Model | Size | `add` | `query` | Notes |
+|---|---|---|---|---|
+| qwen3.5 (cloud) | 397B | ✅ | ✅ grounded | Stable across all runs |
+| gemma4 | 12B | ✅ | ⚠️ unstable | Grounded answer on some runs; "could not retrieve" on others |
+| qwen2.5-coder | 14B | ✅ | ❌ | Model ignores nudge retries; sanitize fallback returns clear message |
+
+### Recommendations
+
+- **For `add` (ingestion):** any Ollama model ≥12B works. Tool-calling
+  is not required.
+- **For `query` / `chat`:** use models with strong instruction-following
+  capability. General-purpose models (e.g. gemma4, qwen3) perform better
+  than code-focused models (e.g. qwen2.5-coder) for tool-use tasks.
+- **For slow hardware:** set `ollama.timeout: 600` or higher. Local 12B
+  models can take 60-730s per request depending on GPU and model size.
+- **Instability:** 12B models may produce different results between runs
+  (grounded answer on one run, "could not retrieve" on the next). This is
+  model-level variance, not an adapter issue. Consider fixing
+  temperature/seed if reproducibility is needed.
+
+## Live Test Results
+
+Tested with Ollama 0.32.5, LiteLLM 1.94.0, openai-agents 0.19.1:
+
+| Model | Test | Result |
+|---|---|---|
+| gemma4:12b | LiteLLM raw tool call | `read_file` with correct name, 14s |
+| gemma4:12b | Full SDK tool-loop | `index.md` → `messages.md` → final answer, 132s |
+| qwen3.5:397b (cloud) | Full tool-loop | Grounded answer with time/equipment details |
+| qwen2.5-coder:14b | Full tool-loop | 3 nudge retries → sanitize fallback (clear message) |
+| Non-Ollama path | Full test suite | 1148 passed, 0 regressions |
+
+## Limitations
+
+- Very small models (1B params) may still fail after retries — the
+  adapter prevents crashes but cannot fix a model's fundamental inability
+  to follow tool-use instructions.
+- The `add` path makes multiple sequential LLM calls (summary +
+  concepts); on very slow hardware the second call may time out. Raise
+  `ollama.timeout` or `litellm.timeout` to accommodate.
+- The adapter does not extract tool-call JSON from `content` (the
+  `ollama_chat/` rewrite makes this unnecessary for models that support
+  native tool calling).

openkb/add_coordinator.py

@@ -8,6 +8,10 @@
 
 import click
 
+# Import ollama_adapter for its module-level side effect: propagating
+# OPENAI_API_BASE → OLLAMA_API_BASE + litellm.api_base so the ollama_chat
+# LiteLLM provider reaches the correct endpoint even on the ``add`` path.
+from openkb.agent.ollama_adapter import is_ollama_backend, rewrite_ollama_model  # noqa: F401
 from openkb.locks import kb_ingest_lock_held
 from openkb.mutation import MutationSnapshot, snapshot_paths