feat: agent tools and local chat for the PageIndex SDK (v0.2.10)

[rejojer]

Adds an agent integration layer to the client — the PageIndex cloud MCP tool contract, runnable in-process against either mode, consumable from every major agent surface — and, built on top of that layer, managed local chat over three industry-standard wire protocols. This is the full SDK 0.2.10 increment over main. (Developed as a stack: the tools layer was #393, kept open as that layer's isolated diff and review record; this PR was retargeted to main to carry the complete change.)

What

The tools layer — one call per framework, identical across local and cloud; the mode is decided by the client constructor alone:

# — inside your async app; agent frameworks are async-native —
from pageindex import PageIndexLocalClient

client = PageIndexLocalClient()                 # or PageIndexCloudClient(api_key="...")
doc = client.submit_document("report.pdf", wait=True)

# ── OpenAI Agents SDK ──
# pip install "pageindex[openai]"
from agents import Agent, Runner

agent = Agent(**client.openai_agent_config())   # instructions + tools (+ the local model default)
result = await Runner.run(agent, "What was total revenue this quarter, vs last year?")
print(result.final_output)

# ── Anthropic SDK tool runner ──
# pip install "pageindex[anthropic]"
import anthropic

runner = anthropic.AsyncAnthropic().beta.messages.tool_runner(
    **client.anthropic_runner_config(model="claude-sonnet-4-6", asynchronous=True),
    messages=[{"role": "user", "content": "What was total revenue this quarter?"}],
)
final = await runner.until_done()
print(final.content[-1].text)

# ── Claude Agent SDK ──
# pip install "pageindex[claude]"
from claude_agent_sdk import ClaudeAgentOptions, ResultMessage, query

options = ClaudeAgentOptions(**client.claude_agent_config())
async for message in query(prompt="What was total revenue this quarter?", options=options):
    if isinstance(message, ResultMessage):
        print(message.result)

# ── Any other framework — no extras needed ──
tools = client.agent_tools()   # plain functions returning JSON envelopes

Each *_config() is pure sugar over the explicit slots — agent_instructions() plus as_openai_tools() / as_anthropic_tools() / as_claude_mcp() — with one include_management / doc_id / naming parameter applied consistently everywhere; drop to the explicit form to customize, and nothing changes underneath.

The chat layer, on the same tools (local mode) — each method = its wire format = its backend protocol, 1:1, no translation layer. Every method takes its protocol's native message form or a bare query string (one user message):

# client and doc from the setup above

# ── Universal — any OpenAI-compatible backend (ollama, vllm, gateways, compat endpoints) ──
answer = client.chat_completions("What was Q3 revenue?", doc_id=doc["doc_id"],
                                 model="gpt-5.4")   # model optional — defaults to the client's retrieve_model
print(answer["choices"][0]["message"]["content"])

# ── Agentic — backends that speak the Responses API ──
result = client.responses("Which section covers risk factors?",
                          doc_id=doc["doc_id"], model="gpt-5.4")
print(result["output"][-1]["content"][0]["text"])

# ── Claude-native — Anthropic Messages protocol ──
# pip install "pageindex[anthropic]"
reply = client.messages("What was Q3 revenue?",
                        doc_id=doc["doc_id"], model="claude-sonnet-4-6")
print(reply["content"][-1]["text"])

Streaming (stream=True) ships on all three chat surfaces: chat_completions keeps the cloud signature's text-pieces/chunk-dicts modes, responses forwards native stream events — one logical response per call: lifecycle events collapsed, sequence numbers monotonic, output_index re-based onto the single logical output, and the terminal event reflects the backend's real terminal state — messages forwards the native Anthropic event stream verbatim. Multi-turn is verbatim append: extend the next call's input with responses()' output items or messages()' new-turn messages (same doc_id each call) and the provider prompt-cache prefix carries over — a tested contract.

Coverage

Bring your own agent — the tools, on every major surface:

Surface	Local	Cloud
agent_tools() — plain functions, any framework	✅ in-process tools	✅ live cloud tool set over MCP
as_openai_tools() / openai_agent_config() — OpenAI Agents SDK	✅	✅ (hosted=True: execution on OpenAI's side, read-only endpoint by default)
as_anthropic_tools() / anthropic_runner_config() — Anthropic SDK tool runner	✅ sync & async	✅ sync & async
as_claude_mcp() / claude_agent_config() — Claude Agent SDK / Claude Code	✅ in-process MCP server	✅ remote MCP config — read-only endpoint by default
Standard MCP, no SDK involved	⬜ stdio entry point (follow-up)	✅ api.pageindex.ai/mcp (read-only: …/mcp?tools=read) — any MCP host, the Anthropic MCP connector, OpenAI hosted MCP

Managed chat — the SDK runs the loop:

Method	Wire format	Engine	Local	Cloud
chat_completions()	OpenAI chatcmpl	openai-agents	✅	✅ hosted endpoint
responses()	OpenAI Responses	openai-agents	✅	⬜ raises — cloud converges toward this later
messages()	Anthropic Messages	anthropic tool_runner	✅	⬜ raises

All three chat methods share the same base: bare-string or native-message input, doc_id targeting, model override, streaming, max_turns, sampling passthrough. Local doc_id is enforcement, not just prompting — the registered tools resolve names inside the target allowlist, so out-of-scope documents stay unreachable even under prompt injection. Tool sets: local = browse_documents · get_document · get_document_structure · get_page_content (plus remove_document behind include_management=True); cloud additionally serves search_documents, folders, and get_document_image — discovered live from the server, never frozen into the SDK.

Document QA, end to end

An actual run (local mode, OpenAI Agents SDK, over examples/documents/q1-fy25-earnings.pdf):

Q: What was Disney's total revenue in Q1 FY2025, and how did it compare to the prior-year quarter? Cite the page you found it on.

[tool] get_document_structure({"doc_name": "q1-fy25-earnings.pdf", "part": 1})
[tool] get_page_content({"doc_name": "q1-fy25-earnings.pdf", "pages": "1,3"})

A: Disney's total revenue in Q1 FY2025 was $24.7 billion, up 5% from $23.5 billion in the prior-year quarter.
Citations — Page 1: "Revenues increased 5% for Q1 to $24.7 billion from $23.5 billion in Q1 fiscal 2024"; Page 3: table shows $24,690M vs $23,549M, +5%.

This is the reasoning-based retrieval loop working as designed: the agent reads the tree structure first, picks tight page ranges, and answers strictly from tool output with page citations — no vector index, no chunking, and the retrieval "intelligence" is the host agent's own model (the navigation tools themselves make no LLM calls).

Design — the tools layer

• The tool surface is the cloud MCP contract: browse_documents / get_document / get_document_structure / get_page_content, doc_name-addressed, same input schemas, descriptions, and JSON response envelopes as the hosted MCP server's tools/list — agent prompts port unchanged between the cloud MCP connection and these in-process tools. Adapters hand the contract/server schema to the framework verbatim (FunctionTool(params_json_schema=…), beta_tool(input_schema=…)) — no regeneration from Python signatures, so items/enum/pattern/bounds survive on every surface. tests/data/cloud_mcp_contract.json freezes the contract; a parity test guards drift.
• Local is an honest subset: tools that don't exist locally (folders, search_documents, get_document_image) are not registered, mirroring the server's gating semantics. Cloud-only parameters (folder_id, sort/query, recursive) are hidden from the local surface entirely — strict-schema frameworks then cannot express the dead-end calls, and the call_tool/MCP path still answers direct calls with a guided "works on PageIndex cloud" envelope as the backstop. Local descriptions and instructions teach only that surface: the exposed schema is the contract minus the documented hidden set (mechanically asserted), and a dead-reference test keeps local guidance from naming cloud-only tools. remove_document is off by default, behind include_management=True.
• The management gate is structural wherever possible. The config-handoff surfaces — as_claude_mcp(), as_openai_tools(hosted=True), the raw connector URL — point at the server's read-only endpoint (/mcp?tools=read) by default, so the URL itself is the gate and works identically in every MCP client. The in-process surfaces (agent_tools(), as_openai_tools(), as_anthropic_tools()) expose only tools the server marks readOnlyHint; local withholds remove_document at registration. include_management=True is the one switch that opens the complete list in either mode, on every surface. All four tool exports stay polymorphic: on a cloud client the live tool set — including new server-side tools — arrives without an SDK release.
• Tools never raise, and failures carry the protocol's own error marking — every failure returns the same {"error", "errorCode", "next_steps"} envelope the cloud emits, flagged through each channel that has one (MCP isError propagated, Anthropic tool runner is_error: true via ToolError), so the model can always tell a failed call from data. Destructive calls validate every argument before acting — a rejection envelope means nothing was deleted.
• agent_instructions(doc_id=None) supplies the retrieval playbook for the agent's system prompt. Cloud: the live instructions the MCP server serves for the key's tool set, captured from the initialize handshake over the same bridge session — server-side guidance updates arrive without an SDK release, and an empty server response raises instead of silently substituting. Local: the built-in playbook for the in-process tools, a trimmed subset with a consistency test that every tool it names exists locally. doc_id (str or list) appends the target documents — in the run above it is what let the agent skip discovery.
• Zero new hard dependencies: openai-agents / claude-agent-sdk / anthropic are imported at call time with actionable errors; the [openai] / [claude] / [anthropic] extras carry floor-only pins. import pageindex and every existing feature work with none installed (covered by tests).
• submit_document(wait=True) polls with growing intervals; returns on completed, raises on failed or after 30 minutes — the manual polling loop cloud callers write today spins forever on a failed document.

Design — chat on the tools

• Basis is industry standards, not the cloud chat endpoint. The cloud /chat/completions quirks (history flattening, bespoke prompt, stateless re-reading, arbitrary caps) are not mirrored; local targets the standard formats and becomes the reference the cloud can later converge toward. responses()/messages() raise on cloud clients until then.
• Passthrough doctrine. Content is never rewritten — the caller's messages, the model's answers, tool outputs, native finish/stop reasons. The SDK owns exactly four things: gatekeeping (structural validation only — no message caps, sampling params pass through), table-setting (thin chat header + the local AGENT_INSTRUCTIONS; caller system content appended, not rejected; the doc_id targeting block leads the conversation and the tool layer enforces it), tool execution (read-only local set, scoped to doc_id), and billing (usage aggregation, envelope ids). Per-run tracing is disabled; prompt-cache routing keys are per-conversation, never pooled across users.
• Engines are the vendors' own loops, never hand-rolled: openai-agents for the two OpenAI protocols (litellm/-prefixed and provider/model names route through the SDK's LiteLLM model, openai/ strips to the OpenAI SDK), the Anthropic SDK's tool_runner for messages() (floor 0.108.0 — the first release whose runner stops at a refusal carrying a tool_use block instead of executing the tool; verified by probing mock transports against 0.84.0 through 0.108.0). Rule of the layer: engines = each vendor's official thin loop; agent hosts (Claude Code et al.) only ever get tools.
• Envelopes report what actually happened. responses() carries the backend's real terminal status/incomplete_details (recorded at the transport layer — the engine discards them), a partial page read names every omitted page, and framework exceptions surface as PageIndexAPIError, never as raw engine types.
• Prompt-cache continuity is a tested contract. Round-tripped history reaches the backend as an item-for-item extension of the previous call's final model input — asserted on both engines against captured payloads. Anthropic's explicit cache_control breakpoints sit on the managed system blocks only.
• enable_citations raises as cloud-only (citations need block-level OCR data local mode does not store).
• messages() resolves its max_tokens default per model (8192, or 4096 for the claude-3 generation), so the simple call needs only a question on any model.

Verification

• 265 tests green (plus 3 skipped without the claude extra and 3 key-gated live tests): tool behavior against a seeded store with no LLM calls; the real chat engines against scripted backends (a Model fake under openai-agents, a mock HTTP transport under the real anthropic SDK); contract parity vs the frozen snapshot; framework-missing/-installed behavior both ways; streaming on all three surfaces; the round-trip prefix-extension assertions on both engines; doc_id scoping, error-marking, and envelope-honesty regressions.
• Live against the real cloud MCP server: agent_tools() and as_anthropic_tools() discovered this key's gated tool set (7 read-only tools; include_management=True adds remove_document); frozen-contract parity letter-for-letter; envelope field parity on the analogous calls; the server serves non-empty initialize.instructions.
• Live against real model backends: OpenAI — chat_completions answered with the structure-first loop; responses round-trip answered the follow-up with zero new tool calls. Anthropic — messages() history was accepted verbatim by the real API, follow-up answered with zero new tool turns, cache_control hit live (cache_read_input_tokens: 1826), native streaming; both the sync and AsyncAnthropic tool runners drove the live cloud tools end-to-end; the Messages API MCP connector reached api.pageindex.ai/mcp server-side (mcp_tool_use/mcp_tool_result in a single call).
• Review record: multiple independent multi-agent review rounds ran over each layer (adversarial runtime probes, claims-vs-code, refactor-equivalence audits, best-practice review against the frameworks' source); every finding was reproduced before being fixed, and deliberate non-changes are documented alongside. The tools layer's rounds are recorded in feat: agent tools — OpenAI Agents SDK, Claude Agent SDK, and any framework, local & cloud #393; the chat and tools-export rounds in this PR's commit messages. A maximum-effort whole-PR review round then ran over this diff — 26 verified findings, 20 fixed in the first pass (d87fa89, b135711, eb1a230), the remainder triaged with rationale. Nine further review rounds followed (commit messages 31c9150 through eebed64), covering argument coercion, protocol terminal states, envelope honesty, provider error containment, CodeQL findings, and SDK dependency floors — each finding reproduced before the fix landed.

Release gate: the default cloud configs point at the read-only MCP endpoint — VectifyAI/pageindex-chat#448 must be deployed before 0.2.10 ships (an older server ignores the tools=read parameter and would silently serve the full set behind a URL that promises read-only).

Follow-ups (not in this PR): an AsyncPageIndexClient twin per the industry dual-client pattern — every layer around the SDK is already async-native (FastAPI server, agent engines, agent frameworks); the async chat path is the engines' native form (drops the sync bridge, streams pass through as async for), and cloud transport gains an httpx track; a stdio pageindex-mcp entry point for non-Python MCP hosts; a public doc_id scope on the BYO tool exports (the chat surfaces already enforce it); the docs-site agent-integration page; cloud /responses·/messages convergence toward these surfaces.

[rejojer]

Code review

No issues found. Checked for bugs and CLAUDE.md compliance.

🤖 Generated with Claude Code

_{- If this code review was useful, please react with 👍. Otherwise, react with 👎.}