feat: agentic-core public API — composable step functions per ADR-03

[ashwing]

Proposal: agentic-core public API — composable step functions per ADR-03

Context

ADR-03 defines the core crate's public API as a set of composable async functions that implement the agentic loop. Each function is a standalone step that can be:

• Called directly in standalone mode (execute() composes them)
• Wrapped in a Praxis HttpFilter for gateway mode
• Tested in isolation without HTTP infrastructure

The ADR intentionally left the function parameters as (...) placeholders. This issue proposes concrete type signatures, informed by:

• The ADR-02 store schema (three-table model: Conversation/Response/Item)
• PR Cache upstream TLS certificates at config time praxis-proxy/praxis#34's working agentic loop implementation (tool dispatch pattern)
• vLLM's own ConversationContext pattern in responses/serving.py
• RFC Epic: Wasm Runtime praxis-proxy/praxis#18's session-aware KV cache requirements (future extensibility)

Design Goals

1. Each step is independently testable — no hidden coupling between steps
2. serde_json::Value at boundaries, typed internally — the gateway is model-agnostic; we don't enforce a fixed item schema at the API level
3. Trait-based backends — store, inference client, and tool dispatch are swappable
4. Streaming-first — inference returns a stream, not a collected response
5. Tool loop is explicit — the caller decides iteration policy, not the library
6. Future-proof for RFC Epic: Wasm Runtime praxis-proxy/praxis#18 — ExecutionContext carries optional cache hints without breaking the API

Proposed Types

ExecutionContext — shared state across steps

pub struct ExecutionContext {
    /// Pluggable response/conversation store (ADR-02).
    pub store: Arc<dyn ResponseStore>,

    /// Inference backend caller.
    pub inference: Arc<dyn InferenceClient>,

    /// Tool registry for dispatch.
    pub tools: Arc<dyn ToolRegistry>,

    /// Gateway configuration (LLM base URL, timeouts, etc.)
    pub config: CoreConfig,
}

Store trait (ADR-02 alignment)

#[async_trait]
pub trait ResponseStore: Send + Sync {
    /// Load a response by ID, returning its metadata and history_item_ids.
    async fn get_response(&self, response_id: &str) -> Result<StoredResponse, Error>;

    /// Bulk-fetch items by IDs. Caller is responsible for re-ordering.
    async fn get_items(&self, item_ids: &[String]) -> Result<Vec<StoredItem>, Error>;

    /// Persist a completed response with its items.
    async fn persist(
        &self,
        response_id: &str,
        conversation_id: Option<&str>,
        previous_response_id: Option<&str>,
        items: &[Item],
        metadata: &ResponseMetadata,
    ) -> Result<(), Error>;
}

Inference trait

#[async_trait]
pub trait InferenceClient: Send + Sync {
    /// Send a request to the LLM and return a stream of output chunks.
    /// The stream yields typed events (delta text, tool calls, done).
    async fn call(
        &self,
        request: &InferenceRequest,
    ) -> Result<Pin<Box<dyn Stream<Item = Result<InferenceEvent, Error>> + Send>>, Error>;
}

Tool dispatch trait

#[async_trait]
pub trait ToolRegistry: Send + Sync {
    /// Execute a tool call, returning the result.
    async fn dispatch(&self, call: &ToolCall) -> Result<ToolResult, Error>;

    /// Check if a tool is registered and available.
    fn has_tool(&self, name: &str) -> bool;
}

Proposed Step Functions

1. rehydrate_conversation

/// Reconstruct conversation history from a previous_response_id.
///
/// Loads the stored response, fetches referenced items, restores ordering,
/// and returns the full conversation ready for the next inference call.
pub async fn rehydrate_conversation(
    previous_response_id: &str,
    ctx: &ExecutionContext,
) -> Result<Conversation, Error>

Returns: Conversation containing ordered history items + metadata from the prior turn.

2. call_inference

/// Send the conversation to the LLM backend and collect the response.
///
/// For non-streaming consumers, collects the full response.
/// For streaming, returns a stream handle.
pub async fn call_inference(
    conversation: &Conversation,
    request: &InferenceRequest,
    ctx: &ExecutionContext,
) -> Result<InferenceResult, Error>

Returns: InferenceResult containing output items + any tool calls detected.

3. dispatch_tools

/// Execute all tool calls from an inference result.
///
/// Dispatches each call through the ToolRegistry, collecting results.
/// Returns the tool outputs ready to be appended to the conversation.
pub async fn dispatch_tools(
    tool_calls: &[ToolCall],
    ctx: &ExecutionContext,
) -> Result<Vec<ToolResult>, Error>

4. assemble_response

/// Build the final API response from inference output and tool results.
///
/// Produces the ResponsesResponse object with stable item IDs,
/// proper lifecycle events, and correct output typing.
pub async fn assemble_response(
    request_id: &str,
    output: &InferenceResult,
    model: &str,
    metadata: &ResponseMetadata,
) -> Result<ResponsesResponse, Error>

5. persist_response

/// Save the completed response and its items to the store.
///
/// Only persists when store is enabled and the request requires it.
pub async fn persist_response(
    response: &ResponsesResponse,
    conversation_id: Option<&str>,
    previous_response_id: Option<&str>,
    ctx: &ExecutionContext,
) -> Result<(), Error>

6. execute — the convenience orchestrator

/// Run the full agentic loop: rehydrate → infer → (tool loop) → assemble → persist.
///
/// This is the default composition of the step functions with standard
/// iteration policy (max_iterations capped, tool results fed back).
pub async fn execute(
    request: ResponsesRequest,
    ctx: &ExecutionContext,
) -> Result<ResponsesResponse, Error>

Key Domain Types

pub struct Conversation {
    pub id: Option<String>,
    pub history: Vec<Item>,
    pub metadata: ConversationMetadata,
}

pub struct Item {
    pub id: String,
    pub data: serde_json::Value,  // model-agnostic
}

pub struct InferenceRequest {
    pub model: String,
    pub input: Vec<serde_json::Value>,  // history + new input
    pub tools: Vec<ToolConfig>,
    pub stream: bool,
    pub params: serde_json::Map<String, serde_json::Value>,  // passthrough
}

pub struct InferenceResult {
    pub output: Vec<OutputItem>,
    pub tool_calls: Vec<ToolCall>,
    pub usage: Option<Usage>,
}

pub struct ToolCall {
    pub id: String,
    pub call_id: String,
    pub name: String,
    pub arguments: String,
}

pub struct ToolResult {
    pub call_id: String,
    pub output: String,
}

Composability Example

Standalone mode (default execute())

let response = agentic_core::execute(request, &ctx).await?;

Custom loop with guardrails

let conversation = rehydrate_conversation(&prev_id, &ctx).await?;
let mut inference_req = build_request(&conversation, &request);

for _ in 0..max_iterations {
    let result = call_inference(&conversation, &inference_req, &ctx).await?;
    
    if result.tool_calls.is_empty() {
        return assemble_response(&req_id, &result, &model, &meta).await;
    }
    
    // Custom guardrail: validate tool calls before dispatch
    validate_tool_calls(&result.tool_calls)?;
    
    let tool_results = dispatch_tools(&result.tool_calls, &ctx).await?;
    append_tool_results(&mut inference_req, &result, &tool_results);
}

Praxis filter chain

filter_chains:
  - name: agentic-loop
    filters:
      - filter: rehydrate
      - filter: inference
      - filter: tool_dispatch
        branch_chains:
          - name: tool-loop
            on_result: { filter: tool_dispatch, key: action, result: loop }
            rejoin: inference
            max_iterations: 10
      - filter: assemble
      - filter: persist

Implementation Plan

Phase 1: Type definitions + traits (this issue)

• Define all types in crates/agentic-core/src/types.rs
• Define traits in crates/agentic-core/src/traits.rs
• Stub all step functions with todo!() bodies
• Ship as a PR so other PRs ([FEAT] agentic-core conversation and responses database CRUD. #33, Cache upstream TLS certificates at config time praxis-proxy/praxis#34) can align their interfaces

Phase 2: rehydrate_conversation + persist_response

• Implement against the ResponseStore trait
• PR Epic: WebSocket and HTTP Upgrade support praxis-proxy/praxis#33's SQLx backend becomes one implementation of the trait
• Include in-memory store for testing

Phase 3: call_inference

• Implement against InferenceClient trait
• Default implementation wraps the existing proxy logic (reqwest to vLLM)
• Handle streaming vs non-streaming

Phase 4: dispatch_tools + execute

• MCP client as first ToolRegistry implementation
• execute() ties all steps together with default iteration policy
• Aligns with PR Cache upstream TLS certificates at config time praxis-proxy/praxis#34's loop pattern

Phase 5: Praxis filter wrappers

• Thin HttpFilter impls in agentic-praxis that delegate to step functions

Relationship to Existing PRs

PR	Relationship
praxis-proxy/praxis#33 (store CRUD)	Becomes a ResponseStore trait implementation. Types align with StoredResponse/StoredItem.
praxis-proxy/praxis#34 (OGX loop)	OgxStore becomes a ResponseStore impl. The agentic loop in agentic.rs becomes the reference for execute().
praxis-proxy/praxis#35 (module reloc)	Reverted. Proxy stays in core for now; step functions live alongside it.
RFC praxis-proxy/praxis#18 (KV cache)	ExecutionContext is extensible — a session_cache: Option<Arc<dyn SessionCache>> field can be added without breaking the API.

Open Questions

1. Streaming + tool loop tension: SSE streaming to clients requires forwarding chunks in real-time. But the tool loop needs to collect the full output to detect tool calls before iterating. vLLM solves this with a ConversationContext that accumulates output while yielding each chunk to the SSE stream — effectively "tee-ing" the stream. Should call_inference return a stream that the caller both forwards and accumulates, or should we have separate streaming/non-streaming paths? Proposal: call_inference returns a Stream<InferenceEvent>. For the tool-loop path, execute() collects internally. For direct streaming, the server layer can forward events as they arrive and buffer for tool detection simultaneously.

2. serde_json::Value vs typed items: PR Cache upstream TLS certificates at config time praxis-proxy/praxis#34 uses Vec<serde_json::Value> for input items (maximum flexibility, no schema enforcement). PR Epic: WebSocket and HTTP Upgrade support praxis-proxy/praxis#33 defines typed InOutItem enum. Proposal: core API uses serde_json::Value at boundaries; typed wrappers are optional convenience in a types module.

3. Error granularity: Should each step have its own error type, or share a unified Error enum? Proposal: single Error enum with variants per failure domain (Store, Inference, ToolDispatch, Serialization).

4. Where does SSE assembly live? assemble_response produces the final API object, but SSE event framing (data: ...\n\n) is HTTP-level. Proposal: assembly produces the typed response; SSE framing lives in agentic-server.

---

cc @leseb — as discussed, happy to drive the implementation. Would appreciate feedback on the trait boundaries and type choices before I start coding.

[leseb]

Enhanced agentic-core Public API — Composable Step Functions for Gateway Integration

Summary

This proposal expands the agentic-core public API from issue praxis-proxy/praxis#42's
6 step functions and 3 traits to 14 step functions and 8 traits.
The expanded surface enables a clean 1:1 mapping between core
functions and gateway filters, fulfilling ADR-03's requirement that
"each loop step is a public function" and that gateway adapters are
"thin wrappers (~10 lines each) that delegate to agentic-core."

The design is informed by the Praxis proxy's Responses API proposal
(praxis#354), which decomposes the same agentic loop
into 14 composable HttpFilter implementations. Each filter needs
exactly one core function to delegate to. Issue praxis-proxy/praxis#42's 6 functions
leave 8 filters with no counterpart, and its monolithic ToolRegistry
trait cannot serve tool types with radically different lifecycles.

Motivation

The Granularity Gap

Issue praxis-proxy/praxis#42 proposes:

#	Function	Covers
1	rehydrate_conversation	Conversation history
2	call_inference	LLM backend call
3	dispatch_tools	Tool execution
4	assemble_response	Response building
5	persist_response	Storage
6	execute	Orchestrator

A real gateway integration requires these additional steps that have
no counterpart in issue praxis-proxy/praxis#42:

Missing Step	Why It's Needed
Request parsing/validation	Extract fields the orchestrator acts on (stream, store, background, model, tools), generate response_id, reject invalid combinations
Store initialization	Create the shared store instance early so downstream steps can read/write
File resolution	Resolve file_id, file_data, image_url references to inline content before inference
Tool parsing	Parse tool definitions, list MCP tools, build mcp_tool_map, normalize tool_choice — separate from execution
Stream transformation	Convert Chat Completions SSE chunks into Responses API events (24 event types). This is a full state machine, not a trivial mapping
Individual tool executors	MCP (session management, approval policy), web search (provider abstraction), file search (vector store fan-out, citation extraction) — each has different dependencies and signatures
Context compaction	Token counting and context window management via summarization
Reasoning summarization	Post-streaming second inference call for reasoning summary generation

The Trait Granularity Problem

Issue praxis-proxy/praxis#42 defines a single ToolRegistry trait:

#[async_trait]
pub trait ToolRegistry: Send + Sync {
    async fn dispatch(&self, call: &ToolCall) -> Result<ToolResult, Error>;
    fn has_tool(&self, name: &str) -> bool;
}

This cannot serve tools with fundamentally different lifecycles:

• MCP tools require session management (create/reuse/close
  sessions keyed on endpoint + headers), approval policies
  (always/never/filter list), and protocol auto-detection
  (Streamable HTTP with SSE fallback).
• Web search requires provider abstraction (Brave, Tavily,
  SearXNG), search_context_size configuration, and result
  formatting.
• File search requires vector store fan-out across multiple
  stores in parallel, ranking options (weighted/rrf/neural),
  citation marker extraction, and result templating.

A single dispatch(&ToolCall) -> ToolResult signature erases these
differences. The gateway needs separate traits to inject different
implementations per tool type.

---

Design Principles

1. One core function per gateway filter. Every gateway filter
   that implements Responses API logic delegates to exactly one
   public function in agentic-core. The filter handles HTTP
   lifecycle; the core function handles domain logic.

2. Traits per concern, not per abstraction layer. Separate
   traits for each tool executor type. Implementations differ
   radically: MCP requires session management; web search requires
   provider abstraction; file search requires vector store fan-out.

3. Streaming as a first-class return type. call_inference
   returns a stream that can be both forwarded to clients AND
   accumulated for tool detection. The tee pattern is explicit.

4. Gateway-agnostic state. AgenticState is a plain data
   struct. The gateway wraps it in whatever synchronization
   primitive it needs (Arc<Mutex<_>>, RefCell, &mut).

5. serde_json::Value at public boundaries. Per ADR-03,
   public function signatures accept and return Value. Internal
   types are strongly typed. The boundary conversion is the core
   function's responsibility.

6. Background jobs are gateway infrastructure. Queue/worker
   machinery (tokio channels, bounded queues, HTTP polling
   endpoints) belongs to the gateway, not the core. The core
   validates constraints (background=true && store=false is
   rejected) and sets status=queued, but does not manage the
   execution.

---

Shared State: AgenticState

Request-scoped struct that flows through the entire loop.
Replaces the implicit state passing in issue praxis-proxy/praxis#42's
ExecutionContext. Every step reads some fields and writes
others.

pub struct AgenticState {
    // -- Identity --
    pub request: Value,
    pub response_id: String,
    pub conversation_id: Option<String>,
    pub tenant_id: Option<String>,

    // -- Routing flags --
    pub store_enabled: bool,
    pub stream_enabled: bool,
    pub background: bool,

    // -- Conversation --
    pub messages: Vec<Value>,

    // -- Tools --
    pub tools: Vec<Value>,
    pub tool_choice: Value,
    pub tool_calls: Vec<Value>,
    pub mcp_tool_map: HashMap<String, Value>,
    pub previous_tools: Vec<Value>,

    // -- Output --
    pub output_items: Vec<Value>,
    pub response_object: Value,
    pub usage: Value,
    pub reasoning: Option<String>,
    pub status: ResponseStatus,

    // -- Loop --
    pub iteration: u32,

    // -- Runtime --
    pub config: AgenticConfig,
}

pub enum ResponseStatus {
    Queued,
    InProgress,
    Completed,
    Incomplete(String),
    Failed(String),
    Cancelled,
}

pub enum LoopDecision {
    Continue,
    Done,
    Incomplete(String),
}

Gateway integration: Praxis wraps this in
Arc<Mutex<AgenticState>> and stores it in filter_metadata.
Each filter locks, calls the core function with &mut state,
and unlocks. Standalone mode uses &mut state directly.

---

Step Functions

Step 0: validate_request

pub fn validate_request(
    body: &Value,
    config: &AgenticConfig,
) -> Result<AgenticState, AgenticError>

Parse the incoming Responses API request. Extract fields the
orchestrator needs. Generate response_id. Create and return
a fully initialized AgenticState.

Validates only what the orchestrator acts on:

• stream — response delivery mode
• store — persistence flag
• background — async processing flag
• model — routing (use default if absent)
• previous_response_id — triggers rehydration downstream
• tools — tool types present (determines which executors to invoke)
• tool_choice — dispatch behavior
• Combination constraints: reject stream=true && background=true,
  reject background=true && store=false

Does NOT validate: inference server requirements, nested field
structures, parameter types or ranges. Unknown fields preserved
in request for forwarding.

Gateway filter: request_validate

---

Step 1: init_store

pub async fn init_store(
    state: &AgenticState,
    store: &dyn ResponseStore,
) -> Result<(), AgenticError>

Initialize store connection, create initial response record if
store_enabled. Makes the store available for downstream steps
(rehydrate reads, compact reads/writes, persist writes).

Gateway filter: response_store (request phase)

---

Step 2: rehydrate_conversation

pub async fn rehydrate_conversation(
    state: &mut AgenticState,
    store: &dyn ResponseStore,
) -> Result<(), AgenticError>

Load conversation history from previous_response_id or
conversation_id. Reconstruct message history. Recover MCP
tool listings from previous responses.

Reads: state.request (previous_response_id, conversation_id),
state.tenant_id

Writes: state.messages (loaded history + current input
appended), state.previous_tools (recovered MCP tool listings)

Behavior:

• If previous_response_id: fetch stored response + hidden
  messages, prefer stored messages over public input/output
  reconstruction, reject if previous response is
  incomplete/background/in-progress, extract previous_usage
  for auto-compaction
• If conversation_id (no previous_response_id): fetch stored
  conversation messages, append current input
• If neither: pass current input through as-is

Gateway filter: rehydrate

---

Step 3: resolve_files

pub async fn resolve_files(
    state: &mut AgenticState,
    file_store: &dyn FileStore,
) -> Result<(), AgenticError>

Walk all messages in state.messages. Resolve file_id to
base64 data URLs via FileStore. Wrap file_data in data URL
format. Pass file_url and image_url through. Preserve
input_file.filename for citations and input_image.detail
level.

Gateway filter: file_resolve

---

Step 4: parse_tools

pub async fn parse_tools(
    state: &mut AgenticState,
    mcp_client: &dyn McpToolProvider,
) -> Result<(), AgenticError>

Parse tool definitions from the request. For MCP tools: check
state.previous_tools for reusable listings, call tools/list
via McpToolProvider if not reusable, reject duplicate tool
names across servers, apply allowed_tools filter. Synthesize
function tool definitions for built-in tools (web_search,
file_search). Build mcp_tool_map (tool_name to server
config). Normalize tool_choice.

Writes: state.tools, state.tool_choice, state.mcp_tool_map

Gateway filter: tool_parse

---

Step 5: call_inference

pub async fn call_inference(
    state: &AgenticState,
    client: &dyn InferenceClient,
) -> Result<(InferenceStream, BackendFormat), AgenticError>

Build the inference request from state.messages, state.tools,
state.tool_choice. Delegate to the InferenceClient trait
which handles both native /v1/responses pass-through and
/v1/chat/completions conversion internally. Return the raw
stream and its format so transform_stream knows whether to
pass through or convert.

InferenceStream is
Pin<Box<dyn Stream<Item = Result<Bytes, AgenticError>> + Send>>
— raw SSE bytes from the backend.

BackendFormat is an enum: Responses | ChatCompletions.

Gateway filter: responses_proxy

---

Step 6: transform_stream

pub fn transform_stream(
    state: &mut AgenticState,
    raw_stream: InferenceStream,
    format: BackendFormat,
) -> Result<ResponsesEventStream, AgenticError>

The SSE state machine. Consumes the raw InferenceStream and
produces a ResponsesEventStream yielding typed
ResponsesEvent values.

When format is Responses: events pass through with
minimal transformation (assign sequence numbers, accumulate
state).

When format is ChatCompletions: full transformation —
track whether message_item_added / content_part_emitted /
reasoning_part_emitted, emit all 24 Responses API event types,
accumulate text content, tool call arguments, usage.

As each event is yielded, the function simultaneously
accumulates into state: tool_calls, output_items,
usage, response_object, reasoning, status. After the
stream is fully consumed, these fields are populated for
downstream steps.

pub enum ResponsesEvent {
    ResponseCreated(Value),
    ResponseInProgress(Value),
    ResponseCompleted(Value),
    ResponseIncomplete(Value),
    ResponseFailed(Value),
    OutputItemAdded(Value),
    OutputItemDone(Value),
    ContentPartAdded(Value),
    ContentPartDone(Value),
    OutputTextDelta { delta: String, sequence_number: u64 },
    OutputTextDone { text: String, annotations: Vec<Value>, sequence_number: u64 },
    OutputTextAnnotationAdded(Value),
    FunctionCallArgumentsDelta { delta: String, sequence_number: u64 },
    FunctionCallArgumentsDone { arguments: String, sequence_number: u64 },
    RefusalDelta { delta: String, sequence_number: u64 },
    RefusalDone { refusal: String, sequence_number: u64 },
    ReasoningDelta { delta: String, sequence_number: u64 },
    ReasoningDone(Value),
    ReasoningSummaryTextDelta { delta: String, sequence_number: u64 },
    ReasoningSummaryTextDone(Value),
    ReasoningSummaryPartAdded(Value),
    ReasoningSummaryPartDone(Value),
    Error(Value),
}

Gateway filter: stream_events

---

Step 7: dispatch_tools

pub async fn dispatch_tools(
    state: &mut AgenticState,
    mcp: &dyn McpToolExecutor,
    web_search: &dyn WebSearchProvider,
    file_search: &dyn VectorStoreClient,
) -> Result<LoopDecision, AgenticError>

Classify each tool call in state.tool_calls:

• Function tool (client-side) — add to output items, do
  NOT loop
• MCP tool — execute via McpToolExecutor
• web_search — execute via WebSearchProvider
• file_search — execute via VectorStoreClient

Mixed tool call handling: if a turn contains both
server-side and client-side tool calls, execute server-side
tools first, then exit the loop so the client can provide
function_call_output items in a follow-up request.

After executing server-side tools: append tool-result messages
to state.messages, add tool result items to
state.output_items, increment state.iteration, reset
state.tool_choice to auto after first iteration.

Returns LoopDecision:

• Continue — all tool calls were server-side, results ready,
  loop back to inference
• Done — no tool calls, or client-side function calls present
• Incomplete(reason) — iteration limit hit or finish_reason == "length"

Loop control in standalone mode: the caller checks
LoopDecision and loops.

Loop control in gateway mode: the filter translates
Continue to filter_results.set("action", "loop") and
Done to filter_results.set("action", "done"). The
gateway's branch chain evaluator handles re-entry at
call_inference.

Gateway filter: tool_dispatch

---

Step 8: execute_mcp_tool

pub async fn execute_mcp_tool(
    tool_call: &Value,
    server_config: &Value,
    sessions: &dyn McpSessionManager,
) -> Result<Value, AgenticError>

Execute a single MCP tool call. Look up server config from
mcp_tool_map. Check approval policy. Reuse or create MCP
session (keyed on endpoint + headers hash). Call tools/call
on MCP server. Parse result (text content, image content).
Return tool result message.

Called by dispatch_tools for each MCP tool call, not directly
by the orchestrator.

Gateway filter: mcp_tool

---

Step 9: execute_web_search

pub async fn execute_web_search(
    tool_call: &Value,
    provider: &dyn WebSearchProvider,
) -> Result<Value, AgenticError>

Execute a web search. Normalize all web_search type variants.
Pass search_context_size to provider. Return formatted
results as tool result message.

Gateway filter: web_search

---

Step 10: execute_file_search

pub async fn execute_file_search(
    tool_call: &Value,
    vector_store: &dyn VectorStoreClient,
) -> Result<(Value, HashMap<String, String>), AgenticError>

Execute file search against vector stores. Fan out across all
configured stores in parallel. Preserve filters,
ranking_options, max_num_results. Extract citation markers
from results. Return (tool result message, citation_files map).

Gateway filter: file_search

---

Step 11: compact_context

pub async fn compact_context(
    state: &mut AgenticState,
    inference: &dyn InferenceClient,
    store: &dyn ResponseStore,
) -> Result<bool, AgenticError>

Count tokens via provider usage (from previous_usage) or
tiktoken estimate. If threshold exceeded: build summarization
prompt from conversation messages (prepend instructions),
call inference with compaction_model, replace conversation
history with compaction item + preserved user messages, store
compaction response as hidden entry.

Returns true if compaction occurred.

Gateway filter: compact

---

Step 12: summarize_reasoning

pub async fn summarize_reasoning(
    state: &mut AgenticState,
    inference: &dyn InferenceClient,
) -> Result<Option<Vec<ResponsesEvent>>, AgenticError>

Run after streaming completes and after the tool loop exits.
If reasoning.generate_summary is configured and reasoning
content was accumulated: build summary prompt, make second
inference call, generate reasoning_summary_text events.
Add reasoning item to output items.

Returns Some(events) if a summary was generated (the caller
forwards these events to the client), None otherwise.

Gateway filter: reasoning

---

Step 13: persist_response

pub async fn persist_response(
    state: &AgenticState,
    store: &dyn ResponseStore,
) -> Result<(), AgenticError>

Final persistence step. Save state.response_object,
state.messages (including hidden messages), and conversation
state to the store. Skip when state.store_enabled is false.

During streaming, incremental upserts are handled separately
(via ResponseStore::upsert_streaming_event). This function
handles the terminal state.

Gateway filter: response_store (response phase)

---

Convenience: execute

pub async fn execute(
    body: Value,
    config: AgenticConfig,
    deps: &AgenticDeps,
) -> Result<Value, AgenticError>

Standalone entry point. Composes all steps with default loop
logic:

state = validate_request(body, config)
init_store(&state, deps.store)
rehydrate_conversation(&mut state, deps.store)
resolve_files(&mut state, deps.file_store)
compact_context(&mut state, deps.inference, deps.store)
parse_tools(&mut state, deps.mcp_provider)

loop {
    let (stream, format) = call_inference(&state, deps.inference)?
    let events = transform_stream(&mut state, stream, format)?
    drain_stream(events)  // consume and accumulate

    match dispatch_tools(&mut state, ...)? {
        LoopDecision::Continue => continue,
        LoopDecision::Done => break,
        LoopDecision::Incomplete(r) => {
            state.status = ResponseStatus::Incomplete(r);
            break;
        }
    }
}

summarize_reasoning(&mut state, deps.inference)
persist_response(&state, deps.store)
assemble_final_response(&state)

AgenticDeps bundles all trait objects:

pub struct AgenticDeps {
    pub store: Arc<dyn ResponseStore>,
    pub file_store: Arc<dyn FileStore>,
    pub inference: Arc<dyn InferenceClient>,
    pub mcp_provider: Arc<dyn McpToolProvider>,
    pub mcp_executor: Arc<dyn McpToolExecutor>,
    pub web_search: Arc<dyn WebSearchProvider>,
    pub vector_store: Arc<dyn VectorStoreClient>,
}

---

Trait Definitions

ResponseStore

Expanded from issue praxis-proxy/praxis#42 with tenant isolation and conversation
message support.

#[async_trait]
pub trait ResponseStore: Send + Sync {
    // Response CRUD
    async fn insert_response(&self, tenant_id: Option<&str>,
        response: &Value) -> Result<(), AgenticError>;
    async fn get_response(&self, tenant_id: Option<&str>,
        response_id: &str) -> Result<Option<Value>, AgenticError>;
    async fn update_response(&self, tenant_id: Option<&str>,
        response_id: &str, update: &Value) -> Result<(), AgenticError>;
    async fn delete_response(&self, tenant_id: Option<&str>,
        response_id: &str) -> Result<bool, AgenticError>;
    async fn list_responses(&self, tenant_id: Option<&str>,
        limit: u32, cursor: Option<&str>) -> Result<Value, AgenticError>;

    // Hidden messages (source of truth for future turns)
    async fn get_messages(&self, tenant_id: Option<&str>,
        response_id: &str) -> Result<Option<Vec<Value>>, AgenticError>;
    async fn store_messages(&self, tenant_id: Option<&str>,
        response_id: &str, messages: &[Value]) -> Result<(), AgenticError>;

    // Conversation-level message cache
    async fn get_conversation_messages(&self, tenant_id: Option<&str>,
        conversation_id: &str) -> Result<Option<Vec<Value>>, AgenticError>;
    async fn store_conversation_messages(&self, tenant_id: Option<&str>,
        conversation_id: &str, messages: &[Value]) -> Result<(), AgenticError>;

    // Input items with pagination
    async fn list_input_items(&self, tenant_id: Option<&str>,
        response_id: &str, limit: u32,
        cursor: Option<&str>) -> Result<Value, AgenticError>;

    // Streaming incremental upsert
    async fn upsert_streaming_event(&self, tenant_id: Option<&str>,
        response_id: &str, event: &ResponsesEvent) -> Result<(), AgenticError>;
}

Implementations: SQLite (dev/single-node), PostgreSQL
(production), InMemory (unit tests).

InferenceClient

Expanded to return stream + backend format.

#[async_trait]
pub trait InferenceClient: Send + Sync {
    async fn call(
        &self,
        request: &Value,
        config: &AgenticConfig,
    ) -> Result<(InferenceStream, BackendFormat), AgenticError>;
}

pub type InferenceStream =
    Pin<Box<dyn Stream<Item = Result<Bytes, AgenticError>> + Send>>;

pub enum BackendFormat {
    Responses,
    ChatCompletions,
}

The conversion between Responses API and Chat Completions
request format lives inside the InferenceClient
implementation, not in the core function. This keeps
call_inference clean — it builds the request from state
and delegates.

Implementations:

• VllmClient — calls vLLM /v1/responses or
  /v1/chat/completions
• HttpProxyClient — generic HTTP proxy to any
  OpenAI-compatible backend
• In Praxis mode: the filter sets upstream headers and
  lets Pingora's proxy handle the HTTP call; the
  InferenceClient trait is not used directly

FileStore

#[async_trait]
pub trait FileStore: Send + Sync {
    async fn get_file(
        &self,
        tenant_id: Option<&str>,
        file_id: &str,
    ) -> Result<Option<FileContent>, AgenticError>;
}

pub struct FileContent {
    pub data: Vec<u8>,
    pub mime_type: String,
    pub filename: Option<String>,
}

Implementations: Local filesystem, S3-compatible,
database BLOB, OpenAI Files API proxy.

McpToolProvider

Tool listing and discovery. Separate from execution.

#[async_trait]
pub trait McpToolProvider: Send + Sync {
    async fn list_tools(
        &self,
        server_url: &str,
        headers: &HashMap<String, String>,
    ) -> Result<Vec<Value>, AgenticError>;
}

Implementations:

• Direct MCP client (calls tools/list over Streamable
  HTTP)
• Praxis MCP gateway catalog (reads from the gateway's
  aggregated catalog)

McpToolExecutor

#[async_trait]
pub trait McpToolExecutor: Send + Sync {
    async fn execute(
        &self,
        tool_name: &str,
        arguments: &Value,
        server_config: &Value,
        sessions: &dyn McpSessionManager,
    ) -> Result<Value, AgenticError>;
}

McpSessionManager

pub trait McpSessionManager: Send + Sync {
    fn get_or_create_session(
        &self,
        endpoint_key: &str,
        server_url: &str,
        headers: &HashMap<String, String>,
    ) -> Result<Arc<dyn McpSession>, AgenticError>;
}

#[async_trait]
pub trait McpSession: Send + Sync {
    async fn call_tool(
        &self,
        name: &str,
        arguments: &Value,
    ) -> Result<Value, AgenticError>;

    async fn close(&self) -> Result<(), AgenticError>;
}

Implementations:

• In-process session pool (standalone)
• Praxis MCP session infrastructure from praxis#24 PR 5

WebSearchProvider

#[async_trait]
pub trait WebSearchProvider: Send + Sync {
    async fn search(
        &self,
        query: &str,
        context_size: ContextSize,
    ) -> Result<Value, AgenticError>;
}

pub enum ContextSize {
    Low,
    Medium,
    High,
}

Implementations: Brave Search API, Tavily API, SearXNG.

VectorStoreClient

#[async_trait]
pub trait VectorStoreClient: Send + Sync {
    async fn search(
        &self,
        store_id: &str,
        query: &str,
        options: &FileSearchOptions,
    ) -> Result<Vec<Value>, AgenticError>;
}

pub struct FileSearchOptions {
    pub max_num_results: u32,
    pub filters: Option<Value>,
    pub ranking_options: Option<Value>,
}

Implementations: OpenAI Vector Stores API, Qdrant,
Weaviate, custom vector store.

---

Gateway Filter Mapping

Every Praxis HttpFilter is a thin wrapper: read from
HttpFilterContext, call one core function, write results
back to context.

#	Gateway Filter	Core Function	Core Trait Deps
0	request_validate	validate_request()	—
1	response_store	init_store() + persist_response()	ResponseStore
2	rehydrate	rehydrate_conversation()	ResponseStore
3	file_resolve	resolve_files()	FileStore
4	tool_parse	parse_tools()	McpToolProvider
5	responses_proxy	call_inference()	InferenceClient
6	stream_events	transform_stream()	—
7	tool_dispatch	dispatch_tools()	McpToolExecutor, WebSearchProvider, VectorStoreClient
8	mcp_tool	execute_mcp_tool()	McpSessionManager
9	web_search	execute_web_search()	WebSearchProvider
10	file_search	execute_file_search()	VectorStoreClient
11	compact	compact_context()	InferenceClient, ResponseStore
12	reasoning	summarize_reasoning()	InferenceClient
13	background_jobs	(gateway-specific)	—

Why background_jobs Is Excluded

Background job processing involves:

• Bounded tokio channel (queue capacity 100)
• Long-lived worker tasks pulling from the queue
• Per-request timeout enforcement
• Cancellation via POST /v1/responses/{id}/cancel
• Status polling via GET /v1/responses/{id}

This is gateway infrastructure (concurrency management, HTTP
endpoint handlers, task lifecycle), not domain logic. The core
provides the building blocks:

• validate_request rejects background=true && store=false
• init_store creates the initial response with status=queued
• The full step function pipeline runs inside the worker
  (calling the same core functions)

The gateway owns the queue, workers, and cancellation. A
different gateway (not Praxis) would implement these differently.

---

Streaming Architecture

The streaming design requires three layers:

 InferenceClient::call()        transform_stream()         Caller
 ─────────────────────────     ───────────────────────     ─────────
 Raw SSE bytes from backend → Typed ResponsesEvent stream → Forward to client
                              + accumulate into state        OR collect internally

Layer 1: InferenceStream — raw SSE bytes from the
backend. What InferenceClient::call() returns. Unparsed
data: lines.

Layer 2: transform_stream — consumes InferenceStream,
produces ResponsesEventStream. For each raw chunk:

1. Parse it (Chat Completions chunk or Responses event)
2. Transform to typed ResponsesEvent
3. Accumulate into state (text, tool calls, usage)
4. Yield the event downstream

Layer 3: Caller consumption — drives the
ResponsesEventStream:

• Standalone streaming: execute() forwards events to
  an SSE writer AND accumulates for tool detection
• Standalone non-streaming (stream=false): execute()
  collects all events internally, returns final
  ResponseResource as JSON
• Gateway mode: the stream_events filter forwards
  each event to the client via on_response_body AND writes
  accumulated state to filter_metadata for tool_dispatch
  and response_store

This is a tee pattern: the stream transforms and accumulates
simultaneously. No separate pass is needed to detect tool calls.

---

Loop Control Architecture

dispatch_tools returns a LoopDecision enum. The caller
translates it to its own control flow mechanism.

Standalone Mode

loop {
    let (stream, format) = call_inference(&state, &deps.inference).await?;
    let events = transform_stream(&mut state, stream, format)?;
    drain_stream(events).await;

    match dispatch_tools(&mut state, &deps.mcp_executor,
        &deps.web_search, &deps.vector_store).await?
    {
        LoopDecision::Continue => continue,
        LoopDecision::Done => break,
        LoopDecision::Incomplete(reason) => {
            state.status = ResponseStatus::Incomplete(reason);
            break;
        }
    }
}

Gateway Mode (Praxis Branch Chains)

- filter: tool_dispatch
  branch_chains:
    - name: tool-loop
      on_result:
        filter: tool_dispatch
        key: action
        result: loop          # matches LoopDecision::Continue
      rejoin: responses_proxy # re-enter at call_inference
      max_iterations: 10

The tool_dispatch filter translates:

• LoopDecision::Continue →
  filter_results.set("action", "loop")
• LoopDecision::Done →
  filter_results.set("action", "done")
• LoopDecision::Incomplete(reason) →
  filter_results.set("action", "done") +
  filter_metadata.set("responses.status", "incomplete")

The core function has no knowledge of branch chains, HTTP,
or Praxis. It returns a typed enum. The filter translates.

---

Error Handling

pub enum AgenticError {
    Validation(String),
    Store(String),
    Inference(String),
    Tool(String),
    Stream(String),
    Compaction(String),
    FileResolution(String),
    Mcp(String),
    IterationLimit(u32),
    Cancelled,
    Internal(String),
}

Each variant maps to Responses API error codes. The gateway
translates AgenticError to the appropriate HTTP status and
error response format.

---

Implementation Tiers

Build order — each tier produces a working system:

Tier	Functions	What Works After	External Deps
0	validate_request, init_store, call_inference, transform_stream, persist_response	Stateless proxy with persistence. Text streaming. CRUD.	Inference backend, SQL
1	+ rehydrate_conversation	Multi-turn via previous_response_id and conversation_id	—
2	+ parse_tools, dispatch_tools	Tool definitions parsed, loop control. Client-side function tools.	—
3	+ execute_web_search	Server-side web search	Search API
4	+ execute_mcp_tool	MCP tool execution with session reuse	MCP servers
5	+ execute_file_search	Vector store search with citations	Vector Store API
6	+ resolve_files	File references resolved to inline content	Files API
7	+ summarize_reasoning	Reasoning summarization	Inference backend
8	+ compact_context	Context window management	Inference backend
9	+ execute	Standalone orchestrator	—

---

What This Adds Over Issue praxis-proxy/praxis#42

Dimension	Issue praxis-proxy/praxis#42	This Proposal
Step functions	6	14
Traits	3 (ResponseStore, InferenceClient, ToolRegistry)	8 (+ FileStore, McpToolProvider, McpToolExecutor, McpSessionManager, WebSearchProvider, VectorStoreClient)
Shared state	None (implicit via ExecutionContext)	AgenticState struct
Streaming type	Stream<InferenceEvent>	InferenceStream (raw) + ResponsesEventStream (typed) + ResponsesEvent enum (24 variants)
Loop control	Caller responsibility (untyped)	LoopDecision enum
Error types	Single Error	AgenticError with 11 variants
Gateway mapping	Implicit	Explicit 1:1 table
Implementation order	5-phase plan	10-tier plan matching Praxis
Background jobs	Not addressed	Explicitly excluded with rationale
Multi-tenancy	Not addressed	tenant_id on all store operations

---

Open Questions

1. CompactionStrategy trait: Should token counting and
   compaction thresholds be a trait (pluggable strategies) or
   hardcoded with configuration? Praxis uses tiktoken + a
   configurable threshold. A trait adds flexibility for
   provider-specific token counting but may be over-engineering
   for the initial implementation.

2. MCP session ownership: McpSessionManager lives on
   AgenticState (per-request scope) in Praxis's design. Should
   it be per-request or per-process? Per-request is simpler but
   means sessions are not reused across requests. Per-process
   requires a shared session pool that outlives individual
   requests.

3. transform_stream mutability: The function needs to
   write to state as it yields events. With the tee pattern,
   this means the stream closure captures &mut state. This
   works in standalone mode but in gateway mode the filter may
   need Arc<Mutex<AgenticState>>. Should the function accept
   Arc<Mutex<AgenticState>> directly, or should the gateway
   handle the synchronization wrapper?

4. Inference request building: Should call_inference build
   the inference request internally from AgenticState, or
   should a separate build_inference_request function exist?
   Praxis's responses_proxy does both (build + call), but
   separating them would make the conversion logic independently
   testable.

5. Event emission hooks: Several steps emit SSE events
   (mcp_call.in_progress, web_search_call.searching, etc.)
   during execution, not just at the end. Should these be
   returned as part of the result, or should there be a callback
   mechanism? Praxis emits them inline via stream_events; the
   core needs a way to surface them without depending on SSE
   framing.

---

Relationship to Existing PRs

PR	Relationship
praxis-proxy/praxis#33 (store CRUD)	Becomes a ResponseStore trait implementation. Schema aligns with insert_response/get_response/get_messages.
praxis-proxy/praxis#34 (OGX loop)	The agentic loop becomes the reference for execute(). Tool dispatch pattern informs dispatch_tools.
RFC praxis-proxy/praxis#18 (KV cache)	AgenticConfig is extensible — a session_cache_hints field can be added without breaking the API.

---

References

• ADR-03: Gateway Integration
• Issue praxis-proxy/praxis#42: agentic-core public API
• Praxis praxis-proxy/ai#93: Responses API Filters

[franciscojavierarceo]

@ashwing any chance we can open a PR about this? sorry i didn't see this until now, i guess I don't get pinged about issue being created

[franciscojavierarceo]

also i guess the long posts are hard to review manually

[ashwing]

@leseb this is a great expansion — the Praxis praxis-proxy/ai#93 alignment and per-tool-type trait split make a lot of sense. A few things I'm adopting from your proposal, a couple of questions, and a phased plan to ship incrementally.

What I'm adopting

• AgenticState (mutable, request-scoped) over my original ExecutionContext. Steps writing directly to shared state is cleaner than passing immutable refs + collecting outputs.
• validate_request as Step 0 — parsing, ID generation, and constraint checking before anything else.
• LoopDecision enum — explicit Continue/Done/Incomplete instead of implicit max-iterations.
• Split tool traits — McpToolExecutor, WebSearchProvider, VectorStoreClient as separate traits. The lifecycle differences are real.
• transform_stream — the SSE state machine with 24 event types and simultaneous accumulation is the hardest piece and clearly belongs as its own step.
• BackendFormat enum — supporting both /v1/responses passthrough and /v1/chat/completions conversion cleanly.

Questions

1. Phasing: Can we land the core loop first (Steps 0, 2, 5-7, 13 + execute) and add Steps 3, 4, 8-12 incrementally? Trying to ship all 14 functions in one PR would be a massive review. Happy to commit to a concrete timeline for the phases.

2. Praxis Epic: Responses API and Agentic Loop Orchestration praxis-proxy/ai#93 status: Is that proposal accepted/merging, or still in design? Asking because it affects how tightly we should couple — I want the design to be compatible with Epic: Responses API and Agentic Loop Orchestration praxis-proxy/ai#93's filter decomposition without being dependent on an unmerged spec.

3. Tool executor granularity: Your proposal has execute_mcp_tool, execute_web_search, execute_file_search as top-level public functions. Would it work to have these as methods on their respective traits (called internally by dispatch_tools) instead? My concern is that adding a new tool type (e.g., code_interpreter) would change the public function surface. If they're trait methods, the dispatch logic stays stable:

   // dispatch_tools calls trait methods internally:
   match tool_type {
       "mcp" => mcp_executor.execute(call, sessions).await,
       "web_search" => web_search.search(query, context_size).await,
       "file_search" => vector_store.search(store_id, query, opts).await,
       _ => Err(AgenticError::UnknownTool(tool_type)),
   }

4. compact_context / summarize_reasoning: These add inference call latency. Should they be mandatory steps in the default execute() loop, or opt-in via AgenticConfig flags? I'd lean toward opt-in for MVP (most deployments won't have compaction models configured initially).

Proposed Phased Plan

Phase A (first PR — core loop):
validate_request → rehydrate_conversation → call_inference → transform_stream → dispatch_tools → persist_response → execute

Types: AgenticState, AgenticConfig, LoopDecision, ResponsesEvent, BackendFormat
Traits: ResponseStore, InferenceClient, McpToolExecutor, WebSearchProvider, VectorStoreClient

Phase B (follow-up — tool executors):
MCP session management, McpToolProvider, web search provider, vector store fan-out

Phase C (follow-up — advanced):
resolve_files + FileStore, compact_context, summarize_reasoning, parse_tools, init_store

Phase D (follow-up — streaming integration):
Wire transform_stream to SSE output in agentic-server, tee pattern, non-streaming collection

---

I'll rework PR praxis-proxy/praxis#44 to align with Phase A once we agree on the questions above. @franciscojavierarceo — the PR is already up as a draft (#44), happy to undraft once the direction is confirmed.

[franciscojavierarceo]

@ashwing i guess my suggestion was that proposals are easier to comment on inline as a PR :)

[ashwing]

@franciscojavierarceo good call — added the full design doc as docs/design/core-public-api.md in PR praxis-proxy/praxis#44 so you can comment inline on specific sections. Should be much easier to review than the issue body.