From 993d3c53489291e2542549244d1b655838d76f6d Mon Sep 17 00:00:00 2001 From: Cursor Agent Date: Thu, 16 Jul 2026 15:04:05 +0000 Subject: [PATCH 01/10] docs: add Axion + PolyVerdict production readiness plan Spec the highest-priority fixes for OSS deploy of Phase 1 Lens, with PolyVerdict sequenced as opt-in enforce mode after the observe path is honest and usable. No runtime code changes yet. Co-authored-by: Moses Man --- PLAN.md | 274 ++++++++++++++++++++++++++++++++++++++++++++ SPEC-PolyVerdict.md | 1 + SPEC.md | 4 +- 3 files changed, 278 insertions(+), 1 deletion(-) create mode 100644 PLAN.md diff --git a/PLAN.md b/PLAN.md new file mode 100644 index 0000000..970af54 --- /dev/null +++ b/PLAN.md @@ -0,0 +1,274 @@ +# Axion + PolyVerdict — production readiness plan + +> Plan only. Do not implement from this doc until the build order below is accepted. +> Scope: open-source Phase 1 (Axion Lens) shippable on Cloudflare Workers, plus an honest sequencing decision for PolyVerdict. + +--- + +## Verdict + +The proxy hot path is the strongest part of the repo: tee the response, return it immediately, extract in `waitUntil`. That part is close to real infrastructure. + +What is not shippable is the product around it. The dashboard cannot list or render beliefs. Auth is an open relay if you put a secret in the Worker. Docs sell a belief DAG, Anthropic drop-in, and key passthrough that the code does not provide. PolyVerdict is a proposal with zero code, and its retry/block model fights Lens’s zero-latency observe path. + +For OSS, the first job is to make Phase 1 true: a working OpenAI-compatible observe proxy, a durable per-session belief timeline, a dashboard that shows it, and docs that match. PolyVerdict comes after that, as an opt-in enforce mode, not as default Lens middleware. + +--- + +## What exists today + +``` +Agent → POST /v1/chat/completions + → Worker tees body (ReadableStream.tee) + → caller gets upstream stream immediately + → waitUntil → regex extract → DO append batch + → dashboard tries /api/sessions (404) and expects flat beliefs (gets nested batches) +``` + +| Piece | Status | +|---|---| +| OpenAI chat proxy + SSE tee | Real | +| `waitUntil` belief extraction | Real (quality thin) | +| Session DO storage | Real append log via Durable Object storage | +| Belief DAG / root-cause | Types only | +| Dashboard | UI exists; API contract broken | +| Anthropic `/v1/messages` | Not implemented | +| Caller key passthrough | Documented, not implemented | +| PolyVerdict | `SPEC-PolyVerdict.md` only | +| `npm test` / CI | Vitest dep present; no test script; one stream test file | + +--- + +## Decisions to lock before build + +These are product calls. Wrong defaults will force rework. + +### D1. Auth model (pick one) + +| Option | Meaning | Fit | +|---|---|---| +| **A. Passthrough (recommended for OSS)** | Forward caller `Authorization` / `x-api-key`. Worker holds no model key. Fail closed if neither caller key nor optional server key is present. | Matches “point agent at Axion” and avoids an open credit relay. | +| **B. Server key + proxy token** | Worker holds `UPSTREAM_API_KEY`. Every call requires an Axion token. | Better for a hosted SaaS later; heavier for self-host OSS. | + +Default for this plan: **A**, with optional server key as fallback only when explicitly configured. + +### D2. Phase 1 data model honesty + +Ship a **flat chronological belief timeline**, not a DAG. + +Keep Durable Objects as the session owner. Persist ordered events (already closer to storage than the “in-memory Map, lost on eviction” story). Defer `parentIds` / edges / root-cause until extract or a post-pass can justify links. + +Delete or clearly mark unused `BeliefDAG` / edge APIs as planned. Do not advertise root-cause until it exists. + +### D3. Provider scope for Phase 1 + +Phase 1 ships **OpenAI-compatible `/v1/chat/completions` only**. + +Anthropic Messages becomes Phase 1.1 behind a provider adapter. Remove Claude Code / `ANTHROPIC_BASE_URL` claims from README until that adapter lands. + +### D4. PolyVerdict placement + +PolyVerdict is Gate-shaped: validate, retry, coerce, possibly block. Lens is observe-shaped: never delay the agent. + +Same Worker package later is fine. Same default request path is not. Sequence PolyVerdict as an **opt-in enforce mode** after Lens contracts are honest, behind a provider/content adapter, with its own latency budget. + +--- + +## Build waves (ordered) + +### Wave 0 — Truth and safety (merge before any feature work) + +Make the repo honest and non-dangerous. + +1. **Auth boundary** + - Implement D1 (passthrough + fail closed). + - Align `Env` typing, `wrangler.toml` comments, README Quick Start, and `TECHNICAL.md`. + - Add `.dev.vars.example` with `UPSTREAM_API_URL` / optional `UPSTREAM_API_KEY`. + +2. **Docs vs code** + - Rewrite README / SPEC Phase 1 status to: OpenAI chat proxy + regex timeline + DO session store + local dashboard. + - Move DAG, root-cause, NLP, Anthropic, key passthrough (once implemented, keep), `/api/sessions` (until built), Loop, Gate, PolyVerdict to explicit Planned sections. + - Fix Known Issues: DO uses Durable Object storage (survives eviction); unbounded growth is the real risk. + +3. **OSS hygiene floor** + - Add `"test": "vitest run"` (and keep `typecheck`). + - Minimal GitHub Actions: typecheck + test on PR. + - Stub `CONTRIBUTING.md` + `SECURITY.md` (how to report, that beliefs API is unauthenticated in Phase 1). + +**Exit:** A stranger can deploy without becoming an open model-key proxy, and the README does not lie. + +--- + +### Wave 1 — Make Lens usable end-to-end + +This is the actual Phase 1 product. + +4. **Canonical session identity** + - Pass `{ sessionId }` into `extractBeliefs` from `runExtraction`. + - Document that agents must send `x-axion-session` for multi-turn correlation (or document a fallback “latest” single-session mode for local demo). + - DO GET returns the human session name (the `idFromName` key), never the opaque DO id as the user-facing id. + +5. **Beliefs API contract** + - Public shape: `{ sessionId: string, beliefs: ExtractedBelief[] }` chronological flatten of batches. + - Keep raw batches internal if useful for debugging; do not leak them to the dashboard. + - Treat this JSON as a typed boundary shared by DO → Worker → dashboard. + +6. **Session discovery** + - Either: + - **6a.** Add a small session registry (KV or registry DO write-on-first-use) + `GET /api/sessions`, or + - **6b.** Drop the dropdown and make the dashboard Phase 1 “single session”: paste / read `x-axion-session` (matches SPEC’s “local single-session dashboard” better). + - Recommendation: **6b for OSS MVP**, **6a when multi-session is real**. SPEC already says multi-session is SaaS-later; the UI overreached. + +7. **Dashboard wire-up** + - Consume flattened beliefs. + - Session UX per 6b or 6a. + - Redefine or remove “wrong beliefs only” until `invalidated` exists (today it is `confidence < 0.4`). + - Confirm Workers Assets serve `/app.js` and `/styles.css` (or route them explicitly). + +8. **Content normalization before extract** + - Streaming: keep OpenAI delta concat (already). + - Non-streaming: parse `choices[0].message.content` (and refuse to scan raw JSON envelopes). + - Isolate behind a small `extractAssistantText(responseMode, bytes)` helper so Anthropic can plug in later. + +**Exit:** Point an OpenAI-compatible agent at the Worker with a stable `x-axion-session`, open the dashboard, see a real timeline. + +--- + +### Wave 2 — Extraction quality and contracts + +Worth doing once the pipes work; do not block Wave 1 on perfect linguistics. + +9. **Pattern / confidence honesty** + - Fix `because of` group capture. + - Either populate `evidence` via `evidenceGroup` or stop claiming the field. + - Pick one confidence formula and document it (prefer documented additive modifiers for readability, or update docs to midpoint bands — do not leave both). + - Pass real `sessionId`; stop random per-belief ids. + +10. **Tests at the seams** + - Lens: pattern fixtures (because / because of / intention nesting / no-punctuation). + - Extraction glue: session stamp + content parse for stream and non-stream. + - DO round-trip: store batches → public flatten shape. + - Auth matrix: passthrough vs server key vs neither (fail closed). + - Router: no silent 404 for whatever session UX Wave 1 chose. + +**Exit:** Regressions in the dashboard contract or auth fail CI. + +--- + +### Wave 3 — Provider adapter (Phase 1.1) + +11. **Adapter boundary** + - Interface covering: route match, auth header map, stream event parse, assistant text extract. + - OpenAI adapter = current behavior. + - Anthropic Messages adapter = new route + SSE shape. + - Then restore README Claude Code / Hermes claims with tested instructions. + +**Exit:** At least one non-OpenAI agent path is real, or docs stay OpenAI-only. + +--- + +### Wave 4 — PolyVerdict (after Lens is honest) + +Do not start until Waves 0–1 are done and Wave 2 tests exist. Prefer Wave 3 adapter first so schema enforcement is not OpenAI-only forever. + +12. **PolyVerdict as opt-in enforce mode** + - Trigger: `x-schema` / `response_format` JSON Schema (draft 2020-12). + - Control flow: hold → validate → retry upstream (max 3) with violation hints → optional coerce → return. + - Explicitly **not** the Lens tee path. New code path that may add latency; success criteria keep `<200ms` for syntax-only pass, zero extra latency only when schema passes first try (meaning: validate after full body for non-stream, or buffered validate for stream — decide and document; do not pretend tee + mutate is free). + - Schema registry DO (named schemas) as a separate binding from session beliefs. + - Hash cache for identical schema+prompt skip. + - Semantic / PolyGnosis verification stays opt-in and budget-capped (Phase later inside PolyVerdict). + +13. **Composition with Lens** + - When enforce mode is on: validate first, then Lens extracts from the **delivered** (possibly coerced) text. + - When off: today’s observe path unchanged. + - Same package, mode switch — not a silent middleware wrap of every request. + +**Exit:** Schema-gated chat completions work on OpenAI path with tests for pass / fail-retry / coerce; Lens still observes. + +--- + +### Explicitly later (do not sneak into OSS Phase 1) + +| Item | Why later | +|---|---| +| Belief DAG + root-cause | Needs justified edges and failure signals | +| Axion Loop | Needs stable multi-turn sessions + embeddings | +| Axion Gate (tool-call block) | Needs plan extraction + intervene path | +| Hosted multi-session SaaS | Out of OSS core per SPEC | +| Langfuse / Arize export | Easy after flat JSON is stable | +| Semantic PolyVerdict | Costly; after syntax path | + +--- + +## Priority matrix + +| Priority | Fix | Wave | +|---|---|---| +| P0 | Auth fail-closed + passthrough (or token gate) | 0 | +| P0 | Docs truth-align (no DAG / Anthropic / passthrough lies) | 0 | +| P0 | Flatten beliefs API + dashboard session UX | 1 | +| P0 | Stamp `sessionId` on extract | 1 | +| P0 | Non-stream content parse before extract | 1 | +| P1 | `npm test` + CI + env example | 0–2 | +| P1 | Pattern / evidence / confidence honesty | 2 | +| P1 | OpenAI-only README until adapter exists | 0 / 3 | +| P2 | Anthropic adapter | 3 | +| P2 | CONTRIBUTING / SECURITY | 0 | +| P3 | PolyVerdict enforce mode | 4 | +| P3 | DAG / Loop / Gate | later | + +--- + +## Architecture sketch after Waves 0–1 + +``` +Agent (OpenAI-compatible, x-axion-session) + ↕ +Axion Worker + ├── Auth: passthrough or configured server key (fail closed) + ├── POST /v1/chat/completions → upstream → tee → waitUntil extract + ├── GET /api/beliefs/:sessionId → flat ExtractedBelief[] + ├── Dashboard: paste/select session → timeline + └── SessionDurableObject: durable ordered belief batches (internal) +``` + +After Wave 4 (optional): + +``` + ├── Observe mode (default): tee + Lens + └── Enforce mode (x-schema): validate/retry/coerce → then Lens on delivered text +``` + +--- + +## Suggested first build PR sequence (when we exit plan mode) + +1. `fix(auth+docs): fail-closed passthrough + honest README/SPEC` +2. `fix(api): flatten beliefs + session UX + stamp sessionId` +3. `fix(extract): non-stream content parse + because-of / evidence` +4. `chore(ci): vitest script + GH Actions` +5. (optional) `feat(providers): Anthropic messages adapter` +6. (later) `feat(polyverdict): opt-in schema enforce mode` + +Do not combine 1–3 with PolyVerdict in one PR. + +--- + +## Open questions for the next turn + +Answer these when we leave plan mode; defaults above apply if silent: + +1. Auth: confirm **passthrough (A)** vs **server key + token (B)**. +2. Dashboard: confirm **paste session id (6b)** vs **registry + `/api/sessions` (6a)**. +3. Is PolyVerdict in the first production OSS tag at all, or clearly “proposal / Phase 2 enforce”? +4. Keep unused DAG types with `@planned` comments, or delete until needed? + +--- + +## References + +- Product: `SPEC.md`, `TECHNICAL.md`, `README.md` +- PolyVerdict proposal: `SPEC-PolyVerdict.md` +- Runtime: `src/proxy/*`, `src/lens/*`, `src/state/SessionDurableObject.ts`, `src/dashboard/*` +) diff --git a/SPEC-PolyVerdict.md b/SPEC-PolyVerdict.md index 0b1f20d..0bc851a 100644 --- a/SPEC-PolyVerdict.md +++ b/SPEC-PolyVerdict.md @@ -2,6 +2,7 @@ > **Status:** Proposal — potential Axion Gate evolution > **Type:** Feature spec (proxy layer for LLM output compliance) +> **Sequencing:** Not part of Phase 1 OSS. See [PLAN.md](./PLAN.md) Wave 4 — opt-in enforce mode after Lens observe path is shippable. Do not implement as default Lens middleware. ## Problem diff --git a/SPEC.md b/SPEC.md index a7e9233..7413ff6 100644 --- a/SPEC.md +++ b/SPEC.md @@ -84,10 +84,12 @@ State: Durable Object per session (belief DAG in memory) | Phase | What | Status | |---|---|---| -| 1 | Axion Lens - proxy + belief extraction + local dashboard | **Next** | +| 1 | Axion Lens - proxy + belief extraction + local dashboard | **Next** (see [PLAN.md](./PLAN.md)) | | 2 | Axion Loop - embedding detection + intervention injection | Future | | 3 | Axion Gate - tool call interception + verification + blocking | Future | +Production OSS readiness (auth, dashboard contract, docs honesty, PolyVerdict sequencing) is specified in [PLAN.md](./PLAN.md). Do not treat Loop/Gate/PolyVerdict as part of the Phase 1 OSS tag until that plan’s Wave 0–1 exits. + ## Open-Source Scope (Phase 1) This repo contains the open-source core: From dadf0782a1ca205fc96bc38b2f26f607cee698ac Mon Sep 17 00:00:00 2001 From: Cursor Agent Date: Thu, 16 Jul 2026 15:05:43 +0000 Subject: [PATCH 02/10] docs: lock BUILD-SPEC for Axion + PolyVerdict implementation All product decisions locked: passthrough auth, flat timeline, OpenAI+Anthropic adapters, paste-session dashboard, PolyVerdict opt-in enforce mode (syntax+coerce+retry). Ready to build. Co-authored-by: Moses Man --- BUILD-SPEC.md | 223 ++++++++++++++++++++++++++++++++++++++++++++++++++ 1 file changed, 223 insertions(+) create mode 100644 BUILD-SPEC.md diff --git a/BUILD-SPEC.md b/BUILD-SPEC.md new file mode 100644 index 0000000..023d06d --- /dev/null +++ b/BUILD-SPEC.md @@ -0,0 +1,223 @@ +# BUILD-SPEC — Axion + PolyVerdict (locked, ready to build) + +> All product decisions are locked below. Implement exactly this. Do not reopen auth/dashboard/provider/PolyVerdict placement questions. +> Supersedes open questions in `PLAN.md`. + +--- + +## Locked decisions + +| ID | Choice | +|---|---| +| D1 Auth | **Passthrough first.** Forward caller `Authorization` or Anthropic `x-api-key` (+ `anthropic-version`). If caller has no key and `UPSTREAM_API_KEY` secret is set, use the secret. If neither → **401** with clear JSON error. Never send `Bearer undefined`. | +| D2 Data model | **Flat chronological timeline.** DO stores batches internally; public API returns `{ sessionId, beliefs: ExtractedBelief[] }`. Keep `BeliefNode`/`BeliefDAG` types marked `@planned` in comments only — do not implement graph APIs. | +| D3 Providers | **OpenAI chat completions + Anthropic Messages** in this build. Shared provider adapter. | +| D4 Dashboard | **Paste / enter session id** (no `/api/sessions`, no registry). Prefill from `?session=` query or `localStorage`. Show copy hint for `x-axion-session`. | +| D5 Confidence | **Documented additive modifiers**, clamp `[0.1, 1.0]`. Update code to match README (not midpoint bands). | +| D6 “Wrong” filter | Rename UI to **Low confidence** (`confidence < 0.4`). Do not claim invalidation. | +| D7 PolyVerdict | **Opt-in enforce mode** in same Worker when `x-axion-schema` header or `response_format.type === "json_schema"` is present. Separate code path (may buffer). Default requests stay zero-latency tee. | +| D8 Scope of PolyVerdict v1 | Syntax validate + type coerce (string↔number↔boolean) + retry ≤3 with violation hints. No semantic/second-model verify. No schema-registry DO yet (inline schema / header JSON only). No LexGateway. | + +--- + +## Target architecture + +``` +POST /v1/chat/completions → OpenAI adapter → observe OR enforce +POST /v1/messages → Anthropic adapter → observe OR enforce + +Observe (default): + upstream → tee → caller + waitUntil → normalizeAssistantText → extractBeliefs({sessionId}) → DO + +Enforce (x-axion-schema | response_format.json_schema): + loop ≤3: upstream (non-stream forced for enforce) → parse JSON → validate/coerce + on fail: retry with schema + violation hint appended to messages + on success: return coerced JSON as OpenAI/Anthropic-shaped response + waitUntil Lens on delivered assistant text + +GET /api/beliefs/:sessionId → flat ExtractedBelief[] +GET /dashboard → paste session id UI +``` + +--- + +## Module map (create / change) + +| Path | Action | +|---|---| +| `src/proxy/auth.ts` | **New** — resolveUpstreamAuth(request, env) → Headers patch or 401 | +| `src/proxy/providers/types.ts` | **New** — ProviderAdapter interface | +| `src/proxy/providers/openai.ts` | **New** — chat completions route, SSE delta, non-stream content | +| `src/proxy/providers/anthropic.ts` | **New** — `/v1/messages`, SSE `content_block_delta`, non-stream content | +| `src/proxy/providers/index.ts` | **New** — matchProvider(pathname) | +| `src/proxy/content.ts` | **New** — extractAssistantText({ isSse, provider, rawAccumulated }) | +| `src/proxy/index.ts` | **Rewrite routing** — providers, auth, enforce branch | +| `src/proxy/stream.ts` | Flush TextDecoder; keep tee; export helpers for Anthropic SSE parse | +| `src/proxy/extraction.ts` | Pass `{ sessionId }` into extractBeliefs | +| `src/proxy/beliefs.ts` | Expect flat shape from DO (or flatten here) | +| `src/proxy/types.ts` | `UPSTREAM_API_KEY?: string`; optional PolyVerdict flags | +| `src/state/SessionDurableObject.ts` | Store `sessionName`; GET flattens + returns human sessionId | +| `src/lens/patterns.ts` | Fix because-of; evidenceGroup on evidence patterns; punctuation optional where safe | +| `src/lens/extract.ts` | Additive confidence + clamp [0.1,1.0] | +| `src/polyverdict/schema.ts` | **New** — minimal JSON Schema subset validator + coerce | +| `src/polyverdict/enforce.ts` | **New** — retry loop, hint injection | +| `src/dashboard/app.js` | Paste session UX; low-confidence rename | +| `package.json` | `test` script | +| `.github/workflows/ci.yml` | typecheck + test | +| `.dev.vars.example`, `CONTRIBUTING.md`, `SECURITY.md` | New | +| `README.md`, `SPEC.md`, `TECHNICAL.md`, `SPEC-PolyVerdict.md`, `PLAN.md` | Truth-align | + +--- + +## Detailed requirements + +### 1. Auth (`src/proxy/auth.ts`) + +``` +resolveUpstreamCredentials(request, env): + callerAuth = Authorization header (trim) + callerAnthropicKey = x-api-key + serverKey = env.UPSTREAM_API_KEY?.trim() + + if callerAuth: use it (forward as-is) + else if callerAnthropicKey: forward x-api-key + anthropic-version (default 2023-06-01 if missing) + else if serverKey: Authorization: Bearer ${serverKey} + else: throw AuthError 401 "Provide Authorization or x-api-key, or configure UPSTREAM_API_KEY" + +Also forward: OpenAI-Organization, anthropic-version (when Anthropic path), content-type. +``` + +### 2. Provider adapters + +**OpenAI** +- Route: `POST /v1/chat/completions` +- Validate: non-empty `messages[]` +- Upstream path: `/v1/chat/completions` +- Stream text: `choices[0].delta.content` +- Non-stream text: `choices[0].message.content` (string or join text parts) + +**Anthropic** +- Route: `POST /v1/messages` +- Validate: non-empty `messages[]` (Anthropic shape) +- Upstream path: `/v1/messages` +- Stream text: SSE events where `type === "content_block_delta"` and `delta.type === "text_delta"` → `delta.text` +- Non-stream text: join `content[]` blocks with `type === "text"` → `.text` +- Headers: `x-api-key`, `anthropic-version`, `content-type` + +### 3. Stream / content + +- Always `decoder.decode(value, { stream: true })` then final `decoder.decode()` flush. +- For SSE: parse provider-specific deltas into assistant text for extraction. +- For non-SSE: **never** feed raw JSON to lens — parse via provider adapter. + +### 4. Session + DO + +- Proxy sessionId from `x-axion-session` or UUID; echo header always. +- `extractBeliefs(text, { sessionId })`. +- DO storage key `"beliefs"` remains batch array; also store `"sessionName"` on first write. +- `GET /beliefs` response: + ```json + { + "sessionId": "", + "beliefs": [ /* ExtractedBelief flattened chronological */ ] + } + ``` +- Flatten: concatenate each batch’s `beliefs` in storage order. +- Remove “STUB” header; document as Phase 1 timeline store. + +### 5. Lens patterns / confidence + +- Split `because of` and `because` into two patterns OR use group resolution that accepts group 1 or 2. +- Evidence patterns: set `evidenceGroup: 1` and put a short claim in `belief` (e.g. label `"cited evidence"` or the capture duplicated into both belief + evidence — prefer `belief` = capture, `evidence` = capture for evidence-type for dashboard usefulness). +- Confidence: scan markers; apply additive deltas from README: + - definitely/certainly/absolutely: +0.2 + - probably/likely: +0.1 + - might/could be/possibly: −0.2 + - not sure/uncertain/unsure: −0.3 + - clamp to [0.1, 1.0] +- Soften trailing punctuation requirement: allow end-of-string as clause end. + +### 6. Dashboard + +- Replace session ` with a text input + Load button. Prefill from ?session= query or the axion.sessionId localStorage key on mount. Rename the 'Wrong beliefs only' filter to 'Low confidence only' (confidence < 0.4) and guard BeliefCard against missing confidence. Co-authored-by: Moses Man --- src/dashboard/app.js | 102 ++++++++++++++++++++++++++++----------- src/dashboard/styles.css | 39 +++++++++++++-- 2 files changed, 109 insertions(+), 32 deletions(-) diff --git a/src/dashboard/app.js b/src/dashboard/app.js index a16681e..6412500 100644 --- a/src/dashboard/app.js +++ b/src/dashboard/app.js @@ -33,7 +33,8 @@ function Stat({ value, label }) { } function BeliefCard({ belief }) { - const level = confidenceLevel(belief.confidence); + const hasConfidence = typeof belief.confidence === 'number'; + const level = hasConfidence ? confidenceLevel(belief.confidence) : null; return React.createElement('div', { className: 'belief-card', 'data-type': belief.type, @@ -42,7 +43,7 @@ function BeliefCard({ belief }) { React.createElement('span', { className: 'belief-type-badge', 'data-type': belief.type, - }, TYPE_LABELS[belief.type]), + }, TYPE_LABELS[belief.type] || belief.type), React.createElement('span', { className: 'belief-timestamp' }, formatTime(belief.timestamp)) ), React.createElement('div', { className: 'belief-text' }, belief.belief), @@ -55,7 +56,7 @@ function BeliefCard({ belief }) { React.createElement('span', { className: 'belief-meta-label' }, 'Action: '), belief.actionTaken ), - React.createElement('div', { className: 'confidence-bar' }, + hasConfidence && React.createElement('div', { className: 'confidence-bar' }, React.createElement('div', { className: 'confidence-track' }, React.createElement('div', { className: 'confidence-fill', @@ -69,26 +70,44 @@ function BeliefCard({ belief }) { ); } +const SESSION_STORAGE_KEY = 'axion.sessionId'; + +function readStoredSession() { + try { + return localStorage.getItem(SESSION_STORAGE_KEY) || ''; + } catch (e) { + return ''; + } +} + +function persistSession(sessionId) { + try { + if (sessionId) localStorage.setItem(SESSION_STORAGE_KEY, sessionId); + } catch (e) { + /* localStorage unavailable (private mode) - non-fatal */ + } +} + +function initialSessionId() { + const fromUrl = new URLSearchParams(window.location.search).get('session'); + if (fromUrl && fromUrl.trim()) return fromUrl.trim(); + return readStoredSession(); +} + function App() { const [beliefs, setBeliefs] = useState([]); - const [sessions, setSessions] = useState([]); - const [selectedSession, setSelectedSession] = useState(''); + const [sessionInput, setSessionInput] = useState(''); + const [activeSession, setActiveSession] = useState(''); const [filterType, setFilterType] = useState('all'); const [minConfidence, setMinConfidence] = useState(0); - const [wrongOnly, setWrongOnly] = useState(false); + const [lowConfidenceOnly, setLowConfidenceOnly] = useState(false); const [loading, setLoading] = useState(false); - useEffect(() => { - fetch('/api/sessions') - .then(r => r.json()) - .then(data => setSessions(data.sessions || [])) - .catch(() => {}); - }, []); - const loadBeliefs = useCallback((sessionId) => { if (!sessionId) return; + setActiveSession(sessionId); setLoading(true); - fetch(`/api/beliefs/${sessionId}`) + fetch(`/api/beliefs/${encodeURIComponent(sessionId)}`) .then(r => r.json()) .then(data => { setBeliefs(data.beliefs || []); @@ -97,19 +116,33 @@ function App() { .catch(() => setLoading(false)); }, []); + const handleLoad = useCallback(() => { + const sessionId = sessionInput.trim(); + if (!sessionId) return; + persistSession(sessionId); + loadBeliefs(sessionId); + }, [sessionInput, loadBeliefs]); + useEffect(() => { - if (selectedSession) loadBeliefs(selectedSession); - }, [selectedSession, loadBeliefs]); + const initial = initialSessionId(); + if (initial) { + setSessionInput(initial); + persistSession(initial); + loadBeliefs(initial); + } + }, [loadBeliefs]); const filtered = beliefs.filter(b => { if (filterType !== 'all' && b.type !== filterType) return false; - if (b.confidence < minConfidence) return false; - if (wrongOnly && b.confidence >= 0.4) return false; + const hasConfidence = typeof b.confidence === 'number'; + if (hasConfidence && b.confidence < minConfidence) return false; + if (lowConfidenceOnly && !(hasConfidence && b.confidence < 0.4)) return false; return true; }); - const avgConfidence = beliefs.length > 0 - ? (beliefs.reduce((s, b) => s + b.confidence, 0) / beliefs.length).toFixed(2) + const scored = beliefs.filter(b => typeof b.confidence === 'number'); + const avgConfidence = scored.length > 0 + ? (scored.reduce((s, b) => s + b.confidence, 0) / scored.length).toFixed(2) : '-'; const typeCounts = beliefs.reduce((acc, b) => { @@ -126,13 +159,24 @@ function App() { ), // Session selector React.createElement('div', { className: 'session-selector' }, - React.createElement('select', { - className: 'filter-select', - value: selectedSession, - onChange: e => setSelectedSession(e.target.value), + React.createElement('form', { + className: 'session-form', + onSubmit: e => { e.preventDefault(); handleLoad(); }, }, - React.createElement('option', { value: '' }, '- Select Session -'), - ...sessions.map(s => React.createElement('option', { key: s, value: s }, s)) + React.createElement('input', { + type: 'text', + className: 'filter-input session-input', + placeholder: 'Paste session id (x-axion-session)', + value: sessionInput, + spellCheck: false, + autoComplete: 'off', + onChange: e => setSessionInput(e.target.value), + }), + React.createElement('button', { + type: 'submit', + className: 'session-load-btn', + disabled: !sessionInput.trim(), + }, 'Load') ) ), // Stats @@ -177,10 +221,10 @@ function App() { React.createElement('label', { className: 'filter-toggle' }, React.createElement('input', { type: 'checkbox', - checked: wrongOnly, - onChange: e => setWrongOnly(e.target.checked), + checked: lowConfidenceOnly, + onChange: e => setLowConfidenceOnly(e.target.checked), }), - 'Wrong beliefs only' + 'Low confidence only' ), ), // Timeline diff --git a/src/dashboard/styles.css b/src/dashboard/styles.css index f70b85c..6388e7a 100644 --- a/src/dashboard/styles.css +++ b/src/dashboard/styles.css @@ -180,9 +180,42 @@ html, body { margin-bottom: 2rem; } -.session-selector select { - width: 100%; - max-width: 400px; +.session-form { + display: flex; + gap: 0.5rem; + align-items: stretch; + max-width: 480px; +} + +.session-input { + flex: 1; + min-width: 0; +} + +.session-load-btn { + background: var(--bg-elevated); + color: var(--text); + border: 1px solid var(--border); + padding: 0.4rem 1rem; + font-family: var(--mono); + font-size: 0.8rem; + border-radius: 2px; + cursor: pointer; + outline: none; + transition: border-color 0.15s, color 0.15s; +} + +.session-load-btn:hover:not(:disabled) { + border-color: var(--border-hover); +} + +.session-load-btn:focus { + border-color: var(--text-secondary); +} + +.session-load-btn:disabled { + color: var(--text-muted); + cursor: not-allowed; } /* Timeline */ From f32f46ca1fbed917aa6b67b7b57d6090f321ce66 Mon Sep 17 00:00:00 2001 From: Cursor Agent Date: Thu, 16 Jul 2026 15:09:15 +0000 Subject: [PATCH 05/10] chore: add test/check scripts, CI, and OSS hygiene files Add 'test' (vitest run) and 'check' (tsc --noEmit && vitest run) npm scripts, a GitHub Actions CI workflow running npm run check on Node 20 for pull_request and push to main, plus .dev.vars.example, CONTRIBUTING.md, and SECURITY.md. SECURITY documents that the beliefs API is unauthenticated in Phase 1 and the session id is a capability URL. Co-authored-by: Moses Man --- .dev.vars.example | 2 ++ .github/workflows/ci.yml | 18 ++++++++++++++++++ CONTRIBUTING.md | 27 +++++++++++++++++++++++++++ SECURITY.md | 33 +++++++++++++++++++++++++++++++++ package.json | 2 ++ 5 files changed, 82 insertions(+) create mode 100644 .dev.vars.example create mode 100644 .github/workflows/ci.yml create mode 100644 CONTRIBUTING.md create mode 100644 SECURITY.md diff --git a/.dev.vars.example b/.dev.vars.example new file mode 100644 index 0000000..51bb63a --- /dev/null +++ b/.dev.vars.example @@ -0,0 +1,2 @@ +UPSTREAM_API_URL=https://api.openai.com +# UPSTREAM_API_KEY= # optional if agents pass their own keys diff --git a/.github/workflows/ci.yml b/.github/workflows/ci.yml new file mode 100644 index 0000000..2693990 --- /dev/null +++ b/.github/workflows/ci.yml @@ -0,0 +1,18 @@ +name: CI + +on: + pull_request: + push: + branches: [main] + +jobs: + check: + runs-on: ubuntu-latest + steps: + - uses: actions/checkout@v4 + - uses: actions/setup-node@v4 + with: + node-version: 20 + cache: npm + - run: npm ci + - run: npm run check diff --git a/CONTRIBUTING.md b/CONTRIBUTING.md new file mode 100644 index 0000000..0c167bd --- /dev/null +++ b/CONTRIBUTING.md @@ -0,0 +1,27 @@ +# Contributing + +Thanks for your interest in Axion. This is an early-phase project, so expect rough edges. + +## Development + +Requirements: Node 20+ and npm. + +```bash +npm ci # install dependencies +npm run dev # run the Worker locally (wrangler dev) +npm test # run the test suite (vitest) +npm run check # typecheck + tests; run this before opening a PR +``` + +Copy `.dev.vars.example` to `.dev.vars` and fill in the values you need for local runs. + +## Pull requests + +- Keep changes focused. One logical change per PR. +- Run `npm run check` and make sure it passes before opening a PR. +- Match the existing code style; don't reformat unrelated files. +- Describe what changed and why in the PR description. + +## Reporting security issues + +Do not open a public issue for security problems. See [SECURITY.md](./SECURITY.md). diff --git a/SECURITY.md b/SECURITY.md new file mode 100644 index 0000000..4a00a31 --- /dev/null +++ b/SECURITY.md @@ -0,0 +1,33 @@ +# Security + +## Reporting a vulnerability + +Please report suspected vulnerabilities privately through GitHub Security Advisories +("Report a vulnerability" under the repository's Security tab). If that is not +available to you, open a regular GitHub issue but leave out exploit details and +ask a maintainer for a private channel. + +We will acknowledge reports and work with you on a fix. There is no bug bounty. + +## Threat model (Phase 1) + +Be aware of these design choices before deploying: + +- **The beliefs API is unauthenticated.** `GET /api/beliefs/:sessionId` has no + auth check. Anyone who knows a session id can read that session's captured + beliefs. +- **The session id is a capability URL.** The session id (`x-axion-session`, a + UUID by default) is the only thing protecting a session's data. Treat it like + a secret: don't paste it into public places, logs, or shared dashboards you + don't control. +- **Upstream credentials pass through.** The proxy forwards the caller's + `Authorization` / `x-api-key` upstream, or uses the `UPSTREAM_API_KEY` secret + if configured. It never logs keys and never sends `Bearer undefined`. + +## Known future work (not implemented yet) + +- Rate limiting on proxy and beliefs endpoints. +- Authentication / access control for the beliefs API. + +If your deployment handles sensitive data, keep session ids secret and put the +Worker behind your own access controls until these land. diff --git a/package.json b/package.json index 07f5bdf..218acb2 100644 --- a/package.json +++ b/package.json @@ -14,6 +14,8 @@ "dev": "wrangler dev", "deploy": "wrangler deploy", "typecheck": "tsc --noEmit", + "test": "vitest run", + "check": "tsc --noEmit && vitest run", "tail": "wrangler tail" }, "devDependencies": { From 7725b63172c466c0a7a51afa2de12f9924229189 Mon Sep 17 00:00:00 2001 From: Cursor Agent Date: Thu, 16 Jul 2026 15:11:17 +0000 Subject: [PATCH 06/10] lens: fix because-of extraction, additive confidence, evidence field - Split because-of / because into two patterns (group 1) so causal beliefs are extracted from "Because of ..." sentences - Set evidenceGroup: 1 on all evidence patterns (belief + evidence hold cited text) - Replace midpoint confidence interpolation with additive delta modifiers, clamped to [0.1, 1.0] - Allow end-of-string as a clause terminator in patterns - Add thorough extract.test.ts (vitest) Co-authored-by: Moses Man --- src/lens/extract.test.ts | 244 +++++++++++++++++++++++++++++++++++++++ src/lens/extract.ts | 20 ++-- src/lens/patterns.ts | 68 +++++++---- src/lens/types.ts | 17 +-- 4 files changed, 311 insertions(+), 38 deletions(-) create mode 100644 src/lens/extract.test.ts diff --git a/src/lens/extract.test.ts b/src/lens/extract.test.ts new file mode 100644 index 0000000..0fd0803 --- /dev/null +++ b/src/lens/extract.test.ts @@ -0,0 +1,244 @@ +/** + * Tests for the Axion Lens belief extraction engine. + * + * Covers the BUILD-SPEC §5 requirements: + * - "because of" now extracts a causal belief (previously silently dropped) + * - evidence patterns populate the `evidence` field + * - additive confidence modifiers, clamped to [0.1, 1.0] + * - end-of-string acts as a clause terminator + * - sessionId is stamped onto every belief when provided + */ +import { describe, it, expect } from 'vitest'; +import { extractBeliefs } from './extract.js'; +import { + BELIEF_PATTERNS, + CONFIDENCE_MARKERS, + CONFIDENCE_MAX, + CONFIDENCE_MIN, +} from './patterns.js'; +import type { ExtractedBelief } from './types.js'; + +/** Deterministic option overrides so ids/timestamps are stable in assertions. */ +function fixedOpts(sessionId?: string) { + let n = 0; + return { + sessionId, + uuid: () => `id-${n++}`, + now: () => 1_700_000_000_000, + }; +} + +function findByType(beliefs: ExtractedBelief[], type: ExtractedBelief['type']) { + return beliefs.filter((b) => b.type === type); +} + +describe('extractBeliefs - empty / trivial input', () => { + it('returns [] for empty, whitespace, or undefined-ish text', async () => { + expect(await extractBeliefs('')).toEqual([]); + expect(await extractBeliefs(' \n ')).toEqual([]); + // no reasoning markers at all + expect(await extractBeliefs('Hello there, nice weather today.')).toEqual([]); + }); +}); + +describe('extractBeliefs - "because of" (BUILD-SPEC §5 fix)', () => { + it('extracts a causal belief from "Because of the missing env var the app crashed."', async () => { + const beliefs = await extractBeliefs( + 'Because of the missing env var the app crashed.', + fixedOpts('s1'), + ); + const causal = findByType(beliefs, 'causal'); + expect(causal.length).toBeGreaterThan(0); + expect(causal[0]!.belief).toContain('missing env var'); + // must not include the connective itself + expect(causal[0]!.belief.toLowerCase()).not.toContain('because of'); + }); + + it('still extracts bare "because X" as causal', async () => { + const beliefs = await extractBeliefs( + 'It failed because the token expired.', + fixedOpts('s1'), + ); + const causal = findByType(beliefs, 'causal'); + expect(causal.length).toBe(1); + expect(causal[0]!.belief).toBe('the token expired'); + }); + + it('does not double-count "because of" as both because-of and bare because', async () => { + const beliefs = await extractBeliefs( + 'Because of the outage the deploy stalled.', + fixedOpts('s1'), + ); + expect(findByType(beliefs, 'causal').length).toBe(1); + }); +}); + +describe('extractBeliefs - evidence field', () => { + it('populates both belief and evidence for "based on" patterns', async () => { + const beliefs = await extractBeliefs( + 'Based on the logs, the request timed out.', + fixedOpts('s1'), + ); + const evidence = findByType(beliefs, 'evidence'); + expect(evidence.length).toBe(1); + expect(evidence[0]!.belief).toBe('logs'); + expect(evidence[0]!.evidence).toBe('logs'); + }); + + it('captures evidence for "according to"', async () => { + const beliefs = await extractBeliefs( + 'According to the changelog, the API was deprecated.', + fixedOpts('s1'), + ); + const evidence = findByType(beliefs, 'evidence'); + expect(evidence.length).toBeGreaterThan(0); + expect(evidence[0]!.belief).toBe('changelog'); + expect(evidence[0]!.evidence).toBe('changelog'); + }); + + it('every evidence pattern declares evidenceGroup = 1', () => { + for (const p of BELIEF_PATTERNS.filter((p) => p.type === 'evidence')) { + expect(p.evidenceGroup).toBe(1); + } + }); +}); + +describe('extractBeliefs - end-of-string terminator', () => { + it('extracts an intention when the sentence has no trailing punctuation', async () => { + const beliefs = await extractBeliefs( + 'I will refactor the auth module', + fixedOpts('s1'), + ); + const intent = findByType(beliefs, 'intention'); + expect(intent.length).toBe(1); + expect(intent[0]!.belief).toBe('refactor the auth module'); + }); + + it('extracts a causal belief ending at end-of-string', async () => { + const beliefs = await extractBeliefs( + 'The build broke because the lockfile drifted', + fixedOpts('s1'), + ); + const causal = findByType(beliefs, 'causal'); + expect(causal.length).toBe(1); + expect(causal[0]!.belief).toBe('the lockfile drifted'); + }); +}); + +describe('extractBeliefs - additive confidence modifiers, clamped [0.1, 1.0]', () => { + it('uses baseline confidence when no marker is present', async () => { + const [b] = await extractBeliefs( + 'Because of the outage the deploy stalled.', + fixedOpts('s1'), + ); + // because-of baseline is 0.85 + expect(b!.confidence).toBeCloseTo(0.85, 5); + }); + + it('adds +0.1 for "probably"', async () => { + const [b] = await extractBeliefs( + 'Because of the outage the deploy probably stalled.', + fixedOpts('s1'), + ); + expect(b!.confidence).toBeCloseTo(0.95, 5); + }); + + it('subtracts 0.2 for "might"', async () => { + const [b] = await extractBeliefs( + 'Because of the outage the deploy might have stalled.', + fixedOpts('s1'), + ); + expect(b!.confidence).toBeCloseTo(0.65, 5); + }); + + it('subtracts 0.3 for "not sure"', async () => { + const [b] = await extractBeliefs( + "Because of the outage I'm not sure the deploy stalled.", + fixedOpts('s1'), + ); + expect(b!.confidence).toBeCloseTo(0.55, 5); + }); + + it('clamps the upper bound to 1.0 (0.85 + 0.2 = 1.05 → 1.0)', async () => { + const [b] = await extractBeliefs( + 'Because of the outage the deploy definitely stalled.', + fixedOpts('s1'), + ); + expect(b!.confidence).toBe(CONFIDENCE_MAX); + expect(b!.confidence).toBe(1.0); + }); + + it('sums multiple distinct markers and clamps the lower bound to 0.1', async () => { + // if-then baseline 0.6; "uncertain" (-0.3) + "might"/"possibly" (-0.2) + // = -0.5 → 0.1 (floor). + const [b] = await extractBeliefs( + 'If the migration is uncertain then it might possibly break.', + fixedOpts('s1'), + ); + expect(b!.confidence).toBe(CONFIDENCE_MIN); + expect(b!.confidence).toBe(0.1); + }); + + it('never returns a confidence outside [0.1, 1.0] for any pattern', async () => { + const text = [ + 'Because of the outage the deploy definitely absolutely stalled.', + 'If the migration is uncertain unsure then it might possibly break.', + 'Based on the logs, it probably failed.', + ].join('\n'); + const beliefs = await extractBeliefs(text, fixedOpts('s1')); + expect(beliefs.length).toBeGreaterThan(0); + for (const b of beliefs) { + expect(b.confidence).toBeGreaterThanOrEqual(0.1); + expect(b.confidence).toBeLessThanOrEqual(1.0); + } + }); + + it('CONFIDENCE_MARKERS expose additive deltas matching the spec', () => { + const byLabel = Object.fromEntries(CONFIDENCE_MARKERS.map((m) => [m.label, m.delta])); + expect(byLabel.certain).toBe(0.2); + expect(byLabel.likely).toBe(0.1); + expect(byLabel.possible).toBe(-0.2); + expect(byLabel.uncertain).toBe(-0.3); + }); +}); + +describe('extractBeliefs - sessionId stamping', () => { + it('stamps the provided sessionId onto every belief', async () => { + const text = [ + 'Because of the outage the deploy stalled.', + 'Based on the logs, it failed.', + 'I will roll back the release.', + ].join('\n'); + const beliefs = await extractBeliefs(text, fixedOpts('session-abc')); + expect(beliefs.length).toBeGreaterThan(1); + for (const b of beliefs) { + expect(b.sessionId).toBe('session-abc'); + } + }); + + it('generates a non-empty sessionId when none is provided', async () => { + const beliefs = await extractBeliefs('Because of the outage the deploy stalled.'); + expect(beliefs.length).toBeGreaterThan(0); + for (const b of beliefs) { + expect(typeof b.sessionId).toBe('string'); + expect(b.sessionId.length).toBeGreaterThan(0); + } + // all beliefs from one call share the same generated session id + const ids = new Set(beliefs.map((b) => b.sessionId)); + expect(ids.size).toBe(1); + }); +}); + +describe('extractBeliefs - belief shape', () => { + it('stamps id, timestamp, rawText and line', async () => { + const beliefs = await extractBeliefs( + 'Line one is filler.\nBecause of the outage the deploy stalled.', + fixedOpts('s1'), + ); + const b = beliefs[0]!; + expect(b.id).toBe('id-0'); + expect(b.timestamp).toBe(1_700_000_000_000); + expect(b.rawText.toLowerCase()).toContain('because of'); + expect(b.line).toBe(2); + }); +}); diff --git a/src/lens/extract.ts b/src/lens/extract.ts index a19fecd..4a3a853 100644 --- a/src/lens/extract.ts +++ b/src/lens/extract.ts @@ -14,6 +14,8 @@ import type { BeliefType, ExtractedBelief, PatternMatch } from './types.js'; import { BELIEF_PATTERNS, CONFIDENCE_MARKERS, + CONFIDENCE_MAX, + CONFIDENCE_MIN, DEFAULT_CONFIDENCE, MARKER_SCAN_RADIUS, } from './patterns.js'; @@ -143,18 +145,18 @@ function surroundingContext(text: string, start: number, length: number): string } /** - * Adjust a pattern's baseline confidence toward the strongest marker band - * found in the surrounding context. The strongest marker (first in - * `CONFIDENCE_MARKERS`, which is ordered by descending strength) wins. + * Adjust a pattern's baseline confidence by summing the additive `delta` of + * every distinct confidence-marker category found in the surrounding context, + * then clamping to [CONFIDENCE_MIN, CONFIDENCE_MAX]. */ function adjustConfidence(baseline: number, context: string): number { + let confidence = baseline; for (const marker of CONFIDENCE_MARKERS) { if (marker.pattern.test(context)) { - // Interpolate baseline halfway toward the marker's target band. - return clamp01((baseline + marker.confidence) / 2); + confidence += marker.delta; } } - return clamp01(baseline); + return clampConfidence(confidence); } /** Return the `pattern` with the `g` flag added (idempotent). */ @@ -172,9 +174,9 @@ function lineNumberAt(text: string, offset: number): number { return line; } -function clamp01(n: number): number { - if (Number.isNaN(n)) return 0; - return Math.min(1, Math.max(0, n)); +function clampConfidence(n: number): number { + if (Number.isNaN(n)) return CONFIDENCE_MIN; + return Math.min(CONFIDENCE_MAX, Math.max(CONFIDENCE_MIN, n)); } /** Fallback session id when none is supplied. */ diff --git a/src/lens/patterns.ts b/src/lens/patterns.ts index 7da82f8..ad50a12 100644 --- a/src/lens/patterns.ts +++ b/src/lens/patterns.ts @@ -46,28 +46,35 @@ export interface BeliefPattern { */ export const BELIEF_PATTERNS: BeliefPattern[] = [ // ── Evidence references ──────────────────────────────────────────────── + // Evidence patterns set `evidenceGroup: 1` so the cited text lands in the + // belief's `evidence` field as well as its `belief` field (both hold the + // cited text - useful for the dashboard, per BUILD-SPEC §5). + // // "based on X, ..." / "based on the X, ..." { label: 'based-on', type: 'evidence', - pattern: /\bbased on (?:the |the )?([^.;!?\n]{2,120}?)(?:[,.;]|\sthen)/i, + pattern: /\bbased on (?:the )?([^.;!?\n]{2,120}?)(?:[,.;]|\sthen|$)/i, group: 1, + evidenceGroup: 1, confidence: 0.8, }, // "according to X, ..." { label: 'according-to', type: 'evidence', - pattern: /\baccording to (?:the )?([^.;!?\n]{2,120}?)(?:[,.;]|\sthen)/i, + pattern: /\baccording to (?:the )?([^.;!?\n]{2,120}?)(?:[,.;]|\sthen|$)/i, group: 1, + evidenceGroup: 1, confidence: 0.8, }, // "from the X, ..." (only when followed by a verb phrase - avoids "from the start") { label: 'from-the', type: 'evidence', - pattern: /\bfrom the ([^.;!?\n]{2,120}?)(?:[,.;]|\sthen)/i, + pattern: /\bfrom the ([^.;!?\n]{2,120}?)(?:[,.;]|\sthen|$)/i, group: 1, + evidenceGroup: 1, confidence: 0.7, }, // "the error says X" / "the error message says X" / "the error indicates X" @@ -76,23 +83,34 @@ export const BELIEF_PATTERNS: BeliefPattern[] = [ type: 'evidence', pattern: /\bthe error(?: message)? (?:says|indicates|shows|states) "?([^";!?\n]{2,140})"?/i, group: 1, + evidenceGroup: 1, confidence: 0.85, }, // ── Causal claims ────────────────────────────────────────────────────── - // "because X" / "because of X" / ", because X" + // "because of X" - split from bare "because" so group 1 always holds the + // belief text (the previous single pattern used group 2 and never fired for + // the "because of" branch). See BUILD-SPEC §5. + { + label: 'because-of', + type: 'causal', + pattern: /\bbecause of\s+([^.;!?\n]{2,120}?)(?:[.;!?\n]|$)/i, + group: 1, + confidence: 0.85, + }, + // "because X" / ", because X" (not "because of", handled above) { label: 'because', type: 'causal', - pattern: /\bbecause of\b\s*([^.;!?\n]{2,120}?)[.;!?\n]|because\s+([^.;!?\n]{2,120}?)[.;!?\n]/i, - group: 2, + pattern: /\bbecause\s+(?!of\b)([^.;!?\n]{2,120}?)(?:[.;!?\n]|$)/i, + group: 1, confidence: 0.85, }, // "since X" - but NOT temporal "since [year]"; require a verb-ish word after. { label: 'since-causal', type: 'causal', - pattern: /\bsince\s+(?!the\s+\d|\d{4})([^.;!?\n]{2,120}?)[.;!?\n]/i, + pattern: /\bsince\s+(?!the\s+\d|\d{4})([^.;!?\n]{2,120}?)(?:[.;!?\n]|$)/i, group: 1, confidence: 0.8, }, @@ -100,7 +118,7 @@ export const BELIEF_PATTERNS: BeliefPattern[] = [ { label: 'due-to', type: 'causal', - pattern: /\b(?:due to|as a result of)\s+([^.;!?\n]{2,120}?)[.;!?\n]/i, + pattern: /\b(?:due to|as a result of)\s+([^.;!?\n]{2,120}?)(?:[.;!?\n]|$)/i, group: 1, confidence: 0.85, }, @@ -110,7 +128,7 @@ export const BELIEF_PATTERNS: BeliefPattern[] = [ { label: 'assuming', type: 'assumption', - pattern: /\b(?:assuming|presumably)\s+(?:that\s+)?([^.;!?\n]{2,120}?)[.;!?\n]/i, + pattern: /\b(?:assuming|presumably)\s+(?:that\s+)?([^.;!?\n]{2,120}?)(?:[.;!?\n]|$)/i, group: 1, confidence: 0.65, }, @@ -118,7 +136,7 @@ export const BELIEF_PATTERNS: BeliefPattern[] = [ { label: 'i-assume', type: 'assumption', - pattern: /\b(?:i(?:'ll| will)|let's|let us) assume\s+(?:that\s+)?([^.;!?\n]{2,120}?)[.;!?\n]/i, + pattern: /\b(?:i(?:'ll| will)|let's|let us) assume\s+(?:that\s+)?([^.;!?\n]{2,120}?)(?:[.;!?\n]|$)/i, group: 1, confidence: 0.65, }, @@ -126,7 +144,7 @@ export const BELIEF_PATTERNS: BeliefPattern[] = [ { label: 'if-then', type: 'assumption', - pattern: /\bif\s+([^,.;!?\n]{2,100}?)\s+then\s+([^.;!?\n]{2,120}?)[.;!?\n]/i, + pattern: /\bif\s+([^,.;!?\n]{2,100}?)\s+then\s+([^.;!?\n]{2,120}?)(?:[.;!?\n]|$)/i, group: 1, actionGroup: 2, confidence: 0.6, @@ -138,7 +156,7 @@ export const BELIEF_PATTERNS: BeliefPattern[] = [ { label: 'i-will', type: 'intention', - pattern: /\b(?:i(?:'ll| will|i'm going to|'m going to)|let me|i should|i'm going to)\s+([^.;!?\n]{2,120}?)[.;!?\n]/i, + pattern: /\b(?:i(?:'ll| will|i'm going to|'m going to)|let me|i should|i'm going to)\s+([^.;!?\n]{2,120}?)(?:[.;!?\n]|$)/i, group: 1, confidence: 0.75, }, @@ -147,7 +165,7 @@ export const BELIEF_PATTERNS: BeliefPattern[] = [ { label: 'i-plan', type: 'intention', - pattern: /\bi (?:plan|intend)\s+to\s+([^.;!?\n]{2,120}?)[.;!?\n]/i, + pattern: /\bi (?:plan|intend)\s+to\s+([^.;!?\n]{2,120}?)(?:[.;!?\n]|$)/i, group: 1, confidence: 0.75, }, @@ -155,27 +173,33 @@ export const BELIEF_PATTERNS: BeliefPattern[] = [ /** * Confidence markers. These are scanned in the *surrounding clause* around a - * match and nudge the baseline confidence up or down. + * match and nudge the baseline confidence up or down by an additive `delta`. * - * Each marker has a target band; the engine interpolates toward it. Order is - * by descending strength so the strongest marker in a clause wins. + * The engine sums the deltas of every distinct marker category found near a + * match, adds them to the pattern baseline, and clamps the result to + * [0.1, 1.0] (see BUILD-SPEC §5 / README). This replaces the older + * "interpolate toward a target band" behaviour. */ export interface ConfidenceMarkerPattern { pattern: RegExp; - /** Target confidence band when this marker is present. */ - confidence: number; + /** Additive nudge applied to the baseline confidence when present. */ + delta: number; label: string; } export const CONFIDENCE_MARKERS: ConfidenceMarkerPattern[] = [ - { label: 'certain', confidence: 1.0, pattern: /\b(?:definitely|certainly|absolutely|without a doubt|clearly|obviously|guaranteed)\b/i }, - { label: 'likely', confidence: 0.85, pattern: /\b(?:probably|likely|most likely|almost certainly|highly likely|strongly)\b/i }, - { label: 'possible', confidence: 0.6, pattern: /\b(?:might|may|could be|could possibly|perhaps|possibly|seems to)\b/i }, - { label: 'uncertain', confidence: 0.35, pattern: /\b(?:not sure|unsure|unclear|uncertain|i think|i believe|i guess|roughly|around)\b/i }, + { label: 'certain', delta: +0.2, pattern: /\b(?:definitely|certainly|absolutely|without a doubt|guaranteed)\b/i }, + { label: 'likely', delta: +0.1, pattern: /\b(?:probably|likely|most likely|almost certainly|highly likely)\b/i }, + { label: 'possible', delta: -0.2, pattern: /\b(?:might|could be|possibly|may|perhaps)\b/i }, + { label: 'uncertain', delta: -0.3, pattern: /\b(?:not sure|uncertain|unsure|unclear)\b/i }, ]; /** Default confidence when no marker is found near a match. */ export const DEFAULT_CONFIDENCE = 0.7; +/** Confidence is clamped to this inclusive range. */ +export const CONFIDENCE_MIN = 0.1; +export const CONFIDENCE_MAX = 1.0; + /** Maximum characters of context to scan on each side of a match for markers. */ export const MARKER_SCAN_RADIUS = 80; diff --git a/src/lens/types.ts b/src/lens/types.ts index 34fd9b2..e78aaca 100644 --- a/src/lens/types.ts +++ b/src/lens/types.ts @@ -13,13 +13,16 @@ /** The kind of reasoning fragment that was detected. */ export type BeliefType = 'causal' | 'assumption' | 'intention' | 'evidence'; -/** Confidence markers map onto a numeric band. See `patterns.ts` for the bands. */ +/** + * Confidence markers apply an additive delta to the pattern baseline, clamped + * to [0.1, 1.0]. See `patterns.ts` for the exact word lists / deltas. + */ export type ConfidenceMarker = - | 'certain' // definitely, certainly, absolutely → ~1.0 - | 'likely' // probably, likely, almost certainly → ~0.85 - | 'possible' // might, could be, may → ~0.6 - | 'uncertain' // not sure, unclear, unsure → ~0.35 - | 'none'; // no marker detected → 0.7 default + | 'certain' // definitely, certainly, absolutely → +0.2 + | 'likely' // probably, likely → +0.1 + | 'possible' // might, could be, possibly → -0.2 + | 'uncertain' // not sure, uncertain, unsure → -0.3 + | 'none'; // no marker detected → baseline (0.7 default) /** * A single belief extracted from one model response. @@ -36,7 +39,7 @@ export interface ExtractedBelief { belief: string; /** Cited evidence, if any (e.g. "the error message"). */ evidence?: string; - /** Confidence score in [0,1]. Derived from confidence markers. */ + /** Confidence score in [0.1, 1.0]. Baseline adjusted by confidence markers. */ confidence: number; /** If an action was stated alongside the belief, the action text. */ actionTaken?: string; From 9bfbfcf097853c25c16c1ed3d5bed3eb91dd5de3 Mon Sep 17 00:00:00 2001 From: Cursor Agent Date: Thu, 16 Jul 2026 15:12:01 +0000 Subject: [PATCH 07/10] feat(polyverdict): v1 schema subset validator, coercion, and enforce helpers Implements BUILD-SPEC section 7 (opt-in enforce path): - schema.ts: zero-dep JSON Schema subset validator + type coercion (type/properties/required/items/enum, nested, unknown keywords ignored), plus stripMarkdownFences and parseJsonFromAssistant. - enforce.ts: detectSchemaTrigger (x-axion-schema header + response_format json_schema), enforceOnce, OpenAI/Anthropic retry-hint injection, MAX_ENFORCE_ATTEMPTS=3, and a pure runEnforceLoop driver. - types.ts, index.ts: shared types and re-exports. - schema.test.ts: valid/missing-required/coerce/enum/nested/fence-strip (28 tests). Co-authored-by: Moses Man --- src/polyverdict/enforce.ts | 281 +++++++++++++++++++++++ src/polyverdict/index.ts | 42 ++++ src/polyverdict/schema.test.ts | 289 +++++++++++++++++++++++ src/polyverdict/schema.ts | 406 +++++++++++++++++++++++++++++++++ src/polyverdict/types.ts | 84 +++++++ 5 files changed, 1102 insertions(+) create mode 100644 src/polyverdict/enforce.ts create mode 100644 src/polyverdict/index.ts create mode 100644 src/polyverdict/schema.test.ts create mode 100644 src/polyverdict/schema.ts create mode 100644 src/polyverdict/types.ts diff --git a/src/polyverdict/enforce.ts b/src/polyverdict/enforce.ts new file mode 100644 index 0000000..479add2 --- /dev/null +++ b/src/polyverdict/enforce.ts @@ -0,0 +1,281 @@ +/** + * PolyVerdict v1 - enforce path: schema-trigger detection, single-shot + * enforcement, and retry-hint injection. + * + * Everything here is intentionally pure (no `fetch`, no globals) so the proxy + * integrator can drive the retry loop around its own upstream call. See + * {@link runEnforceLoop} for a ready-made driver that takes an injected + * upstream callback, and the "Integrator loop" note below for the manual + * shape. + * + * Integrator loop (manual): + * 1. const trigger = detectSchemaTrigger(request.headers, body); + * if (!trigger) → fall through to the normal observe/tee path. + * 2. Force `stream: false` upstream (enforce always buffers). + * 3. for attempt in 1..MAX_ENFORCE_ATTEMPTS: + * text = + * result = enforceOnce(text, trigger.schema) + * if (result.ok) → return provider-shaped JSON (result.jsonText) + * else if (attempt < MAX) → messages = buildRetryMessages( + * messages, { schema: trigger.schema, errors: result.errors, + * assistantText: text }) // Anthropic variant for /v1/messages + * else → return last text + surface result.errors + */ + +import { + parseJsonFromAssistant, + validateAndCoerce, +} from './schema.js'; +import type { + AnthropicMessage, + OpenAiMessage, + SchemaTrigger, +} from './types.js'; +export type { SchemaTrigger } from './types.js'; + +/** Maximum total upstream attempts (initial call + retries) in enforce mode. */ +export const MAX_ENFORCE_ATTEMPTS = 3; + +/** Result of a single enforcement attempt on one assistant message. */ +export interface EnforceResult { + /** True when the text parsed and validated (after coercion). */ + ok: boolean; + /** The coerced value (present only when `ok`). */ + value?: unknown; + /** Canonical serialization of `value` to hand back to the caller. */ + jsonText?: string; + /** Violation messages (empty when `ok`). */ + errors: string[]; +} + +/** + * Inspect an incoming request for a PolyVerdict schema trigger. + * + * Two triggers are recognised (header takes precedence): + * - `x-axion-schema`: inline JSON Schema. Parsed directly; if that fails we + * retry after `decodeURIComponent` (headers are often URL-encoded). + * - body `response_format: { type: "json_schema", json_schema: { schema } }` + * (OpenAI structured-output style). + * + * Returns `null` when no valid trigger is present, so the caller can skip the + * enforce path entirely. + */ +export function detectSchemaTrigger( + requestHeaders: Headers, + body: unknown, +): SchemaTrigger | null { + const headerTrigger = triggerFromHeader(requestHeaders); + if (headerTrigger) return headerTrigger; + return triggerFromBody(body); +} + +function triggerFromHeader(headers: Headers): SchemaTrigger | null { + const raw = headers.get('x-axion-schema'); + if (!raw || raw.trim() === '') return null; + + const schema = parseSchemaHeader(raw); + if (schema === undefined) return null; + return { schema }; +} + +/** Parse a header value as JSON, falling back to a URL-decoded parse. */ +function parseSchemaHeader(raw: string): unknown | undefined { + try { + return JSON.parse(raw); + } catch { + // fall through to decode attempt + } + try { + return JSON.parse(decodeURIComponent(raw)); + } catch { + return undefined; + } +} + +function triggerFromBody(body: unknown): SchemaTrigger | null { + if (!isObject(body)) return null; + const rf = (body as Record).response_format; + if (!isObject(rf)) return null; + if (rf.type !== 'json_schema') return null; + + const js = rf.json_schema; + if (!isObject(js)) return null; + if (!('schema' in js)) return null; + + const name = typeof js.name === 'string' ? js.name : undefined; + return { schema: js.schema, name }; +} + +/** + * Run one enforcement pass over an assistant text: strip fences, parse JSON, + * then validate + coerce against `schema`. + */ +export function enforceOnce( + assistantText: string, + schema: unknown, +): EnforceResult { + const parsed = parseJsonFromAssistant(assistantText); + if (!parsed.ok) { + return { ok: false, errors: [`JSON parse failed: ${parsed.error}`] }; + } + + const validated = validateAndCoerce(parsed.value, schema); + if (!validated.ok) { + return { ok: false, errors: validated.errors }; + } + + return { + ok: true, + value: validated.value, + jsonText: JSON.stringify(validated.value), + errors: [], + }; +} + +// ── Retry-hint injection ──────────────────────────────────────────────────── + +/** Context used to build a corrective retry message. */ +export interface RetryContext { + /** The JSON Schema the output must satisfy. */ + schema: unknown; + /** Violations from the previous attempt. */ + errors: string[]; + /** The assistant text that failed (echoed back so the model can self-correct). */ + assistantText?: string; + /** Optional schema name for the instruction. */ + name?: string; +} + +/** + * Build the corrective instruction text appended on a retry. Shared by both + * provider variants. + */ +export function buildViolationHint(ctx: RetryContext): string { + const label = ctx.name ? ` "${ctx.name}"` : ''; + const schemaJson = safeStringify(ctx.schema); + const violations = ctx.errors.length + ? ctx.errors.map((e) => `- ${e}`).join('\n') + : '- output was not valid JSON'; + + return [ + `Your previous response did not satisfy the required JSON schema${label}.`, + '', + 'Schema violations:', + violations, + '', + 'Required JSON schema:', + schemaJson, + '', + 'Respond again with ONLY a single JSON value that satisfies the schema.', + 'Do not include any prose, explanation, or Markdown code fences.', + ].join('\n'); +} + +/** + * Append retry turns to an OpenAI-style `messages` array: the failed assistant + * output (when available) followed by a user correction message. Returns a new + * array; the input is not mutated. + */ +export function buildRetryMessages( + messages: OpenAiMessage[], + ctx: RetryContext, +): OpenAiMessage[] { + const next: OpenAiMessage[] = [...messages]; + if (typeof ctx.assistantText === 'string' && ctx.assistantText.trim() !== '') { + next.push({ role: 'assistant', content: ctx.assistantText }); + } + next.push({ role: 'user', content: buildViolationHint(ctx) }); + return next; +} + +/** + * Anthropic Messages variant of {@link buildRetryMessages}. Text content is + * emitted as plain strings, which the Messages API accepts. + */ +export function buildRetryMessagesAnthropic( + messages: AnthropicMessage[], + ctx: RetryContext, +): AnthropicMessage[] { + const next: AnthropicMessage[] = [...messages]; + if (typeof ctx.assistantText === 'string' && ctx.assistantText.trim() !== '') { + next.push({ role: 'assistant', content: ctx.assistantText }); + } + next.push({ role: 'user', content: buildViolationHint(ctx) }); + return next; +} + +// ── Optional driver ───────────────────────────────────────────────────────── + +/** Outcome of {@link runEnforceLoop}. */ +export interface EnforceLoopResult extends EnforceResult { + /** Number of upstream attempts actually made (1..MAX_ENFORCE_ATTEMPTS). */ + attempts: number; + /** The final assistant text seen (last attempt). */ + finalText: string; +} + +/** + * Drive the enforce retry loop with an injected upstream callback. Keeps + * PolyVerdict free of any transport concerns: the integrator supplies a + * function that sends the current `messages` upstream (non-streaming) and + * resolves to the assistant text. + * + * @param messages Initial OpenAI-style messages. + * @param schema The JSON Schema to enforce. + * @param callUpstream Sends messages upstream, returns assistant text. + * @param opts Optional schema name / max attempts / message builder. + */ +export async function runEnforceLoop( + messages: OpenAiMessage[], + schema: unknown, + callUpstream: (messages: OpenAiMessage[], attempt: number) => Promise, + opts: { + name?: string; + maxAttempts?: number; + buildRetry?: (messages: OpenAiMessage[], ctx: RetryContext) => OpenAiMessage[]; + } = {}, +): Promise { + const maxAttempts = clampAttempts(opts.maxAttempts ?? MAX_ENFORCE_ATTEMPTS); + const buildRetry = opts.buildRetry ?? buildRetryMessages; + + let current = messages; + let lastText = ''; + let last: EnforceResult = { ok: false, errors: ['no upstream attempt made'] }; + + for (let attempt = 1; attempt <= maxAttempts; attempt++) { + lastText = await callUpstream(current, attempt); + last = enforceOnce(lastText, schema); + if (last.ok) { + return { ...last, attempts: attempt, finalText: lastText }; + } + if (attempt < maxAttempts) { + current = buildRetry(current, { + schema, + errors: last.errors, + assistantText: lastText, + name: opts.name, + }); + } + } + + return { ...last, attempts: maxAttempts, finalText: lastText }; +} + +// ── utilities ──────────────────────────────────────────────────────────────── + +function isObject(v: unknown): v is Record { + return typeof v === 'object' && v !== null && !Array.isArray(v); +} + +function clampAttempts(n: number): number { + if (!Number.isFinite(n) || n < 1) return 1; + return Math.min(Math.floor(n), MAX_ENFORCE_ATTEMPTS); +} + +function safeStringify(v: unknown): string { + try { + return JSON.stringify(v, null, 2); + } catch { + return String(v); + } +} diff --git a/src/polyverdict/index.ts b/src/polyverdict/index.ts new file mode 100644 index 0000000..f766b7b --- /dev/null +++ b/src/polyverdict/index.ts @@ -0,0 +1,42 @@ +/** + * PolyVerdict v1 - public entrypoint. + * + * Opt-in structured-output enforcement for the Axion proxy. Import surface: + * + * import { + * detectSchemaTrigger, enforceOnce, buildRetryMessages, + * validateAndCoerce, parseJsonFromAssistant, MAX_ENFORCE_ATTEMPTS, + * } from 'axion/polyverdict'; + */ + +export { + validateAndCoerce, + stripMarkdownFences, + parseJsonFromAssistant, +} from './schema.js'; + +export { + detectSchemaTrigger, + enforceOnce, + buildViolationHint, + buildRetryMessages, + buildRetryMessagesAnthropic, + runEnforceLoop, + MAX_ENFORCE_ATTEMPTS, + type EnforceResult, + type EnforceLoopResult, + type RetryContext, +} from './enforce.js'; + +export type { + SchemaTrigger, + Ok, + Err, + ValidationResult, + ParseResult, + JsonSchema, + JsonSchemaType, + OpenAiMessage, + AnthropicMessage, + AnthropicTextBlock, +} from './types.js'; diff --git a/src/polyverdict/schema.test.ts b/src/polyverdict/schema.test.ts new file mode 100644 index 0000000..19b569c --- /dev/null +++ b/src/polyverdict/schema.test.ts @@ -0,0 +1,289 @@ +/** + * Tests for the PolyVerdict minimal JSON Schema subset: validation, type + * coercion, and JSON-from-assistant extraction. Also exercises the enforce + * trigger + single-shot enforcement helpers. + */ +import { describe, it, expect } from 'vitest'; +import { + validateAndCoerce, + stripMarkdownFences, + parseJsonFromAssistant, +} from './schema'; +import { + detectSchemaTrigger, + enforceOnce, + buildRetryMessages, + MAX_ENFORCE_ATTEMPTS, +} from './enforce'; + +describe('validateAndCoerce', () => { + it('accepts a valid object against a properties/required schema', () => { + const schema = { + type: 'object', + properties: { name: { type: 'string' }, age: { type: 'integer' } }, + required: ['name', 'age'], + }; + const r = validateAndCoerce({ name: 'Ada', age: 36 }, schema); + expect(r.ok).toBe(true); + if (r.ok) expect(r.value).toEqual({ name: 'Ada', age: 36 }); + }); + + it('reports a missing required property', () => { + const schema = { + type: 'object', + properties: { name: { type: 'string' }, age: { type: 'integer' } }, + required: ['name', 'age'], + }; + const r = validateAndCoerce({ name: 'Ada' }, schema); + expect(r.ok).toBe(false); + if (!r.ok) { + expect(r.errors.some((e) => e.includes('age'))).toBe(true); + } + }); + + it('coerces a numeric string to a number', () => { + const schema = { type: 'object', properties: { age: { type: 'number' } } }; + const r = validateAndCoerce({ age: '42' }, schema); + expect(r.ok).toBe(true); + if (r.ok) expect(r.value).toEqual({ age: 42 }); + }); + + it('coerces a numeric string to an integer', () => { + const r = validateAndCoerce('7', { type: 'integer' }); + expect(r.ok).toBe(true); + if (r.ok) expect(r.value).toBe(7); + }); + + it('rejects a non-integer numeric string for integer type', () => { + const r = validateAndCoerce('7.5', { type: 'integer' }); + expect(r.ok).toBe(false); + }); + + it('coerces "true"/"false" strings to booleans', () => { + const schema = { + type: 'object', + properties: { active: { type: 'boolean' }, deleted: { type: 'boolean' } }, + }; + const r = validateAndCoerce({ active: 'true', deleted: 'false' }, schema); + expect(r.ok).toBe(true); + if (r.ok) expect(r.value).toEqual({ active: true, deleted: false }); + }); + + it('coerces a number to a string when the schema says string', () => { + const r = validateAndCoerce({ id: 42 }, { + type: 'object', + properties: { id: { type: 'string' } }, + }); + expect(r.ok).toBe(true); + if (r.ok) expect(r.value).toEqual({ id: '42' }); + }); + + it('fails when an enum value is not allowed', () => { + const schema = { type: 'string', enum: ['red', 'green', 'blue'] }; + const r = validateAndCoerce('purple', schema); + expect(r.ok).toBe(false); + if (!r.ok) expect(r.errors[0]).toMatch(/enum|one of/i); + }); + + it('accepts an allowed enum value', () => { + const r = validateAndCoerce('green', { + type: 'string', + enum: ['red', 'green', 'blue'], + }); + expect(r.ok).toBe(true); + }); + + it('validates and coerces nested objects and arrays', () => { + const schema = { + type: 'object', + properties: { + user: { + type: 'object', + properties: { id: { type: 'integer' }, name: { type: 'string' } }, + required: ['id'], + }, + tags: { type: 'array', items: { type: 'string' } }, + scores: { type: 'array', items: { type: 'number' } }, + }, + required: ['user'], + }; + const r = validateAndCoerce( + { + user: { id: '5', name: 'Grace' }, + tags: ['a', 'b'], + scores: ['1', '2.5'], + }, + schema, + ); + expect(r.ok).toBe(true); + if (r.ok) { + expect(r.value).toEqual({ + user: { id: 5, name: 'Grace' }, + tags: ['a', 'b'], + scores: [1, 2.5], + }); + } + }); + + it('reports errors from deep inside nested structures', () => { + const schema = { + type: 'object', + properties: { + items: { + type: 'array', + items: { + type: 'object', + properties: { qty: { type: 'integer' } }, + required: ['qty'], + }, + }, + }, + }; + const r = validateAndCoerce({ items: [{ qty: 1 }, { name: 'x' }] }, schema); + expect(r.ok).toBe(false); + if (!r.ok) { + expect(r.errors.some((e) => e.includes('[1]') && e.includes('qty'))).toBe( + true, + ); + } + }); + + it('ignores unknown keywords', () => { + const schema = { + type: 'object', + properties: { n: { type: 'number' } }, + additionalProperties: false, + $schema: 'https://json-schema.org/draft/2020-12/schema', + description: 'ignored', + }; + const r = validateAndCoerce({ n: 1, extra: true }, schema); + expect(r.ok).toBe(true); + }); +}); + +describe('stripMarkdownFences', () => { + it('strips a ```json fenced block', () => { + const text = '```json\n{"a":1}\n```'; + expect(stripMarkdownFences(text)).toBe('{"a":1}'); + }); + + it('strips a bare ``` fenced block', () => { + const text = '```\n{"a":1}\n```'; + expect(stripMarkdownFences(text)).toBe('{"a":1}'); + }); + + it('extracts a fenced block embedded in prose', () => { + const text = 'Here you go:\n```json\n{"ok":true}\n```\nHope that helps!'; + expect(stripMarkdownFences(text)).toBe('{"ok":true}'); + }); + + it('returns trimmed text unchanged when no fence is present', () => { + expect(stripMarkdownFences(' {"a":1} ')).toBe('{"a":1}'); + }); +}); + +describe('parseJsonFromAssistant', () => { + it('parses fenced JSON', () => { + const r = parseJsonFromAssistant('```json\n{"a":1}\n```'); + expect(r.ok).toBe(true); + if (r.ok) expect(r.value).toEqual({ a: 1 }); + }); + + it('extracts a JSON object from surrounding prose', () => { + const r = parseJsonFromAssistant('Sure! {"a": 1, "b": [2, 3]} done.'); + expect(r.ok).toBe(true); + if (r.ok) expect(r.value).toEqual({ a: 1, b: [2, 3] }); + }); + + it('fails on non-JSON text', () => { + const r = parseJsonFromAssistant('no json here at all'); + expect(r.ok).toBe(false); + }); +}); + +describe('detectSchemaTrigger', () => { + it('detects an inline x-axion-schema header', () => { + const schema = { type: 'object', properties: { a: { type: 'number' } } }; + const headers = new Headers({ 'x-axion-schema': JSON.stringify(schema) }); + const t = detectSchemaTrigger(headers, {}); + expect(t).not.toBeNull(); + expect(t?.schema).toEqual(schema); + }); + + it('URL-decodes a percent-encoded header', () => { + const schema = { type: 'string' }; + const encoded = encodeURIComponent(JSON.stringify(schema)); + const headers = new Headers({ 'x-axion-schema': encoded }); + const t = detectSchemaTrigger(headers, {}); + expect(t?.schema).toEqual(schema); + }); + + it('detects response_format.json_schema in the body', () => { + const schema = { type: 'object' }; + const body = { + response_format: { + type: 'json_schema', + json_schema: { name: 'Widget', schema }, + }, + }; + const t = detectSchemaTrigger(new Headers(), body); + expect(t?.schema).toEqual(schema); + expect(t?.name).toBe('Widget'); + }); + + it('returns null when no trigger is present', () => { + expect(detectSchemaTrigger(new Headers(), { messages: [] })).toBeNull(); + }); +}); + +describe('enforceOnce', () => { + it('accepts + coerces valid fenced JSON', () => { + const schema = { + type: 'object', + properties: { n: { type: 'number' } }, + required: ['n'], + }; + const r = enforceOnce('```json\n{"n":"5"}\n```', schema); + expect(r.ok).toBe(true); + expect(r.value).toEqual({ n: 5 }); + expect(r.jsonText).toBe('{"n":5}'); + }); + + it('reports parse failures', () => { + const r = enforceOnce('definitely not json', { type: 'object' }); + expect(r.ok).toBe(false); + expect(r.errors[0]).toMatch(/parse failed/i); + }); + + it('reports schema violations', () => { + const r = enforceOnce('{"n": "abc"}', { + type: 'object', + properties: { n: { type: 'number' } }, + }); + expect(r.ok).toBe(false); + expect(r.errors.length).toBeGreaterThan(0); + }); +}); + +describe('buildRetryMessages', () => { + it('appends the failed output and a corrective user hint', () => { + const messages = [{ role: 'user' as const, content: 'give me json' }]; + const next = buildRetryMessages(messages, { + schema: { type: 'object' }, + errors: ['$.n: expected number, got string'], + assistantText: '{"n":"x"}', + }); + expect(next).toHaveLength(3); + expect(next[1]).toEqual({ role: 'assistant', content: '{"n":"x"}' }); + expect(next[2]?.role).toBe('user'); + expect(next[2]?.content).toContain('$.n: expected number'); + // Original array is not mutated. + expect(messages).toHaveLength(1); + }); +}); + +describe('MAX_ENFORCE_ATTEMPTS', () => { + it('is capped at 3 per BUILD-SPEC', () => { + expect(MAX_ENFORCE_ATTEMPTS).toBe(3); + }); +}); diff --git a/src/polyverdict/schema.ts b/src/polyverdict/schema.ts new file mode 100644 index 0000000..51fea73 --- /dev/null +++ b/src/polyverdict/schema.ts @@ -0,0 +1,406 @@ +/** + * PolyVerdict v1 - minimal JSON Schema subset validator + type coercion. + * + * Zero npm dependencies. Implements just enough of JSON Schema to enforce + * structured LLM output: + * + * - `type`: object | array | string | number | integer | boolean | null + * - `properties` (nested), `required`, `items` (schema or tuple), `enum` + * - unknown keywords are ignored (never an error) + * + * Coercion (BUILD-SPEC §7): string "42" → number/integer, "true"/"false" → + * boolean, number/boolean → string when the schema says string. Coercion is + * best-effort: a value is only rewritten when the target type is unambiguous. + * + * Public API: + * validateAndCoerce(data, schema): { ok, value } | { ok:false, errors } + * stripMarkdownFences(text): string + * parseJsonFromAssistant(text): { ok, value } | { ok:false, error } + */ + +import type { + JsonSchema, + JsonSchemaType, + ParseResult, + ValidationResult, +} from './types.js'; + +const KNOWN_TYPES: readonly JsonSchemaType[] = [ + 'object', + 'array', + 'string', + 'number', + 'integer', + 'boolean', + 'null', +]; + +/** + * Validate `data` against `schema`, coercing primitive types where the schema + * makes the intent unambiguous. Returns the (possibly rewritten) value on + * success, or a flat list of violation messages on failure. + */ +export function validateAndCoerce( + data: unknown, + schema: unknown, +): ValidationResult { + const errors: string[] = []; + const value = coerceNode(data, schema, '$', errors); + if (errors.length > 0) return { ok: false, errors }; + return { ok: true, value }; +} + +/** + * Recursively validate + coerce a single node. Pushes any violations onto + * `errors` (keyed by JSON path) and returns the best-effort coerced value. + */ +function coerceNode( + data: unknown, + schema: unknown, + path: string, + errors: string[], +): unknown { + // A non-object schema (or `true`) imposes no constraints. + if (!isPlainObject(schema)) return data; + const s = schema as JsonSchema; + + let value = data; + + // 1. Type coercion / checking. + const type = s.type; + if (typeof type === 'string' && (KNOWN_TYPES as string[]).includes(type)) { + value = coerceType(value, type as JsonSchemaType, path, errors); + } else if (Array.isArray(type)) { + value = coerceUnionType(value, type, path, errors); + } + + // 2. Structural recursion (independent of an explicit `type`, so a bare + // `{ properties: ... }` schema still validates nested objects). + if (isPlainObject(value) && isPlainObject(s.properties)) { + value = coerceObject(value, s, path, errors); + } else if (Array.isArray(value) && s.items !== undefined) { + value = coerceArray(value, s.items, path, errors); + } else if (isPlainObject(value) && Array.isArray(s.required)) { + // `required` without `properties`: still enforce presence. + checkRequired(value, s.required, path, errors); + } + + // 3. Enum membership (checked after coercion so "42" → 42 can still match). + if (Array.isArray(s.enum)) { + if (!s.enum.some((candidate) => deepEqual(candidate, value))) { + errors.push( + `${path}: value ${short(value)} is not one of ${short(s.enum)}`, + ); + } + } + + return value; +} + +/** Coerce/validate a value against a single primitive type. */ +function coerceType( + value: unknown, + type: JsonSchemaType, + path: string, + errors: string[], +): unknown { + switch (type) { + case 'string': + if (typeof value === 'string') return value; + // number → string, boolean → string. + if (typeof value === 'number' && Number.isFinite(value)) { + return String(value); + } + if (typeof value === 'boolean') return String(value); + errors.push(`${path}: expected string, got ${typeName(value)}`); + return value; + + case 'number': + if (typeof value === 'number' && Number.isFinite(value)) return value; + if (typeof value === 'string' && isNumericString(value)) { + return Number(value); + } + errors.push(`${path}: expected number, got ${typeName(value)}`); + return value; + + case 'integer': + if (typeof value === 'number' && Number.isInteger(value)) return value; + if (typeof value === 'string' && isIntegerString(value)) { + return Number(value); + } + errors.push(`${path}: expected integer, got ${typeName(value)}`); + return value; + + case 'boolean': + if (typeof value === 'boolean') return value; + if (value === 'true') return true; + if (value === 'false') return false; + errors.push(`${path}: expected boolean, got ${typeName(value)}`); + return value; + + case 'null': + if (value === null) return null; + errors.push(`${path}: expected null, got ${typeName(value)}`); + return value; + + case 'object': + if (isPlainObject(value)) return value; + errors.push(`${path}: expected object, got ${typeName(value)}`); + return value; + + case 'array': + if (Array.isArray(value)) return value; + errors.push(`${path}: expected array, got ${typeName(value)}`); + return value; + + default: + return value; + } +} + +/** + * Coerce against a union `type: [...]`. If the value already matches one of + * the listed types it is kept; otherwise we attempt coercion to the first + * type and only report the failure if none succeed. + */ +function coerceUnionType( + value: unknown, + types: JsonSchemaType[], + path: string, + errors: string[], +): unknown { + const known = types.filter((t) => (KNOWN_TYPES as string[]).includes(t)); + if (known.length === 0) return value; + + if (known.some((t) => matchesType(value, t))) return value; + + for (const t of known) { + const local: string[] = []; + const coerced = coerceType(value, t, path, local); + if (local.length === 0) return coerced; + } + + errors.push(`${path}: expected one of [${known.join(', ')}], got ${typeName(value)}`); + return value; +} + +/** Validate an object's `properties` + `required`, returning a coerced copy. */ +function coerceObject( + value: Record, + schema: JsonSchema, + path: string, + errors: string[], +): Record { + const props = schema.properties as Record; + const out: Record = { ...value }; + + if (Array.isArray(schema.required)) { + checkRequired(value, schema.required, path, errors); + } + + for (const key of Object.keys(props)) { + if (!(key in value)) continue; // presence handled by `required` + const childPath = `${path}.${key}`; + out[key] = coerceNode(value[key], props[key], childPath, errors); + } + + return out; +} + +/** Validate array `items` (single schema or positional tuple). */ +function coerceArray( + value: unknown[], + items: unknown, + path: string, + errors: string[], +): unknown[] { + if (Array.isArray(items)) { + // Tuple validation: schema per position. + return value.map((el, i) => + i < items.length ? coerceNode(el, items[i], `${path}[${i}]`, errors) : el, + ); + } + // Single schema applied to every element. + return value.map((el, i) => coerceNode(el, items, `${path}[${i}]`, errors)); +} + +/** Push an error for each missing required property. */ +function checkRequired( + value: Record, + required: string[], + path: string, + errors: string[], +): void { + for (const name of required) { + if (typeof name !== 'string') continue; + if (!(name in value)) { + errors.push(`${path}: missing required property "${name}"`); + } + } +} + +// ── JSON extraction helpers ───────────────────────────────────────────────── + +/** + * Remove Markdown code-fence wrappers from an assistant message. + * + * Handles ```` ```json … ``` ````, ```` ``` … ``` ````, and fenced blocks + * embedded in surrounding prose. If no fence is present the trimmed input is + * returned unchanged. + */ +export function stripMarkdownFences(text: string): string { + if (typeof text !== 'string') return ''; + const trimmed = text.trim(); + + // A fenced block anywhere in the text: ```lang\n … \n``` + const fenced = trimmed.match(/```[ \t]*([A-Za-z0-9_-]+)?[ \t]*\r?\n?([\s\S]*?)```/); + if (fenced) return fenced[2]!.trim(); + + return trimmed; +} + +/** + * Parse JSON from an assistant response, tolerating Markdown fences and + * surrounding prose. Falls back to extracting the first balanced-looking + * `{...}` / `[...]` span before giving up. + */ +export function parseJsonFromAssistant(text: string): ParseResult { + const stripped = stripMarkdownFences(text); + + const direct = tryParse(stripped); + if (direct.ok) return direct; + + const candidate = extractFirstJsonSpan(stripped); + if (candidate !== null) { + const fromSpan = tryParse(candidate); + if (fromSpan.ok) return fromSpan; + } + + return { + ok: false, + error: direct.ok ? 'unreachable' : direct.error, + }; +} + +function tryParse(text: string): ParseResult { + const t = text.trim(); + if (t === '') return { ok: false, error: 'empty response' }; + try { + return { ok: true, value: JSON.parse(t) }; + } catch (err) { + return { + ok: false, + error: err instanceof Error ? err.message : String(err), + }; + } +} + +/** + * Extract the first plausibly-complete JSON object/array from `text` by + * matching the outermost brackets. Best-effort only; the real parse still runs + * afterward. Skips brackets inside string literals. + */ +function extractFirstJsonSpan(text: string): string | null { + const start = firstBracketIndex(text); + if (start === -1) return null; + + const open = text[start]; + const close = open === '{' ? '}' : ']'; + let depth = 0; + let inString = false; + let escaped = false; + + for (let i = start; i < text.length; i++) { + const ch = text[i]; + if (inString) { + if (escaped) escaped = false; + else if (ch === '\\') escaped = true; + else if (ch === '"') inString = false; + continue; + } + if (ch === '"') inString = true; + else if (ch === open) depth++; + else if (ch === close) { + depth--; + if (depth === 0) return text.slice(start, i + 1); + } + } + return null; +} + +function firstBracketIndex(text: string): number { + const obj = text.indexOf('{'); + const arr = text.indexOf('['); + if (obj === -1) return arr; + if (arr === -1) return obj; + return Math.min(obj, arr); +} + +// ── small utilities ───────────────────────────────────────────────────────── + +/** True for a plain (non-array, non-null) object. */ +function isPlainObject(v: unknown): v is Record { + return typeof v === 'object' && v !== null && !Array.isArray(v); +} + +/** Does `value` already satisfy the given primitive type (no coercion)? */ +function matchesType(value: unknown, type: JsonSchemaType): boolean { + switch (type) { + case 'string': + return typeof value === 'string'; + case 'number': + return typeof value === 'number' && Number.isFinite(value); + case 'integer': + return typeof value === 'number' && Number.isInteger(value); + case 'boolean': + return typeof value === 'boolean'; + case 'null': + return value === null; + case 'object': + return isPlainObject(value); + case 'array': + return Array.isArray(value); + default: + return false; + } +} + +/** A string that Number() maps to a finite value (rejecting "" / whitespace). */ +function isNumericString(s: string): boolean { + if (s.trim() === '') return false; + return Number.isFinite(Number(s)); +} + +/** A numeric string whose value is an integer. */ +function isIntegerString(s: string): boolean { + return isNumericString(s) && Number.isInteger(Number(s)); +} + +/** Human-readable type name for error messages. */ +function typeName(v: unknown): string { + if (v === null) return 'null'; + if (Array.isArray(v)) return 'array'; + return typeof v; +} + +/** Structural equality via JSON serialization; good enough for enum checks. */ +function deepEqual(a: unknown, b: unknown): boolean { + if (a === b) return true; + try { + return JSON.stringify(a) === JSON.stringify(b); + } catch { + return false; + } +} + +/** Compact, length-capped JSON preview for error messages. */ +function short(v: unknown): string { + let str: string; + try { + str = JSON.stringify(v); + } catch { + str = String(v); + } + if (str === undefined) str = String(v); + return str.length > 80 ? `${str.slice(0, 77)}...` : str; +} diff --git a/src/polyverdict/types.ts b/src/polyverdict/types.ts new file mode 100644 index 0000000..d40533d --- /dev/null +++ b/src/polyverdict/types.ts @@ -0,0 +1,84 @@ +/** + * PolyVerdict v1 - shared type definitions. + * + * PolyVerdict is Axion's opt-in *enforce* path: when a caller supplies a JSON + * Schema (via the `x-axion-schema` header or an OpenAI-style + * `response_format.json_schema`), the proxy validates and type-coerces the + * model's output against that schema, retrying with violation hints up to + * {@link MAX_ENFORCE_ATTEMPTS} times. + * + * Scope (BUILD-SPEC §7 / D8): syntax validate + type coerce + retry ≤ 3. + * No semantic verification, no second-model check, no schema-registry DO. + */ + +/** + * A detected PolyVerdict enforce trigger: the JSON Schema the output must + * satisfy, plus an optional caller-supplied name (from `response_format`). + */ +export interface SchemaTrigger { + schema: unknown; + name?: string; +} + +/** Successful result carrying a (possibly coerced) value. */ +export interface Ok { + ok: true; + value: T; +} + +/** Failure result carrying a list of human-readable violation messages. */ +export interface Err { + ok: false; + errors: string[]; +} + +/** Result of {@link validateAndCoerce}: coerced value or a list of errors. */ +export type ValidationResult = Ok | Err; + +/** Result of {@link parseJsonFromAssistant}: parsed value or a single error. */ +export type ParseResult = Ok | { ok: false; error: string }; + +/** + * The JSON-primitive type names understood by the minimal schema subset. + * Anything else in a schema's `type` is treated as "no type constraint". + */ +export type JsonSchemaType = + | 'object' + | 'array' + | 'string' + | 'number' + | 'integer' + | 'boolean' + | 'null'; + +/** + * The subset of JSON Schema keywords PolyVerdict v1 understands. All other + * keywords are ignored (not an error). `schema` inputs are typed as `unknown` + * at the public boundary; this interface documents the recognised shape. + */ +export interface JsonSchema { + type?: JsonSchemaType | JsonSchemaType[]; + properties?: Record; + required?: string[]; + items?: unknown | unknown[]; + enum?: unknown[]; + [keyword: string]: unknown; +} + +/** A single chat message in the OpenAI chat-completions format. */ +export interface OpenAiMessage { + role: 'system' | 'user' | 'assistant' | 'tool' | string; + content: string; +} + +/** A text block in the Anthropic Messages format. */ +export interface AnthropicTextBlock { + type: 'text'; + text: string; +} + +/** A single message in the Anthropic Messages format. */ +export interface AnthropicMessage { + role: 'user' | 'assistant' | string; + content: string | AnthropicTextBlock[]; +} From 98b98309450a1f62936f9ea64d66e2a8939ca380 Mon Sep 17 00:00:00 2001 From: Cursor Agent Date: Thu, 16 Jul 2026 15:14:27 +0000 Subject: [PATCH 08/10] Add content normalization + provider adapters - content.ts: extractOpenAIAssistantText, extractAnthropicAssistantText, extractAssistantText (transport-aware dispatcher) - stream.ts: parseAnthropicSseData, final TextDecoder flush, provider-aware tee (backward-compatible openai default) - providers/: ProviderAdapter interface, openai + anthropic adapters, matchProvider registry - content.test.ts + stream.test.ts coverage Co-authored-by: Moses Man --- src/proxy/content.test.ts | 114 +++++++++++++++++++++++++++++++ src/proxy/content.ts | 102 +++++++++++++++++++++++++++ src/proxy/providers/anthropic.ts | 39 +++++++++++ src/proxy/providers/index.ts | 43 ++++++++++++ src/proxy/providers/openai.ts | 39 +++++++++++ src/proxy/providers/types.ts | 32 +++++++++ src/proxy/stream.test.ts | 75 ++++++++++++++++++++ src/proxy/stream.ts | 57 ++++++++++++++-- 8 files changed, 497 insertions(+), 4 deletions(-) create mode 100644 src/proxy/content.test.ts create mode 100644 src/proxy/content.ts create mode 100644 src/proxy/providers/anthropic.ts create mode 100644 src/proxy/providers/index.ts create mode 100644 src/proxy/providers/openai.ts create mode 100644 src/proxy/providers/types.ts diff --git a/src/proxy/content.test.ts b/src/proxy/content.test.ts new file mode 100644 index 0000000..3b0f774 --- /dev/null +++ b/src/proxy/content.test.ts @@ -0,0 +1,114 @@ +/** + * Tests for assistant text normalization: the OpenAI/Anthropic non-streaming + * extractors and the transport-aware extractAssistantText dispatcher. + */ +import { describe, it, expect } from "vitest"; +import { + extractOpenAIAssistantText, + extractAnthropicAssistantText, + extractAssistantText, +} from "./content"; + +describe("extractOpenAIAssistantText", () => { + it("reads string content from choices[0].message.content", () => { + const body = JSON.stringify({ + choices: [{ message: { role: "assistant", content: "Hello world" } }], + }); + expect(extractOpenAIAssistantText(body)).toBe("Hello world"); + }); + + it("joins array content parts (text objects and bare strings)", () => { + const body = JSON.stringify({ + choices: [ + { + message: { + role: "assistant", + content: [{ type: "text", text: "a" }, { text: "b" }, "c"], + }, + }, + ], + }); + expect(extractOpenAIAssistantText(body)).toBe("abc"); + }); + + it("returns empty string for malformed JSON", () => { + expect(extractOpenAIAssistantText("not json")).toBe(""); + }); + + it("returns empty string when content is missing", () => { + expect(extractOpenAIAssistantText(JSON.stringify({ choices: [] }))).toBe(""); + expect(extractOpenAIAssistantText(JSON.stringify({}))).toBe(""); + }); +}); + +describe("extractAnthropicAssistantText", () => { + it("joins content[] text blocks", () => { + const body = JSON.stringify({ + content: [ + { type: "text", text: "Hello" }, + { type: "text", text: " world" }, + ], + }); + expect(extractAnthropicAssistantText(body)).toBe("Hello world"); + }); + + it("skips non-text blocks (e.g. tool_use)", () => { + const body = JSON.stringify({ + content: [ + { type: "text", text: "before" }, + { type: "tool_use", id: "t1", name: "x", input: {} }, + { type: "text", text: " after" }, + ], + }); + expect(extractAnthropicAssistantText(body)).toBe("before after"); + }); + + it("returns empty string for malformed JSON or missing content", () => { + expect(extractAnthropicAssistantText("nope")).toBe(""); + expect(extractAnthropicAssistantText(JSON.stringify({}))).toBe(""); + }); +}); + +describe("extractAssistantText", () => { + it("returns accumulated delta text as-is (trimmed) for SSE openai", () => { + const out = extractAssistantText({ + provider: "openai", + isSse: true, + accumulated: " streamed delta ", + }); + expect(out).toBe("streamed delta"); + }); + + it("returns accumulated delta text as-is (trimmed) for SSE anthropic", () => { + const out = extractAssistantText({ + provider: "anthropic", + isSse: true, + accumulated: "anthropic stream", + }); + expect(out).toBe("anthropic stream"); + }); + + it("parses non-SSE openai body via the OpenAI extractor", () => { + const body = JSON.stringify({ + choices: [{ message: { content: "final answer" } }], + }); + const out = extractAssistantText({ + provider: "openai", + isSse: false, + accumulated: body, + }); + expect(out).toBe("final answer"); + }); + + it("parses non-SSE anthropic body via the Anthropic extractor", () => { + const body = JSON.stringify({ + content: [{ type: "text", text: "final answer" }], + }); + const out = extractAssistantText({ + provider: "anthropic", + isSse: false, + accumulated: body, + }); + expect(out).toBe("final answer"); + }); +}); diff --git a/src/proxy/content.ts b/src/proxy/content.ts new file mode 100644 index 0000000..85d9dd6 --- /dev/null +++ b/src/proxy/content.ts @@ -0,0 +1,102 @@ +/** + * Axion Lens - Assistant text normalization. + * + * Turns a provider's completed (non-streaming) response body, or the already + * accumulated SSE delta text, into a single plain-text string suitable for + * belief extraction. We never feed raw JSON to the lens: non-SSE bodies are + * parsed via the provider-specific extractor below. + * + * These functions are intentionally defensive - a malformed or unexpected body + * yields "" rather than throwing, because extraction runs in the background and + * must never break the proxy. + */ + +import type { ProviderId } from "./providers/types"; + +/** + * Extract the assistant text from an OpenAI chat completion (non-streaming) + * response body. Reads `choices[0].message.content`, which is either a string + * or an array of content parts (`{ type: "text", text }` or bare strings). + */ +export function extractOpenAIAssistantText(rawBody: string): string { + let json: unknown; + try { + json = JSON.parse(rawBody); + } catch { + return ""; + } + + const content = (json as any)?.choices?.[0]?.message?.content; + return contentToText(content); +} + +/** + * Extract the assistant text from an Anthropic Messages (non-streaming) + * response body. Joins every `content[]` block where `type === "text"`. + */ +export function extractAnthropicAssistantText(rawBody: string): string { + let json: unknown; + try { + json = JSON.parse(rawBody); + } catch { + return ""; + } + + const content = (json as any)?.content; + if (!Array.isArray(content)) { + // Some payloads may carry a bare string content; be lenient. + return typeof content === "string" ? content : ""; + } + + let text = ""; + for (const block of content) { + if (block && block.type === "text" && typeof block.text === "string") { + text += block.text; + } + } + return text; +} + +/** + * Normalize assistant text for a response, regardless of transport. + * + * - SSE: `accumulated` already holds the joined delta text (produced by the + * stream tee), so we just return it (trimmed of surrounding whitespace). + * - Non-SSE: `accumulated` is the raw response body; parse it with the + * provider-specific extractor. + */ +export function extractAssistantText(opts: { + provider: ProviderId; + isSse: boolean; + accumulated: string; +}): string { + const { provider, isSse, accumulated } = opts; + + if (isSse) { + return accumulated.trim(); + } + + const raw = provider === "anthropic" + ? extractAnthropicAssistantText(accumulated) + : extractOpenAIAssistantText(accumulated); + + return raw.trim(); +} + +/** + * Coerce an OpenAI-style `content` value (string | array of parts) into text. + */ +function contentToText(content: unknown): string { + if (typeof content === "string") return content; + if (!Array.isArray(content)) return ""; + + let text = ""; + for (const part of content) { + if (typeof part === "string") { + text += part; + } else if (part && typeof part.text === "string") { + text += part.text; + } + } + return text; +} diff --git a/src/proxy/providers/anthropic.ts b/src/proxy/providers/anthropic.ts new file mode 100644 index 0000000..8685085 --- /dev/null +++ b/src/proxy/providers/anthropic.ts @@ -0,0 +1,39 @@ +/** + * Axion Lens - Anthropic Messages adapter. + * + * Routes `POST /v1/messages`, validates a non-empty `messages[]`, and + * normalizes non-streaming responses via the shared content extractor. + */ + +import { extractAnthropicAssistantText } from "../content"; +import type { ProviderAdapter, ValidationResult } from "./types"; + +const UPSTREAM_PATH = "/v1/messages"; + +export const anthropicAdapter: ProviderAdapter = { + id: "anthropic", + + match(pathname: string, method: string): boolean { + return method.toUpperCase() === "POST" && pathname === UPSTREAM_PATH; + }, + + upstreamPath: UPSTREAM_PATH, + + validateRequest(body: unknown): ValidationResult { + const messages = (body as any)?.messages; + if (!Array.isArray(messages) || messages.length === 0) { + return { + ok: false, + message: "Request must include a non-empty 'messages' array", + }; + } + return { ok: true }; + }, + + extractAssistantText(rawBody: string): string { + return extractAnthropicAssistantText(rawBody); + }, +}; + +// Re-export the content helper for callers that want it directly. +export { extractAnthropicAssistantText }; diff --git a/src/proxy/providers/index.ts b/src/proxy/providers/index.ts new file mode 100644 index 0000000..0f7a5cb --- /dev/null +++ b/src/proxy/providers/index.ts @@ -0,0 +1,43 @@ +/** + * Axion Lens - Provider registry + matcher. + * + * `matchProvider(pathname, method)` returns the adapter that handles a request, + * or null when no provider claims it (the caller then falls through to 404 / + * other routing). + */ + +import { anthropicAdapter } from "./anthropic"; +import { openaiAdapter } from "./openai"; +import type { ProviderAdapter, ProviderId } from "./types"; + +/** All provider adapters, in match priority order. */ +export const PROVIDERS: readonly ProviderAdapter[] = [ + openaiAdapter, + anthropicAdapter, +]; + +/** Find the adapter that handles a request path + method, or null. */ +export function matchProvider( + pathname: string, + method: string +): ProviderAdapter | null { + for (const provider of PROVIDERS) { + if (provider.match(pathname, method)) return provider; + } + return null; +} + +/** Look up an adapter by its id. */ +export function getProvider(id: ProviderId): ProviderAdapter { + const found = PROVIDERS.find((p) => p.id === id); + if (!found) throw new Error(`Unknown provider: ${id}`); + return found; +} + +export { openaiAdapter } from "./openai"; +export { anthropicAdapter } from "./anthropic"; +export type { + ProviderAdapter, + ProviderId, + ValidationResult, +} from "./types"; diff --git a/src/proxy/providers/openai.ts b/src/proxy/providers/openai.ts new file mode 100644 index 0000000..3b1ba5c --- /dev/null +++ b/src/proxy/providers/openai.ts @@ -0,0 +1,39 @@ +/** + * Axion Lens - OpenAI chat completions adapter. + * + * Routes `POST /v1/chat/completions`, validates a non-empty `messages[]`, and + * normalizes non-streaming responses via the shared content extractor. + */ + +import { extractOpenAIAssistantText } from "../content"; +import type { ProviderAdapter, ValidationResult } from "./types"; + +const UPSTREAM_PATH = "/v1/chat/completions"; + +export const openaiAdapter: ProviderAdapter = { + id: "openai", + + match(pathname: string, method: string): boolean { + return method.toUpperCase() === "POST" && pathname === UPSTREAM_PATH; + }, + + upstreamPath: UPSTREAM_PATH, + + validateRequest(body: unknown): ValidationResult { + const messages = (body as any)?.messages; + if (!Array.isArray(messages) || messages.length === 0) { + return { + ok: false, + message: "Request must include a non-empty 'messages' array", + }; + } + return { ok: true }; + }, + + extractAssistantText(rawBody: string): string { + return extractOpenAIAssistantText(rawBody); + }, +}; + +// Re-export the content helper for callers that want it directly. +export { extractOpenAIAssistantText }; diff --git a/src/proxy/providers/types.ts b/src/proxy/providers/types.ts new file mode 100644 index 0000000..e8a7ee1 --- /dev/null +++ b/src/proxy/providers/types.ts @@ -0,0 +1,32 @@ +/** + * Axion Lens - Provider adapter interface. + * + * A ProviderAdapter describes how to route, validate, and normalize a single + * upstream model API (OpenAI chat completions, Anthropic Messages). The proxy + * matches an incoming request to an adapter, validates the body, forwards to + * `upstreamPath`, and later normalizes the response text for belief extraction. + */ + +export type ProviderId = "openai" | "anthropic"; + +/** Result of validating an inbound request body. */ +export type ValidationResult = + | { ok: true } + | { ok: false; message: string }; + +export interface ProviderAdapter { + /** Stable identifier for this provider. */ + id: ProviderId; + + /** True if this adapter handles the given request path + method. */ + match(pathname: string, method: string): boolean; + + /** Path to forward to upstream (appended to the configured base URL). */ + upstreamPath: string; + + /** Validate the (already JSON-parsed) request body for this provider. */ + validateRequest(body: unknown): ValidationResult; + + /** Extract assistant text from a non-streaming response body. */ + extractAssistantText(rawBody: string): string; +} diff --git a/src/proxy/stream.test.ts b/src/proxy/stream.test.ts index 056523e..5a1e4bb 100644 --- a/src/proxy/stream.test.ts +++ b/src/proxy/stream.test.ts @@ -6,6 +6,7 @@ import { describe, it, expect } from "vitest"; import { parseSseData, + parseAnthropicSseData, SseLineParser, teeResponseForExtraction, } from "./stream"; @@ -42,6 +43,37 @@ describe("parseSseData", () => { }); }); +describe("parseAnthropicSseData", () => { + it("extracts text from a content_block_delta text_delta", () => { + const payload = JSON.stringify({ + type: "content_block_delta", + index: 0, + delta: { type: "text_delta", text: "Hello" }, + }); + const r = parseAnthropicSseData(payload); + expect(r.text).toBe("Hello"); + expect(r.done).toBe(false); + }); + + it("marks done=true on message_stop", () => { + const r = parseAnthropicSseData(JSON.stringify({ type: "message_stop" })); + expect(r.done).toBe(true); + expect(r.text).toBe(""); + }); + + it("ignores non-text deltas and other event types", () => { + const inputJson = JSON.stringify({ + type: "content_block_delta", + delta: { type: "input_json_delta", partial_json: "{" }, + }); + expect(parseAnthropicSseData(inputJson).text).toBe(""); + expect( + parseAnthropicSseData(JSON.stringify({ type: "message_start" })).text + ).toBe(""); + expect(parseAnthropicSseData("not json").text).toBe(""); + }); +}); + describe("SseLineParser", () => { it("parses complete records delimited by \\n\\n", () => { const p = new SseLineParser(); @@ -117,6 +149,49 @@ describe("teeResponseForExtraction", () => { expect(await accumulatedText).toBe("Hello world"); }); + it("tees an Anthropic SSE stream and accumulates text_delta text", async () => { + const sse = + 'event: content_block_delta\n' + + 'data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":"Hello"}}\n\n' + + 'event: content_block_delta\n' + + 'data: {"type":"content_block_delta","index":0,"delta":{"type":"text_delta","text":" world"}}\n\n' + + 'event: message_stop\n' + + 'data: {"type":"message_stop"}\n\n'; + const original = new Response(sse, { + headers: { "content-type": "text/event-stream" }, + }); + const { response, accumulatedText } = teeResponseForExtraction( + original, + true, + "anthropic" + ); + + expect(await response.text()).toBe(sse); + expect(await accumulatedText).toBe("Hello world"); + }); + + it("flushes trailing multi-byte UTF-8 split across chunk boundaries", async () => { + // "é" is 0xC3 0xA9; split the two bytes across two stream chunks so the + // decoder must hold the first byte until the final flush. + const bytes = new TextEncoder().encode("café"); + const splitAt = bytes.length - 1; // last byte of the "é" sequence + const stream = new ReadableStream({ + start(c) { + c.enqueue(bytes.slice(0, splitAt)); + c.enqueue(bytes.slice(splitAt)); + c.close(); + }, + }); + const original = new Response(stream); + const { response, accumulatedText } = teeResponseForExtraction( + original, + false + ); + + expect(await response.text()).toBe("café"); + expect(await accumulatedText).toBe("café"); + }); + it("handles a null body (returns empty accumulated text)", async () => { const original = new Response(null, { status: 204 }); const { response, accumulatedText } = teeResponseForExtraction(original, false); diff --git a/src/proxy/stream.ts b/src/proxy/stream.ts index 7ae7539..8a81008 100644 --- a/src/proxy/stream.ts +++ b/src/proxy/stream.ts @@ -10,6 +10,7 @@ */ import type { StreamChunk } from "./types"; +import type { ProviderId } from "./providers/types"; /** * Parse a single SSE `data:` payload (without the `data: ` prefix) into text + done. @@ -44,6 +45,39 @@ export function parseSseData(payload: string): StreamChunk { return { raw: payload, text, done: false }; } +/** + * Parse a single Anthropic Messages SSE `data:` payload into text + done. + * + * Anthropic streams emit typed events; assistant text arrives as + * `content_block_delta` events whose `delta.type === "text_delta"`. The stream + * terminates with a `message_stop` event (there is no `[DONE]` sentinel). + */ +export function parseAnthropicSseData(payload: string): StreamChunk { + const trimmed = payload.trim(); + + let text = ""; + let done = false; + try { + const json = JSON.parse(trimmed); + const type = json?.type; + if (type === "content_block_delta" && json?.delta?.type === "text_delta") { + const t = json.delta.text; + if (typeof t === "string") text = t; + } else if (type === "message_stop") { + done = true; + } + } catch { + // Not JSON or not a shape we recognize - contribute nothing. + } + + return { raw: payload, text, done }; +} + +/** Select the SSE payload parser for a provider. */ +function sseParserFor(provider: ProviderId): (payload: string) => StreamChunk { + return provider === "anthropic" ? parseAnthropicSseData : parseSseData; +} + /** * A tiny stateful SSE line parser. Feed it decoded chunks of text; it yields * complete `data:` payloads as they arrive (handling the `\n\n` SSE record @@ -117,12 +151,14 @@ export class SseLineParser { * Promise that resolves with the full accumulated text once the body * has been fully consumed. * - * For SSE responses we parse `data:` lines to extract just the delta text. + * For SSE responses we parse `data:` lines to extract just the delta text, + * using the parser for `provider` (defaults to OpenAI for backward compat). * For non-SSE responses we accumulate raw text. */ export function teeResponseForExtraction( response: Response, - isSse: boolean + isSse: boolean, + provider: ProviderId = "openai" ): { response: Response; accumulatedText: Promise } { const body = response.body; if (!body) { @@ -130,6 +166,7 @@ export function teeResponseForExtraction( } const [callerStream, extractionStream] = body.tee(); + const parseSse = sseParserFor(provider); const accumulatedText = (async () => { let text = ""; @@ -143,16 +180,28 @@ export function teeResponseForExtraction( const decoded = decoder.decode(value, { stream: true }); if (isSse) { for (const payload of parser.feed(decoded)) { - text += parseSseData(payload).text; + text += parseSse(payload).text; } } else { text += decoded; } } + // Final flush of the decoder to emit any bytes held back mid multi-byte + // sequence at the end of the stream (decode() without stream: true). + const tail = decoder.decode(); + if (tail) { + if (isSse) { + for (const payload of parser.feed(tail)) { + text += parseSse(payload).text; + } + } else { + text += tail; + } + } // Flush any trailing SSE data the parser still has buffered. if (isSse) { for (const payload of parser.flush()) { - text += parseSseData(payload).text; + text += parseSse(payload).text; } } } catch { From 2b2368bdaf9040da84ec3bf00202ef9804ce7cf6 Mon Sep 17 00:00:00 2001 From: Cursor Agent Date: Thu, 16 Jul 2026 15:15:28 +0000 Subject: [PATCH 09/10] feat(state): flatten belief timeline and stamp sessionId on extract DO stores sessionName and returns flat ExtractedBelief[]; extraction passes sessionId into the lens. Includes pure flatten helpers + tests. Co-authored-by: Moses Man --- package-lock.json | 543 +++++++++++++++++++++++-- src/proxy/beliefs.ts | 5 +- src/proxy/extraction.ts | 2 +- src/state/SessionDurableObject.test.ts | 103 +++++ src/state/SessionDurableObject.ts | 50 ++- src/state/sessionBeliefs.test.ts | 74 ++++ src/state/sessionBeliefs.ts | 53 +++ 7 files changed, 782 insertions(+), 48 deletions(-) create mode 100644 src/state/SessionDurableObject.test.ts create mode 100644 src/state/sessionBeliefs.test.ts create mode 100644 src/state/sessionBeliefs.ts diff --git a/package-lock.json b/package-lock.json index 72ff659..0e806d6 100644 --- a/package-lock.json +++ b/package-lock.json @@ -7,6 +7,7 @@ "": { "name": "axion", "version": "0.1.0", + "license": "MIT", "devDependencies": { "@cloudflare/workers-types": "^4.20240620.0", "typescript": "^5.5.3", @@ -206,6 +207,24 @@ "esbuild": "*" } }, + "node_modules/@esbuild/aix-ppc64": { + "version": "0.28.1", + "resolved": "https://registry.npmjs.org/@esbuild/aix-ppc64/-/aix-ppc64-0.28.1.tgz", + "integrity": "sha512-Svl7tq8k/08+p6CXPpRjQ1fKX+1odH/BQbb48fV6fj3CWHhsoIOoY87w1oHXm0qEpkIK3ZfVgp0hed3XBXzXMQ==", + "cpu": [ + "ppc64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "aix" + ], + "peer": true, + "engines": { + "node": ">=18" + } + }, "node_modules/@esbuild/android-arm": { "version": "0.17.19", "resolved": "https://registry.npmjs.org/@esbuild/android-arm/-/android-arm-0.17.19.tgz", @@ -478,6 +497,24 @@ "node": ">=12" } }, + "node_modules/@esbuild/netbsd-arm64": { + "version": "0.28.1", + "resolved": "https://registry.npmjs.org/@esbuild/netbsd-arm64/-/netbsd-arm64-0.28.1.tgz", + "integrity": "sha512-oks0DYbLwWMmaakTsCb+zL4E+aHRVLom9IJZOAthMQEPiQmydXHkziYEsGYRx0uNV/IjEKGAV941JzH02pflqw==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "netbsd" + ], + "peer": true, + "engines": { + "node": ">=18" + } + }, "node_modules/@esbuild/netbsd-x64": { "version": "0.17.19", "resolved": "https://registry.npmjs.org/@esbuild/netbsd-x64/-/netbsd-x64-0.17.19.tgz", @@ -495,6 +532,24 @@ "node": ">=12" } }, + "node_modules/@esbuild/openbsd-arm64": { + "version": "0.28.1", + "resolved": "https://registry.npmjs.org/@esbuild/openbsd-arm64/-/openbsd-arm64-0.28.1.tgz", + "integrity": "sha512-MEFJe5C3R8pwXdZ5Y21oo6m7ePiS0d9pWucn99O/wvyJZChoIQKrQDxKrGeW8F5+T0okTHesAmDeiHDTIq0V/Q==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "openbsd" + ], + "peer": true, + "engines": { + "node": ">=18" + } + }, "node_modules/@esbuild/openbsd-x64": { "version": "0.17.19", "resolved": "https://registry.npmjs.org/@esbuild/openbsd-x64/-/openbsd-x64-0.17.19.tgz", @@ -512,6 +567,24 @@ "node": ">=12" } }, + "node_modules/@esbuild/openharmony-arm64": { + "version": "0.28.1", + "resolved": "https://registry.npmjs.org/@esbuild/openharmony-arm64/-/openharmony-arm64-0.28.1.tgz", + "integrity": "sha512-ge+Z7EXFNt2BO1oAMsVpiQ8EwndV9i1xXerAeTIK7AtPs3bKFXQM7nlRxDSIUIMeueR1CNXxqztLzdNeReKBJg==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "openharmony" + ], + "peer": true, + "engines": { + "node": ">=18" + } + }, "node_modules/@esbuild/sunos-x64": { "version": "0.17.19", "resolved": "https://registry.npmjs.org/@esbuild/sunos-x64/-/sunos-x64-0.17.19.tgz", @@ -1120,9 +1193,6 @@ "arm64" ], "dev": true, - "libc": [ - "glibc" - ], "license": "MIT", "optional": true, "os": [ @@ -1140,9 +1210,6 @@ "arm64" ], "dev": true, - "libc": [ - "musl" - ], "license": "MIT", "optional": true, "os": [ @@ -1160,9 +1227,6 @@ "ppc64" ], "dev": true, - "libc": [ - "glibc" - ], "license": "MIT", "optional": true, "os": [ @@ -1180,9 +1244,6 @@ "s390x" ], "dev": true, - "libc": [ - "glibc" - ], "license": "MIT", "optional": true, "os": [ @@ -1200,9 +1261,6 @@ "x64" ], "dev": true, - "libc": [ - "glibc" - ], "license": "MIT", "optional": true, "os": [ @@ -1220,9 +1278,6 @@ "x64" ], "dev": true, - "libc": [ - "musl" - ], "license": "MIT", "optional": true, "os": [ @@ -1895,9 +1950,6 @@ "arm64" ], "dev": true, - "libc": [ - "glibc" - ], "license": "MPL-2.0", "optional": true, "os": [ @@ -1919,9 +1971,6 @@ "arm64" ], "dev": true, - "libc": [ - "musl" - ], "license": "MPL-2.0", "optional": true, "os": [ @@ -1943,9 +1992,6 @@ "x64" ], "dev": true, - "libc": [ - "glibc" - ], "license": "MPL-2.0", "optional": true, "os": [ @@ -1967,9 +2013,6 @@ "x64" ], "dev": true, - "libc": [ - "musl" - ], "license": "MPL-2.0", "optional": true, "os": [ @@ -2588,6 +2631,402 @@ } } }, + "node_modules/vitest/node_modules/@esbuild/android-arm": { + "version": "0.28.1", + "resolved": "https://registry.npmjs.org/@esbuild/android-arm/-/android-arm-0.28.1.tgz", + "integrity": "sha512-0k2F129Xdio1TdJfzJ8sy1Q47vUD2NnwdhiAf7drUN1EBTfPf4hsFCtmMgu/6m8JSzsBrlmVjudMBQqOfG8usQ==", + "cpu": [ + "arm" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "android" + ], + "peer": true, + "engines": { + "node": ">=18" + } + }, + "node_modules/vitest/node_modules/@esbuild/android-arm64": { + "version": "0.28.1", + "resolved": "https://registry.npmjs.org/@esbuild/android-arm64/-/android-arm64-0.28.1.tgz", + "integrity": "sha512-34EGEbCIAgosYz6goLcopX6Mo7NyGv9tfwEM2/7Ce2VcVRk568iSvniGWcUXIy7wEDR1wzolcxcriFVrWYcwBg==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "android" + ], + "peer": true, + "engines": { + "node": ">=18" + } + }, + "node_modules/vitest/node_modules/@esbuild/android-x64": { + "version": "0.28.1", + "resolved": "https://registry.npmjs.org/@esbuild/android-x64/-/android-x64-0.28.1.tgz", + "integrity": "sha512-dbwY7ltSMDWsRatcRpCnES4F+im88OCUgGZjy52shC7GqHRE/cYlxNbB4Z4UpJswpcc4Qxd2oE/ufM0p61IKng==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "android" + ], + "peer": true, + "engines": { + "node": ">=18" + } + }, + "node_modules/vitest/node_modules/@esbuild/darwin-arm64": { + "version": "0.28.1", + "resolved": "https://registry.npmjs.org/@esbuild/darwin-arm64/-/darwin-arm64-0.28.1.tgz", + "integrity": "sha512-TZbWkQY7kvTAXbXUT7uVACR5cMHsDiSz9z7ZKAX/RTq/WJEk3QyRr0wZpNhBDX+/0CtdqUIJlOiodQcta6tY3Q==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "darwin" + ], + "peer": true, + "engines": { + "node": ">=18" + } + }, + "node_modules/vitest/node_modules/@esbuild/darwin-x64": { + "version": "0.28.1", + "resolved": "https://registry.npmjs.org/@esbuild/darwin-x64/-/darwin-x64-0.28.1.tgz", + "integrity": "sha512-zfdzgK9ACBNZLI/CyHTOx81SyNbM6YXn7rxSgX97VjyiPl9W1i4Ka4fgKECEoFCKGpvBj5qArWIGgQjOwkgskQ==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "darwin" + ], + "peer": true, + "engines": { + "node": ">=18" + } + }, + "node_modules/vitest/node_modules/@esbuild/freebsd-arm64": { + "version": "0.28.1", + "resolved": "https://registry.npmjs.org/@esbuild/freebsd-arm64/-/freebsd-arm64-0.28.1.tgz", + "integrity": "sha512-wG2EA8ENdEI0qhkSZMjfqrdY+ziCYCPMmtZjjIwOmXFjmyzEHn+UUxk5of+SYsjtfs3VpnlC7QLzSI5hY/rOAw==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "freebsd" + ], + "peer": true, + "engines": { + "node": ">=18" + } + }, + "node_modules/vitest/node_modules/@esbuild/freebsd-x64": { + "version": "0.28.1", + "resolved": "https://registry.npmjs.org/@esbuild/freebsd-x64/-/freebsd-x64-0.28.1.tgz", + "integrity": "sha512-i7dZ9vQgnvSCzi/rYCXNgtF/U+eKZNJBzu3eTQbRgHnM7tNSizLOkRFAl3qzVc/Op/u5YkHHa4pf/3DOYHthLQ==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "freebsd" + ], + "peer": true, + "engines": { + "node": ">=18" + } + }, + "node_modules/vitest/node_modules/@esbuild/linux-arm": { + "version": "0.28.1", + "resolved": "https://registry.npmjs.org/@esbuild/linux-arm/-/linux-arm-0.28.1.tgz", + "integrity": "sha512-qVXBOHQS+d5Y722GwJzJUtOLlX7km3CraOaGormF1pDtPd2C/l1SHRPgjLunLGe51Sh5YYWKMFDyV4SxgMQYTQ==", + "cpu": [ + "arm" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "peer": true, + "engines": { + "node": ">=18" + } + }, + "node_modules/vitest/node_modules/@esbuild/linux-arm64": { + "version": "0.28.1", + "resolved": "https://registry.npmjs.org/@esbuild/linux-arm64/-/linux-arm64-0.28.1.tgz", + "integrity": "sha512-yHs+0uc8+nvEAfAfxrWQKK5peSNzBc4PegcMO0EJ2hT71uA7vB8Ihg2e77R2P7SG5uYjPbHlLLmve4LLLRCf0g==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "peer": true, + "engines": { + "node": ">=18" + } + }, + "node_modules/vitest/node_modules/@esbuild/linux-ia32": { + "version": "0.28.1", + "resolved": "https://registry.npmjs.org/@esbuild/linux-ia32/-/linux-ia32-0.28.1.tgz", + "integrity": "sha512-d1z4ZuP0ajrfz/FhGT4vv278rX8KnPPJx8i5+AtK7TYbx9Le9F1hyzurZpkEyjkGa9dUGhQow4C1NmeGvqxN2w==", + "cpu": [ + "ia32" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "peer": true, + "engines": { + "node": ">=18" + } + }, + "node_modules/vitest/node_modules/@esbuild/linux-loong64": { + "version": "0.28.1", + "resolved": "https://registry.npmjs.org/@esbuild/linux-loong64/-/linux-loong64-0.28.1.tgz", + "integrity": "sha512-M5sRjUVZrkm1OAPR3dlOYzNmN+loZKGVi1VUQGrwuqLcbR6qeAz+famMhjASeH3YVKvZz+zT1jlh/keC3Rj/lg==", + "cpu": [ + "loong64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "peer": true, + "engines": { + "node": ">=18" + } + }, + "node_modules/vitest/node_modules/@esbuild/linux-mips64el": { + "version": "0.28.1", + "resolved": "https://registry.npmjs.org/@esbuild/linux-mips64el/-/linux-mips64el-0.28.1.tgz", + "integrity": "sha512-mRObBZeHh2OxcBFPWE/FjylkRgZdYuiTR3vaTozquCGOH14iP9oN4x4Ge81CoIDYQrXmIxpFumJBu5MtZpnQJQ==", + "cpu": [ + "mips64el" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "peer": true, + "engines": { + "node": ">=18" + } + }, + "node_modules/vitest/node_modules/@esbuild/linux-ppc64": { + "version": "0.28.1", + "resolved": "https://registry.npmjs.org/@esbuild/linux-ppc64/-/linux-ppc64-0.28.1.tgz", + "integrity": "sha512-slScBsMAb3GFDcdrCgLwZtPYRoH2H/youv10QiZyRjmsP48fznoveWytSgCI/R0ZcUgpc0ZhIUEx6LHts8yrfQ==", + "cpu": [ + "ppc64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "peer": true, + "engines": { + "node": ">=18" + } + }, + "node_modules/vitest/node_modules/@esbuild/linux-riscv64": { + "version": "0.28.1", + "resolved": "https://registry.npmjs.org/@esbuild/linux-riscv64/-/linux-riscv64-0.28.1.tgz", + "integrity": "sha512-kw0owk1o0GFETUJyW0jc0G4Yzs0BHZn0JDZ8JRT088vjJYX777BAs1fDGxAC+q831qOs2DTC96mNsG2opdfyyQ==", + "cpu": [ + "riscv64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "peer": true, + "engines": { + "node": ">=18" + } + }, + "node_modules/vitest/node_modules/@esbuild/linux-s390x": { + "version": "0.28.1", + "resolved": "https://registry.npmjs.org/@esbuild/linux-s390x/-/linux-s390x-0.28.1.tgz", + "integrity": "sha512-/lAIjX8aYFRByhh6L5rYtPEDRqa9de/4V/juOXcta5frjvzXO4/sqEtyytse0g3zZFuWu5cDN0MkLz2qRDD2Ag==", + "cpu": [ + "s390x" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "peer": true, + "engines": { + "node": ">=18" + } + }, + "node_modules/vitest/node_modules/@esbuild/linux-x64": { + "version": "0.28.1", + "resolved": "https://registry.npmjs.org/@esbuild/linux-x64/-/linux-x64-0.28.1.tgz", + "integrity": "sha512-u/anNYF2mmVOEDwLtnQ1wOr3EZ9sTNGLWrsYGYwHWzGA3Si84IOkHXlbWTD1NB+9/1lcnweYKO54uhxZydNzfA==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "linux" + ], + "peer": true, + "engines": { + "node": ">=18" + } + }, + "node_modules/vitest/node_modules/@esbuild/netbsd-x64": { + "version": "0.28.1", + "resolved": "https://registry.npmjs.org/@esbuild/netbsd-x64/-/netbsd-x64-0.28.1.tgz", + "integrity": "sha512-aeL6lAnN89Hz43Mlh1G8ARasbuoYvSITDEx0tHh5b7jJnHcssqgjy9Yx430GDpmCa6OyrKoS0aNRjKundRizGg==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "netbsd" + ], + "peer": true, + "engines": { + "node": ">=18" + } + }, + "node_modules/vitest/node_modules/@esbuild/openbsd-x64": { + "version": "0.28.1", + "resolved": "https://registry.npmjs.org/@esbuild/openbsd-x64/-/openbsd-x64-0.28.1.tgz", + "integrity": "sha512-i/ZLIOafE0Z8cI/XANJAixoJL/uRAoS2xOA3rb0xN+KK0K177cMAsQYkzHtBrtMXAKuAc7HGgcWiZ/sRC1Nxgw==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "openbsd" + ], + "peer": true, + "engines": { + "node": ">=18" + } + }, + "node_modules/vitest/node_modules/@esbuild/sunos-x64": { + "version": "0.28.1", + "resolved": "https://registry.npmjs.org/@esbuild/sunos-x64/-/sunos-x64-0.28.1.tgz", + "integrity": "sha512-BEjgtECkL3vY+SaSQ6nzVfiALUeFxpawyp8Jmf5PtYhf1Ug40N1h/hxlhts+f1FvSvarEigdxS3BlSMI2PJLcQ==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "sunos" + ], + "peer": true, + "engines": { + "node": ">=18" + } + }, + "node_modules/vitest/node_modules/@esbuild/win32-arm64": { + "version": "0.28.1", + "resolved": "https://registry.npmjs.org/@esbuild/win32-arm64/-/win32-arm64-0.28.1.tgz", + "integrity": "sha512-lCv9eK/H6ZJWbE7bh2nw54CZ9M2nupBxJcTsdk/QQnWkdSjKGuxmmH8/GWrlT1eMmZfn4dGcCjRte397WqfQXA==", + "cpu": [ + "arm64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "win32" + ], + "peer": true, + "engines": { + "node": ">=18" + } + }, + "node_modules/vitest/node_modules/@esbuild/win32-ia32": { + "version": "0.28.1", + "resolved": "https://registry.npmjs.org/@esbuild/win32-ia32/-/win32-ia32-0.28.1.tgz", + "integrity": "sha512-zvb/mB2bSCoJOpoCBgYKKpX6YM6mJBlBUVUtVj41DlZJVEB6/0CKlRYxP5wWl1C1ILiCoAU5wZZ4q1P3qeS6Eg==", + "cpu": [ + "ia32" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "win32" + ], + "peer": true, + "engines": { + "node": ">=18" + } + }, + "node_modules/vitest/node_modules/@esbuild/win32-x64": { + "version": "0.28.1", + "resolved": "https://registry.npmjs.org/@esbuild/win32-x64/-/win32-x64-0.28.1.tgz", + "integrity": "sha512-bm4Mowrv+GXMlpWX++EcXw/iLyd1o3+bJkC2DkWXYVvgZCqD/bSj9ctZeAMC3cIxgjRVR2Dufaiu4YPxr5gW1A==", + "cpu": [ + "x64" + ], + "dev": true, + "license": "MIT", + "optional": true, + "os": [ + "win32" + ], + "peer": true, + "engines": { + "node": ">=18" + } + }, "node_modules/vitest/node_modules/@vitest/mocker": { "version": "4.1.9", "resolved": "https://registry.npmjs.org/@vitest/mocker/-/mocker-4.1.9.tgz", @@ -2615,6 +3054,50 @@ } } }, + "node_modules/vitest/node_modules/esbuild": { + "version": "0.28.1", + "resolved": "https://registry.npmjs.org/esbuild/-/esbuild-0.28.1.tgz", + "integrity": "sha512-HrJrvZv5ayxBzPfwphOoNzkzOIIlifzk0KJrGK2c8R4+LKpMtpYLQeUdjnwjWv/LZlkH2laZk+4w78pi99D4Vw==", + "dev": true, + "hasInstallScript": true, + "license": "MIT", + "optional": true, + "peer": true, + "bin": { + "esbuild": "bin/esbuild" + }, + "engines": { + "node": ">=18" + }, + "optionalDependencies": { + "@esbuild/aix-ppc64": "0.28.1", + "@esbuild/android-arm": "0.28.1", + "@esbuild/android-arm64": "0.28.1", + "@esbuild/android-x64": "0.28.1", + "@esbuild/darwin-arm64": "0.28.1", + "@esbuild/darwin-x64": "0.28.1", + "@esbuild/freebsd-arm64": "0.28.1", + "@esbuild/freebsd-x64": "0.28.1", + "@esbuild/linux-arm": "0.28.1", + "@esbuild/linux-arm64": "0.28.1", + "@esbuild/linux-ia32": "0.28.1", + "@esbuild/linux-loong64": "0.28.1", + "@esbuild/linux-mips64el": "0.28.1", + "@esbuild/linux-ppc64": "0.28.1", + "@esbuild/linux-riscv64": "0.28.1", + "@esbuild/linux-s390x": "0.28.1", + "@esbuild/linux-x64": "0.28.1", + "@esbuild/netbsd-arm64": "0.28.1", + "@esbuild/netbsd-x64": "0.28.1", + "@esbuild/openbsd-arm64": "0.28.1", + "@esbuild/openbsd-x64": "0.28.1", + "@esbuild/openharmony-arm64": "0.28.1", + "@esbuild/sunos-x64": "0.28.1", + "@esbuild/win32-arm64": "0.28.1", + "@esbuild/win32-ia32": "0.28.1", + "@esbuild/win32-x64": "0.28.1" + } + }, "node_modules/vitest/node_modules/estree-walker": { "version": "3.0.3", "resolved": "https://registry.npmjs.org/estree-walker/-/estree-walker-3.0.3.tgz", diff --git a/src/proxy/beliefs.ts b/src/proxy/beliefs.ts index b234f8b..cb16f50 100644 --- a/src/proxy/beliefs.ts +++ b/src/proxy/beliefs.ts @@ -32,7 +32,10 @@ export async function fetchBeliefs( try { const id = env.SESSION.idFromName(sessionId); const stub = env.SESSION.get(id); - doRes = await stub.fetch("https://internal/beliefs"); + // Pass the path sessionId as a hint so the DO can echo a human-readable + // sessionId even if no beliefs have been written yet (DO returns flat). + const hint = `https://internal/beliefs?sessionId=${encodeURIComponent(sessionId)}`; + doRes = await stub.fetch(hint); } catch (err) { return jsonError( 502, diff --git a/src/proxy/extraction.ts b/src/proxy/extraction.ts index b0b4865..9cfffdd 100644 --- a/src/proxy/extraction.ts +++ b/src/proxy/extraction.ts @@ -27,7 +27,7 @@ export async function runExtraction( let result: ExtractionResult; try { - const beliefs = await extractBeliefs(responseText); + const beliefs = await extractBeliefs(responseText, { sessionId }); result = { sessionId, beliefs, diff --git a/src/state/SessionDurableObject.test.ts b/src/state/SessionDurableObject.test.ts new file mode 100644 index 0000000..ca5ca80 --- /dev/null +++ b/src/state/SessionDurableObject.test.ts @@ -0,0 +1,103 @@ +/** + * Tests for SessionDurableObject using an in-memory mock of DurableObjectState. + * Exercises the store → flatten GET round trip and, critically, that GET + * returns the human-readable sessionName rather than the opaque DO id. + */ +import { describe, it, expect } from "vitest"; +import { SessionDurableObject } from "./SessionDurableObject"; +import type { ExtractionResult } from "../proxy/types"; +import type { ExtractedBelief } from "../lens/types"; + +function belief(id: string): ExtractedBelief { + return { + id, + sessionId: "ignored", + type: "causal", + belief: `belief-${id}`, + confidence: 0.7, + timestamp: 0, + rawText: "", + line: 1, + }; +} + +function makeResult(sessionId: string, ids: string[]): ExtractionResult { + return { sessionId, beliefs: ids.map(belief), rawText: "raw", timestamp: Date.now() }; +} + +/** Minimal in-memory DurableObjectState stand-in. */ +function makeState(idString = "opaque-do-id-abc123") { + const store = new Map(); + return { + id: { toString: () => idString }, + storage: { + get: async (key: string) => store.get(key), + put: async (key: string, value: unknown) => { + store.set(key, value); + }, + }, + } as unknown as DurableObjectState; +} + +function post(session: string, ids: string[]): Request { + return new Request("https://internal/store-beliefs", { + method: "POST", + headers: { "Content-Type": "application/json" }, + body: JSON.stringify(makeResult(session, ids)), + }); +} + +describe("SessionDurableObject", () => { + it("stores batches and returns a flat chronological list on GET", async () => { + const doInstance = new SessionDurableObject(makeState()); + + await doInstance.fetch(post("my-session", ["a", "b"])); + await doInstance.fetch(post("my-session", ["c"])); + + const res = await doInstance.fetch(new Request("https://internal/beliefs")); + const body = (await res.json()) as { sessionId: string; beliefs: ExtractedBelief[] }; + + expect(body.beliefs.map((b) => b.id)).toEqual(["a", "b", "c"]); + }); + + it("returns the stored human sessionName, not the DO id", async () => { + const doInstance = new SessionDurableObject(makeState("opaque-do-id-abc123")); + await doInstance.fetch(post("human-friendly-name", ["a"])); + + const res = await doInstance.fetch(new Request("https://internal/beliefs")); + const body = (await res.json()) as { sessionId: string }; + + expect(body.sessionId).toBe("human-friendly-name"); + expect(body.sessionId).not.toBe("opaque-do-id-abc123"); + }); + + it("falls back to the request hint before any write", async () => { + const doInstance = new SessionDurableObject(makeState()); + + const res = await doInstance.fetch( + new Request("https://internal/beliefs?sessionId=hint-name") + ); + const body = (await res.json()) as { sessionId: string; beliefs: ExtractedBelief[] }; + + expect(body.sessionId).toBe("hint-name"); + expect(body.beliefs).toEqual([]); + }); + + it("prefers the stored sessionName over the request hint", async () => { + const doInstance = new SessionDurableObject(makeState()); + await doInstance.fetch(post("stored-name", ["a"])); + + const res = await doInstance.fetch( + new Request("https://internal/beliefs?sessionId=hint-name") + ); + const body = (await res.json()) as { sessionId: string }; + + expect(body.sessionId).toBe("stored-name"); + }); + + it("404s on unknown routes", async () => { + const doInstance = new SessionDurableObject(makeState()); + const res = await doInstance.fetch(new Request("https://internal/nope")); + expect(res.status).toBe(404); + }); +}); diff --git a/src/state/SessionDurableObject.ts b/src/state/SessionDurableObject.ts index e00a911..da4bb57 100644 --- a/src/state/SessionDurableObject.ts +++ b/src/state/SessionDurableObject.ts @@ -1,25 +1,32 @@ /** - * STUB - Axion Lens session Durable Object. + * Axion Lens - Session state Durable Object (Phase 1 timeline store). * - * This file is a placeholder so the proxy module typechecks before the real - * Durable Object (src/state/SessionDurableObject.ts, built by another agent) - * lands. The real implementation stores the per-session belief graph in DO - * storage and serves it at the /beliefs route. + * Phase 1 stores beliefs as an append-only chronological timeline, NOT a graph. + * Each POST /store-beliefs call appends one batch (the beliefs extracted from a + * single response) to the `"beliefs"` storage key. GET /beliefs flattens every + * batch into one ordered list and returns the public shape + * `{ sessionId, beliefs: ExtractedBelief[] }`. + * + * The `sessionId` returned is the human-readable session name (the value the + * caller passed via `x-axion-session`, stored under `"sessionName"` on write), + * never the opaque Durable Object id. * * The proxy talks to this DO via: * env.SESSION.idFromName(sessionId) → stub → POST https://internal/store-beliefs * → GET https://internal/beliefs * * Wrangler binds this class as the `SESSION` Durable Object in wrangler.toml. + * + * @planned BeliefNode / BeliefDAG graph APIs (parent/child edges, root-cause + * routes) are intentionally not implemented here. See BUILD-SPEC decision D2. */ import type { ExtractionResult } from "../proxy/types"; - -interface StoredBeliefs { - beliefs: ExtractionResult["beliefs"]; - rawText: string; - timestamp: number; -} +import { + flattenBeliefBatches, + resolveSessionId, + type BeliefBatch, +} from "./sessionBeliefs"; export class SessionDurableObject implements DurableObject { private state: DurableObjectState; @@ -31,25 +38,36 @@ export class SessionDurableObject implements DurableObject { async fetch(request: Request): Promise { const url = new URL(request.url); - // POST /store-beliefs - persist extracted beliefs for this session. + // POST /store-beliefs - append one batch of beliefs for this session. if (url.pathname === "/store-beliefs" && request.method === "POST") { const result = (await request.json()) as ExtractionResult; - const stored: StoredBeliefs[] = (await this.state.storage.get("beliefs")) || []; + const stored: BeliefBatch[] = (await this.state.storage.get("beliefs")) || []; stored.push({ beliefs: result.beliefs, rawText: result.rawText, timestamp: result.timestamp, }); await this.state.storage.put("beliefs", stored); + // Persist the human-readable session name so GET can echo it back + // instead of the opaque DO id. Refresh on every write. + if (result.sessionId) { + await this.state.storage.put("sessionName", result.sessionId); + } return new Response(JSON.stringify({ ok: true, count: result.beliefs.length }), { headers: { "Content-Type": "application/json" }, }); } - // GET /beliefs - return the full belief graph for this session. + // GET /beliefs - return the flat chronological timeline for this session. if (url.pathname === "/beliefs" && request.method === "GET") { - const stored: StoredBeliefs[] = (await this.state.storage.get("beliefs")) || []; - return new Response(JSON.stringify({ sessionId: this.state.id.toString(), beliefs: stored }), { + const stored: BeliefBatch[] = (await this.state.storage.get("beliefs")) || []; + const sessionName = (await this.state.storage.get("sessionName")) ?? null; + // Fall back to a request hint (the id from the incoming path) when + // nothing has been written yet. Never leak the opaque DO id. + const hint = url.searchParams.get("sessionId"); + const beliefs = flattenBeliefBatches(stored); + const sessionId = resolveSessionId(sessionName, hint); + return new Response(JSON.stringify({ sessionId, beliefs }), { headers: { "Content-Type": "application/json" }, }); } diff --git a/src/state/sessionBeliefs.test.ts b/src/state/sessionBeliefs.test.ts new file mode 100644 index 0000000..28bb071 --- /dev/null +++ b/src/state/sessionBeliefs.test.ts @@ -0,0 +1,74 @@ +/** + * Tests for the pure session-timeline helpers: flattenBeliefBatches and + * resolveSessionId. These run without any Cloudflare runtime. + */ +import { describe, it, expect } from "vitest"; +import { flattenBeliefBatches, resolveSessionId, type BeliefBatch } from "./sessionBeliefs"; +import type { ExtractedBelief } from "../lens/types"; + +function belief(id: string, overrides: Partial = {}): ExtractedBelief { + return { + id, + sessionId: "s", + type: "causal", + belief: `belief-${id}`, + confidence: 0.7, + timestamp: 0, + rawText: "", + line: 1, + ...overrides, + }; +} + +function batch(ids: string[], timestamp = 0): BeliefBatch { + return { beliefs: ids.map((id) => belief(id)), rawText: "", timestamp }; +} + +describe("flattenBeliefBatches", () => { + it("concatenates every batch's beliefs in storage order", () => { + const batches = [batch(["a", "b"], 1), batch(["c"], 2), batch(["d", "e"], 3)]; + const flat = flattenBeliefBatches(batches); + expect(flat.map((b) => b.id)).toEqual(["a", "b", "c", "d", "e"]); + }); + + it("returns an empty array for no batches", () => { + expect(flattenBeliefBatches([])).toEqual([]); + }); + + it("tolerates malformed batches without throwing", () => { + const messy = [ + batch(["a"]), + // deliberately malformed entries + null as unknown as BeliefBatch, + { rawText: "x", timestamp: 0 } as unknown as BeliefBatch, + batch(["b"]), + ]; + expect(flattenBeliefBatches(messy).map((b) => b.id)).toEqual(["a", "b"]); + }); + + it("returns [] when given a non-array", () => { + expect(flattenBeliefBatches(undefined as unknown as BeliefBatch[])).toEqual([]); + }); +}); + +describe("resolveSessionId", () => { + it("prefers the stored sessionName over the request hint", () => { + expect(resolveSessionId("human-name", "hint-name")).toBe("human-name"); + }); + + it("falls back to the request hint when no sessionName is stored", () => { + expect(resolveSessionId(null, "hint-name")).toBe("hint-name"); + expect(resolveSessionId(undefined, "hint-name")).toBe("hint-name"); + expect(resolveSessionId(" ", "hint-name")).toBe("hint-name"); + }); + + it("returns empty string when neither is available", () => { + expect(resolveSessionId(null, null)).toBe(""); + expect(resolveSessionId(undefined, undefined)).toBe(""); + }); + + it("trims whitespace from the chosen value", () => { + expect(resolveSessionId(" human ", null)).toBe("human"); + expect(resolveSessionId(null, " hint ")).toBe("hint"); + }); +}); diff --git a/src/state/sessionBeliefs.ts b/src/state/sessionBeliefs.ts new file mode 100644 index 0000000..f0cd58d --- /dev/null +++ b/src/state/sessionBeliefs.ts @@ -0,0 +1,53 @@ +/** + * Axion Lens - Session belief timeline helpers. + * + * Phase 1 stores beliefs as an append-only list of batches (one batch per + * extracted response). The public API exposes a single flat chronological + * list, so these pure helpers do the flattening and sessionId resolution + * without any Cloudflare runtime dependency, which keeps them unit-testable. + */ + +import type { ExtractedBelief } from "../lens/types.js"; + +/** One stored batch: the beliefs extracted from a single response. */ +export interface BeliefBatch { + beliefs: ExtractedBelief[]; + rawText: string; + timestamp: number; +} + +/** + * Concatenate every batch's `beliefs` array in storage order, producing the + * flat chronological timeline the public API returns. Tolerant of malformed + * input (non-array batches / missing `beliefs`) so a corrupt storage read can + * never throw. + */ +export function flattenBeliefBatches(batches: BeliefBatch[]): ExtractedBelief[] { + if (!Array.isArray(batches)) return []; + const out: ExtractedBelief[] = []; + for (const batch of batches) { + if (batch && Array.isArray(batch.beliefs)) { + out.push(...batch.beliefs); + } + } + return out; +} + +/** + * Resolve the human-readable sessionId for a GET response. + * + * Preference: the stored `sessionName` (the human name the caller used, saved + * on the first write) wins. If nothing has been stored yet, fall back to the + * request hint (the id from the incoming path). Never return the opaque + * Durable Object id. + */ +export function resolveSessionId( + storedName?: string | null, + hint?: string | null +): string { + const stored = storedName?.trim(); + if (stored) return stored; + const hinted = hint?.trim(); + if (hinted) return hinted; + return ""; +} From d7e223d4a3c9a2710d2225764b05c5790787632a Mon Sep 17 00:00:00 2001 From: Cursor Agent Date: Thu, 16 Jul 2026 15:23:04 +0000 Subject: [PATCH 10/10] feat: wire providers, PolyVerdict enforce, and honest docs Integrate OpenAI + Anthropic observe paths with passthrough auth, content-normalized Lens extraction, and opt-in PolyVerdict schema enforcement. Truth-align README/SPEC/TECHNICAL/PLAN with shipped code. Co-authored-by: Moses Man --- BUILD-SPEC.md | 16 +- PLAN.md | 290 ++++++-------------------- README.md | 292 +++++++++++++------------- SPEC-PolyVerdict.md | 116 +++++------ SPEC.md | 179 ++++++++-------- TECHNICAL.md | 495 +++++++++++++++++++------------------------- src/proxy/index.ts | 315 +++++++++++++++++++++------- wrangler.toml | 9 +- 8 files changed, 829 insertions(+), 883 deletions(-) diff --git a/BUILD-SPEC.md b/BUILD-SPEC.md index 023d06d..ab793e4 100644 --- a/BUILD-SPEC.md +++ b/BUILD-SPEC.md @@ -198,14 +198,14 @@ Also forward: OpenAI-Organization, anthropic-version (when Anthropic path), cont ## Acceptance checklist -- [ ] `npm run check` passes -- [ ] OpenAI proxy works with caller Bearer key and no worker secret -- [ ] Anthropic `/v1/messages` routed and text extracted for Lens -- [ ] Dashboard loads beliefs by pasted session id -- [ ] `GET /api/beliefs/:id` returns flat `ExtractedBelief[]` -- [ ] Enforce mode rejects invalid JSON schema outputs and retries -- [ ] Docs do not claim unimplemented DAG/NLP/passthrough-wrongly -- [ ] No `Bearer undefined` path +- [x] `npm run check` passes (100 tests + tsc) +- [x] OpenAI proxy works with caller Bearer key and no worker secret +- [x] Anthropic `/v1/messages` routed and text extracted for Lens +- [x] Dashboard loads beliefs by pasted session id +- [x] `GET /api/beliefs/:id` returns flat `ExtractedBelief[]` +- [x] Enforce mode rejects invalid JSON schema outputs and retries +- [x] Docs do not claim unimplemented DAG/NLP/passthrough-wrongly +- [x] No `Bearer undefined` path --- diff --git a/PLAN.md b/PLAN.md index 970af54..5e9198c 100644 --- a/PLAN.md +++ b/PLAN.md @@ -1,274 +1,116 @@ -# Axion + PolyVerdict — production readiness plan +# Axion + PolyVerdict: production readiness plan -> Plan only. Do not implement from this doc until the build order below is accepted. -> Scope: open-source Phase 1 (Axion Lens) shippable on Cloudflare Workers, plus an honest sequencing decision for PolyVerdict. +> **Status:** decisions locked; Waves 0 through 4 (syntax path) implemented. This file is now a record of how the current state was reached, not an open plan. +> **Source of truth for scope:** [BUILD-SPEC.md](./BUILD-SPEC.md). The locked decisions there supersede the "decisions to lock" section below. Where this file and BUILD-SPEC disagree, BUILD-SPEC wins. --- -## Verdict +## Where things stand -The proxy hot path is the strongest part of the repo: tee the response, return it immediately, extract in `waitUntil`. That part is close to real infrastructure. - -What is not shippable is the product around it. The dashboard cannot list or render beliefs. Auth is an open relay if you put a secret in the Worker. Docs sell a belief DAG, Anthropic drop-in, and key passthrough that the code does not provide. PolyVerdict is a proposal with zero code, and its retry/block model fights Lens’s zero-latency observe path. - -For OSS, the first job is to make Phase 1 true: a working OpenAI-compatible observe proxy, a durable per-session belief timeline, a dashboard that shows it, and docs that match. PolyVerdict comes after that, as an opt-in enforce mode, not as default Lens middleware. - ---- - -## What exists today - -``` -Agent → POST /v1/chat/completions - → Worker tees body (ReadableStream.tee) - → caller gets upstream stream immediately - → waitUntil → regex extract → DO append batch - → dashboard tries /api/sessions (404) and expects flat beliefs (gets nested batches) -``` +The plan below was written against an earlier repo where the docs oversold the code. That gap is closed. The decisions were locked in [BUILD-SPEC.md](./BUILD-SPEC.md) and the build was carried out. Current state: | Piece | Status | |---|---| -| OpenAI chat proxy + SSE tee | Real | -| `waitUntil` belief extraction | Real (quality thin) | -| Session DO storage | Real append log via Durable Object storage | -| Belief DAG / root-cause | Types only | -| Dashboard | UI exists; API contract broken | -| Anthropic `/v1/messages` | Not implemented | -| Caller key passthrough | Documented, not implemented | -| PolyVerdict | `SPEC-PolyVerdict.md` only | -| `npm test` / CI | Vitest dep present; no test script; one stream test file | +| OpenAI chat proxy + SSE tee | Implemented | +| Anthropic `/v1/messages` proxy | Implemented | +| Passthrough auth + fail-closed 401 | Implemented (`src/proxy/auth.ts`) | +| `waitUntil` belief extraction, sessionId stamped | Implemented | +| DO session store, flat `GET /api/beliefs/:id` | Implemented (append batches, flatten on read) | +| Dashboard paste-session UX + low-confidence filter | Implemented | +| Additive confidence, clamp `[0.1, 1.0]`, because-of / evidence | Implemented | +| PolyVerdict enforce (syntax validate + coerce + retry <=3) | Implemented, opt-in | +| Vitest suite + `npm run check` + GitHub Actions | Implemented | +| `.dev.vars.example`, `CONTRIBUTING`, `SECURITY` | Present | +| Belief DAG / root-cause | Not built (types only, `@planned`) | +| `/api/sessions` registry | Not built (by decision) | +| Axion Loop / Gate | Not built | +| Semantic PolyVerdict / schema registry DO | Not built | --- -## Decisions to lock before build - -These are product calls. Wrong defaults will force rework. - -### D1. Auth model (pick one) - -| Option | Meaning | Fit | -|---|---|---| -| **A. Passthrough (recommended for OSS)** | Forward caller `Authorization` / `x-api-key`. Worker holds no model key. Fail closed if neither caller key nor optional server key is present. | Matches “point agent at Axion” and avoids an open credit relay. | -| **B. Server key + proxy token** | Worker holds `UPSTREAM_API_KEY`. Every call requires an Axion token. | Better for a hosted SaaS later; heavier for self-host OSS. | - -Default for this plan: **A**, with optional server key as fallback only when explicitly configured. - -### D2. Phase 1 data model honesty - -Ship a **flat chronological belief timeline**, not a DAG. - -Keep Durable Objects as the session owner. Persist ordered events (already closer to storage than the “in-memory Map, lost on eviction” story). Defer `parentIds` / edges / root-cause until extract or a post-pass can justify links. - -Delete or clearly mark unused `BeliefDAG` / edge APIs as planned. Do not advertise root-cause until it exists. - -### D3. Provider scope for Phase 1 - -Phase 1 ships **OpenAI-compatible `/v1/chat/completions` only**. - -Anthropic Messages becomes Phase 1.1 behind a provider adapter. Remove Claude Code / `ANTHROPIC_BASE_URL` claims from README until that adapter lands. - -### D4. PolyVerdict placement - -PolyVerdict is Gate-shaped: validate, retry, coerce, possibly block. Lens is observe-shaped: never delay the agent. - -Same Worker package later is fine. Same default request path is not. Sequence PolyVerdict as an **opt-in enforce mode** after Lens contracts are honest, behind a provider/content adapter, with its own latency budget. - ---- - -## Build waves (ordered) - -### Wave 0 — Truth and safety (merge before any feature work) - -Make the repo honest and non-dangerous. +## Decisions (locked) -1. **Auth boundary** - - Implement D1 (passthrough + fail closed). - - Align `Env` typing, `wrangler.toml` comments, README Quick Start, and `TECHNICAL.md`. - - Add `.dev.vars.example` with `UPSTREAM_API_URL` / optional `UPSTREAM_API_KEY`. +These were open product calls. They are now locked in [BUILD-SPEC.md](./BUILD-SPEC.md). Recorded here for context. -2. **Docs vs code** - - Rewrite README / SPEC Phase 1 status to: OpenAI chat proxy + regex timeline + DO session store + local dashboard. - - Move DAG, root-cause, NLP, Anthropic, key passthrough (once implemented, keep), `/api/sessions` (until built), Loop, Gate, PolyVerdict to explicit Planned sections. - - Fix Known Issues: DO uses Durable Object storage (survives eviction); unbounded growth is the real risk. +### D1: auth model. Locked: passthrough first. -3. **OSS hygiene floor** - - Add `"test": "vitest run"` (and keep `typecheck`). - - Minimal GitHub Actions: typecheck + test on PR. - - Stub `CONTRIBUTING.md` + `SECURITY.md` (how to report, that beliefs API is unauthenticated in Phase 1). +Forward the caller's `Authorization` or `x-api-key`. Use `UPSTREAM_API_KEY` only when it is set. Fail closed with 401 otherwise. No `Bearer undefined`. This matches "point your agent at Axion" and avoids an open model-key relay. -**Exit:** A stranger can deploy without becoming an open model-key proxy, and the README does not lie. +### D2: Phase 1 data model. Locked: flat chronological timeline. ---- - -### Wave 1 — Make Lens usable end-to-end - -This is the actual Phase 1 product. - -4. **Canonical session identity** - - Pass `{ sessionId }` into `extractBeliefs` from `runExtraction`. - - Document that agents must send `x-axion-session` for multi-turn correlation (or document a fallback “latest” single-session mode for local demo). - - DO GET returns the human session name (the `idFromName` key), never the opaque DO id as the user-facing id. +The Durable Object owns the session and appends ordered batches to DO storage. The public API flattens them to `{ sessionId, beliefs: ExtractedBelief[] }`. `BeliefNode` / `BeliefDAG` types stay marked `@planned`; no graph API is built. -5. **Beliefs API contract** - - Public shape: `{ sessionId: string, beliefs: ExtractedBelief[] }` chronological flatten of batches. - - Keep raw batches internal if useful for debugging; do not leak them to the dashboard. - - Treat this JSON as a typed boundary shared by DO → Worker → dashboard. +### D3: provider scope. Locked: OpenAI + Anthropic. -6. **Session discovery** - - Either: - - **6a.** Add a small session registry (KV or registry DO write-on-first-use) + `GET /api/sessions`, or - - **6b.** Drop the dropdown and make the dashboard Phase 1 “single session”: paste / read `x-axion-session` (matches SPEC’s “local single-session dashboard” better). - - Recommendation: **6b for OSS MVP**, **6a when multi-session is real**. SPEC already says multi-session is SaaS-later; the UI overreached. +Both `POST /v1/chat/completions` and `POST /v1/messages` ship behind a shared provider adapter. This resolves the earlier plan's "OpenAI-only until an adapter lands" hedge: the adapter landed. -7. **Dashboard wire-up** - - Consume flattened beliefs. - - Session UX per 6b or 6a. - - Redefine or remove “wrong beliefs only” until `invalidated` exists (today it is `confidence < 0.4`). - - Confirm Workers Assets serve `/app.js` and `/styles.css` (or route them explicitly). +### D4: PolyVerdict placement. Locked: opt-in enforce mode, separate path. -8. **Content normalization before extract** - - Streaming: keep OpenAI delta concat (already). - - Non-streaming: parse `choices[0].message.content` (and refuse to scan raw JSON envelopes). - - Isolate behind a small `extractAssistantText(responseMode, bytes)` helper so Anthropic can plug in later. - -**Exit:** Point an OpenAI-compatible agent at the Worker with a stable `x-axion-session`, open the dashboard, see a real timeline. +Same Worker, different code path. It runs only on a schema trigger and buffers by design. It is never a silent wrap of the observe path. --- -### Wave 2 — Extraction quality and contracts +## Build waves (as executed) -Worth doing once the pipes work; do not block Wave 1 on perfect linguistics. +### Wave 0: truth and safety. Done. -9. **Pattern / confidence honesty** - - Fix `because of` group capture. - - Either populate `evidence` via `evidenceGroup` or stop claiming the field. - - Pick one confidence formula and document it (prefer documented additive modifiers for readability, or update docs to midpoint bands — do not leave both). - - Pass real `sessionId`; stop random per-belief ids. +- Passthrough auth implemented and fail-closed. +- Docs rewritten to match code (this pass): no DAG, no root-cause, no `/api/sessions`, no Anthropic claim without the route. +- `npm test` / `npm run check`, GitHub Actions, `.dev.vars.example`, `CONTRIBUTING.md`, `SECURITY.md` added. -10. **Tests at the seams** - - Lens: pattern fixtures (because / because of / intention nesting / no-punctuation). - - Extraction glue: session stamp + content parse for stream and non-stream. - - DO round-trip: store batches → public flatten shape. - - Auth matrix: passthrough vs server key vs neither (fail closed). - - Router: no silent 404 for whatever session UX Wave 1 chose. +### Wave 1: Lens usable end-to-end. Done. -**Exit:** Regressions in the dashboard contract or auth fail CI. +- `{ sessionId }` passed into `extractBeliefs`; `x-axion-session` correlates a run and is echoed on every response. +- Public beliefs shape is the flat `{ sessionId, beliefs }` boundary shared by DO, Worker, and dashboard. +- Session discovery is paste-a-session-id (option 6b), not a registry. The dashboard consumes the flat list; the "wrong" filter became "low confidence only" (`confidence < 0.4`). +- Non-streaming bodies are parsed to assistant text before extraction; raw JSON is never scanned. ---- +### Wave 2: extraction quality and tests. Done. -### Wave 3 — Provider adapter (Phase 1.1) +- `because of` capture fixed (split from bare `because`). Evidence patterns populate the `evidence` field. +- One documented confidence formula: additive markers, clamp `[0.1, 1.0]`. +- Tests at the seams: auth matrix, content parse (stream + non-stream, both providers), lens fixtures, DO flatten, schema validate/coerce. -11. **Adapter boundary** - - Interface covering: route match, auth header map, stream event parse, assistant text extract. - - OpenAI adapter = current behavior. - - Anthropic Messages adapter = new route + SSE shape. - - Then restore README Claude Code / Hermes claims with tested instructions. +### Wave 3: provider adapter. Done. -**Exit:** At least one non-OpenAI agent path is real, or docs stay OpenAI-only. +- `ProviderAdapter` interface with OpenAI and Anthropic adapters. Anthropic SSE parses `content_block_delta` text deltas. Docs restore the Anthropic base-URL instructions because the route now exists. ---- +### Wave 4: PolyVerdict enforce. Syntax path done. -### Wave 4 — PolyVerdict (after Lens is honest) - -Do not start until Waves 0–1 are done and Wave 2 tests exist. Prefer Wave 3 adapter first so schema enforcement is not OpenAI-only forever. - -12. **PolyVerdict as opt-in enforce mode** - - Trigger: `x-schema` / `response_format` JSON Schema (draft 2020-12). - - Control flow: hold → validate → retry upstream (max 3) with violation hints → optional coerce → return. - - Explicitly **not** the Lens tee path. New code path that may add latency; success criteria keep `<200ms` for syntax-only pass, zero extra latency only when schema passes first try (meaning: validate after full body for non-stream, or buffered validate for stream — decide and document; do not pretend tee + mutate is free). - - Schema registry DO (named schemas) as a separate binding from session beliefs. - - Hash cache for identical schema+prompt skip. - - Semantic / PolyGnosis verification stays opt-in and budget-capped (Phase later inside PolyVerdict). - -13. **Composition with Lens** - - When enforce mode is on: validate first, then Lens extracts from the **delivered** (possibly coerced) text. - - When off: today’s observe path unchanged. - - Same package, mode switch — not a silent middleware wrap of every request. - -**Exit:** Schema-gated chat completions work on OpenAI path with tests for pass / fail-retry / coerce; Lens still observes. +- Trigger on `x-axion-schema` or `response_format.json_schema`. Buffered enforce loop: validate, coerce, retry <=3 with violation hints, 422 on exhaustion. Lens runs on delivered text. +- Not the tee path. Enforce forces non-streaming. +- Semantic verification, schema registry DO, and hash cache are deferred. --- -### Explicitly later (do not sneak into OSS Phase 1) +## Deferred (not in the current build) -| Item | Why later | +| Item | Why deferred | |---|---| | Belief DAG + root-cause | Needs justified edges and failure signals | | Axion Loop | Needs stable multi-turn sessions + embeddings | -| Axion Gate (tool-call block) | Needs plan extraction + intervene path | -| Hosted multi-session SaaS | Out of OSS core per SPEC | -| Langfuse / Arize export | Easy after flat JSON is stable | -| Semantic PolyVerdict | Costly; after syntax path | - ---- - -## Priority matrix - -| Priority | Fix | Wave | -|---|---|---| -| P0 | Auth fail-closed + passthrough (or token gate) | 0 | -| P0 | Docs truth-align (no DAG / Anthropic / passthrough lies) | 0 | -| P0 | Flatten beliefs API + dashboard session UX | 1 | -| P0 | Stamp `sessionId` on extract | 1 | -| P0 | Non-stream content parse before extract | 1 | -| P1 | `npm test` + CI + env example | 0–2 | -| P1 | Pattern / evidence / confidence honesty | 2 | -| P1 | OpenAI-only README until adapter exists | 0 / 3 | -| P2 | Anthropic adapter | 3 | -| P2 | CONTRIBUTING / SECURITY | 0 | -| P3 | PolyVerdict enforce mode | 4 | -| P3 | DAG / Loop / Gate | later | - ---- - -## Architecture sketch after Waves 0–1 - -``` -Agent (OpenAI-compatible, x-axion-session) - ↕ -Axion Worker - ├── Auth: passthrough or configured server key (fail closed) - ├── POST /v1/chat/completions → upstream → tee → waitUntil extract - ├── GET /api/beliefs/:sessionId → flat ExtractedBelief[] - ├── Dashboard: paste/select session → timeline - └── SessionDurableObject: durable ordered belief batches (internal) -``` - -After Wave 4 (optional): - -``` - ├── Observe mode (default): tee + Lens - └── Enforce mode (x-schema): validate/retry/coerce → then Lens on delivered text -``` - ---- - -## Suggested first build PR sequence (when we exit plan mode) - -1. `fix(auth+docs): fail-closed passthrough + honest README/SPEC` -2. `fix(api): flatten beliefs + session UX + stamp sessionId` -3. `fix(extract): non-stream content parse + because-of / evidence` -4. `chore(ci): vitest script + GH Actions` -5. (optional) `feat(providers): Anthropic messages adapter` -6. (later) `feat(polyverdict): opt-in schema enforce mode` - -Do not combine 1–3 with PolyVerdict in one PR. +| Axion Gate | Needs plan extraction + an intervene path | +| Semantic PolyVerdict | Costly; comes after the syntax path proves out | +| Schema registry DO, hash cache | Optimisations, not needed for correctness | +| Hosted multi-session SaaS | Out of the OSS core per SPEC | +| Langfuse / Arize export | Straightforward once the flat JSON is stable | --- -## Open questions for the next turn +## Open questions (resolved) -Answer these when we leave plan mode; defaults above apply if silent: +The plan's original open questions, with their locked answers: -1. Auth: confirm **passthrough (A)** vs **server key + token (B)**. -2. Dashboard: confirm **paste session id (6b)** vs **registry + `/api/sessions` (6a)**. -3. Is PolyVerdict in the first production OSS tag at all, or clearly “proposal / Phase 2 enforce”? -4. Keep unused DAG types with `@planned` comments, or delete until needed? +1. Auth: passthrough (A). Locked. +2. Dashboard: paste session id (6b). Locked. +3. PolyVerdict in the first OSS tag: yes, as opt-in syntax enforce. Semantic stays future. +4. Unused DAG types: kept with `@planned` comments, not deleted. --- ## References -- Product: `SPEC.md`, `TECHNICAL.md`, `README.md` -- PolyVerdict proposal: `SPEC-PolyVerdict.md` -- Runtime: `src/proxy/*`, `src/lens/*`, `src/state/SessionDurableObject.ts`, `src/dashboard/*` -) +- Locked scope: [BUILD-SPEC.md](./BUILD-SPEC.md) +- Product: [SPEC.md](./SPEC.md), [TECHNICAL.md](./TECHNICAL.md), [README.md](./README.md) +- PolyVerdict: [SPEC-PolyVerdict.md](./SPEC-PolyVerdict.md) +- Runtime: `src/proxy/*`, `src/lens/*`, `src/polyverdict/*`, `src/state/*`, `src/dashboard/*` diff --git a/README.md b/README.md index 178991c..551863d 100644 --- a/README.md +++ b/README.md @@ -2,7 +2,7 @@ # Axion -Agent cognitive middleware - inspect, detect, and verify agent reasoning in real time. +Agent cognitive middleware. A proxy that reads what an agent believes from its own model output, in real time, with no code changes to the agent. **by LatticeAG** @@ -15,244 +15,234 @@ Agent cognitive middleware - inspect, detect, and verify agent reasoning in real --- -## Overview +## What this is -> **Agent cognitive middleware - a proxy layer that makes agent reasoning visible, diagnosable, and verifiable.** -> Observe beliefs. Detect loops. Block bad actions. +Axion is a Cloudflare Worker that sits in front of a model API. Point an agent at it by overriding the base URL. The Worker forwards each request upstream and streams the response straight back with zero added latency. After the response is delivered, it runs a rule-based parser over the assistant text to pull out reasoning fragments (causal claims, assumptions, intentions, cited evidence), stamps them with a confidence score, and stores them per session. A local dashboard reads them back as a timeline. -Axion sits between an AI agent and the outside world. It intercepts model responses, agent outputs, and tool calls - then reconstructs the agent's decision-making chain in real time. Existing observability tools show *what* happened. Axion shows *why* it happened, *when* it's going wrong, and stops it before damage is done. - -Named after the axion particle: theorized to exist, never directly observed, detected only through its effects on surrounding matter. Agent beliefs are the same - invisible, but they shape every decision. Axion makes them visible. - -Built for any agent that supports `base_url` override. Zero code changes. +Named after the axion particle: theorized to exist, never directly observed, detected only through its effects. Agent beliefs are the same. They are invisible but they shape every decision, and this makes them visible. ``` -Agent ←→ Axion (CF Worker) ←→ Model API - ←→ Tools (file, shell, API) +Agent <-> Axion (CF Worker) <-> Model API ``` +> **Honest scope.** This repo is the Phase 1 observe path plus an opt-in schema enforce mode. It does not detect loops, block tool calls, or build a belief graph. See [What is not built](#what-is-not-built) before you form expectations. The full product plan lives in [BUILD-SPEC.md](./BUILD-SPEC.md), which is the source of truth for scope. + --- -## Core Features +## What is shipped -| Feature | Description | -| --- | --- | -| **Belief Extraction** | Rule-based parser extracts causal claims, assumptions, intentions, and evidence from each model response. No model dependency, sub-millisecond. | -| **Belief DAG** | Beliefs are linked into a directed acyclic graph across the session. When an action fails, backtrack to the root-cause belief. | -| **SSE Streaming Proxy** | Forwards model responses with zero added latency. Belief extraction runs in `waitUntil()` after the stream completes. | -| **Session State** | Durable Object per session holds the full belief graph in memory. No external database required. | -| **Timeline Dashboard** | Visual timeline of every decision point, beliefs behind it, confidence level, and evidence cited. Filter by type, confidence, or wrong beliefs only. | -| **Agent-Agnostic** | Works with Claude Code, Codex CLI, Cursor, Gemini CLI, Hermes, LangChain, and any agent that supports `base_url` override. | +- **OpenAI-compatible proxy.** `POST /v1/chat/completions`, streaming and non-streaming. +- **Anthropic Messages proxy.** `POST /v1/messages`, streaming and non-streaming. +- **Passthrough auth.** Forward the caller's `Authorization` or `x-api-key`. Fall back to the `UPSTREAM_API_KEY` secret only when it is set. Otherwise return 401. No `Bearer undefined` is ever sent upstream. +- **Zero-latency observe path.** The response body is tee'd with `ReadableStream.tee()`. One branch streams to the caller untouched; the other accumulates text for extraction in `waitUntil()` after delivery. +- **Belief extraction.** Regex patterns pull causal/assumption/intention/evidence fragments. Confidence starts at a per-pattern baseline and is nudged by additive markers, clamped to `[0.1, 1.0]`. Every belief is stamped with the session id. +- **Durable session store.** A Durable Object appends each response's beliefs as a batch to Durable Object storage. `GET /api/beliefs/:sessionId` returns a flat chronological `ExtractedBelief[]`. +- **Local dashboard.** Paste or link a session id, see its timeline. Filter by type, minimum confidence, or low-confidence only. +- **PolyVerdict enforce mode (opt-in).** Send a JSON Schema and the Worker validates, coerces types, and retries the model up to 3 times before returning. Off by default. +- **Tests and CI.** Vitest suite, `npm test` / `npm run check`, GitHub Actions on push and PR. ---- +## What is not built -## Advanced Capabilities +These appear in older drafts and in the roadmap. None of them are in the code: -| Feature | Description | -| --- | --- | -| **Confidence Scoring** | Each extracted belief gets a confidence score (0.0–1.0) based on linguistic markers: "definitely" (0.9), "probably" (0.7), "might" (0.4), "not sure" (0.3). | -| **Belief Type Classification** | Four types: causal (why), assumption (what's taken as given), intention (what the agent will do), evidence (what was cited). Color-coded in dashboard. | -| **Root-Cause Backtracking** | When a failure occurs (error, wrong output, user correction), trace back through the belief DAG to the exact belief that caused the wrong action. | -| **Observability Integration** | Belief data exports as structured JSON - feed into Langfuse, Arize, Braintrust, or any OpenTelemetry-compatible tool as span metadata. | -| **Zero-Code Integration** | Change `base_url`. That's it. No SDK, no imports, no code changes. The agent runs normally while Axion observes in the background. | +- Belief DAG, parent/child edges, root-cause backtracking. The store is a flat timeline. `BeliefNode`/`BeliefDAG` types exist but are marked `@planned` and have no runtime. +- `/api/sessions` session registry. The dashboard takes a pasted session id instead. +- Axion Loop (loop detection) and Axion Gate (tool-call blocking). +- Semantic PolyVerdict, second-model verification, hallucination checks. +- Schema registry Durable Object, schema hash cache. +- Hosted multi-session SaaS dashboard. --- -## The Three Layers +## The three layers -| Layer | Name | Phase | What it does | +| Layer | Name | Status | What it does | | :---: | --- | :---: | --- | -| 1 | **Axion Lens** | Shipping | Belief inspection - extracts the agent's reasoning chain, assumptions, and confidence from each response. Builds a belief DAG across the session. | -| 2 | **Axion Loop** | Planned | Revision loop breaker - embeds agent outputs, detects when an agent is stuck cycling the same reasoning, intervenes with targeted feedback instead of a crude kill signal. | -| 3 | **Axion Gate** | Planned | Runtime verification - intercepts tool calls before execution, checks plan alignment, contradiction, and failure patterns, then blocks or allows. | +| 1 | **Axion Lens** | Shipping (observe) | Extracts reasoning fragments from each response into a per-session timeline. Read-only. | +| 2 | **Axion Loop** | Planned | Detect when an agent is cycling the same reasoning and intervene with feedback. Not implemented. | +| 3 | **Axion Gate** | Planned | Verify tool calls before execution and block bad ones. Not implemented. | -> Lens is read-only and cannot break anything. It ships first and powers the other two: Loop uses the belief graph to classify loops, Gate uses the belief graph plus the stated plan to verify actions. +Lens is read-only and cannot change agent behaviour. PolyVerdict enforce mode is a separate opt-in path that does change output (it can retry the model and coerce types), triggered only when the caller supplies a schema. --- ## Architecture ``` -Phase 1 (Lens): Observe - - Agent - ↕ HTTP - Axion Proxy (CF Worker) - ├── stream.ts → SSE passthrough, zero added latency - ├── extract.ts → belief extraction (regex + NLP, <1ms) - └── SessionDurableObject → belief DAG in memory - ↕ - Model API - -Phase 2 (Loop): Detect Phase 3 (Gate): Block - [planned] [planned] +Agent (OpenAI- or Anthropic-compatible, sends x-axion-session) + | + v +Axion Worker (Cloudflare) + |- auth.ts resolve passthrough / server-key credentials, or 401 + |- providers/ match POST /v1/chat/completions or POST /v1/messages + |- stream.ts ReadableStream.tee: caller branch + extraction branch + |- content.ts normalize SSE deltas / non-stream body to assistant text + |- extraction.ts waitUntil -> extractBeliefs({ sessionId }) -> DO + |- polyverdict/ opt-in enforce: validate + coerce + retry <=3 + | + v +Model API (UPSTREAM_API_URL, default https://api.openai.com) + +State: SessionDurableObject appends belief batches to DO storage. +Read: GET /api/beliefs/:sessionId -> { sessionId, beliefs: ExtractedBelief[] } +UI: GET /dashboard ``` --- -## Quick Start +## Quick start -Requires Node.js and a Cloudflare account. +Requires Node.js 20+ and a Cloudflare account for deploy. ```bash -# Clone -git clone https://github.com/LatticeAG/Axion.git -cd axion - -# Install npm install - -# Run locally +cp .dev.vars.example .dev.vars # optional: set UPSTREAM_API_KEY npm run dev -# → http://localhost:8787 - -# Point any agent at Axion -export ANTHROPIC_BASE_URL=http://localhost:8787 export OPENAI_BASE_URL=http://localhost:8787 +# send header x-axion-session: my-session on your agent's requests +# dashboard: http://localhost:8787/dashboard?session=my-session +npm run check +``` -# Open the dashboard -# → http://localhost:8787/dashboard +The proxy uses passthrough auth. If your agent already sends its own API key, you do not need `UPSTREAM_API_KEY`. Set it only if you want the Worker to hold the key and let callers omit it. + +Anthropic agents route through `POST /v1/messages`: + +```bash +export ANTHROPIC_BASE_URL=http://localhost:8787 +# Claude Code and other Anthropic Messages clients hit POST /v1/messages ``` Deploy your own instance: ```bash npx wrangler deploy -# → https://your-axion-worker.dev +# -> https://your-axion-worker.dev ``` --- -## Integration +## Sessions and the dashboard -Axion works with any agent that supports `base_url` override. Zero code changes - set the environment variable and the agent runs normally. +Beliefs are grouped by session. Send `x-axion-session: ` on agent requests to correlate a multi-turn run. If the header is absent the Worker generates a UUID per request and returns it in the `x-axion-session` response header, so a single call is still captured but multi-turn correlation needs the header. -```bash -# Claude Code -export ANTHROPIC_BASE_URL=https://your-axion-worker.dev +Open `http://localhost:8787/dashboard`, paste the session id, and press Load. The id also reads from `?session=` in the URL and from `localStorage` (`axion.sessionId`). -# Codex / OpenAI-compatible agents -export OPENAI_BASE_URL=https://your-axion-worker.dev -``` +The beliefs API is unauthenticated in Phase 1. Anyone with a session id can read that session's beliefs. Treat the id like a capability token. See [SECURITY.md](./SECURITY.md). -```yaml -# Hermes - config.yaml -providers: - anthropic: - base_url: https://your-axion-worker.dev -``` +--- -``` -# Cursor -# Settings → Models → set "Custom API base URL" -# → https://your-axion-worker.dev -``` +## PolyVerdict enforce mode -Axion observes in the background. The agent behaves exactly as before - except every decision now has a visible, traceable belief behind it. +Enforce mode is off unless the request carries a schema. Two triggers: ---- +- Header `x-axion-schema: ` (URL-decoded if needed), or +- Body `response_format: { "type": "json_schema", "json_schema": { "schema": { ... } } }`. -## Belief Extraction Patterns +When triggered, the Worker forces a non-streaming upstream call, parses the assistant JSON (stripping Markdown fences), validates it against the schema, and coerces primitive types (`"42"` to number, `"true"`/`"false"` to boolean, number to string). On a violation it appends the errors as a correction message and retries, up to 3 attempts total. On success it returns a provider-shaped JSON response. After 3 failed attempts it returns HTTP 422 with the violations. Lens still extracts from the delivered text. -| Type | Pattern Examples | Confidence | -| --- | --- | --- | -| **Causal** | "because X", "since X", "due to X", "as a result of X" | 0.6–0.9 | -| **Assumption** | "assuming X", "I'll assume X", "presumably X", "if X then Y" | 0.3–0.7 | -| **Intention** | "I'll do X", "I'm going to X", "let me X", "I should X" | 0.5–0.8 | -| **Evidence** | "based on X", "from the X", "according to X", "the error says X" | 0.6–0.9 | +The schema subset covers `type`, `properties`, `required`, `items`, `enum`, and nesting. Unknown keywords are ignored. There is no semantic or second-model verification. -Confidence modifiers: "definitely" (+0.2), "certainly" (+0.2), "probably" (+0.1), "might" (−0.2), "could be" (−0.1), "not sure" (−0.3). +Example (OpenAI path): + +```bash +curl http://localhost:8787/v1/chat/completions \ + -H "Authorization: Bearer $OPENAI_API_KEY" \ + -H "x-axion-schema: {\"type\":\"object\",\"properties\":{\"score\":{\"type\":\"number\"}},\"required\":[\"score\"]}" \ + -d '{"model":"gpt-4o-mini","messages":[{"role":"user","content":"Rate this 1-10 as JSON."}]}' +``` --- -## Roadmap +## Belief extraction -| Phase | Layer | Status | Ships | -| :---: | --- | :---: | --- | -| 1 | **Axion Lens** | In Progress | Proxy + belief extraction + local dashboard | -| 2 | **Axion Loop** | Planned | Embedding detection + intervention injection | -| 3 | **Axion Gate** | Planned | Tool-call interception + verification + blocking | +| Type | Trigger phrases | Baseline confidence | +| --- | --- | --- | +| Causal | "because X", "because of X", "since X", "due to X", "as a result of X" | 0.8 to 0.85 | +| Assumption | "assuming X", "presumably X", "I'll assume X", "if X then Y" | 0.6 to 0.65 | +| Intention | "I'll X", "I'm going to X", "let me X", "I should X", "I plan/intend to X" | 0.75 | +| Evidence | "based on X", "according to X", "from the X", "the error says X" | 0.7 to 0.85 | + +Confidence starts at the baseline and each marker found near the match adds its delta: -**Phase 1 open-source scope:** the proxy, the belief extraction engine, session state, and a local single-session dashboard. A hosted multi-session SaaS dashboard (cross-session analysis, team sharing, alerting, community pattern library) comes later. +- definitely / certainly / absolutely: +0.2 +- probably / likely: +0.1 +- might / could be / possibly / may: -0.2 +- not sure / uncertain / unsure: -0.3 + +The result is clamped to `[0.1, 1.0]`. This is a linguistic heuristic, not a truth signal. It reflects how the model hedged, nothing more. --- -## File Structure +## File structure ``` axion/ -├── src/ -│ ├── proxy/ CF Worker: stream proxy + interception -│ │ ├── index.ts Worker entry point + routing -│ │ ├── stream.ts SSE streaming + response teeing -│ │ ├── routes.ts API route handlers (/dashboard, /api/beliefs) -│ │ ├── beliefs.ts Belief storage coordination -│ │ ├── extraction.ts Triggers belief extraction via waitUntil() -│ │ └── types.ts Proxy-specific types -│ ├── lens/ Belief extraction engine -│ │ ├── types.ts ExtractedBelief, BeliefNode, BeliefDAG -│ │ ├── patterns.ts Regex pattern definitions -│ │ ├── extract.ts Main extraction function -│ │ └── index.ts Re-exports -│ ├── state/ Durable Object: session belief graph -│ │ └── SessionDurableObject.ts -│ └── dashboard/ React timeline UI -│ ├── index.html -│ ├── app.js -│ └── styles.css -├── SPEC.md Full architecture specification -├── TECHNICAL.md Technical deep-dive -├── wrangler.toml CF Worker config + DO bindings -├── tsconfig.json -└── package.json +|- src/ +| |- proxy/ +| | |- index.ts Worker entry: routing, observe + enforce branches +| | |- auth.ts passthrough / server-key credential resolution +| | |- stream.ts ReadableStream.tee + SSE parsing (OpenAI + Anthropic) +| | |- content.ts assistant-text normalization per provider +| | |- extraction.ts waitUntil glue to extractBeliefs + DO store +| | |- beliefs.ts GET /api/beliefs/:id -> DO +| | |- routes.ts dashboard static asset handler +| | |- providers/ openai + anthropic adapters, matcher, interface +| | |- types.ts Env + proxy request/response types +| |- lens/ +| | |- patterns.ts regex belief patterns + confidence markers +| | |- extract.ts extraction engine (additive confidence, clamp) +| | |- types.ts ExtractedBelief; BeliefNode/BeliefDAG (@planned) +| |- polyverdict/ +| | |- schema.ts JSON Schema subset validator + coercion +| | |- enforce.ts trigger detection, retry loop, hint injection +| | |- types.ts enforce types +| |- state/ +| | |- SessionDurableObject.ts append batches, flatten on GET +| | |- sessionBeliefs.ts pure flatten / sessionId helpers +| |- dashboard/ React via CDN, no build step +|- BUILD-SPEC.md locked scope (source of truth) +|- SPEC.md TECHNICAL.md SPEC-PolyVerdict.md PLAN.md +|- wrangler.toml tsconfig.json package.json ``` --- -## Tech Stack +## Tech stack -- **Runtime** - Cloudflare Workers (edge proxy, zero cold start, global) -- **State** - Durable Objects (per-session belief graph, in memory) -- **Extraction** - Rule-based parser (regex + lightweight NLP), no model dependency, <1ms -- **Dashboard** - React (CDN, no build step), served from Worker static assets -- **External dependencies** - Zero in the open-source core +- **Runtime:** Cloudflare Workers. +- **State:** Durable Objects, one per session, backed by Durable Object storage. +- **Extraction:** regex rules only, no model call, sub-millisecond. +- **Dashboard:** React from CDN, no bundler, served as static assets. +- **Runtime dependencies:** none. `wrangler`, `typescript`, and `vitest` are dev-only. --- -## Key Technical Details +## Known issues -- **Streaming:** Responses are streamed through via `ReadableStream` with a `TransformStream` tee - one stream goes to the caller, the other accumulates for belief extraction. Zero added latency on the hot path. -- **Extraction timing:** Belief extraction runs in `waitUntil()` after the response stream completes. The agent never waits for extraction. -- **Session isolation:** Each agent session gets its own Durable Object instance. Belief graphs are isolated per session, held in memory. -- **Pattern engine:** Regex patterns are defined as an extensible array in `patterns.ts`. New belief types or confidence modifiers can be added without touching the extraction logic. -- **Dashboard:** Uses React via CDN (no build step, no bundler). Served as static assets from the Worker. Dark theme, monospace, LatticeAG brand. +- Loop (Phase 2) and Gate (Phase 3) are not implemented. +- The beliefs API is unauthenticated. A session id is a read capability. See [SECURITY.md](./SECURITY.md). +- Session storage is unbounded. Beliefs append to Durable Object storage and are never trimmed. Long-lived session ids will grow without limit. There is no rate limiting yet. +- Extraction is a regex heuristic. It misses reasoning that does not use the trigger phrases and will mis-parse unusual phrasing. Confidence reflects hedging words, not correctness. +- Without an `x-axion-session` header each request lands under a fresh UUID, so multi-turn correlation depends on the caller sending a stable id. +- The dashboard loads React from a CDN and needs internet access for that page. The proxy itself does not. +- Enforce mode always returns non-streaming JSON, even if the client asked to stream. This is intentional so the full payload can be validated. --- ## Links -- **Product page:** [latticeag.vercel.app/products/axion](https://latticeag.vercel.app/products/axion) -- **Source code:** [github.com/LatticeAG/Axion](https://github.com/LatticeAG/Axion) +- **Source:** [github.com/LatticeAG/Axion](https://github.com/LatticeAG/Axion) - **LatticeAG:** [latticeag.vercel.app](https://latticeag.vercel.app) -## Known Issues - -- Phase 2 (Axion Loop) and Phase 3 (Axion Gate) are not yet implemented -- Dashboard uses CDN-hosted React - requires internet access for the dashboard page only (proxy works offline) -- Durable Object belief graph is in-memory - sessions are lost on DO eviction (acceptable for Phase 1) - ---- - ## License -MIT - see [LICENSE](./LICENSE). +MIT. See [LICENSE](./LICENSE). ---
-**LatticeAG** - *Agents, together.* - -[github.com/LatticeAG/Axion](https://github.com/LatticeAG/Axion) +**LatticeAG** - Agents, together.
diff --git a/SPEC-PolyVerdict.md b/SPEC-PolyVerdict.md index 0bc851a..8c2cf6e 100644 --- a/SPEC-PolyVerdict.md +++ b/SPEC-PolyVerdict.md @@ -1,81 +1,77 @@ -# SPEC-PolyVerdict: Structured Output Firewall +# PolyVerdict: structured output enforcement -> **Status:** Proposal — potential Axion Gate evolution -> **Type:** Feature spec (proxy layer for LLM output compliance) -> **Sequencing:** Not part of Phase 1 OSS. See [PLAN.md](./PLAN.md) Wave 4 — opt-in enforce mode after Lens observe path is shippable. Do not implement as default Lens middleware. +> **Status:** partially implemented. The syntax path ships (opt-in). Semantic verification is future work. +> **Type:** feature spec for an opt-in enforce path in the Axion Worker. +> **Scope lock:** [BUILD-SPEC.md](./BUILD-SPEC.md) D7/D8. This spec must not describe the enforce path as default Lens middleware; it runs only when a caller supplies a schema. -## Problem - -Apps calling LLMs get malformed JSON, hallucinated fields, type errors, and schema violations. Existing solutions (vLLM structured outputs, guidance, outlines) are inference-engine-integrated — you must use their runtime. There is no model-agnostic middleware. +## What ships today -## Relationship to Axion +PolyVerdict v1 is an opt-in enforce path in the same Worker as Axion Lens. It runs only when a request carries a JSON Schema. Code lives in `src/polyverdict/` and is wired into `src/proxy/index.ts`. -| Axion Layer | PolyVerdict Relationship | -|-------------|-------------------------| -| Axion Lens | Lens extracts beliefs from model output. PolyVerdict ensures the output is structurally valid *before* belief extraction. | -| Axion Loop | Loop detects stuck agents. PolyVerdict prevents schema-violation loops by enforcing correct format. | -| Axion Gate | Gate blocks bad tool calls pre-execution. PolyVerdict blocks bad LLM responses pre-delivery. **Same pattern, different layer.** | +Implemented: -## Solution +1. **Schema trigger detection.** `x-axion-schema` header (JSON, URL-decoded as a fallback), or body `response_format: { type: "json_schema", json_schema: { schema, name? } }`. Header wins. No trigger means the request stays on the observe path. +2. **Non-streaming enforce.** The upstream call is forced to `stream: false` so the full payload can be validated. The client `response_format` is stripped upstream; the Worker owns validation. +3. **Parse.** Strip Markdown fences, then `JSON.parse`, with a balanced-bracket fallback for JSON embedded in prose. +4. **Validate against a minimal JSON Schema subset.** `type` (single or union), `properties`, `required`, `items` (schema or tuple), `enum`, and nesting. Unknown keywords are ignored. +5. **Type coercion.** `"42"` to number/integer, `"true"`/`"false"` to boolean, finite number/boolean to string. Coercion runs before enum checks. +6. **Retry with hints.** On a violation the errors are appended as a correction message and the model is called again, up to 3 attempts total. OpenAI and Anthropic message shapes are both handled. +7. **Result.** On success, a provider-shaped 200 whose assistant content is the coerced JSON string; Lens then extracts from that delivered text. After 3 failed attempts, HTTP 422 with the violation list. -A proxy layer that intercepts LLM responses and enforces: +## What is not built -1. **JSON Schema Compliance** — parse response against user-provided schema. Auto-retry with schema in prompt. -2. **Semantic Content Diff** — for critical fields, run a second model to verify content semantics. -3. **Type Coercion** — silently cast types (string "42" → int 42) instead of failing. -4. **Hallucination Detection** — use PolyGnosis-style adversarial verification on high-risk fields. +These appeared in the original proposal and are not implemented: -## Architecture +- **Semantic content diff / second-model verification.** No field is checked by a second model. +- **Hallucination detection (PolyGnosis-style).** Not present. +- **Schema registry Durable Object.** Named schemas are not supported. Schemas are inline (header JSON or `response_format`) only. +- **Hash cache** for identical schema plus prompt. Not present; every enforce request calls upstream. +- **LexGateway cost-optimised retry routing.** Retries hit the same upstream as the first attempt. +- **Streaming enforce.** Enforce always returns non-streaming JSON, even if the client asked to stream. -``` -App → POST /v1/chat/completions (with x-schema header) - → PolyVerdict Proxy (wraps same Worker pattern as Axion) - → Schema Enforcement Layer - → Retry Loop (max 3, with progressively stricter schema hints) - → LLM (upstream API) - ← Validated response - → Semantic Verification (opt-in, per-field) - → Type Coercion - ← Clean typed JSON to app -``` +## Problem -## Key Design Decisions +Apps calling LLMs get malformed JSON, missing fields, and type errors. Inference-side tools (structured decoding in vLLM, guidance, outlines) require you to run their engine. PolyVerdict is model-agnostic middleware: change the base URL, add a schema, get validated JSON back or a 422. -- **Drop-in replacement** for OpenAI-compatible endpoints (change `base_url` only) -- **Schema format:** JSON Schema (draft 2020-12) via `response_format` extension -- **Same streaming architecture as Axion Lens** — `TransformStream` tee pattern, zero added latency on pass-through -- **Hash cache** — identical schema + identical prompt → skip verification -- **Cost control:** semantic verification is opt-in per field with a budget cap +## Relationship to Axion -## Integration With Axion +| Axion layer | Relationship | +|---|---| +| Lens | Lens observes; PolyVerdict enforces. Enforce runs Lens on the delivered (coerced) text, so beliefs come from validated output. | +| Loop | Planned. A format-enforced output removes one class of retry loop, but Loop itself is not built. | +| Gate | Planned. PolyVerdict is Gate-shaped (validate, retry, block) but scoped to response format, not tool calls. | -PolyVerdict runs in the same proxy as Axion: +## Architecture (as built) ``` -Agent ↔ Axion Proxy - ├── PolyVerdict (on response stream) - │ └── Validates structure, enforces schema - ├── Axion Lens (on completed response) - │ └── Extracts beliefs from validated output - ├── Axion Loop (on agent's next action) - │ └── Detects loops using cleaner data - └── Axion Gate (on tool calls) - └── Blocks bad actions using verified beliefs +App -> POST /v1/chat/completions or /v1/messages + with x-axion-schema OR response_format.json_schema + | + v +detectSchemaTrigger -> present? + |- no -> observe path (tee + Lens), unchanged + |- yes -> enforce loop (<=3): + force stream:false -> upstream + extract assistant text + strip fences -> parse JSON + validateAndCoerce(schema) + ok -> provider-shaped 200 (coerced JSON) + Lens on delivered text + fail -> append violation hint, retry + exhausted -> 422 with violations ``` -PolyVerdict feeds cleaner, more predictable data to Lens, which means Loop and Gate make better decisions. +## Design decisions -## Implementation Notes +- **Opt-in, separate path.** Enforce is never applied to a request without a schema. The default observe path stays zero added latency. +- **Buffered by design.** Validating a partial stream is not meaningful, so enforce forces a full non-streaming payload. The zero-latency guarantee does not apply here. +- **Drop-in.** Same base-URL override as the observe path; the schema is the only extra input. +- **Zero dependencies.** The validator is hand-written; no schema library is pulled in. -- Same `wrangler.toml` / `Durable Object` / `TransformStream` pattern as Axion Lens -- Schema registry: Durable Object that stores named schemas (avoids sending full JSON Schema on every request) -- `x-schema` header: `"schema-name"` or inline JSON Schema -- Retry: uses the failed schema violation as a hint in the retry prompt -- Models: use LexGateway for cost-optimised retry routing on the second attempt +## Future work -## Success Criteria +- Semantic verification, opt-in and budget-capped per field. +- Schema registry Durable Object for named schemas, keyed separately from session state. +- Hash cache to skip upstream when an identical schema plus prompt already passed. +- Wider JSON Schema coverage (formats, patterns, numeric bounds). -- 100% schema compliance on valid model outputs -- <200ms overhead for syntax-only path (same budget as Axion Lens) -- <5% false positive rate on semantic verification -- Zero user-visible latency when schema passes on first try +These are deferred until the syntax path and the Lens contracts have proven out. See [PLAN.md](./PLAN.md) for sequencing. diff --git a/SPEC.md b/SPEC.md index 7413ff6..171bc26 100644 --- a/SPEC.md +++ b/SPEC.md @@ -1,67 +1,60 @@ # Axion -> Agent cognitive middleware - a proxy layer that inspects, detects, and verifies agent reasoning. -> Open-source core. Hosted SaaS dashboard later. +> Agent cognitive middleware. A proxy that reads an agent's reasoning from its own model output. +> Open-source core. The vision is three layers; today one of them ships. ---- +> **Source of truth for scope:** [BUILD-SPEC.md](./BUILD-SPEC.md). All product decisions there are locked. This file keeps the brand and architecture at a high level. Where the two disagree, BUILD-SPEC wins. -## What It Is +--- -Axion is a proxy that sits between an AI agent and the outside world. It intercepts model responses, agent outputs, and tool calls - then inspects, detects, and verifies agent reasoning in real time. +## What it is -Any agent that supports `base_url` override works. Zero code changes. +Axion is a proxy between an AI agent and a model API. Override the agent's base URL and Axion forwards each request upstream, streams the response back with zero added latency, then parses the assistant text for reasoning fragments and stores them per session. Any agent that supports a `base_url` override works, with no code changes. ``` -Agent ←→ Axion ←→ Model API -Agent ←→ Axion ←→ Tools (file, shell, API) +Agent <-> Axion <-> Model API ``` -## What Problem It Solves +## The problem + +Agents make decisions you cannot see. Observability tools show what an agent did (the calls, the tokens, the latency). They do not show the reasoning behind a choice. Axion extracts that reasoning as it streams past and lays it out as a per-session timeline you can read after the fact. -Agents make decisions developers can't understand, get stuck in loops they can't break, and take actions they can't verify. Existing tools show **what** happened (observability). Axion shows **why** it happened, **when** it's going wrong, and **stops** it before damage is done. +The longer-term goal is to act on that reasoning: catch loops, block bad tool calls. Those layers are not built. See status below. --- -## The Three Layers +## The three layers + +### Layer 1: Axion Lens (belief inspection) + +**Status: shipping (observe path).** Read-only. Cannot change agent behaviour. + +Intercepts each model response and extracts reasoning fragments (causal claims, assumptions, intentions, cited evidence) with a confidence score, then stores them as a flat chronological timeline per session. A local dashboard reads the timeline back. -### Layer 1 - Axion Lens (belief inspection) -**Observe. Read-only. Cannot break anything.** +- Rule-based regex parsing, no model call, sub-millisecond. +- Emits `ExtractedBelief` records: `{ id, sessionId, type, belief, evidence?, confidence, actionTaken?, timestamp, rawText, line }`. +- Confidence is a per-pattern baseline nudged by additive markers, clamped to `[0.1, 1.0]`. +- Stored in a Durable Object as append-only batches; the public API flattens them. -Intercepts model responses and extracts the agent's beliefs, assumptions, and reasoning chain. Builds a belief DAG across the session. When something goes wrong, you trace back to the exact belief that caused the wrong action. +There is no belief graph and no root-cause backtracking. Those were in earlier drafts and are not implemented. -- Extracts `{belief, evidence, confidence, action_taken}` from each model response -- Rule-based parsing (regex + NLP), not model-based - fast and cheap -- Builds a belief graph: beliefs → decisions → outcomes -- Backtracks from failures to root-cause beliefs -- Serves a timeline dashboard: every decision point, beliefs behind it, confidence, correct/wrong -- Feeds into existing observability tools (Langfuse, Arize) as structured metadata +### Layer 2: Axion Loop (revision loop breaker) -**This is the MVP. Ships first.** +**Status: planned. Not implemented.** -### Layer 2 - Axion Loop (revision loop breaker) -**Detect + intervene. Uses belief graph from Layer 1.** +The intent is to detect when an agent repeats the same reasoning and inject targeted feedback instead of a hard kill. This needs stable multi-turn sessions and an embedding step, neither of which exists yet. -Detects when an agent is stuck in a revision loop and intervenes with targeted feedback - not a crude kill signal. +### Layer 3: Axion Gate (runtime verification) -- Embeds each agent output, maintains sliding window of last 10-20 outputs -- If cosine similarity exceeds threshold (0.85), flags potential loop -- Classifies: productive iteration vs stuck loop vs thrashing (uses belief graph) -- Injects targeted feedback: "You've tried [X] 3 times with the same result. Consider: [alternatives not yet tried]." -- Escalation ladder: soft nudge → hard nudge (force re-read task) → escalate to human +**Status: planned. Not implemented.** -### Layer 3 - Axion Gate (runtime self-verification) -**Block + correct. Uses belief graph + plan from Layer 1.** +The intent is to check tool calls before they run (plan alignment, contradiction, known failure patterns) and block bad ones. No tool-call interception exists in the code. -Verification gate that checks agent actions **before** they execute. Not post-hoc evals - real-time blocking with corrections fed back to the agent. +### PolyVerdict (structured output enforce) -- Intercepts tool calls (file writes, shell, API) before execution -- Three checks per call: - - **Plan alignment:** does this action match the stated plan? - - **Contradiction detection:** does this contradict a prior decision? - - **Pattern matching:** does this match a known failure anti-pattern? -- Blocks bad actions, injects correction: "Blocked: you decided to use customer_uuid in step 3 but are writing user_id." -- Uses cheap flash models via OpenCode Zen for verification (target: <500ms, <$0.001 per check) -- Logs every blocked action as training data for the rules engine +**Status: partial. Syntax path shipped, opt-in.** + +A separate enforce path in the same Worker. When a caller supplies a JSON Schema it validates and type-coerces the model output and retries up to 3 times. It is not part of the default observe path and only runs when a schema is present. See [SPEC-PolyVerdict.md](./SPEC-PolyVerdict.md). --- @@ -69,82 +62,90 @@ Verification gate that checks agent actions **before** they execute. Not post-ho ``` Agent - ↕ -Axion Proxy (CF Worker) - ├── Axion Lens → intercepts model responses → extracts beliefs → waitUntil() - ├── Axion Loop → intercepts agent outputs → embeds → detects loops - └── Axion Gate → intercepts tool calls → verifies → blocks/allows - ↕ -Model API / Tools - -State: Durable Object per session (belief DAG in memory) + | +Axion Worker (Cloudflare) + |- Auth: passthrough caller key, or configured server key, else 401 + |- POST /v1/chat/completions OpenAI adapter -> observe or enforce + |- POST /v1/messages Anthropic adapter -> observe or enforce + |- GET /api/beliefs/:id flat ExtractedBelief[] + |- GET /dashboard paste-session timeline UI + | +Model API + +State: SessionDurableObject, one per session, append-only belief batches in DO storage. ``` -## Build Order +## Build order and status | Phase | What | Status | |---|---|---| -| 1 | Axion Lens - proxy + belief extraction + local dashboard | **Next** (see [PLAN.md](./PLAN.md)) | -| 2 | Axion Loop - embedding detection + intervention injection | Future | -| 3 | Axion Gate - tool call interception + verification + blocking | Future | +| 1 | Axion Lens: OpenAI + Anthropic observe proxy, belief timeline, DO store, dashboard | **Shipped** | +| 1 | PolyVerdict enforce mode (syntax validate + coerce + retry), opt-in | **Shipped** | +| 2 | Axion Loop: loop detection + intervention | Planned, not started | +| 3 | Axion Gate: tool-call interception + verification + blocking | Planned, not started | +| later | Semantic PolyVerdict, schema registry, belief graph, hosted SaaS | Not started | + +The locked build order and module map are in [BUILD-SPEC.md](./BUILD-SPEC.md). [PLAN.md](./PLAN.md) records how the current state was reached and what is deferred. -Production OSS readiness (auth, dashboard contract, docs honesty, PolyVerdict sequencing) is specified in [PLAN.md](./PLAN.md). Do not treat Loop/Gate/PolyVerdict as part of the Phase 1 OSS tag until that plan’s Wave 0–1 exits. +## Open-source scope (Phase 1) -## Open-Source Scope (Phase 1) +This repo is the open-source core: -This repo contains the open-source core: +- CF Worker proxy for OpenAI chat completions and Anthropic Messages, zero added latency. +- Rule-based belief extraction. +- Per-session Durable Object store. +- Local single-session dashboard. +- Opt-in PolyVerdict syntax enforce mode. -- CF Worker proxy (streams model responses, zero added latency) -- Belief extraction engine (rule-based parser) -- Session state (Durable Object) -- Local dashboard (single session, served by Worker) +Not in the open-source core (possible SaaS later): -**Not in open-source core (SaaS later):** -- Hosted multi-session dashboard -- Cross-session belief analysis -- Team sharing + alerting -- Community belief pattern library +- Hosted multi-session dashboard. +- Cross-session analysis. +- Team sharing and alerting. +- Community pattern library. -## Tech Stack +## Tech stack -- **Runtime:** Cloudflare Workers -- **Session state:** Durable Objects -- **Extraction:** Rule-based (regex + lightweight NLP), no model dependency -- **Dashboard:** React, served from Worker static assets -- **Zero external dependencies** for the open-source core +- **Runtime:** Cloudflare Workers. +- **State:** Durable Objects (per-session append-only storage). +- **Extraction:** regex rules, no model dependency. +- **Dashboard:** React from CDN, served as static assets from the Worker. +- **Runtime dependencies:** none. ## Integration ```bash -# Claude Code -export ANTHROPIC_BASE_URL=https://your-axion-worker.dev - -# Codex / OpenAI agents +# OpenAI-compatible agents export OPENAI_BASE_URL=https://your-axion-worker.dev -# Cursor - set custom API base URL in settings -# Hermes - set provider base_url in config.yaml +# Anthropic Messages clients (e.g. Claude Code) +export ANTHROPIC_BASE_URL=https://your-axion-worker.dev ``` -Agent works normally. Axion observes in the background. Dashboard at `https://your-axion-worker.dev/dashboard`. +Send `x-axion-session: ` to correlate a multi-turn run, then open the dashboard at `https://your-axion-worker.dev/dashboard` and paste the id. Only these two provider routes are implemented; other agents work only if they speak one of these two API shapes. --- ## Brand -**Axion** - a particle theorized to exist but never directly observed. Agent beliefs are the same: invisible, but they shape every decision. Axion makes them visible. +**Axion** is a particle theorized to exist but never directly observed, detected only through its effects. Agent beliefs are the same. They are invisible but they drive every decision. -**LatticeAG** - *"Agents, together."* +**LatticeAG** - Agents, together. ``` axion/ -├── SPEC.md ← this file -├── README.md -├── src/ -│ ├── proxy/ ← CF Worker: stream proxy + interception -│ ├── lens/ ← belief extraction engine -│ ├── state/ ← Durable Object: session belief graph -│ └── dashboard/ ← React: belief timeline -├── wrangler.toml -└── package.json +|- BUILD-SPEC.md <- locked scope, source of truth +|- SPEC.md <- this file +|- README.md +|- TECHNICAL.md +|- SPEC-PolyVerdict.md +|- PLAN.md +|- src/ +| |- proxy/ CF Worker: routing, auth, tee, providers, enforce wiring +| |- lens/ belief extraction engine +| |- polyverdict/ schema validate + coerce + retry +| |- state/ Durable Object session store +| |- dashboard/ React timeline UI +|- wrangler.toml +|- package.json ``` diff --git a/TECHNICAL.md b/TECHNICAL.md index feb9898..54fc070 100644 --- a/TECHNICAL.md +++ b/TECHNICAL.md @@ -1,255 +1,259 @@ -# Axion - Technical Specification +# Axion technical reference -> Deep-dive into the architecture, data flow, and implementation details. -> For a high-level overview, see [README.md](./README.md). For the full build plan, see [SPEC.md](./SPEC.md). +> How the Worker actually behaves, matched to the code in `src/`. +> High-level overview: [README.md](./README.md). Locked scope: [BUILD-SPEC.md](./BUILD-SPEC.md). + +This document describes what is implemented. Planned layers (Loop, Gate, belief graph, semantic PolyVerdict) are called out as planned and have no runtime. --- -## System Architecture +## Request routing -``` - ┌─────────────────────────────────────────────┐ - │ Axion Worker │ - │ (Cloudflare Workers) │ - │ │ - Agent ──HTTP──→ fetch() → route handler │ - │ │ │ - │ ┌───────────┼───────────┐ │ - │ │ │ │ │ - │ ▼ │ ▼ │ - │ stream.ts │ routes.ts │ - │ (SSE proxy) │ /dashboard │ - │ │ │ /api/beliefs/:id │ - │ │ │ /api/sessions │ - │ ▼ │ │ - │ extraction.ts │ SessionDurableObject │ - │ (waitUntil) │ .fetch() → belief DAG │ - │ │ │ │ - Model API ←────┘ ▼ │ │ - │ lens/extract │ │ - │ (regex+NLP) │ │ - │ │ │ │ - │ ▼ │ │ - │ SessionDurableObject.addBelief() │ - │ (persisted in DO memory) │ - └─────────────────────────────────────────────┘ -``` +The Worker entry point is `src/proxy/index.ts`. It routes on method and pathname: -### Request Flow (Phase 1 - Axion Lens) +| Method + path | Handler | Result | +|---|---|---| +| `GET /api/beliefs/:sessionId` | `fetchBeliefs` | flat belief timeline for a session | +| `GET /dashboard`, `/dashboard/*` | `handleDashboard` | dashboard static assets | +| `POST /v1/chat/completions` | OpenAI adapter | proxy (observe or enforce) | +| `POST /v1/messages` | Anthropic adapter | proxy (observe or enforce) | +| `GET /` | redirect | 302 to `/dashboard` | +| anything else | fallthrough | 404 | -``` -1. Agent sends POST /v1/chat/completions with messages[] -2. Worker receives request, extracts session ID from x-axion-session header -3. Worker forwards request to upstream Model API (UPSTREAM_API_URL) -4. Response streams back through two paths via TransformStream tee: - ├── Path A: streamed to agent immediately (zero added latency) - └── Path B: accumulated in buffer for belief extraction -5. When stream completes: - ├── waitUntil() triggers extractBeliefs(fullResponseText, sessionId) - ├── Beliefs extracted via regex patterns (sub-millisecond) - └── Beliefs stored in Durable Object via addBelief() -6. Dashboard fetches /api/beliefs/:sessionId → DO returns belief graph JSON -``` +There is no `/api/sessions` route. Session discovery is by pasting an id into the dashboard, not by listing. -### Latency Budget +Provider matching lives in `src/proxy/providers/index.ts`. `matchProvider(pathname, method)` walks the adapter list and returns the first adapter whose `match()` claims the request, or null. -| Stage | Time | Blocking? | -|---|---|---| -| Request forwarding | <1ms | Yes (unavoidable) | -| SSE streaming | Passthrough | No (streamed to caller) | -| Full response accumulation | Stream duration | No (parallel with streaming) | -| Belief extraction | <1ms | No (waitUntil) | -| Belief storage in DO | <5ms | No (waitUntil) | -| **Added latency to agent** | **<1ms** | - | +--- + +## Auth (`src/proxy/auth.ts`) + +`resolveUpstreamHeaders(request, env, provider)` builds the upstream headers or returns a ready-to-send 401. It follows the passthrough-first model (BUILD-SPEC D1), in this order: + +1. Caller `Authorization` header present: forward it as-is. This also covers gateway tokens. +2. Else caller `x-api-key` present: forward it (Anthropic-style key). +3. Else `env.UPSTREAM_API_KEY` set and non-empty after trim: use the server key. On the Anthropic path it is sent as `x-api-key`; otherwise as `Authorization: Bearer `. +4. Else: return 401 with `{ "error": { "message": "Provide Authorization or x-api-key, or configure UPSTREAM_API_KEY" } }`. + +The server key is only used when it trims to a non-empty string, so `Bearer undefined` is never emitted. `OpenAI-Organization` is forwarded when present. On the Anthropic path, `anthropic-version` is set from the caller header or defaults to `2023-06-01`. + +The result is a discriminated union: `{ ok: true, headers }` or `{ ok: false, response }`. The proxy branches on `ok` rather than catching exceptions. --- -## Module Deep-Dive +## Observe path (default) -### src/proxy/stream.ts - SSE Streaming Proxy +For a request with no schema trigger, `observeProviderRequest` in `index.ts` runs: -The core of Axion Lens. Handles both streaming (`stream: true`) and non-streaming responses. +1. Read the session id from `x-axion-session`, or generate a UUID. +2. Parse and validate the JSON body via the provider adapter (non-empty `messages[]`). +3. Resolve upstream headers (auth above). +4. `fetch` the upstream URL (`UPSTREAM_API_URL` + the adapter's `upstreamPath`) with the original raw body. +5. If upstream is not OK, pass the response through unchanged (with the session header added). +6. Tee the response body with `teeResponseForExtraction` (`src/proxy/stream.ts`). One branch streams to the caller untouched. The other accumulates assistant text. +7. In `ctx.waitUntil`, await the accumulated text, normalize it, run `extractBeliefs({ sessionId })`, and store the result in the Durable Object. +8. Return the caller branch with an `x-axion-session` response header. -**Streaming flow:** -1. Forward the request to the upstream API -2. Create a `TransformStream` that tees the response body -3. One readable stream goes to the caller (the agent) - immediate, zero buffering -4. The other stream accumulates chunks into a buffer string -5. When the readable stream to the caller ends, `waitUntil()` fires -6. The accumulated buffer is passed to `extractBeliefs()` +The agent never waits for extraction. The only synchronous cost is forwarding the request and the tee, which does not buffer the caller branch. -**Non-streaming flow:** -1. Forward the request to the upstream API -2. Clone the response (response can only be read once) -3. One clone goes to the caller immediately -4. The other clone is read as text and passed to `extractBeliefs()` via `waitUntil()` +### Streaming vs non-streaming -**Key invariant:** The agent never waits for belief extraction. The response is forwarded immediately regardless of streaming mode. +`isSse` is true when the request body had `stream: true` or the upstream `content-type` includes `text/event-stream`. -### src/lens/patterns.ts - Belief Extraction Patterns +- **SSE:** the extraction branch parses `data:` records and pulls delta text per provider. OpenAI deltas come from `choices[0].delta.content`; Anthropic text comes from `content_block_delta` events with `delta.type === "text_delta"`. +- **Non-SSE:** the extraction branch accumulates the raw body, and `extractAssistantText` parses it via the provider adapter. Raw JSON is never fed to the lens. -Patterns are defined as an extensible array: +### Tee and decoding (`src/proxy/stream.ts`) -```typescript -interface BeliefPattern { - regex: RegExp; - type: 'causal' | 'assumption' | 'intention' | 'evidence'; - confidence: number; // base confidence - groupIndex: number; // which regex group contains the belief text -} -``` +`teeResponseForExtraction(response, isSse, provider)` calls `response.body.tee()`. The extraction reader decodes with `decoder.decode(value, { stream: true })` on each chunk and a final `decoder.decode()` flush to release any bytes held mid multi-byte sequence. For SSE, an `SseLineParser` splits on the blank-line record delimiter (tolerating `\n\n` and `\r\n\r\n`), joins `data:` lines per the SSE spec, and a `flush()` handles a trailing record with no terminator. Accumulation is best-effort: a read error is swallowed so extraction never breaks the proxy. -Each pattern matches a linguistic construct and extracts the relevant text. The extracted text becomes the `belief` field of an `ExtractedBelief`. +--- -**Confidence modifiers** are applied after pattern matching: -- "definitely", "certainly" → +0.2 -- "probably", "likely" → +0.1 -- "might", "could be" → -0.1 -- "not sure", "uncertain" → -0.3 +## Content normalization (`src/proxy/content.ts`) -Final confidence is clamped to [0.1, 1.0]. +`extractAssistantText({ provider, isSse, accumulated })` returns the assistant text for extraction: -### src/lens/extract.ts - Extraction Pipeline +- SSE: `accumulated` already holds the joined delta text from the tee, so it is returned trimmed. +- Non-SSE OpenAI: `choices[0].message.content`, either a string or an array of text parts, joined. +- Non-SSE Anthropic: every `content[]` block with `type === "text"`, joined. -``` -Input: responseText (string), sessionId (string) - -1. Split response into sentences (split on . ! ? \n) -2. For each sentence, test against all patterns in patterns.ts -3. For each match: - a. Extract belief text from regex group - b. Look for confidence modifiers in surrounding context - c. Apply modifiers to base confidence - d. Create ExtractedBelief object with UUID + timestamp -4. Link beliefs to parent (previous belief in session) for DAG -5. Return ExtractedBelief[] - -Output: ExtractedBelief[] -``` +All extractors are defensive. A malformed body yields `""` rather than throwing. + +--- -### src/state/SessionDurableObject.ts - Belief DAG +## Provider adapters (`src/proxy/providers/`) -Each agent session gets one Durable Object instance. The DO holds beliefs in memory as a `Map`. +`ProviderAdapter` (in `providers/types.ts`) defines the seam: ```typescript -interface BeliefNode { - id: string; - type: 'causal' | 'assumption' | 'intention' | 'evidence'; - belief: string; - evidence?: string; - confidence: number; - actionTaken?: string; - timestamp: number; - parentId: string | null; - childrenIds: string[]; +interface ProviderAdapter { + id: "openai" | "anthropic"; + match(pathname: string, method: string): boolean; + upstreamPath: string; + validateRequest(body: unknown): ValidationResult; + extractAssistantText(rawBody: string): string; } ``` -**DAG construction:** When `addBelief(belief)` is called, if `parentId` is provided, the new node is linked as a child of the parent. This creates a tree structure within the session - each belief is connected to the one that preceded it, enabling root-cause backtracking. +- `openaiAdapter`: matches `POST /v1/chat/completions`, upstream path `/v1/chat/completions`. +- `anthropicAdapter`: matches `POST /v1/messages`, upstream path `/v1/messages`. -**DO fetch handler:** -- `GET /` → returns full belief graph as `{ nodes: BeliefNode[], edges: [{parent, child}] }` -- `POST /` → adds a belief to the graph, body is `ExtractedBelief` -- `GET /type/:type` → filter by belief type -- `GET /root-cause/:failedActionId` → backtrack from a failed action to root-cause belief +Both validate that `messages` is a non-empty array and delegate non-stream text extraction to the shared `content.ts` helpers. --- -## TypeScript Types +## Session state (`src/state/SessionDurableObject.ts`) -### Core Types (src/lens/types.ts) +Phase 1 stores beliefs as an append-only chronological timeline, not a graph. There is no in-memory `Map`, no parent/child edges, and no root-cause route. -```typescript -type BeliefType = 'causal' | 'assumption' | 'intention' | 'evidence'; +The DO handles two internal routes: +- `POST /store-beliefs`: appends one batch (`{ beliefs, rawText, timestamp }`) to the `"beliefs"` storage key and refreshes the `"sessionName"` key with the human session id. +- `GET /beliefs`: reads all batches, flattens them into one ordered `ExtractedBelief[]`, and returns `{ sessionId, beliefs }`. The `sessionId` is the stored human name (or the request hint), never the opaque Durable Object id. + +Flattening and id resolution are pure functions in `src/state/sessionBeliefs.ts` (`flattenBeliefBatches`, `resolveSessionId`), which keeps them unit-testable and tolerant of corrupt reads. + +`GET /api/beliefs/:sessionId` (`src/proxy/beliefs.ts`) resolves the DO by `idFromName(sessionId)`, fetches `/beliefs` with the path id as a hint, and passes the JSON through with permissive CORS headers. + +Because beliefs live in Durable Object storage, they survive DO eviction. The real risk is unbounded growth: nothing trims old batches. + +--- + +## Lens patterns and confidence (`src/lens/`) + +### Patterns (`patterns.ts`) + +`BELIEF_PATTERNS` is an ordered list. Each entry is `{ label, type, pattern, group, evidenceGroup?, actionGroup?, confidence }`. The engine walks patterns in order and the first match wins a span. + +- Causal: `because of X`, `because X` (split so group 1 always holds the text), `since X` (non-temporal), `due to X`, `as a result of X`. +- Assumption: `assuming X`, `presumably X`, `I'll assume X`, `if X then Y` (X as belief, Y as action). +- Intention: `I'll X`, `I'm going to X`, `let me X`, `I should X`, `I plan/intend to X`. +- Evidence: `based on X`, `according to X`, `from the X`, `the error says X`. Evidence patterns set `evidenceGroup: 1`, so the cited text lands in both the `belief` and `evidence` fields. + +Clause ends at `. ; ! ?` a newline, or end of string, so a match at the very end of a response is not dropped for lack of trailing punctuation. + +### Confidence (`extract.ts` + `patterns.ts`) + +Confidence starts at the pattern baseline. `CONFIDENCE_MARKERS` are scanned in a window of `MARKER_SCAN_RADIUS` (80) characters on each side of the match. Each marker category found adds its delta: + +| Marker category | Words | Delta | +|---|---|---| +| certain | definitely, certainly, absolutely, without a doubt, guaranteed | +0.2 | +| likely | probably, likely, most likely, almost certainly, highly likely | +0.1 | +| possible | might, could be, possibly, may, perhaps | -0.2 | +| uncertain | not sure, uncertain, unsure, unclear | -0.3 | + +The sum is added to the baseline and clamped to `[0.1, 1.0]` (`CONFIDENCE_MIN`, `CONFIDENCE_MAX`). This is additive, not midpoint-band interpolation. `DEFAULT_CONFIDENCE` is 0.7. + +### Extraction pipeline + +``` +extractBeliefs(text, { sessionId, uuid?, now? }): Promise + +1. Empty / whitespace text -> []. +2. scanPatterns: run every pattern globally, collect matches with capture, + evidence, action, source index, and line number. +3. Sort by source position, then pattern precedence. +4. dedupeOverlaps: drop a match whose span is wholly inside an earlier one. +5. For each survivor: baseline confidence, adjust by markers in context, clamp. +6. Shape into ExtractedBelief with a UUID and a shared timestamp. +``` + +Every belief is stamped with the passed `sessionId`. There is no parent linking and no DAG construction. + +--- + +## Types (`src/lens/types.ts`) + +`ExtractedBelief` is the record the whole system agrees on: + +```typescript interface ExtractedBelief { id: string; sessionId: string; - type: BeliefType; + type: 'causal' | 'assumption' | 'intention' | 'evidence'; belief: string; evidence?: string; - confidence: number; // 0.0–1.0 + confidence: number; // clamped to [0.1, 1.0] actionTaken?: string; - timestamp: number; // Unix ms + timestamp: number; // Unix ms rawText: string; + line: number; } +``` -interface BeliefNode extends ExtractedBelief { - parentId: string | null; - childrenIds: string[]; -} +`BeliefNode` (adds `parentIds`, `childIds`, `invalidated?`), `BeliefEdge`, and `BeliefDAG` also exist in this file. They are **planned types only** (BUILD-SPEC D2). No code constructs, stores, or serves them. Do not read their presence as a shipped graph. -interface BeliefDAG { - sessionId: string; - nodes: Map; - rootIds: string[]; -} +--- -interface BeliefPattern { - regex: RegExp; - type: BeliefType; - confidence: number; - groupIndex: number; -} -``` +## PolyVerdict enforce path (`src/polyverdict/`) ---- +Enforce mode is opt-in. `detectSchemaTrigger(headers, body)` returns a trigger when either: -## Durable Object: Design Decisions +- `x-axion-schema` header holds a JSON Schema (parsed directly, then with `decodeURIComponent` as a fallback), or +- the body has `response_format: { type: "json_schema", json_schema: { schema, name? } }`. -### Why Durable Objects (not KV, not D1)? +The header takes precedence. When there is no trigger, the request never enters this path. -| Option | Chosen? | Why | -|---|---|---| -| Durable Objects | Yes | In-memory belief graph, single-writer consistency, per-session isolation, sub-ms reads | -| KV | No | Eventually consistent, no complex queries, no in-memory graph | -| D1 | No | SQL is overkill for a graph, adds cold starts, not per-session isolated | -| Workers Cache | No | Not durable across requests, no transactional guarantees | +When triggered, `enforceProviderRequest` in `index.ts` runs a loop up to `MAX_ENFORCE_ATTEMPTS` (3): + +1. Force `stream: false` on the upstream body and strip the client `response_format` (the Worker owns validation). +2. `fetch` upstream and extract assistant text via the provider adapter. +3. `enforceOnce(text, schema)`: strip Markdown fences, `parseJsonFromAssistant`, then `validateAndCoerce`. +4. On success, run Lens on the delivered text in `waitUntil` and return a provider-shaped 200 (OpenAI `chat.completion` or Anthropic `message`) whose assistant content is the coerced JSON string. +5. On failure with attempts left, append the violations as a correction message (`buildRetryMessages` / `buildRetryMessagesAnthropic`) and retry. +6. After 3 failures, run Lens on the last text and return HTTP 422 with `{ error: { message, errors, attempts } }`. + +Enforce always returns non-streaming JSON, even if the client asked to stream, so the full payload can be validated. -### Memory Limits +### Schema subset (`schema.ts`) -Durable Objects have a 128MB memory limit. A single belief node is ~500 bytes. That's ~256K beliefs per session - far beyond what any reasonable agent session produces (typical: 50–500 beliefs). +`validateAndCoerce(data, schema)` implements a minimal JSON Schema subset with zero dependencies: -If sessions grow beyond this, the DO can flush old beliefs to Durable Storage (persistent disk) and lazy-load on access. Not needed for Phase 1. +- Keywords: `type` (single or union), `properties`, `required`, `items` (single schema or tuple), `enum`, and nesting. Unknown keywords are ignored. +- Coercion: `"42"` to number/integer, `"true"`/`"false"` to boolean, finite number/boolean to string. Coercion runs before enum checks so `"42"` can still match an enum of `42`. +- Returns `{ ok: true, value }` (coerced) or `{ ok: false, errors }` (path-keyed messages). + +There is no semantic verification, no second model, and no schema registry. Those are future work (see [SPEC-PolyVerdict.md](./SPEC-PolyVerdict.md)). + +`enforce.ts` also exports `runEnforceLoop`, a transport-agnostic driver that takes an injected upstream callback. The Worker uses its own inline loop; `runEnforceLoop` exists for tests and reuse. --- -## Dashboard Architecture +## Latency + +| Stage | Observe path | Enforce path | +|---|---|---| +| Request forwarding | one upstream round trip | one upstream round trip per attempt | +| Response to caller | streamed as it arrives, no buffering | buffered, returned after validation | +| Belief extraction | `waitUntil`, after delivery | `waitUntil`, after delivery | +| Added latency | effectively none | validation plus up to 2 retries | -The dashboard is a single-page React app served as static assets from the Worker. +The zero-added-latency claim applies to the observe path only. Enforce mode buffers by design. -### Design Constraints +--- -- **No build step** - React is loaded from CDN via `