From ad3bfbc7f17e247298910808a44e764acdb44fd7 Mon Sep 17 00:00:00 2001 From: Yordis Prieto Date: Thu, 6 Aug 2026 00:29:09 -0400 Subject: [PATCH 1/2] chore(docs): challenge the corpus with a product that has no agent record Signed-off-by: Yordis Prieto --- docs/research/README.md | 1 + docs/research/agent-platform/index.md | 1 + .../agent-platform/products/ironclaw.md | 506 ++++++++++ docs/research/agent-platform/synthesis.md | 192 +++- docs/research/session-store/index.md | 13 +- .../session-store/products/ironclaw/index.md | 952 ++++++++++++++++++ docs/research/session-store/synthesis.md | 282 +++++- 7 files changed, 1894 insertions(+), 53 deletions(-) create mode 100644 docs/research/agent-platform/products/ironclaw.md create mode 100644 docs/research/session-store/products/ironclaw/index.md diff --git a/docs/research/README.md b/docs/research/README.md index 5df8904c1..f23926005 100644 --- a/docs/research/README.md +++ b/docs/research/README.md @@ -13,6 +13,7 @@ Repos we lean on most for research. One deduplicated list across all corpora. - [block/buzz](https://github.com/block/buzz) - [xai-org/grok-build](https://github.com/xai-org/grok-build) - [NousResearch/hermes-agent](https://github.com/NousResearch/hermes-agent) +- [nearai/ironclaw](https://github.com/nearai/ironclaw) - [anomalyco/opencode](https://github.com/anomalyco/opencode) - [pingdotgg/t3code](https://github.com/pingdotgg/t3code) - [cloudflare/agents](https://github.com/cloudflare/agents) diff --git a/docs/research/agent-platform/index.md b/docs/research/agent-platform/index.md index 11c6de9f0..206c4474c 100644 --- a/docs/research/agent-platform/index.md +++ b/docs/research/agent-platform/index.md @@ -22,6 +22,7 @@ and evidence rules behind each product dossier remain reproducible. - [CrewAI](./products/crewai.md) - [Devin](./products/devin.md) - [Hermes Agent](./products/hermes-agent.md) +- [IronClaw (NEAR AI)](./products/ironclaw.md) - [Jido](./products/jido.md) - [kagent](./products/kagent.md) - [LangGraph Platform](./products/langgraph-platform.md) diff --git a/docs/research/agent-platform/products/ironclaw.md b/docs/research/agent-platform/products/ironclaw.md new file mode 100644 index 000000000..7aef2fe40 --- /dev/null +++ b/docs/research/agent-platform/products/ironclaw.md @@ -0,0 +1,506 @@ +# IronClaw (NEAR AI): what "agent" means + +Part of Agent Definition Research. +Produced by running [RESEARCH_PROMPT](../RESEARCH_PROMPT.md). +Evidence from the `nearai/ironclaw` source: the in-repo architecture and +contract documents plus the implementing Rust workspace. There is no published +product documentation site, so every source below is in-repository. IronClaw is +a Rust monorepo (dual MIT OR Apache-2.0) whose current architecture is called +"Reborn"; the pre-Reborn v1 monolith has been deleted from the tree. + +## Source anchors + +Sources retrieved 2026-08-05, pinned to commit +[`2ae6621`](https://github.com/nearai/ironclaw/commit/2ae66212fe80208524179047878916dafc0538ee) +(committed 2026-08-05T18:20:35Z). Citations use repo-relative `path:line`. + +- `crates/Architecture.md` (1019 lines), the crates-level architecture map. +- `docs/reborn/contracts/`: `kernel-boundary.md`, `capabilities.md`, + `capability-access.md`, `runtime-profiles.md`, `turns-agent-loop.md`, + `turn-persistence.md`, `memory.md`, `skills-extension.md`, `triggers.md`, + `storage-placement.md`, `host-api.md`. +- `docs/reborn/subagent-spawn/{README.md,phase-2-mechanisms.md,phase-3-integration.md}`. +- `crates/contracts/ironclaw_host_api/src/ids.rs`, + `crates/contracts/ironclaw_loop_contracts/src/snapshot.rs`, + `crates/domains/ironclaw_threads/src/contract.rs`. +- `profiles/{local,local-sandbox,server,server-multitenant}.toml`, + `registry/`, `skills/`. + +Precedence note the repo states about itself: contract docs under +`docs/reborn/contracts/` and crate-local `AGENTS.md` are authoritative for +behavior-changing work (`crates/Architecture.md:8-9`), and where a frozen +contract has drifted from the code, "**THE CODE WINS**" +(`docs/reborn/target-architecture/CHECKLIST.md:355`). Places where the two +disagree are flagged inline. + +## The `agent` noun (primary-source quotes) + +- **There is no agent object.** This is the most consequential finding, and it + is a negative one. `AgentId` is declared as a validated string id alongside + tenant and user (`crates/contracts/ironclaw_host_api/src/ids.rs:208-210`): + + ```rust + string_id!(TenantId, "tenant", validate_scope_id); + string_id!(UserId, "user", validate_scope_id); + string_id!(AgentId, "agent", validate_scope_id); + ``` + + A repo-wide search for an agent record, an agent registry, or an agent + definition type (`AgentRecord`, `AgentDefinition`, `agent_registry`) returns + nothing across `crates/` and `docs/`. There is no create-agent API, no agent + CRUD, no agent table in `migrations/V1..V34`. The agent is a **scope + coordinate**, and `validate_scope_id` bounds it at 256 bytes with no path + separators, control characters, or reserved `__ironclaw_` prefix because the + id becomes a storage path segment. + +- **The scope coordinate is load-bearing everywhere.** `AgentId` is a + first-class axis of the resource scope model + (`docs/reborn/contracts/storage-placement.md`), of the transcript scope + (`ThreadScope { tenant_id, agent_id, project_id?, owner_user_id?, mission_id? }`, + `crates/domains/ironclaw_threads/src/contract.rs`), of the durable concurrency + key ("Active-lock key is the canonical `TurnScope`: tenant, agent, optional + project, and thread", `turn-persistence.md:49`), and of trigger definitions + ("`agent_id` | Captured agent scope at create time", `triggers.md:3`). What an + "agent" owns is therefore precise: a partition of threads, memory, triggers, + locks, and quota buckets. + +- **What behaves like an agent is assembled per run, from four separable + parts.** The architecture thesis names them (`crates/Architecture.md:14-20`): + + ```text + Products own UX. + Loops own agent behavior. + The kernel boundary owns authority, recovery, and side-effect mediation. + Substrates own durable, reusable primitives. + ``` + + The layers "are not peers. Products and loops are replaceable userland code. + The kernel boundary is the narrow authority surface they must use for side + effects" (`Architecture.md:101-104`). + +- **The loop is explicitly not the agent's security identity** + (`Architecture.md:78-80`): + + > The loop is intentionally not the security perimeter. It asks for effects + > through host ports, and those host ports eventually route privileged effects + > through the host runtime and `CapabilityHost` boundary. + + The kernel contract inverts the usual framing: "The Reborn kernel is the + security perimeter. It is defined by what it mediates and secures, not by how + much product behavior it performs" (`kernel-boundary.md:9-11`). Loop strategy, + prompt assembly, "routine engines and mission orchestration", skill selection, + and provider heuristics are all listed as **userland non-responsibilities** of + the kernel (`kernel-boundary.md:59-69`). + +- **Identity is file-shaped data, resolved through a kernel-mediated memory + substrate.** A reference prompt assembler reads + (`docs/reborn/contracts/memory.md:250-263`): `BOOTSTRAP.md`, `AGENTS.md`, + `SOUL.md`, `USER.md`, `IDENTITY.md`, `SYSTEM.md`, `MEMORY.md`, `TOOLS.md`, + `HEARTBEAT.md`, `context/profile.json`, `context/assistant-directives.md`. + The kernel does not own their meaning but does own their safety: "identity + files are primary-scope only"; the stable set (`AGENTS.md`, `SOUL.md`, + `IDENTITY.md`, `TOOLS.md`, `BOOTSTRAP.md`) "may be considered for default + prompt assembly"; the personal set (`USER.md`, + `context/assistant-directives.md`) is "excluded unless the resolved run + profile explicitly allows personal context"; admin `SYSTEM.md` is admin-scope + only; and "writes to prompt-injected files are scanned by the safety + sanitizer" because "these files can affect future execution context" + (`memory.md:245-278`). Prompt assembly itself is declared replaceable: + "reference loop prompt assemblers are replaceable behavior, not memory backend + source of truth." + +- **Persistent versus ephemeral.** Both, split cleanly. Persistent: the scope + coordinate and everything filed under it (threads, memory documents, triggers, + quota buckets). Ephemeral: the executing thing, which is a `TurnRun` claimed + by a runner under a lease, executing one loop driver against a checkpointable + `LoopExecutionState`. + +- **Conceptual model.** Agent-as-scope, with agent-as-config assembled onto it + per run. Not agent-as-identity (no record to point at), not agent-as-process + (the process is the run), not agent-as-session (one agent has many threads). + The closest single label: **agent-as-scope-plus-resolved-profile**. + +## Subagents + +- **Name and shape.** Subagents, spawned through a capability rather than an + API: `spawn_subagent(flavor_id, task, handoff?)` + (`docs/reborn/subagent-spawn/README.md`). The public v1 schema is + blocking-only. The design's central claim is that a subagent is not a new kind + of thing (`README.md:§5.4`): "a child agent loop is **not** an OS process". It + is another run on the same runner and driver plane, which the architecture + lists as a non-goal to violate: "separate subagent execution machinery outside + the normal runner/driver loop" (`Architecture.md:38-40`). + +- **Creation is dynamic, from a static catalog.** The parent spawns at runtime, + but only from a compile-time flavor table (`General`, `Explorer`, `Coder`, + `Planner`) whose direction prompts are embedded with `include_str!`. Callers + choose a `flavor_id`; they cannot author a child's prompt or tool set inline. + +- **What the child inherits, and what it does not.** It inherits the scope: + `tenant_id`, `agent_id`, and `project_id` verbatim, plus `owner_user_id`, and + only `thread_id` is fresh. It inherits **no authority**: the child begins with + an *empty* grant and lease set, and the flavor's capability allowlist is + described as "a surface *ceiling*, not authority". Context is not inherited by + default: the seed is `Fresh` (goal only) or `Handoff(String)` (a curated + parent blob re-materialized into the child scope). `Fork` (full parent-context + copy) is "enum variant reserved, unimplemented" (`README.md:74`) and requesting + it is denied at runtime (`phase-3-integration.md:677`). + +- **The goal lands as a user message, deliberately.** "Never the system message + — the goal is model-generated and may carry upstream-tainted content" + (`README.md`). A parent's instruction to a child is treated as untrusted + input, not as configuration. + +- **Nesting is bounded four ways, all before submission.** `allow_nesting = + false` by default, a depth cap, a per-turn fan-out cap, and an atomic + `reserve_tree_descendants(scope, root, delta, cap)` against + `MAX_TREE_DESCENDANTS` backed by a durable `SpawnTreeReservation`. The threat + model is named: "**Fork-bomb via depth × fan-out.** Three caps — depth, + per-turn fan-out, per-tree descendants — all enforced before `submit_turn`, + all rejecting without queuing" (`phase-2-mechanisms.md:1884`). + +- **Communication is return-value only, mediated by the host.** The capability + returns `CapabilityOutcome::AwaitDependentRun`, which awaits the whole child + set and resolves inline when all children are already terminal. Children get + their own thread, so there is no shared transcript and no message passing + between parent and child. Approvals raised by a child surface on the child + thread, which is why inheriting `owner_user_id` matters. + +- **Lifetime coupling is recorded, not silent.** When a parent cancel discards a + child's result, a durable `SubagentResultTombstone { child_run_id, disposition: + "discarded_by_parent_cancel", terminal_status }` is written. The parent-child + link lives on the run (`parent_run_id`, `subagent_depth`, + `spawn_tree_root_run_id`), not on the thread. + +- **Status caveat: off in every shipped profile at this commit.** + `builtin.spawn_subagent` is deny-filtered via a `TEMP(disable-spawn-subagents)` + marker in `crates/loop/ironclaw_turn_runner/src/runtime.rs`, per the 2026-07 + status note in `crates/Architecture.md`. The design is detailed and + contract-frozen; the feature is not on. + +## Configuration surface (what, where, why) + +Configuration is stratified by *who is allowed to change it and what it can +affect*, and each stratum has a stated reason. + +**1. Deployment envelope, in TOML profiles at the repo root.** +`profiles/{local,local-sandbox,server,server-multitenant}.toml` hold coarse +deployment settings: `database_backend`, `[channels] gateway_enabled`/`cli_mode`, +`[sandbox] enabled`, and the proactive-behavior toggles `[heartbeat]`, +`[routines]`, `[hygiene]`. Overrides come from `~/.ironclaw/config.toml` or +environment variables, selected by `IRONCLAW_PROFILE` +(`profiles/local.toml:1-10`). The rationale is deployment-shape independence +without forking the architecture (`runtime-profiles.md:9-19`): + +> ```text +> same agent loop +> same CapabilityHost +> same RuntimeDispatcher +> same events/audit/resource model +> different filesystem/process/network/approval backends +> ``` + +`DeploymentMode` (`LocalSingleUser`, `HostedMultiTenant`, `EnterpriseDedicated`) +crossed with `RuntimeProfile` (twelve presets from `SecureDefault` through +`LocalYolo` and `HostedYoloTenantScoped`) resolves to an +`EffectiveRuntimePolicy` naming the filesystem backend, process backend, network +mode, secret mode, approval policy, and audit mode +(`runtime-profiles.md:36-140`). The invariant is stated as the reason the knob is +safe to expose (`runtime-profiles.md:27-33`): + +> ```text +> DeploymentMode constrains the maximum authority available. +> Profile changes backend permissiveness within that deployment. +> Profile does not bypass CapabilityHost. +> ``` + +**2. Per-run behavior, in a resolved run profile.** This is the closest thing +IronClaw has to "an agent definition", and it is a value captured on a run +rather than a stored template +(`crates/contracts/ironclaw_loop_contracts/src/snapshot.rs:19-41`): + +```rust +pub struct ResolvedRunProfile { + pub run_class_id: RunClassId, + pub profile_id: RunProfileId, + pub profile_version: RunProfileVersion, + pub loop_driver: AgentLoopDriverDescriptor, + pub checkpoint_schema_id: CheckpointSchemaId, + pub checkpoint_schema_version: RunProfileVersion, + pub model_profile_id: ModelProfileId, + pub capability_surface_profile_id: CapabilitySurfaceProfileId, + pub context_profile_id: ContextProfileId, + pub steering_policy: SteeringPolicy, + pub cancellation_policy: CancellationPolicy, + pub checkpoint_policy: CheckpointPolicy, + pub resource_budget_policy: ResourceBudgetPolicy, + #[serde(default)] + pub personal_context_policy: PersonalContextPolicy, // Excluded by default + pub runtime_constraints: RuntimeProfileConstraints, + pub runner_pool_id: Option, + pub scheduling_class: SchedulingClass, + pub concurrency_class: ConcurrencyClass, + pub resolution_fingerprint: RunProfileFingerprint, + pub provenance: RedactedRunProfileProvenance, +} +``` + +The stated reason each of these is a *selection* rather than a grant +(`Architecture.md:459-474`): + +> Profiles do not grant authority by themselves. They choose bounded surfaces +> and policies that host/kernel services enforce later: +> +> ```text +> profile selects visible capability surface +> but CapabilityHost still authorizes exact invocation +> profile selects model/context policies +> but host ports still enforce safety, scope, and redaction +> profile selects checkpoint policy +> but runner still validates durable checkpoint/result evidence +> profile selects runtime constraints +> but deployment mode and host runtime policy may only reduce authority +> ``` + +Note `PersonalContextPolicy::Excluded` as the serde default: the privacy-safe +value is what an old or partial record deserializes to. + +**3. Instructions and knowledge, as portable file bundles.** Skills are +`SKILL.md` bundles owned by a first-party in-process extension, and the contract +opens by refusing them authority (`skills-extension.md:11-27`): + +> This contract intentionally keeps skills out of the kernel. [...] The +> first-party skills extension is userland code, even when it ships with +> IronClaw and runs in process. +> +> ```text +> skills can provide instructions and supporting files; +> skills cannot grant authority. +> ``` + +The tree ships roughly thirty of them under `skills/` (`coding`, `code-review`, +`commit`, `delegation`, `llm-council`, `plan-mode`, `product-prioritization`, +various `*-setup` bundles), which is where most of the product's apparent +"personality" per task lives. + +**4. Tool and channel inventory, as a static registry.** `registry/` holds +`tools/`, `mcp-servers/`, `channels/`, and `_bundles.json`. Registration is +explicitly not authorization (`capability-access.md:28`): "A registered +capability is only a possibility. It is not authority." + +**5. Identity and memory, as markdown under the memory substrate** (the file +list quoted above), with layer scopes (`private`, `shared/team`, custom named +layers) declaring readable/writable flags, writes to read-only layers failing +closed, and optional privacy classifiers able to redirect sensitive shared-layer +writes to private layers with the redirect visible in the write result +(`memory.md:225-240`). + +**6. Proactive work, as trigger records.** `TriggerRecord` carries +`trigger_id`, `tenant_id`, `creator_user_id`, `agent_id`, `project_id`, `name`, +`source`, `schedule`, and a materialized `prompt`, managed through +`trigger_create` / `trigger_list` / `trigger_remove` capabilities +(`triggers.md:19-45`). The design constraint is that a trigger does not get its +own execution path (`triggers.md:13`): "It does **not** own a parallel agent +loop [...] A trigger fire is routed into the normal Reborn turn pipeline and then +persists through the same turn, run, and recovery machinery as any other inbound +submission." + +**Cross-cutting rationale, stated once and repeated per surface:** a knob may +narrow authority and may never widen it. `TrustClass` "is an authority ceiling, +not a permission grant and not a bypass"; "user-installed packages cannot +self-declare `TrustClass::FirstParty` or `TrustClass::System`"; those ceilings +"do not grant authority by themselves" +(`kernel-boundary.md:94-105`, `host-api.md:293-295`). There is also no privileged +exemption for the vendor's own code: "There is no private back door for shipped +loops or first-party code" (`kernel-boundary.md:47`). + +## Binding time + +- **Definition time (compile/deploy).** Loop families and drivers are registered + in code (`DriverRegistry`), subagent flavors are a compile-time table with + `include_str!` prompts, and the skills/tools/channels registries are on-disk + inventories loaded at composition. Adding a loop family is a code change with + a listed blast radius (`Architecture.md:939`). +- **Turn submission time.** The run profile is resolved once, during + `submit_turn`, as step 3 of the turn flow: the coordinator "persists turn/run + state, enforces active-thread ownership, resolves the run profile, and emits a + wake hint" (`Architecture.md:487-493`). Identity/system context, personal-context + policy, and the model route are resolved in the same window + (`Architecture.md:748`), and the executor "persists a model-route snapshot + before invoking the driver" (`Architecture.md:496-497`). +- **Captured, not re-resolved.** This is the binding-time rule that matters + most (`Architecture.md:440-442`): + + > A resolved run profile is captured on the run so execution can be recovered + > without re-resolving a different driver or policy after restart. + + So a definition change cannot affect in-flight work: a resumed run replays + against the profile snapshot, the same `loop_driver` id/version, and the same + `checkpoint_schema_id`/version. Config edits apply to the next submission. + Triggers behave the same way at a longer horizon, capturing `agent_id` and + `project_id` "at create time" (`triggers.md:3`). +- **Mid-run mutability is narrow and typed.** Not the profile: what can change + mid-run is bounded to steering (a queued user message under + `SteeringPolicy`), cancellation intent, and approval or auth resolution, each + of which moves the run through the documented state machine rather than + editing its configuration. An approval lease is scoped to an exact invocation + fingerprint and is "resume-only authority" that must be claimed through + `CapabilityHost::resume_json` (`capability-access.md:88`). +- **Versioning.** Run profiles carry `profile_version`, a + `resolution_fingerprint`, and a redacted `provenance` record, and checkpoint + schemas carry their own id and version. That is versioning of the *resolution* + rather than of a stored agent definition; there is no agent definition object + to version. `ResolvedRunProfile::legacy_compatibility` exists so turn rows + persisted before profiles existed still project into the current type + (`snapshot.rs:60-64`). + +## Relationships between nouns + +Cardinalities, as the code and contracts express them: + +| Relationship | Cardinality and ownership | +| --- | --- | +| tenant → agent | 1:N. `AgentId` is a scope axis under `TenantId`, not an object. | +| (tenant, agent, project?, owner?, mission?) → thread | 1:N. `ThreadScope` is the path prefix; threads live under it. | +| thread → turn | 1:N, but at most one *active*: "One active run per canonical thread is enforced before model/tool side effects" (`Architecture.md`, Key Invariants). | +| turn → turn run | 1:N over retries/resumes; `TurnId` is the accepted message, `TurnRunId` the execution attempt. | +| turn run → runner | 1:1 at a time, by lease. Claim stores runner id plus lease token; heartbeats renew only for a matching, unexpired pair (`turn-persistence.md:88-89`). | +| turn run → loop driver | 1:1, fixed by the captured profile. | +| loop driver → capability invocation | 1:N, every one mediated by `CapabilityHost`. | +| capability → runtime lane | 1:1 per invocation (WASM, script-process, MCP, first-party; system deferred), chosen by `RuntimeDispatcher` *below* authorization. | +| parent run → child run | 1:N, linked by `parent_run_id` / `spawn_tree_root_run_id`, bounded by depth, fan-out, and descendant reservation. | +| child run → thread | 1:1 fresh thread; siblings of the parent's thread, not nested under it. | +| trigger → turn | 1:N synthetic inbound submissions through the normal pipeline. | +| skill → authority | 0. Skills inject instructions and files only. | + +Three answers worth stating directly, because the corpus varies most on these: + +- **Agent to session: one agent, many threads, and the agent is the scope, not a + participant.** A thread cannot exist outside an agent scope, and it cannot + move between agents: scope is baked into the storage path, and `ensure_thread` + rejects a scope/thread mismatch with `ThreadScopeMismatch`. +- **Agent to sandbox: an agent does not imply an environment.** Containment is + selected per capability invocation via `SandboxBackend` (`None`, `Srt`, + `SmolVm`, `Docker`) under the deployment mode's ceiling + (`runtime-profiles.md:36-48`), and `[sandbox] enabled = false` in + `profiles/local.toml` shows the same agent running unsandboxed locally. There + is no long-lived per-agent VM or workspace container in the model. +- **Agent to subagent: ownership without shared state, and death is recorded.** + The parent owns the spawn tree reservation and the cancel decision; the child + owns its own thread and starts with zero grants. A cancelled parent does not + silently orphan a finished child; it writes a tombstone naming the + disposition. + +## Lifecycle + +- **The agent is never created or destroyed**, because there is nothing to + create. Using a new `AgentId` brings a scope into existence implicitly the + first time something is filed under it; `MemorySeedService` "owns initial and + upgrade seeding" of `README.md`, `MEMORY.md`, `IDENTITY.md`, `SOUL.md`, + `AGENTS.md`, `USER.md`, `HEARTBEAT.md` (`memory.md:288-300`), which is the + nearest thing to provisioning. There is no delete-agent operation; deletion + exists per thread (`delete_thread`) and per memory document. +- **The run is what has a lifecycle**, and it is a durable state machine + (`Architecture.md:540-582`): `submit_turn` creates queued work, "but no + model/tool side effect runs before the process claim succeeds"; a runner claims + and heartbeats; a capability needing approval or auth writes gate and + checkpoint refs and the driver returns `LoopExit::Blocked`, keeping the + active-thread lock; `resume_turn` requeues the same run against the same + checkpoint; validated exits move to `Completed`/`Failed`/`Cancelled`. +- **Pause and resume are checkpoint-based and evidence-gated.** `LoopExit` is + "a driver claim, not trusted durable state", and `LoopExitApplier` "validates + host-owned evidence before mapping the exit to a trusted transition" + (`Architecture.md:580-582`). An unverifiable exit becomes a sanitized terminal + failure (`driver_protocol_violation` / `interrupted_unexpectedly`), because "A + syntactically valid ref is not evidence by itself" (`Architecture.md:644-646`). +- **Crash recovery prefers giving up over guessing** (`Architecture.md:620-632`): + + > ```text + > runner crashes or stops heartbeating + > -> reconciler sees expired Running/CancelRequested lease + > -> Running => terminal Failed (sanitized "lease_expired") + > -> CancelRequested => terminal Cancelled + > ``` + > + > Reborn does not automatically retry uncertain side-effecting work after a + > lost lease — expiry is terminal, and the user resubmits explicitly. + + (`turn-persistence.md:91` still describes expiry as moving to + `RecoveryRequired` and keeping the lock; `Architecture.md:578-579` says that + variant "survives only as a legacy variant". The code wins.) +- **Who owns the loop: the product does, in the sense that matters here.** + IronClaw runs the loop itself (managed), but the loop is a replaceable userland + plug-in rather than a fixed brain: "Loop diversity is an expected feature" + (`kernel-boundary.md:71-76`), with lightweight, CodeAct, model-specific, and + subagent families named. What is *not* bring-your-own is the authority path; + a custom loop cannot reach the dispatcher directly, and "direct dispatcher + calls from loops or product entry points" is a listed non-goal + (`Architecture.md:33-40`). +- **What persists across runs.** Memory and identity documents; the thread + transcript, summary artifacts, and out-of-band tool-result records; triggers + and their schedule state; audit and lifecycle events with replay cursors; + admission reservations while a run is non-terminal. What does not persist: + grants and leases (per-invocation), secret material (leased once and consumed + before use), and loop execution state beyond a bounded checkpoint payload. +- **Proactive existence is a deployment toggle, not an agent property.** + `[heartbeat]`, `[routines]`, and `[hygiene]` in `profiles/local.toml` turn on + background self-directed work, and `HEARTBEAT.md` is explicitly "volatile + routine/proactive context, not stable default-loop identity context" + (`memory.md:270-272`). Whether this agent wakes up on its own is decided by + the deployment profile, not by anything filed under its id. + +## What makes it "an agent" here (our inference) + +Our inference: in IronClaw, an agent is **a durable scope coordinate plus a +per-run resolved profile**: an `AgentId` that partitions threads, memory, +triggers, locks, and quotas, onto which each turn binds a loop driver, model +profile, capability surface, and context policy that the kernel then enforces +independently of whatever the loop believes. What makes it an agent rather than +an LLM call is not autonomy or tool use; it is that every effect crosses a +mediated authority boundary that survives the loop being replaced, and that +enough state is journaled for an interrupted run to be resumed or failed +deliberately rather than retried blindly. + +Two design commitments follow from that and are worth carrying into our own +work. First, **visibility is not authority, at every layer**: a registered +capability "is only a possibility", a profile "selects visible capability +surface" while `CapabilityHost` "still authorizes exact invocation", a +`TrustClass` is "an authority ceiling, not a permission grant", a subagent +flavor's allowlist is "a surface *ceiling*, not authority", and skills "cannot +grant authority". Layering four independent narrowing surfaces means no single +misconfiguration escalates. Second, **the thing you can replace is not the thing +you must trust**: products and loops are userland and swappable precisely +because the kernel boundary, not the loop, holds the perimeter. An agent +definition in this model is deliberately thin, because a thick one would be +authority in disguise. + +## Open questions + +- **Where does an agent's identity actually get bound to an `AgentId`?** The + memory substrate holds identity files and the scope holds a string, but we + found no artifact that says "agent `X` is this persona with these defaults". + Is multi-agent-per-tenant a supported product shape today, or is `AgentId` + currently a single well-known constant per deployment? (`reborn_cli()` defaults + to `tenant_id: "reborn-cli"`, `agent_id: "reborn-cli-agent"`, + `crates/app/ironclaw_composition/src/runtime_input.rs:49-71`, which suggests + the latter for local use.) +- **How is a run profile requested in practice?** `RunProfileRequest` → + `RunProfileResolver` is documented as a pipeline, but the mapping from product + intent (a CLI invocation, a Slack message, a trigger fire) to a + `RunProfileId`, and whether end users can select one, is not stated in the + contracts we read. +- **Is there any agent-level lifecycle operation at all** (archive, disable, + delete everything under an `AgentId`)? Deletion exists per thread and per + document; a scope-wide teardown path is not documented. +- **When does `spawn_subagent` ship?** The design is contract-frozen across + three phase documents while the capability is deny-filtered off in every + profile. What is the remaining blocker, and does the empty-grant-set model + survive contact with real delegation? +- **Mission as a noun.** `mission_id` is a `ThreadScope` axis and "mission + orchestration" is listed as a userland responsibility, but no contract we read + defines a mission's lifecycle or its relationship to threads and triggers. +- **Trust-class assignment mechanics.** Ceilings come from "host policy, + signed/bundled package metadata, or admin configuration" + (`host-api.md:295`). How an operator actually assigns one, and whether + signature verification is implemented at this commit, is unclear from the + contracts. diff --git a/docs/research/agent-platform/synthesis.md b/docs/research/agent-platform/synthesis.md index 1d109bf63..dfb0402be 100644 --- a/docs/research/agent-platform/synthesis.md +++ b/docs/research/agent-platform/synthesis.md @@ -1,13 +1,21 @@ # Synthesis: what the industry means by "agent" Part of Agent Definition Research. -Seventeen product dossiers, one question: what does the noun "agent" +Every product dossier, one question: what does the noun "agent" operationally refer to? Purpose: extract the invariant core our own agent service must model, and the axes where products deliberately diverge. This synthesis is frozen as decision-time input: where a conclusion here differs from an accepted record in the [ADR index](../../adr/index.md), the ADR is authoritative. +> [IronClaw](./products/ironclaw.md) was researched and added after this +> synthesis was first frozen. Its evidence revised Convergence #2 (the trio), +> #3 (the definition's content), #4 (pinning), and #7 (tool restriction), +> Divergences A, B, C, and D, and Design decisions 1 and 4; those revisions +> are marked inline. It is the first product in the corpus with *no* agent +> object of any kind, so it is the sharpest available test of the working +> definition below, and the definition did not survive intact. + ## Convergence **1. The behavioral definition is settled.** Every product that states one @@ -41,6 +49,20 @@ session / knowledge. Even the personal daemons follow it: (files), sessions (transcripts), and memory (markdown) as separate artifacts. **This trio is the invariant core.** +*Revised after IronClaw.* The trio survives, but IronClaw shows the first +member can be absent as a *stored resource* and still be present as a +concept. There is no agent record anywhere: `AgentId` is a validated scope +string and nothing else, verified by the absence of any `AgentRecord`, +`AgentDefinition`, or agent registry in the repository. Identity lives in +markdown (`BOOTSTRAP.md`, `AGENTS.md`, `SOUL.md`, `IDENTITY.md`, `SYSTEM.md`, +`MEMORY.md`, `TOOLS.md`, `HEARTBEAT.md`, `context/*`) read per turn, execution +splits into two persisted boundaries (transcript threads and turn runs), and +memory is a file. So the correct statement of the convergence is that the +three *roles* are always separated, not that all three are always resources. +IronClaw makes the definition role a *coordinate plus a resolution*: the +`ThreadScope` tuple names who, and a `ResolvedRunProfile` captured on the run +records what was in effect. + **3. The definition's content converges.** Wherever the agent is declarable, the same fields recur: instructions/prompt + model + tools + limits, with skills, credentials, and delegation roster as the common @@ -49,6 +71,18 @@ Agents ("model, system prompt, tools, MCP servers, skills"), Claude Code frontmatter, OpenAI's constructor, CrewAI's triad + llm + tools, ADK's LlmAgent, and eve's `agent/` directory: same shape, different serialization. +*Revised after IronClaw.* The field set converges even where the *agent* is +not declarable, which is stronger evidence for the convergence than another +agreeing agent record would be. IronClaw has no agent config, yet its +`ResolvedRunProfile` carries exactly this content one level down: a +`model_profile_id`, a `capability_surface_profile_id`, a `context_profile_id`, +a `loop_driver`, plus steering, cancellation, checkpoint, resource-budget, and +personal-context policies, a `runtime_constraints` block, and scheduling and +concurrency classes. The recurring fields are real; what varies is which noun +owns them. IronClaw attaches them to the *run*, so two runs under the same +`AgentId` can legitimately differ in model, capability surface, and loop +driver with nothing to reconcile. + **4. Sessions pin; definitions version.** Every managed platform freezes the definition into the execution at creation time and versions the definition linearly: OpenComputer ("freezes the agent's active revision... @@ -63,6 +97,19 @@ flows into live sessions (OpenComputer), and Code](./products/claude-code-agent-sdk.md) binds per *invocation* (live file reload) rather than per session, so the file-based products trade pinning for git. +*Revised after IronClaw*, which supplies the purest form of the pin and the +reason it matters. There is no definition to version, so IronClaw pins the +*resolution* instead: a `ResolvedRunProfile` is computed once at run admission +and captured on the run, carrying a `profile_id` and `profile_version`, a +`resolution_fingerprint`, and a `provenance` record of what was consulted. The +stated motivation is recovery correctness, not auditability: a resumed or +recovered run must replay under the same `loop_driver` and +`checkpoint_schema_id` it started under, so re-resolving at resume time is a +correctness bug rather than a convenience. That is the argument for pinning +that every managed platform implies and none states this plainly. It also +generalizes the convergence: what gets frozen into an execution is not +necessarily a version number, it is whatever makes the execution replayable. + **5. Delegation has one output contract.** However subagents are spawned, the parent receives only the child's final result, never its intermediate reasoning. Claude Code ("final message verbatim as the tool result"), @@ -73,6 +120,10 @@ principle ("without needing to share their internal thoughts, plans, or tool implementations"). Fresh child context is likewise near-universal, and CrewAI's delegation prompt states the reason plainly: "they know nothing about the task, so share absolutely everything you know." +IronClaw agrees and adds the durability half nobody else has: a child that +finishes after the parent has stopped caring is not dropped silently, it is +recorded as a `SubagentResultTombstone` naming the disposition, so "the parent +only sees the result" does not become "the result vanishes." **6. The `description` field is the routing protocol.** LLM-driven delegation is steered by a natural-language description everywhere it @@ -80,7 +131,11 @@ exists: Claude Code ("Claude uses each subagent's description to decide when to delegate"), ADK ("primarily used by *other* LLM agents to determine if they should route a task to this agent"), CrewAI's coworker tool text, A2A's AgentSkill descriptions. An agent's description is not documentation; -it is its API. +it is its API. The one product that breaks the pattern breaks it by not +having the noun: IronClaw has no agent registry to describe, so nothing +routes by description. Its triggers route by binding into the ordinary turn +pipeline, which is a reminder that description-based routing is a consequence +of LLM-selected delegation targets, not a universal requirement. **7. Tool restriction is the safety primitive.** Constraining which tools an agent (especially a child) may touch is the first control every product @@ -89,6 +144,21 @@ orchestrator roles, Managed Agents permission policies (MCP defaults `always_ask`), and OpenClaw's inherited allow/deny lists. Sandboxes come second; tool scoping comes first. +*Revised after IronClaw*, which pushes this from a primitive to an +architecture and states the invariant the other products leave implicit: +registration is not authority. Capability access says a registered capability +"is only a possibility. It is not authority." Skills "can provide instructions +and supporting files" but "cannot grant authority." Runtime profiles "change +backend permissiveness" but do "not bypass CapabilityHost," while +`DeploymentMode` sets the ceiling. `TrustClass` caps authority independently. +And mediation is not bypassable by trusted code: "There is no private back +door for shipped loops or first-party code," because "the loop is +intentionally not the security perimeter." Four independent narrowing surfaces +that only ever intersect, none of which can widen another, is the design the +allow/deny-list products approximate with one list. The transferable idea is +that "which tools can this agent touch" should be the *intersection* of +separately-owned ceilings rather than a single field on a definition. + ## Divergence The axes where products make opposite choices (i.e., the decisions our @@ -109,6 +179,21 @@ went the other way historically: it *had* the versioned server-side agent resource (Assistants), deprecated it, and decomposed it into config (Prompts) + state (Conversations) + loop (SDK). +*Revised after IronClaw*, which adds a new low end to the spectrum: **nothing +at all**. `AgentId` is declared alongside `TenantId` and `UserId` by the same +`string_id!` macro with the same scope-id validator, and that is the whole of +it. The agent is a coordinate in a scope tuple that gets projected into a +storage path and used as an authorization axis. Every property one would +expect on an agent record lives somewhere else: persona in markdown files read +per turn, runtime shape in the `ResolvedRunProfile` on the run, authority in +the deployment mode and trust class. Placing this next to Cloudflare's fusion +is the useful contrast: Cloudflare collapses definition, state, and process +into *one* object, IronClaw dissolves the definition into *none*, and both +work, which tells us the definition record is a modeling convenience rather +than a necessity. Its cost is also visible: with no record there is no place +to enumerate agents, no natural home for a description, and no per-agent +default anything, which is a real product gap and not just a purity choice. + **B. Who owns the loop.** Three positions: platform-managed loop (OpenComputer's runtimes, Managed Agents, Devin, and the AgentCore harness where "Who owns the loop: AWS"), customer loop behind an infrastructure @@ -123,6 +208,19 @@ point is the **turn contract**: OpenComputer's `POST /turn`, AgentCore's industry has effectively standardized the *shell* (identity, sessions, durability, isolation) without standardizing the *brain*. +*Revised after IronClaw*, which names the shell and draws it as a hard +boundary rather than an API surface. Its four layers are products (UX), +userland loops (agent behavior), a kernel boundary (authority, recovery, +side-effect mediation), and substrates (durable primitives), with loops +explicitly *demoted*: "the loop is intentionally not the security perimeter," +and no private back door for first-party loops. That is a fourth position on +this axis, distinct from all three above: the loop is customer-replaceable +*and* untrusted, selected by a `loop_driver` on the run profile rather than +supplied over a network contract. It vindicates the shell/brain split by +making it an enforcement boundary instead of an integration point, and it is +the strongest available argument that the shell must mediate side effects +rather than merely host the loop. + **C. Binding time.** Freeze-at-session (all managed platforms), live-reload per invocation (Claude Code), everything-at-runtime (Cloudflare), per-*step* rebinding as a designed feature (Vercel's @@ -130,6 +228,16 @@ live-reload per invocation (Claude Code), everything-at-runtime string interpolation (CrewAI). Hermes adds a constraint nobody else surfaces: prompt-cache economics as the reason mid-run mutation must be rare ("per-conversation prompt caching is sacred"). +*Revised after IronClaw*, which occupies both ends at once and is coherent +about why. Persona is late-bound in the extreme, since the markdown identity +files are read per turn with no version pinning, exactly Claude Code's trade +of pinning for git. But everything that affects *replay* is bound once at +admission and frozen on the run: loop driver, checkpoint schema id and +version, model profile, capability surface, budgets. The line IronClaw draws +is the useful one to steal, and it is neither per-session nor per-invocation: +bind text late, bind mechanism early. Anything a recovery path must agree with +its original run about cannot be re-resolved; anything the model merely reads +can be. **D. Subagents, the least settled axis.** Declared roster with depth-1 cap (Managed Agents: 20 agents/25 threads; Hermes default; OpenClaw default, @@ -146,19 +254,49 @@ work when they "contribute intelligence rather than actions": **fan out reads, single-thread writes**, and fresh-context verifiers *beat* shared-context ones for review. +*Revised after IronClaw*, which is the most conservative position in the +corpus and the only one that treats subagents as an authority problem before a +topology problem. Children are ordinary child runs with lineage on the run +record, they start with **empty grant sets** rather than inherited ones, the +tree is bounded by an atomic descendant reservation taken before any child is +queued, and in the shipped profiles the spawn capability is deny-filtered off +entirely. So the answer to "how deep can delegation go" is currently "it does +not," with the machinery built to turn it on safely later. Read against +Cognition's verdict this is the same conclusion reached from the other +direction: Devin learned empirically that parallel children must not act, +IronClaw arranges structurally that a child *cannot* act until authority is +explicitly granted. Two products, one from production experience and one from +first principles, both landing on children-are-readers-by-default is the +strongest signal on this otherwise unsettled axis. + **E. Session semantics.** Session-as-task-run (OpenComputer, Managed Agents, Devin, LangGraph runs) vs session-as-conversation-lane (OpenClaw's routing-scoped lanes, Hermes' session keys, Cloudflare instances that may *be* a room). Who names it also splits: platform-minted IDs vs caller-supplied keys (AgentCore's client-named `runtimeSessionId`, OpenComputer's get-or-create `key`, OpenClaw's deterministic routing keys). +IronClaw refuses the choice by splitting the noun: a `SessionThread` is the +conversation lane (durably sequenced messages and summaries under a scope), +and a `turn_run` is the task run (lifecycle, locks, checkpoints, admission +reservations), persisted separately with a redaction boundary between them so +lifecycle records hold "metadata and references only." Both halves of the +divergence exist, and neither is asked to do the other's job. Inbound naming +is idempotency-keyed rather than either minted or caller-named: a SHA-256 over +`(scope, source_binding_id, external_event_id)`. **F. Identity scope.** Everyone has intra-org identity; only [A2A](./products/adk-a2a.md) defines cross-org identity: the AgentCard (name, skills, interfaces, security schemes, JWS signatures, well-known URI). It is the only serious interoperable definition, and Vertex + LangGraph + AgentCore + CrewAI all already carry -A2A hooks. +A2A hooks. IronClaw is at the far opposite end and deliberately so: its agent +identity is an internal scope axis in a `ThreadScope` tuple +(`tenant_id`, `agent_id`, optional `project_id`, `owner_user_id`, +`mission_id`), meaningful only inside the deployment and used for storage +placement and authorization rather than for discovery. With no agent record +there is nothing to project into an AgentCard, which makes the cost of the +no-record design concrete: cross-org identity would have to be synthesized +from the scope plus a run profile rather than published from a definition. ## Conceptual models in play @@ -174,8 +312,11 @@ A2A hooks. | agent-as-teammate/product | Devin | | agent-as-learning-identity | Hermes (memory + self-authored skills as the definition) | | agent-as-interface (anything that runs the loop) | Vercel AI SDK 6, Claude Code harness framing | +| agent-as-scope-coordinate (a validated axis in a scope tuple; no stored object, persona in files, runtime shape resolved onto each run) | IronClaw | These are not mutually exclusive; most products stack two or three. +IronClaw stacks agent-as-scope-coordinate with agent-as-file, which is what +makes it legible: the scope answers "whose," the files answer "who." ## Comparison table @@ -198,6 +339,7 @@ These are not mutually exclusive; most products stack two or three. | Netclaw (added post-synthesis) | daemon with file-shaped soul; event-sourced actor sessions | spawn_agent → ephemeral child actors, depth 1 (recursive spawn denied), fail-closed audience inheritance | daemon-start validation; validate-before-restart reload with session drain; identity re-read per session actor | user-run daemon (systemd) | 1:N channel+thread-keyed persistent actors | | kagent (added post-synthesis) | namespaced K8s custom resource reconciled into an A2A service | agent-as-tool by CRD reference; DAG capped at depth 10; fresh child session, identity-only inheritance | reconcile-time resolution into a config Secret; rebinding = pod roll | platform deploys; in-pod ADK runtime owns the loop | 1:N DB sessions; delegation mints child sessions | | AgentCore harness (added post-synthesis) | versioned config record over an AWS-owned loop | no subagent noun; agent-as-tool via Gateway; compose above via Step Functions | auto-versioned config; per-call overrides | AWS owns the loop (managed harness on managed Runtime) | 1:N, session = microVM | +| [IronClaw](./products/ironclaw.md) (added post-synthesis) | scope coordinate with no stored object; persona in markdown, runtime shape in a `ResolvedRunProfile` on the run | child runs, lineage on the run, empty grant sets, atomic descendant reservation, deny-filtered off in shipped profiles | persona per turn (file read); mechanism resolved once at admission and frozen on the run | userland loop above a kernel boundary owning authority and recovery; loop is not the security perimeter | 1:N threads under the scope; thread (transcript) and turn run (lifecycle) are separate resources | ## Working definition @@ -216,13 +358,34 @@ declarations in the revision while assigning limits, credential bindings, work contracts, resolved session context, and observations to their owning resources. +*Revised after IronClaw.* The working definition assumes its own conclusion in +one place: "a named, versioned declaration" presumes the declaration is a +stored resource. IronClaw is a working system where it is not, so the honest +generalization is that an agent is a **named scope plus a resolved +configuration**, and whether that configuration is a versioned record, a set +of files, or a resolution captured per run is a product decision. Our ADRs +already choose the versioned record, and the evidence still supports that +choice for a multi-tenant platform that must enumerate, describe, and share +agents. What IronClaw changes is the *justification*: the record earns its +place by giving us discovery, description-based routing, and per-agent +defaults, not by being the only way to make executions replayable. Replay +needs a captured resolution on the execution, which we should have whether or +not the definition is versioned. + Design decisions the evidence forces, with the industry's answer where one exists: 1. **Model the trio as three first-class resources**: AgentDefinition (versioned), Session (pins a definition version at create), Memory (attachable N:M). Do not embed memory or environment in the definition; - nobody who scaled did. + nobody who scaled did. *Revised after IronClaw*: pinning a definition + version is necessary but not sufficient. Also capture the *resolved* + runtime shape on the execution (loop/driver identity, checkpoint schema + version, model and capability-surface selections, budgets, and a + fingerprint of what was consulted), because a recovery path that + re-resolves can legally land on a different driver or checkpoint schema + than the run it is recovering. A version pointer alone does not prevent + that when resolution depends on anything outside the definition. 2. **Version linearly and immutably; sessions freeze.** Allow per-session overrides that never write back (Managed Agents), staging/rollback (OpenComputer, LangGraph), and exactly one live-mutation exception: @@ -235,6 +398,17 @@ exists: restricted-tool inheritance.** Depth and fan-out are cost controls (Hermes) as much as safety ones. Enforce Cognition's rule structurally if possible: parallel children for reads/analysis; single writer. + *Revised after IronClaw* on two points. First, invert the inheritance + default: children should start with an **empty** grant set that the parent + must explicitly narrow *into*, rather than inheriting the parent's tools + minus a deny list, because a deny list has to anticipate every dangerous + capability while an allow list only has to name the needed ones. Second, + bound the tree by **reserving descendant slots atomically before queueing + any child**, not by checking a depth counter at spawn time, which is the + only form of the limit that holds under concurrent fan-out. IronClaw also + demonstrates the shippable intermediate state worth copying: build the + lineage, reservation, and tombstone machinery, then keep the spawn + capability denied by default until the authority story is finished. 5. **Make `description` a first-class, prompt-visible field**: it is the delegation routing contract, not metadata. 6. **Name sessions with caller-supplied idempotency keys** (get-or-create), @@ -257,3 +431,13 @@ The one-line reading of the whole study: the industry agrees on the lives, who runs the loop, how deep delegation goes) is a product decision, and the most successful designs are the ones that made those decisions explicit rather than inheriting them. + +Revised after IronClaw: the trio is a set of *roles*, not necessarily a set of +resources, and the eighteenth product is the one that proves it by shipping +without the first member. What that reframing buys us is a sharper test for +our own design. Every property we are tempted to put on the agent definition +should have to answer why it belongs to the agent rather than to the scope +(authorization), the files (persona), or the run (resolved mechanism). The +properties that survive that test are the ones a definition record genuinely +owns; the rest are there because a record was the first place we had to put +them. diff --git a/docs/research/session-store/index.md b/docs/research/session-store/index.md index 3642685a0..d5da3c02f 100644 --- a/docs/research/session-store/index.md +++ b/docs/research/session-store/index.md @@ -26,7 +26,9 @@ as draft [ADR#0035: Session Store as a Decider Aggregate on NATS JetStream](../../adr/0035-session-store-decider-aggregate.md). The fx artifacts (the dossier, the session detail JSON reference, and the comparison against our event catalog) were added after the synthesis and are -not yet folded into it. +not yet folded into it. The IronClaw dossier was also added after the +synthesis was frozen, but its evidence is folded in and marked inline +wherever it revised a frozen claim. The corpus is being extended to twenty-eight products. The queue, its ordering, and the verification state of each candidate are in the @@ -55,11 +57,11 @@ at once. Run per-artifact as each landed, it reported clean on files that were not: a sweep of every artifact with a local clone found flattened paths, wrong crates, and bare basenames in six files that individual waves had already passed. The current state of that sweep is 1912 citations, all resolved, no -missing file and no out-of-range line, across all eighteen products whose +missing file and no out-of-range line, across every product whose source is checked out locally. The remaining dossiers (fx, Claude Agent SDK, -Codex CLI, Gemini CLI, Goose, Grok Build, Hermes, LangGraph, OpenCode, T3 Code) -predate that sweep and were verified by hand only, so they carry weaker -mechanical guarantees than the number above suggests. +Codex CLI, Gemini CLI, Goose, Grok Build, Hermes, IronClaw, LangGraph, +OpenCode, T3 Code) predate that sweep and were verified by hand only, so they +carry weaker mechanical guarantees than the number above suggests. The second layer is the one that earns its cost. It is what caught a migration ratchet described as content-hashed when the source compares stored text, a @@ -139,6 +141,7 @@ artifact; this corpus nests because every product here has at least two. - [Goose (Block)](./products/goose/index.md) - [Grok Build](./products/grok-build/index.md) - [Hermes (Nous Research)](./products/hermes-agent/index.md) +- [IronClaw (NEAR AI)](./products/ironclaw/index.md) - [LangGraph (LangChain)](./products/langgraph/index.md) - [Letta](./products/letta/index.md) - [Letta compared to our session event catalog](./products/letta/vs-session-events.md) diff --git a/docs/research/session-store/products/ironclaw/index.md b/docs/research/session-store/products/ironclaw/index.md new file mode 100644 index 000000000..e4d4ae1c0 --- /dev/null +++ b/docs/research/session-store/products/ironclaw/index.md @@ -0,0 +1,952 @@ +# IronClaw: how session transcripts are stored and resumed + +Part of Session Store Research. +Produced by running [RESEARCH_PROMPT](../../RESEARCH_PROMPT.md). +Evidence snapshot retrieved 2026-08-05. Version-sensitive claims were checked +against these authoritative anchors: + +- Repository `github.com/nearai/ironclaw` at commit + `2ae66212fe80208524179047878916dafc0538ee` (committed 2026-08-05T18:20:35Z, + "feat(product): add new, stop, and interrupt commands (#6969)"). Rust + workspace, dual MIT OR Apache-2.0. +- `crates/domains/ironclaw_threads/src/{service.rs,contract.rs,filesystem_service.rs,stored_message.rs,summary_artifacts.rs}` + for the transcript boundary. +- `crates/substrates/ironclaw_filesystem/src/{root.rs,record.rs}` for the + durable substrate the transcript is written through. +- `crates/Architecture.md` and `docs/reborn/contracts/{turn-persistence.md,conversation-binding.md,storage-placement.md,agent-loop-protocol.md}` + for the architectural contracts. +- `docs/reborn/subagent-spawn/README.md` for child-session storage. +- `migrations/V1..V34` for the SQL shape of the backing store. + +Citations use repo-relative `path:line` shorthand against that commit. + +> Scope note. IronClaw has no published product documentation site; every +> primary source below is in-repository (design contracts under `docs/reborn/` +> plus the implementing Rust code). The tree is mid-transition from a legacy +> pre-"Reborn" design to the Reborn architecture, and the two disagree in +> places. Where they do, `crates/Architecture.md` and the code win: the repo +> states this rule explicitly at +> `docs/reborn/target-architecture/CHECKLIST.md:355` ("**THE CODE WINS**"). +> This dossier documents the Reborn transcript boundary and flags legacy +> survivals where they are still reachable. +> +> Naming trap, called out by the repo itself: there are *two* types named +> `SessionThreadService`. The transcript one lives in `ironclaw_threads`; the +> inbound-routing one in `ironclaw_conversations` was renamed +> `InboundConversationService` precisely to end the collision +> (`docs/reborn/contracts/conversation-binding.md`). Everything below is the +> `ironclaw_threads` one unless stated. + +## The storage model + +IronClaw splits durable session state across **two** boundaries with different +owners, and the split is the most load-bearing fact about its model. + +1. **The transcript boundary** (`SessionThreadService`): threads, messages, + and summary artifacts. Source of truth for what was said. +2. **The turn/process boundary** (`ProcessJournalStore`, projected as + `turn_runs` and friends): run lifecycle, leases, checkpoints, admission + reservations, idempotency outcomes. Source of truth for what ran. + +`docs/reborn/contracts/turn-persistence.md:22` draws the line in one sentence: + +> It does **not** own canonical transcript/message storage. Transcript and +> thread-message history remain in the transcript/thread storage boundary. + +And the reverse direction is enforced as a redaction rule rather than a +convention (`turn-persistence.md:106-108`): + +> Turn persistence stores metadata and references only. It must not persist raw +> prompts, assistant content, tool input, secrets, host paths, or backend error +> details in turn/run/checkpoint/idempotency records. + +Within the transcript boundary, the durable session is **a set of individually +CAS-versioned records at virtual paths**, not an append-only log and not one +mutable document. The record layout is stated verbatim in the module doc of the +production implementation (`filesystem_service.rs:22-31`): + +```text +/threads[/agents/][/projects/][/owners/][/missions/]/threads//thread.json +/threads[/.../...]/threads//messages/.json +/threads[/.../...]/threads//summaries/.json +/threads/idempotency/.json +``` + +Three further paths exist under the same thread root +(`filesystem_service.rs:2918-2970`): `messages/` (directory), `tool_results/ +sha256-.bin` for out-of-band tool payloads, and `message_sequence` for the +native per-thread counter row. + +That "virtual filesystem" is not necessarily a filesystem. It is the +`RootFilesystem` substrate, whose canonical inhabitant is a SQL row. Every +entry is either an opaque byte file or a typed record +(`crates/substrates/ironclaw_filesystem/src/record.rs:258-279`): + +> - **Opaque file**: `body` carries arbitrary bytes, `kind` is `None`, +> `indexed` is empty. [...] +> - **Record**: `body` carries the serialized payload (typically JSON), `kind` +> names the schema family [...] and `indexed` declares the projection that +> backends should expose to [`query`]. +> +> Backends never look inside `body` for indexing; everything queryable lives in +> `indexed`. + +The SQL migration that introduced this shape shows the physical form +(`migrations/V28__root_filesystem_records.sql:1-19`): `root_filesystem_entries` +gains `content_type`, `kind`, `indexed JSONB`, and `version BIGINT`, where +"`version` enables compare-and-swap semantics on `put`". Thread records declare +four kinds (`filesystem_service.rs:101-104`): `session_thread`, +`thread_message`, `thread_summary`, `thread_idempotency`. Setting `kind` is +load-bearing for safety, not just typing (`filesystem_service.rs`, module doc): + +> Setting `entry.kind` makes writes record-shaped so `DiskFilesystem` [...] +> triggers the fail-closed path on the CAS gate instead of accepting a +> byte-only first write without CAS enforcement. + +**Authoritative versus derived.** The per-record JSON entries are +authoritative. Everything that makes listing and range reads cheap is an +explicitly rebuildable projection: the thread index rows +(`filesystem_service/thread_index.rs`), the ordered message index, and the +exact-lookup projections, all rebuilt idempotently by a one-time migration pass +(`filesystem_service/transcript_migration.rs:1-60`). The storage-placement +contract states the general rule +(`docs/reborn/contracts/storage-placement.md`): "these views are projections, +not the source of truth". An in-process one-shot context-window cache +(`ONE_SHOT_CONTEXT_WINDOW_CACHE_MAX_ENTRIES: usize = 4096`, +`filesystem_service.rs:110`) is pure cache, seeded on first accept and +invalidated on any later write. + +**Conceptual model.** Session-as-directory-of-records, where the "directory" is +a scoped virtual path tree that is usually SQL rows, and each record carries its +own version token. It is neither session-as-log (no append-only event stream is +the source of truth for the transcript) nor session-as-document (no whole-thread +rewrite) nor session-as-row (the thread record is a small metadata row; the +messages are siblings, not columns). `RootFilesystem` *does* expose an +append/tail event plane (`root.rs:29-30`, `append`/`tail` with `SeqNo`), and +`crates/events/` holds a full event-log/projection stack, but the transcript +does not ride on it: transcript writes go through `put` with a CAS expectation. + +## Keying and identity + +A thread is addressed by **scope plus thread id**, and the scope is structural, +not a string blob (`contract.rs`, `ThreadScope`): + +```rust +pub struct ThreadScope { + pub tenant_id: TenantId, + pub agent_id: AgentId, + pub project_id: Option, + pub owner_user_id: Option, + pub mission_id: Option, +} +``` + +Those axes are projected directly into the path prefix +(`filesystem_service.rs:2997-3003`, `thread_root_string` = `scope_axes_string` ++ `/threads/`), which is why the on-disk layout above shows optional +`agents/`, `projects/`, `owners/`, `missions/` segments. Note what is *not* in +the key: no working directory, no cwd hash, no git worktree. IronClaw keys +sessions by tenant, agent, project, and owner. There is nothing to reconcile on +a moved directory because the filesystem path was never part of the identity. +`AgentId` being a first-class scope axis (`storage-placement.md`) is the +distinctive part: a thread belongs to an *agent*, not merely to a user. + +`ThreadId` is a validated string id, not a mandated UUID +(`crates/contracts/ironclaw_host_api/src/ids.rs:213`): + +```rust +string_id!(ThreadId, "thread", validate_scope_id); +``` + +`validate_scope_id` bounds it at 256 bytes and rejects path separators, control +characters, and the reserved `__ironclaw_` sentinel prefix, because ids become +path segments. Callers may therefore supply their own id; the service mints one +only when absent, as UUIDv4 (`filesystem_service.rs:3137`): + +```rust +fn generated_thread_id() -> ThreadId { ThreadId::new(uuid::Uuid::new_v4().to_string()) } +``` + +So the id scheme encodes neither ordering nor location. Ordering comes from the +per-thread `sequence`, and location from the scope prefix. + +**Listing is scoped, and scoping is a security control rather than a +convenience.** `list_threads_for_scope` carries a normative requirement +(`service.rs:352-372`): + +> Implementations MUST scope the listing by `owner_user_id` (or equivalent +> caller-binding fields on the scope) — otherwise a caller could enumerate +> threads owned by other users in the same `(tenant, agent, project)` triple. + +There is no cross-tenant or cross-agent enumeration path. Reads are further +non-enumerating: `read_thread` must return the *same* `UnknownThread` error for +"does not exist" and "exists but is owned by another scope" +(`service.rs:285-289`) "so callers cannot use the response as an existence +oracle". `resolve_scope`/`read_thread_by_id` exist for the trusted-internal +case where a bare `ThreadId` must be mapped back to its scope, and both are +opt-in per backend, with `supports_resolve_scope()` defaulting to `false` +(`service.rs:317-331`). + +## The store interface + +The interface is genuinely pluggable: `SessionThreadService` is a public +`#[async_trait]` trait, and the on-disk/SQL implementation is one implementor +among several (there are in-memory and stub implementors for tests, and a +blanket `impl SessionThreadService for Arc` at `service.rs:375-380`). Below +is the trait verbatim, with default method bodies elided and marked; a method +with a default body is optional for a backend, and the defaults either fail +closed with `SessionThreadError::Backend` or compose other required methods. + +```rust +/// Canonical Reborn session thread and transcript boundary. +#[async_trait] +pub trait SessionThreadService: Send + Sync { + async fn ensure_thread(&self, request: EnsureThreadRequest) + -> Result; // required + + async fn accept_inbound_message(&self, request: AcceptInboundMessageRequest) + -> Result; // required + + async fn replay_accepted_inbound_message(&self, request: ReplayAcceptedInboundMessageRequest) + -> Result, SessionThreadError>; // required + + async fn mark_message_submitted(&self, scope: &ThreadScope, thread_id: &ThreadId, + message_id: ThreadMessageId, turn_id: String, turn_run_id: String) + -> Result; // required + + async fn mark_message_rejected_busy(&self, scope: &ThreadScope, thread_id: &ThreadId, + message_id: ThreadMessageId) + -> Result; // required + + async fn mark_message_queued(&self, scope: &ThreadScope, thread_id: &ThreadId, + message_id: ThreadMessageId, active_run_id: String) + -> Result; // default: fails closed + + async fn read_thread_message(&self, scope: &ThreadScope, thread_id: &ThreadId, + message_id: ThreadMessageId) + -> Result, SessionThreadError>; // default: fails closed + + async fn append_assistant_draft(&self, request: AppendAssistantDraftRequest) + -> Result; // required + + async fn append_finalized_assistant_message(&self, request: AppendFinalizedAssistantMessageRequest) + -> Result; // default: draft + finalize + + async fn append_tool_result_reference(&self, request: AppendToolResultReferenceRequest) + -> Result; // required + + async fn append_capability_display_preview(&self, request: AppendCapabilityDisplayPreviewRequest) + -> Result; // required + + async fn update_tool_result_reference(&self, request: UpdateToolResultReferenceRequest) + -> Result; // required + + async fn put_tool_result_record(&self, request: PutToolResultRecordRequest) + -> Result<(), SessionThreadError>; // default: fails closed + async fn read_tool_result_record(&self, request: ReadToolResultRecordRequest) + -> Result, SessionThreadError>; // default: fails closed + async fn update_tool_result_record(&self, request: UpdateToolResultRecordRequest) + -> Result<(), SessionThreadError>; // default: fails closed + async fn delete_tool_result_record(&self, request: DeleteToolResultRecordRequest) + -> Result<(), SessionThreadError>; // default: fails closed + + async fn update_assistant_draft(&self, request: UpdateAssistantDraftRequest) + -> Result; // required + + async fn finalize_assistant_message(&self, scope: &ThreadScope, thread_id: &ThreadId, + message_id: ThreadMessageId, content: MessageContent) + -> Result; // required + + async fn redact_message(&self, request: RedactMessageRequest) + -> Result; // required + + async fn load_context_window(&self, request: LoadContextWindowRequest) + -> Result; // required + async fn load_context_messages(&self, request: LoadContextMessagesRequest) + -> Result; // required + + async fn list_thread_history(&self, request: ThreadHistoryRequest) + -> Result; // required + + async fn list_thread_messages_bounded(&self, request: BoundedThreadMessagesRequest) + -> Result; // default: fails closed + async fn list_thread_messages_range(&self, request: ThreadMessageRangeRequest) + -> Result; // default: filter full history + async fn latest_thread_message(&self, request: LatestThreadMessageRequest) + -> Result, SessionThreadError>; // default: scan full history + async fn finalized_assistant_message_by_run(&self, request: FinalizedAssistantMessageByRunRequest) + -> Result, SessionThreadError>; // default: scan full history + + async fn read_thread(&self, request: ThreadHistoryRequest) + -> Result; // default: full history, take thread + async fn delete_thread(&self, scope: &ThreadScope, thread_id: &ThreadId) + -> Result<(), SessionThreadError>; // default: fails closed + + async fn create_summary_artifact(&self, request: CreateSummaryArtifactRequest) + -> Result; // required + + fn supports_resolve_scope(&self) -> bool; // default: false + async fn resolve_scope(&self, thread_id: ThreadId) + -> Result; // default: fails closed + + async fn update_thread_goal(&self, request: UpdateThreadGoalRequest) + -> Result; // default: fails closed + async fn read_thread_by_id(&self, thread_id: ThreadId) + -> Result; // default: fails closed + async fn list_threads_for_scope(&self, request: ListThreadsForScopeRequest) + -> Result; // default: fails closed +} +``` + +Source: `crates/domains/ironclaw_threads/src/service.rs:22-373`. Thirty-two +methods, of which sixteen are required. Three properties are worth extracting: + +- **Fail-closed defaults over silent lies.** The default + `list_threads_for_scope` returns `Backend(...)` deliberately, "so backends + that do not yet implement enumeration surface a clear `503 Service + Unavailable` at the gateway instead of pretending the caller has zero + threads" (`service.rs:352-357`). +- **Performance contracts written into the trait.** `read_thread` exists purely + because using `list_thread_history` as an ownership probe "on a large thread + is hundreds of rows per second per active stream" (`service.rs:271-284`). + `list_thread_messages_bounded` requires the budget to be enforced *while* + reading: "Implementations must enforce the budget while reading, rather than + materializing an unbounded transcript and checking afterward" + (`service.rs:200-203`). +- **No fork, no rewind, no branch, no message edit.** Not deferred behind a + flag: the operations do not exist. `grep -i "rewind\|fork"` over + `crates/domains/ironclaw_threads/src` and over `docs/reborn/` returns nothing + for the transcript boundary. The only retroactive operation is + `redact_message`. + +Below the domain trait sits the substrate contract that supplies the durability +primitives. `RootFilesystem` (`crates/substrates/ironclaw_filesystem/src/root.rs:37`) +is one trait implemented by every backend (local disk, Postgres, libSQL, HSM, +in-memory) *and* by the routing dispatcher, since "the dispatcher *is* a backend +that routes by longest-prefix mount" (`root.rs:11-18`). Its planes +(`root.rs:19-35`): a unified entry plane (`put`, `get`, `delete`, +`delete_if_version`, `list_dir`, `list_dir_bounded`, `query`, `query_ordered`, +`ensure_index`, `stat`, `read_file_bounded`), an atomicity plane (`begin` → +`StorageTxn`), an event plane (`append`, `tail`), and a legacy bytes plane being +removed. The floor is CAS, not transactions (`root.rs:26-28`): + +> **Atomicity** — [`begin`] for backends that natively support multi-key +> transactions. Stores must always work with CAS (`put` + +> `CasExpectation::Version`) as the floor. + +`CasExpectation` is the precondition type the transcript writes on +(`record.rs:236-256`), with a deliberate design note: + +> All multi-step store operations (lease claim, lease consume, status +> transitions) are implemented with `CasExpectation::Version` and retry on +> [`FilesystemError::VersionMismatch`]. Closure-based transactions across async +> boundaries are intentionally absent [...] consumers must continue to work when +> only CAS is available. +> +> ```rust +> pub enum CasExpectation { +> Absent, // Path must not currently hold an entry. Used for issue/create. +> Version(RecordVersion), // Path must currently hold the named version. +> Any, // Overwrite regardless. Used only by backfills / admin flows. +> } +> ``` + +`RecordVersion` is backend-minted and unforgeable by consumers +(`record.rs:180-192`): "Consumers obtain versions only by reading existing +entries — they cannot fabricate one." Reads return it attached +(`VersionedEntry { path, entry, version }`, `record.rs:345-350`) so a +read-modify-write always has a precondition available. The store also documents +the ABA hazard on version-conditioned deletes rather than papering over it +(`root.rs:184-191`): "Version tokens are not generation-stable: a path's version +restarts at 1 on a fresh put after a prior delete." + +## Write and append path (ordering, durability, concurrency, delivery) + +**Commit shape.** New messages are created with `put(..., CasExpectation::Absent)` +(`filesystem_service.rs:491`, `2120`, `633`), i.e. insert-if-absent. Mutations +in place (draft → finalize, status transitions, redaction) go through +`apply_message_update`, which reads the versioned record and writes back with +`CasExpectation::Version(versioned.version)` (`filesystem_service.rs:1223-1250`). +Nothing rewrites the whole thread. There is no append-to-a-log step. + +**Ordering** is a durable per-thread `u64` sequence, and IronClaw has two +mechanisms for allocating it. On backends that expose the native +`ReserveSeq` operation, a path-local counter row at +`/message_sequence` is bumped atomically; otherwise it falls back +to a CAS loop on the thread record's `next_sequence` +(`filesystem_service.rs:1083-1130`). The thread record itself carries the +counter (`filesystem_service.rs:125-130`): + +```rust +struct StoredThreadRecord { + #[serde(flatten)] + record: SessionThreadRecord, + next_sequence: u64, +} +``` + +The migration reasoning is instructive about why a "just switch to the faster +counter" change is unsafe once sequences exist +(`filesystem_service.rs:1098-1109`): + +> Migration safety: a thread that already assigned message sequences under the +> legacy per-thread-record counter (`next_sequence > 1`) must keep using it. The +> native path-local counter starts at 1 for a path with no row, so switching an +> *existing* thread onto it would restart at 1 and collide with messages already +> at sequences 1..N — corrupting ordering and clobbering the sequence index [...] +> No thread ever switches counters mid-stream. + +The legacy fallback is explicitly named as a bottleneck: "This preserves +compatibility but retains the old shared-thread-record CAS bottleneck" +(`filesystem_service.rs:1132-1134`), with `FILESYSTEM_CAS_RETRIES: usize = 8` +(`filesystem_service.rs:94`) before surfacing "filesystem CAS retries +exhausted". + +**Atomicity.** Where the backend supports transactions, the accept path commits +the counter, the message record, the sequence index, and the idempotency record +together (`filesystem_service.rs:519-560`, `1455-1458`): + +> On transactional backends the thread counter, message, sequence index, and +> idempotency record commit together; fallback backends reserve immediately +> before the legacy message write. + +The non-transactional path degrades to a documented ordering, with the message +as the authority (`filesystem_service.rs:1530-1532`): "the message is +authoritative, and the idempotency record accelerates later replays when it can +be written." + +**Concurrency.** Three layers, worth separating: + +1. At the record level, optimistic concurrency with a real expected-version + precondition (`CasExpectation::Version`) and bounded retry. This is a genuine + expected-version write boundary, not advisory. +2. At the *sequence* level, an atomic counter reservation, so two concurrent + writers get distinct sequences rather than one losing. +3. At the *turn* level, a single-active-run lock keyed by the canonical + `TurnScope` of tenant, agent, optional project, thread + (`turn-persistence.md:47-55`). `crates/Architecture.md` lists it among the + key invariants: "One active run per canonical thread is enforced before + model/tool side effects." Concurrent user messages during an active run are + not merged into a race; they are either steered into a queue + (`MessageStatus::Queued` via `mark_message_queued`) or refused + (`RejectedBusy`), which is a store-visible status, not just an HTTP code. + +**Delivery semantics.** At-least-once inbound with an idempotency record that +makes acceptance effectively exactly-once per external event. The key is a +SHA-256 of the full tuple (`filesystem_service.rs:1437-1440`): + +> First, check idempotency. The on-disk key SHA-256s the full (scope, +> source_binding_id, external_event_id) tuple, so a same-binding/event from a +> different scope hashes to a different key (and we only see records under the +> current MountView). + +If the tuple is incomplete (`source_binding_id` or `external_event_id` absent), +no idempotency key is formed at all and the write is unguarded +(`filesystem_service.rs:1421-1428`), a deliberate choice that shifts +responsibility to callers that omit event ids. On a transactional backend, a +duplicate surfaces as `TransactionalMessageWrite::IdempotencyAlreadyAccepted` +and the prior `AcceptedInboundMessage` is returned instead +(`filesystem_service.rs:1506-1523`). The returned value carries +`idempotent_replay: bool` so the caller can tell a fresh accept from a replay. + +**Advisory writes are marked as such.** After a durable accept, the recency +stamp and derived sidebar title are best-effort, with the reason stated +(`filesystem_service.rs:1572-1574`): "silent-ok: the message is already durable; +the recency stamp and derived label are advisory, and failing the accept here +could make an un-idempotent caller retry and duplicate the message." The +counter-example is redaction, where the same class of derived copy is *not* +best-effort (`filesystem_service.rs:2425-2429`): "Propagating the removal is a +redaction obligation, not best-effort: failing here is correct if the copy +cannot be cleared." + +## Read and resume path + +There is no single "resume" call that rehydrates a session. There are four +distinct read shapes, chosen by what the caller needs: + +- **`load_context_window`** is the model-facing read. Reads the latest N messages + via the ordered index, loads summary artifacts, and applies summary + replacement over the range before truncating to `max_messages` + (`filesystem_service.rs:2435-2467`). This is the only path that interprets + summaries. It checks the one-shot cache first (seeded when the accepted message + is `sequence == 1`, so a brand-new thread's first turn avoids a read round + trip: `filesystem_service.rs:1546-1550`). +- **`load_context_messages`** is an explicit by-id fetch of a caller-chosen set, + read concurrently with `join_all` and returned in the requested order + (`filesystem_service.rs:2469-2493`). +- **`list_thread_history`** is the full transcript plus summary artifacts plus + the thread record with its index overlay (`filesystem_service.rs:2495-2518`). + Unbounded, and the trait steers callers away from it for hot paths. +- **`list_thread_messages_range` / `list_thread_messages_bounded` / + `latest_thread_message`** are cursor-ish reads served from the ordered index + (`list_thread_messages_range_indexed`, `filesystem_service.rs:731`), with a + byte budget for export. + +Resume reads the durable store; there is no local-cache-first path, and the +only cache is the in-process one-shot window described above. Every read begins +by resolving the thread record under the caller's exact scope +(`read_thread_versioned`), so an unauthorized read fails before any transcript +materializes. + +The byte-budget denominator for bounded reads is the *stored* representation, +not the domain type (`stored_message.rs`): "Both durable and in-memory bounded +reads use this stored representation as their byte-budget denominator." That +matters because `StoredThreadMessageRecord` re-adds a field the wire type skips +(`tool_result_provider_call` is `#[serde(skip_serializing)]` on +`ThreadMessageRecord`), so the two would otherwise disagree about size. + +Loop-execution resume is a separate mechanism on the other boundary: the driver +returns `LoopExit::Blocked`, the host persists an opaque checkpoint ref plus a +bounded payload, and `resume_turn` requeues the *same* run against that +checkpoint (`Architecture.md:589-604`). Checkpoints are scoped and +non-transferable (`turn-persistence.md:96-98`): "Reads with a matching ref but +foreign scope or run return no state." + +## Listing, summaries, and search + +Listing is served by a **maintained index projection**, not a directory scan. +`ThreadIndexRecord` flattens the thread record and adds derived fields +(`filesystem_service/thread_index.rs:1-90`): + +```rust +struct ThreadIndexRecord { + #[serde(flatten)] + record: SessionThreadRecord, + next_sequence: u64, + flags: ..., + derived_title: Option, +} +``` + +with indexed keys `scope_key`, `activity_sort`, and `thread_id`, and +`THREAD_INDEX_KNOWN_ROW_MAX = 100_000`. The index row is declared the recency +authority for listing (`filesystem_service.rs:1176-1181`): "The index row is +the recency authority for listing; avoiding a full `thread.json` CAS here keeps +activity writes row-shaped." + +The denormalized field that carries the most weight is `derived_title`, written +at message-accept time specifically to avoid an N+1 on the sidebar +(`filesystem_service.rs:1552-1562`). Only the first user message may seed it, +and the code explains the alternative it rejected: seeding from whatever arrives +next "would show the newest message where the sidebar contract promises the +first". Threads that predate the field are healed lazily on listing, with a +bounded read fan-out (`TITLE_DERIVATION_READ_CONCURRENCY: usize = 8`, +`filesystem_service.rs:107`) deriving titles for at most the page being returned +(`filesystem_service.rs:2782-2810`). Pagination is capped: +`LIST_THREADS_DEFAULT_PAGE_SIZE = 50`, `LIST_THREADS_MAX_PAGE_SIZE = 200` +(`filesystem_service.rs:2859-2860`), clamped rather than rejected. + +Consistency between index and record is maintained by writing the index row on +the same events that change the record, plus the idempotent rebuild pass in +`transcript_migration.rs` (`TRANSCRIPT_PAGE_CONFLICT_RETRIES = 5`, conflicts +being `VersionMismatch | BackendBusy`, with completion markers so the rebuild +runs once). Index divergence is therefore recoverable by design: the rows are +derivable from the records. + +**Search: absent.** There is no FTS index, no vector index, and no +`search_threads` operation on the trait. Listing is by scope ordered by +activity, and that is the whole retrieval surface for transcripts. Memory +documents are a separate substrate (`docs/reborn/contracts/memory.md`), not a +transcript search index. + +## Entry/message structure and versioning + +The stored message is a typed record, not an opaque blob +(`contract.rs`, `ThreadMessageRecord`): + +```rust +pub struct ThreadMessageRecord { + pub message_id: ThreadMessageId, + pub thread_id: ThreadId, + pub sequence: u64, + pub kind: MessageKind, + pub status: MessageStatus, + pub created_at: Option>, + pub updated_at: Option>, + pub actor_id: Option, + pub source_binding_id: Option, + pub reply_target_binding_id: Option, + pub turn_id: Option, + pub turn_run_id: Option, + pub tool_result_ref: Option, + #[serde(skip_serializing)] + pub tool_result_provider_call: Option, + pub content: Option, + pub attachments: Vec, + pub redaction_ref: Option, +} +``` + +with two enums doing the discrimination: + +```rust +pub enum MessageKind { User, Assistant, System, Summary, CheckpointReference, + ToolResultReference, CapabilityDisplayPreview } + +pub enum MessageStatus { Accepted, Queued, Submitted, RejectedBusy, DeferredBusy, + Draft, Finalized, Interrupted, Superseded, Redacted, Deleted } +``` + +`DeferredBusy` is explicitly legacy, superseded by `Queued` steering +(`conversation-binding.md`), and there is a test-only injector for it +(`inject_legacy_deferred_busy_for_test`, `filesystem_service.rs:1299`) so the +read path's handling of old rows stays covered. + +The store **parses and interprets** entries. It relies on `sequence` for +ordering, on `kind`/`status` for every projection and query +(`latest_thread_message` filters on both), on `turn_run_id` for +`finalized_assistant_message_by_run`, and on `message_id` for identity. Dedup, +by contrast, keys off the separate idempotency record, not off a field of the +message. + +Payload boundaries are enforced in the contract layer: `MessageContent` carries +text plus `Vec` where "Attachments are carried as references only +— never raw bytes", extracted text is capped at +`MAX_EXTRACTED_TEXT_CHARS = 200_000`, and `GoalStatement` at 4000 chars. Large +tool outputs go out-of-band: the message holds a `tool_result_ref`, and the +payload lives at `tool_results/sha256-.bin` read back in chunks +(`ToolResultRecordChunk`), addressed by SHA-256 of the ref +(`filesystem_service.rs:2928-2949`). + +**Format evolution** is handled three ways, all visible in the code rather than +in a version field: + +1. **Additive serde with optional legacy fields.** `created_at`/`updated_at` are + `Option>` for rows that predate durable timestamps, guarded by + validators named for the two failure modes (`MissingDurableTimestamps`, + `ClearedDurableTimestamps`) so new writes cannot regress to `None`. +2. **Behavioral migration gates rather than rewrites.** The `next_sequence > 1` + check that pins a thread to its original counter for life is the clearest + example: old and new coexist per-thread, forever, with no rewrite. +3. **SQL migrations for the substrate**, `V1..V34`, additive with + `ADD COLUMN IF NOT EXISTS` and safe defaults + (`V28__root_filesystem_records.sql`), plus projection-adding migrations + (`V30__root_filesystem_events.sql`, + `V33__root_filesystem_ordered_index_rows.sql`). + +There is **no session-format version number** on the thread or message record, +and no format-version negotiation. The ratchet is one-way in practice (a +migrated thread keeps its counter; rebuilt projections assume the current +shape), but nothing declares it as such. Flagged under open questions. + +## Compaction and history management + +Compaction produces a **new record and never touches the compacted messages**. +`SummaryArtifact` (`contract.rs`) is stored at +`summaries/.json` and spans a closed sequence range: + +```rust +pub struct SummaryArtifact { + pub summary_id: SummaryArtifactId, + pub thread_id: ThreadId, + pub start_sequence: u64, + pub end_sequence: u64, + pub summary_kind: SummaryKind, // Compaction + pub content: String, + pub model_context_policy: SummaryModelContextPolicy, // ReplaceRangeWhenSelected +} +``` + +`create_summary_artifact` validates that the range is non-empty and that both +endpoints actually exist as messages, then writes with +`CasExpectation::Absent` (`filesystem_service.rs:2682-2752`). Overlap handling +is idempotent-by-content: replaying the identical compaction returns the +existing artifact, while a genuinely different overlapping range is rejected +with `OverlappingSummaryRange`, and only `ReplaceRangeWhenSelected` summaries +are overlap-checked at all (`summary_artifacts.rs:1-70`). + +The **model-visible view** shrinks only inside `load_context_window`, where +`context_messages_with_summary_replacements` substitutes the summary for its +range (`filesystem_service.rs:2458`). Nothing else in the system sees a +shortened thread: `list_thread_history` returns both the messages and the +summary artifacts, so a UI or an export crosses a compaction boundary by seeing +*both* the original messages and the summary that stands in for them. Resume +across a compaction boundary is therefore lossless in the durable record and +lossy only in the assembled prompt. + +Compaction placement is unusual and worth extracting. It is a **host** concern, +not the loop's, and it is treated as a security boundary as much as a context +one (`docs/reborn/contracts/agent-loop-protocol.md:200-270`): "Host-managed +compaction is a typed retention boundary." Compaction scans for leaked secrets +with actions `Block` / `Redact` / `Warn`, replaces matches in place, rescans +before inference and before persistence, and fails closed if matches remain; +it reports an additive `redacted_leak_count` and one safe +`CompactionLeakDetected` milestone. No other product in this corpus runs a leak +scanner inside the compaction path. + +## Rewind, checkpoints, and fork + +**Rewind: absent. Fork: absent. Branch: absent. Message edit: absent.** These +are not deferred features with reserved enum variants in the transcript layer; +the operations do not appear in the trait, the implementation, or the design +docs. The one place `Fork` appears anywhere adjacent is the *subagent* context +seed mode, and it is reserved and unimplemented +(`docs/reborn/subagent-spawn/README.md:74`): "`Fork` seed mode (full +parent-context copy) — enum variant reserved, unimplemented", with +`phase-3-integration.md:677` making the runtime consequence explicit: "`Fork` is +reserved/unimplemented -> Denied." + +The only retroactive operation on the transcript is `redact_message`, and it is +a **destructive in-place edit**, not an appended tombstone +(`filesystem_service.rs:2399-2432`): status becomes `Redacted`, and `content`, +`attachments`, and `tool_result_provider_call` are all set to `None`/empty, +leaving a `redaction_ref` pointer. Summary content gets the same treatment via +`REDACTED_SUMMARY_CONTENT = "[redacted]"` (`filesystem_service.rs:3259`). A +reader after redaction cannot reconstruct what was there, which is the point, +and which is the exact opposite of what an append-only log would give you. + +**Checkpoints exist, but they check point the loop, not the transcript.** They +live on the process journal, hold the loop driver's serialized +`LoopExecutionState` as bounded opaque payload bytes, and are written when a +capability requires approval or auth (`turn-persistence.md:86-101`, +`Architecture.md:589-604`). Two properties matter for comparison with +file-state checkpointing elsewhere in this corpus: + +- They are **not** environment or file-state snapshots. There is no + content/diff/hash store of workspace files tied to a turn. +- The payload is opaque to the store, bounded, and debug-redacted, and only the + ref and schema metadata are ever projected + (`turn-persistence.md:99-101`). + +Lease expiry, the closest thing to an involuntary rewind, is deliberately +terminal rather than resumable (`Architecture.md:620-632`): + +> ```text +> runner crashes or stops heartbeating +> -> reconciler sees expired Running/CancelRequested lease +> -> Running => terminal Failed (sanitized "lease_expired") +> -> CancelRequested => terminal Cancelled +> -> the terminal transition releases the active-thread lock +> ``` +> +> Reborn does not automatically retry uncertain side-effecting work after a lost +> lease — expiry is terminal, and the user resubmits explicitly. + +Note a live doc conflict here: `turn-persistence.md:91` still says expired +leases "transition to `RecoveryRequired` [...] and keep the active lock", while +`Architecture.md:578-579` states `RecoveryRequired` "survives only as a legacy +variant" and expiry is terminal. Per the repo's own precedence rule, the +architecture doc and the code win. + +## Subagents and nested sessions + +A subagent gets a **first-class sibling thread**, not a nested transcript and +not entries in the parent's transcript. The child's `TurnScope` copies +tenant, agent, and project verbatim, and only `thread_id` is fresh +(`docs/reborn/subagent-spawn/README.md`), so the child's records land under the +same scope prefix as any other thread of that agent. The child also inherits +`owner_user_id`, which is what makes approvals surface on the child thread +rather than leaking to another user. + +The durable parent-child link lives on the **run**, not the thread: +`parent_run_id`, `subagent_depth`, and `spawn_tree_root_run_id` are fields of +the child run in the process journal. So the transcript store itself does not +know it is holding a child session; the lineage is a property of the +coordination plane. The gate is a capability call +(`spawn_subagent(flavor_id, task, handoff?)`) returning +`CapabilityOutcome::AwaitDependentRun`, which awaits the entire child set and +resolves inline if all children are already terminal. + +Transcript isolation is total by default: the child starts with an **empty grant +and lease set**, and the seed is either `Fresh` (goal only) or +`Handoff(String)` (a curated blob re-materialized into the child scope). The +capability allowlist of the child's profile is described as "a surface *ceiling*, +not authority". The goal is placed as the child's first **user** message, and the +reason is stated: "Never the system message — the goal is model-generated and may +carry upstream-tainted content." + +Nesting is bounded four ways, all enforced before `submit_turn`: +`allow_nesting = false` by default, a depth cap, a per-turn fan-out cap, and an +atomic `reserve_tree_descendants(scope, root, delta, cap)` against +`MAX_TREE_DESCENDANTS` backed by a durable `SpawnTreeReservation` +(`phase-2-mechanisms.md:1884` names the threat model directly: "Fork-bomb via +depth × fan-out"). + +Cascade behavior on parent cancellation is the part most relevant to us: the +child's result is not silently dropped but recorded, via +`SubagentResultTombstone { child_run_id, disposition: "discarded_by_parent_cancel", +terminal_status }`. That is a durable record of a discarded child, which is +strictly more than "orphan" or "cascade delete". + +Two caveats. First, spawning is **off in every shipped profile** at this commit: +`builtin.spawn_subagent` is deny-filtered via `TEMP(disable-spawn-subagents)` in +`crates/loop/ironclaw_turn_runner/src/runtime.rs`, per the 2026-07 status note in +`crates/Architecture.md`. Second, parent *delete* cascade is unspecified: +`delete_thread` deletes one thread's subtree and index row and says nothing about +children spawned from it (see open questions). + +## Retention, deletion, and multi-host + +**Retention.** There is no TTL, lifecycle policy, or scheduled cleanup for +transcripts. Threads and messages persist until explicitly deleted. Retention +exists only on the *other* boundary, and only for released admission +reservations (`turn-persistence.md:80`): "Released reservation evidence is +retained only while the corresponding terminal run remains within the bounded +terminal-record retention window; active capacity accounting must not scan +unbounded released history." Bounding, elsewhere, is applied to reads (page +sizes, byte budgets, `THREAD_INDEX_KNOWN_ROW_MAX`) rather than to stored +history. + +**Deletion** is scope-checked, subtree-wide, and index-cascading +(`filesystem_service.rs:2643-2680`): probe ownership through `read_thread` +(preserving the non-enumerating error shape), delete the thread root +recursively, invalidate the context cache, then delete the index record. A +missing thread still triggers an index-record delete before returning +`UnknownThread`, which self-heals a stale row. Messages, summaries, tool-result +payloads, and the sequence counter all live under the thread root and go with +it. + +One asymmetry follows from the layout and is worth stating plainly as our +reading of the code, not as a documented behavior: inbound idempotency records +live at `/threads/idempotency/.json` +(`filesystem_service.rs:2993-2995`), which is *outside* any thread root +(`thread_root_string` = scope axes + `/threads/`, +`filesystem_service.rs:2997-3003`). A `delete_thread` therefore leaves the +thread's idempotency records behind, pointing at a `thread_id` and `message_id` +that no longer exist. The replay path reads such a record and reconstructs an +`AcceptedInboundMessage` from it (`accepted_message_from_idempotency_record`, +`filesystem_service.rs:1043`), so a redelivery of a pre-deletion external event +can still be reported as an idempotent replay of a deleted message. We found no +sweeper for these records and no documented TTL on them. + +**Multi-host is a first-class path, not a workaround**, and this is the sharpest +contrast with the file-per-session products in this corpus. Nothing in the +design assumes a shared local filesystem: `RootFilesystem` backends include +libSQL and Postgres (`profiles/local.toml` sets +`database_backend = "libsql"`; `server.toml` and `server-multitenant.toml` are +the multi-tenant deployments), and correctness rests on primitives that work +across processes: + +- CAS with backend-minted versions, so two hosts writing the same record cannot + lose an update (`record.rs:236-256`). +- Atomic path-local sequence reservation, so two hosts appending to one thread + get distinct sequences. +- Runner leases with lease tokens and heartbeats + (`DEFAULT_TURN_RUNNER_HEARTBEAT_INTERVAL = 5s`, + `DEFAULT_TURN_RUNNER_POLL_INTERVAL = 200ms`, + `crates/app/ironclaw_composition/src/runtime_input.rs`), where "Heartbeats + only renew metadata for matching, unexpired runner ID/lease token" + (`turn-persistence.md:89`), and liveness decisions must use durable lease + metadata rather than one event per heartbeat (`turn-persistence.md:90`). +- A durable active-thread lock, so crash detection is a lease-expiry + reconciliation rather than a stale-PID heuristic. + +The local-disk backend is treated as the constrained case rather than the +reference case: it must fail closed on the CAS gate, and where it cannot serve +an operation (`ReserveSeq`) the domain layer degrades to the CAS fallback. + +## Interop with foreign session stores + +No. IronClaw does not discover, import, or resume any other product's session +store, and there is no converter. The only foreign-agent artifact in the tree is +a legacy pre-Reborn sandbox path that persisted streamed events from an external +coding agent for job-detail replay +(`migrations/V5__claude_code.sql`: a `claude_code_events` table keyed by +`job_id`, alongside `agent_jobs.job_mode`). That is job telemetry for a sandbox +runner, not a session transcript store, and it is not part of the Reborn +transcript boundary. + +## What this implies for our Session Store (our inference) + +Our reading: in IronClaw, a stored session is **a scope-owned set of +individually versioned records under a virtual path prefix**: one metadata +record holding a durable sequence counter, one record per message, one record +per summary artifact, plus rebuildable index projections, where every write +carries a compare-and-swap precondition and every read is filtered through the +caller's exact scope. It is deliberately *not* an append-only log: history is +mutable in place under CAS (draft → finalize, status transitions, redaction), +and the durable record is expected to shrink under a redaction obligation. + +Five things this changes or sharpens for our design. + +1. **A real expected-version write boundary is achievable without event + sourcing, and IronClaw is the corpus's proof.** `CasExpectation::Version` + with backend-minted, unforgeable `RecordVersion` tokens and a bounded retry + loop gives lost-update safety at the store's write boundary. Our synthesis + previously observed that no product implemented optimistic concurrency at + that boundary; IronClaw does, and it does so while keeping the store + mutable. Worth noting that CAS-per-record is *weaker* than a per-aggregate + expected-sequence append: it protects each record, not the invariant across + records, which is exactly why IronClaw needs a *separate* active-run lock at + the turn level. If our Session Store is a decider aggregate with an + expected-version append, we get both properties from one mechanism, and that + is a concrete argument for our chosen shape rather than against it. +2. **Separate the lifecycle log from the transcript log, and enforce the + separation as a redaction rule.** IronClaw's strongest structural idea is + that "what ran" (leases, checkpoints, admission, idempotency) and "what was + said" (messages) are different stores with a stated one-way information flow: + turn records hold refs and metadata only, never prompt or assistant content. + That gives them a lifecycle log they can retain, project, and replay freely + without it becoming a shadow copy of the transcript, and it makes transcript + redaction tractable because there is only one place content lives. Our + event-sourced Session Store should adopt the same rule explicitly: lifecycle + events carry references, content lives in exactly one place. +3. **Redaction is the requirement that punishes append-only designs, and + IronClaw shows the cost of answering it destructively.** They satisfy + redaction by destroying content in place and propagating removal to every + derived copy as a hard obligation (the derived title is cleared + non-best-effort; summary content is replaced with `[redacted]`). We cannot + copy that directly on an append-only log, so we need the equivalent + guarantee by other means: content addressed indirectly so a single + crypto-erase or payload delete satisfies redaction across all events that + reference it, plus the same non-best-effort propagation rule for + projections. Any projection holding a copy of content is a redaction + liability, and IronClaw's `derived_title` handling is the pattern to follow. +4. **Idempotency records must be owned by the aggregate they protect.** + IronClaw's inbound dedup by SHA-256 of `(scope, source_binding_id, + external_event_id)` is the right key shape, and the fact that those records + live outside the thread root, survive thread deletion, and have no sweeper is + a defect we should avoid by construction: dedup state belongs inside the + aggregate's key space so deletion and retention cascade to it. It also argues + for our dedup key to be part of the appended event rather than a sidecar, + which an event-sourced design makes natural. +5. **Cascade semantics for children are answerable, and the answer is a durable + tombstone.** IronClaw links children at the *run* level + (`parent_run_id`, `spawn_tree_root_run_id`), bounds trees with an atomic + descendant reservation, and records a `SubagentResultTombstone` when a parent + cancel discards a completed child's result. That is the closest thing in the + corpus to closing the subagent-cascade gap, and it is a good model for us: + record the discard, do not orphan it. What IronClaw does *not* answer is + transcript-level cascade on parent delete, which leaves that half of the gap + open for us too. + +One negative result is also useful: IronClaw ships no rewind, no fork, no +branch, and no message edit, and pays no visible architectural cost for their +absence. Combined with the rest of the corpus, that is evidence that fork and +rewind are product features to be scheduled on demand, not properties the +durable model must be shaped around from day one. + +## Open questions + +- **Format versioning.** There is no schema-version field on the thread, + message, or summary record, and no documented ratchet direction. How is a + breaking transcript-shape change intended to roll out, given the + `next_sequence` precedent of pinning old threads to old behavior forever? +- **Idempotency-record lifecycle.** Who deletes + `/threads/idempotency/.json`? We found no sweeper, no TTL, and no + cascade from `delete_thread`. Is unbounded growth accepted, or is a sweeper + planned elsewhere? +- **Parent-delete cascade for subagent threads.** Child threads are siblings + linked only at the run level. What is the intended behavior for a child's + transcript when the parent thread is deleted, and is there any enumeration + path from parent thread to child threads at all? +- **Retention ownership for transcripts.** Nothing enforces transcript + retention. Is that intended to be a deployment concern (SQL-side policy on + `root_filesystem_entries`), a future product feature, or a deliberate + never-delete stance? +- **`RecoveryRequired` status.** `turn-persistence.md:91` and + `Architecture.md:578-579` disagree on whether an expired lease is terminal. + Which document reflects the current code path in every backend, and is + `turn-persistence.md` simply stale? +- **Cross-scope thread movement.** `ensure_thread` rejects a scope/thread + mismatch with `ThreadScopeMismatch` (`filesystem_service.rs:1320`), and scope + is baked into the path. Is re-parenting a thread (new project, new owner) + supported at all, or is it a copy-and-abandon operation? +- **Sequence gaps.** A reserved sequence whose message write then fails leaves a + hole in the sequence space. Do readers, the ordered index, and summary-range + validation all treat gaps as benign, and is that stated anywhere as a + contract? +- **Multi-writer transcript ordering under steering.** With `Queued` messages + and a single active run per `TurnScope`, what guarantees the model sees queued + user messages in accept order after a busy period, given that + `load_context_window` selects by sequence over the latest N? diff --git a/docs/research/session-store/synthesis.md b/docs/research/session-store/synthesis.md index ae76927be..7f827b7ab 100644 --- a/docs/research/session-store/synthesis.md +++ b/docs/research/session-store/synthesis.md @@ -1,7 +1,7 @@ # Synthesis: what the industry means by a "stored session" Part of Session Store Research. -Nine product dossiers, one question: when an agent product persists, +Every product dossier, one question: when an agent product persists, resumes, lists, and retires a session, what does it actually keep on disk and how close is that shape to an append-only log with derived projections? Purpose: extract the invariant core our own event-sourced @@ -28,6 +28,18 @@ lack. [LangGraph](./products/langgraph/index.md) is the odd one out structurally: its unit is a parent-linked chain of immutable state snapshots, not a message transcript, and it is the second cleanest event-sourcing analog in the corpus after T3 Code. +[IronClaw](./products/ironclaw/index.md) sits off the spectrum's midpoint rather +than along it: its durable session is neither one log nor one mutable row +but a **set of individually compare-and-swap-versioned records** at virtual +paths (one per message, one per summary artifact, one thread metadata +record holding a durable sequence counter), which makes it the only product +in the corpus with a real expected-version precondition on the write +boundary *and* a mutable history. + +> IronClaw was researched and added after this synthesis was first frozen. +> Its evidence revised Convergence #8 (optimistic concurrency), Divergence +> A (the central axis), Divergence F (multi-host), and Design decisions 3 +> and 6; those revisions are marked inline. Every other claim held. ## Convergence @@ -45,6 +57,10 @@ durable source of truth." [T3 Code](./products/t3code/index.md) and Even [Goose](./products/goose/index.md), the corpus's most mutable store, keeps pre-compaction turns as `agent_invisible` rows rather than deleting them. +[IronClaw](./products/ironclaw/index.md) holds the line in one method: summary +substitution happens only inside `load_context_window`, while +`list_thread_history` returns the original messages *and* the summary +artifacts that stand in for them, so only the assembled prompt is lossy. **2. JSONL append-only transcripts are the majority default for CLI products, and the append discipline is remarkably specific.** [Claude Agent @@ -73,6 +89,15 @@ mutation)." OpenCode's v2 `revert.commit` truncates only the *projection*; "the underlying event rows are not deleted." The two exceptions prove the axis: Goose's rewind is "a destructive delete" (`truncate_conversation`), and Hermes's is an in-place flag flip (`active=0`). +[IronClaw](./products/ironclaw/index.md) is a third kind of exception, and a +useful one: it ships **no rewind at all** (no rewind, fork, branch, or +message edit exists in its transcript trait or design docs), and its single +retroactive operation is deliberately destructive: `redact_message` nulls +content, attachments, and the provider-call payload in place, leaving only a +`redaction_ref`. A product can therefore decline the entire rewind axis and +pay no visible architectural cost, which reframes rewind as a schedulable +product feature rather than a property the durable model must be shaped +around. **4. Compaction is universally an upstream/agent-loop concern that the store merely records, never triggers or understands.** [LangGraph](./products/langgraph/index.md): @@ -84,7 +109,14 @@ compaction produces "another appended entry," an `isCompactSummary` marker. upstream of the store... but leaves a durable marker in the log." Even [Goose](./products/goose/index.md), which rewrites rows for compaction, treats the summarization call itself as an agent-loop decision the store just -persists the result of. +persists the result of. The one partial counterexample is +[IronClaw](./products/ironclaw/index.md), where compaction is explicitly *not* the +loop's business: it is host-managed and treated as a security boundary, +"Host-managed compaction is a typed retention boundary", with secret-leak +scanning (`Block`/`Redact`/`Warn`) inside the compaction path, a rescan +before both inference and persistence, and a fail-closed on residual +matches. The store still only records the artifact, but *who decides to +compact* moves below the loop rather than above it. **5. Fork/branch always mints a new identity; nobody reuses the source session id.** [Claude Agent SDK](./products/claude-agent-sdk/index.md)'s @@ -99,6 +131,9 @@ shared-prefix reference." [Goose](./products/goose/index.md) mints a fresh `YYYYMMDD_N` id via `copy_session`. Only [LangGraph](./products/langgraph/index.md) gets a genuinely cheap fork, because content-addressed channel blobs are shared by reference across the copied chain. +[IronClaw](./products/ironclaw/index.md) abstains: the only `Fork` in the tree is a +subagent context-seed mode that is "enum variant reserved, unimplemented" +and denied at runtime if requested. **6. Subagents are (almost) always a sibling stream/session linked by a parent pointer, never entries inlined in the parent's transcript.** @@ -114,7 +149,14 @@ crash-safe result reconciliation. The sole structural exception is [Claude Agent SDK](./products/claude-agent-sdk/index.md), which nests subagent `.jsonl` files physically *inside* the parent's session directory rather than as a database-level sibling, still a separate transcript, just a -different addressing scheme. +different addressing scheme. [IronClaw](./products/ironclaw/index.md) agrees on +the sibling shape but moves the pointer: the child gets a fresh +`thread_id` under the parent's exact tenant/agent/project/owner scope, and +the lineage (`parent_run_id`, `subagent_depth`, +`spawn_tree_root_run_id`) lives on the **run** record, not the thread. The +transcript store does not know it is holding a child session at all, which +is the cleanest separation in the corpus and also the reason IronClaw has no +parent-to-child enumeration path. **7. Cascade-on-delete for subagents is inconsistent and mostly unhandled, and nobody has a clean answer.** [Codex CLI](./products/codex-cli/index.md): "a @@ -127,10 +169,16 @@ found... children keep their `parentThreadId` and would be orphaned." [Grok Build](./products/grok-build/index.md): "No GC for orphaned subagent session directories was found." This is a convergence in the sense that every product that has subagents has the *same unresolved gap*. - -**8. No product implements true optimistic-concurrency control at the -store's write boundary, and the exceptions that come closest are the two -purest event-sourced designs.** [Claude Agent SDK](./products/claude-agent-sdk/index.md): +[IronClaw](./products/ironclaw/index.md) is the first product in the corpus to +close *half* of it: a parent cancel that discards an already-finished child +writes a durable `SubagentResultTombstone { child_run_id, disposition: +"discarded_by_parent_cancel", terminal_status }`, so a discarded child is a +recorded fact rather than an orphan. Parent *delete* cascade is still +unspecified there, so the gap narrows rather than closes. + +**8. Optimistic concurrency at the store's write boundary is rare, and the +one product that implements it fully does so *without* event sourcing.** +Revised after IronClaw. [Claude Agent SDK](./products/claude-agent-sdk/index.md): "there is no expected-position precondition on `append`." [Goose](./products/goose/index.md): "no expected-version precondition anywhere; there is no CAS." [Hermes](./products/hermes-agent/index.md): "there is no optimistic-concurrency / expected-position precondition @@ -144,6 +192,21 @@ conflict-detection teeth; [LangGraph](./products/langgraph/index.md)'s unique `(thread_id, checkpoint_id)` key is unrelated to conflict detection, its own dossier flags "no expected-version/OCC in the OSS savers" and notes `put` is last-write-wins on a given checkpoint id. +[IronClaw](./products/ironclaw/index.md) is the counterexample that forced this +convergence to be rewritten: every transcript write carries a +`CasExpectation` (`Absent` for creates, `Version(RecordVersion)` for +read-modify-write, `Any` reserved for admin backfills), the version token is +backend-minted and unforgeable ("Consumers obtain versions only by reading +existing entries — they cannot fabricate one"), and the retry policy is +explicit (`FILESYSTEM_CAS_RETRIES = 8`, then a hard error). Its substrate +contract makes CAS the floor rather than an optimization: "Stores must always +work with CAS (`put` + `CasExpectation::Version`) as the floor." Two caveats +keep this from being a free win. First, per-record CAS protects each record, +not an invariant across records, which is exactly why IronClaw still needs a +separate durable per-thread active-run lock and an atomic sequence +reservation; a per-aggregate expected-sequence append gets all three +properties from one mechanism. Second, CAS is a lost-update defense on a +mutable store, not an immutability guarantee. ## Divergence @@ -162,7 +225,16 @@ plus a pid registry rather than a store-level contract (Grok Build). Mutable row: [Goose](./products/goose/index.md) ("a mutable row plus an in-place-editable ordered message table... explicitly not session-as-log") and [Hermes](./products/hermes-agent/index.md) ("session-as-mutable-relational- -record... the least event-sourced of the products studied"). **Our +record... the least event-sourced of the products studied"). A third pole, +added after IronClaw: **CAS-versioned record per message** +([IronClaw](./products/ironclaw/index.md)), where history is mutable in place +(draft → finalize, status transitions, redaction) but every mutation +requires the reader's version token, and creates require +`CasExpectation::Absent`. That combination is worth naming because it +separates two properties the rest of the corpus conflates: append-only-ness +(immutability of what was written) and lost-update safety (no writer +silently clobbers another). IronClaw buys the second without the first. +**Our service must decide: is the append-only guarantee enforced by the store (reject any operation that isn't an append), or merely a convention the caller can violate?** T3 Code and OpenCode enforce it structurally (no @@ -194,6 +266,16 @@ sequence column carry order?** The two cleanest event-sourced designs (T3 Code, OpenCode) use opaque ids for identity and a *separate* strictly monotonic sequence for order; they do not conflate the two concerns the way UUIDv7-as-directory-name products do. +[IronClaw](./products/ironclaw/index.md) lands on the same answer from a different +direction: `ThreadId` is a *validated string* (≤256 bytes, no path +separators, no control characters, no reserved `__ironclaw_` prefix, because +the id becomes a path segment) that the caller may supply and the service +mints as UUIDv4 only when absent, while order comes from a durable per-thread +`u64` sequence allocated either by an atomic path-local counter row or by a +CAS loop on the thread record. Its migration note is the best argument in the +corpus for keeping identity and order separate: switching an existing thread +onto the faster counter "would restart at 1 and collide with messages already +at sequences 1..N", so "No thread ever switches counters mid-stream." **C. Scope of the store: per-project directory vs single global database.** Directory-per-project, no cross-project store: [Claude Agent @@ -216,6 +298,18 @@ on a plain column; OpenCode is the middle case, still appending a `session.next.moved` event that projects into `directory`/`path`/ `workspace_id` in the same transaction, so relocation stays cheap and non-migratory without becoming a bare out-of-band UPDATE). +[IronClaw](./products/ironclaw/index.md) dissolves the question instead of +answering it: no working directory, cwd hash, or worktree appears in the key +at all. The path prefix is a *logical* scope of tenant, agent, optional +project, owner, and mission over a virtual filesystem that is usually SQL rows, +so there is no relocation to reconcile, and scoping doubles as an +authorization boundary (listing "MUST scope the listing by `owner_user_id` +[...] otherwise a caller could enumerate threads owned by other users in the +same `(tenant, agent, project)` triple"; reads return the same +`UnknownThread` for absent and cross-scope threads "so callers cannot use the +response as an existence oracle"). The cost is that scope is baked into the +path: re-parenting a thread to a new project or owner has no supported +operation. **D. Compaction's durable shape: in-place row rewrite vs external snapshot file vs pure append marker.** Rewrite in place: [Goose](./products/goose/index.md) @@ -230,6 +324,15 @@ closed if the file is missing). Pure append, no external file needed: entry in the same log), [Codex CLI](./products/codex-cli/index.md) (`Compacted` item with `replacement_history` inline), [T3 Code](./products/t3code/index.md) (no compaction of the log at all, it is unbounded and grows forever). +Sibling record in the same store, addressed by sequence range: +[IronClaw](./products/ironclaw/index.md) writes a `SummaryArtifact { start_sequence, +end_sequence, summary_kind, content, model_context_policy }` at +`summaries/.json` with `CasExpectation::Absent`, validates that +both range endpoints exist as real messages, and makes replay +idempotent-by-content (an identical re-compaction returns the existing +artifact; a different overlapping range is rejected with +`OverlappingSummaryRange`, and only `ReplaceRangeWhenSelected` summaries are +overlap-checked at all). **Divergence to resolve: does a compaction boundary require a sidecar artifact recoverable independently of the log (Grok Build's model, with an explicit fail-closed on missing sidecar), or is a same-stream marker @@ -250,10 +353,22 @@ CLI (`cleanupPeriodDays`, default 30), [Gemini CLI](./products/gemini-cli/index. retention story at all, log grows forever: [T3 Code](./products/t3code/index.md) ("no retention or log-truncation/snapshotting, the log grows unbounded") and [OpenCode](./products/opencode/index.md) ("none found... the log is retained -indefinitely"). **The two purest event-sourced designs are also the two -with zero retention story**, an event-sourced Session Store gets rewind -and audit for free but inherits an explicit obligation to design retention -deliberately, since nothing in the pattern forces it. +indefinitely"), and [IronClaw](./products/ironclaw/index.md), which has no +transcript TTL, lifecycle policy, or sweep, and bounds *reads* (page-size +caps, byte budgets, a 100k index-row ceiling) instead of stored history. Its +only retention rule lives on the run-lifecycle store, not the transcript: +released admission-reservation evidence is kept "only while the corresponding +terminal run remains within the bounded terminal-record retention window", +because "active capacity accounting must not scan unbounded released +history." IronClaw also supplies the corpus's clearest example of what +un-owned retention costs: its inbound-idempotency records live at +`/threads/idempotency/.json`, *outside* any thread root, so +`delete_thread` leaves them behind pointing at a deleted message, with no +sweeper found. **Retention is unowned in the majority of the corpus, and +dedup state that lives outside the aggregate's key space is where the cost +shows up first**, an event-sourced Session Store gets rewind and audit for +free but inherits an explicit obligation to design retention deliberately, +since nothing in the pattern forces it. **F. Multi-host / multi-writer posture.** Single-host by design, no coordination: [Codex CLI](./products/codex-cli/index.md), [Gemini CLI](./products/gemini-cli/index.md), @@ -278,10 +393,26 @@ high-water history-fetch API, "a real distributed event-sync design layered on the same append-only log"). Multi-host via a shared database instance: [LangGraph](./products/langgraph/index.md) (Postgres backend, content-addressed blob upserts are conflict-free by -construction). **This is the widest-open axis**: only OpenCode has -actually solved cross-host replication of an event-sourced session on top -of an ownership-claim protocol; everyone else either assumes single-host or -punts the problem to the storage substrate. +construction). Multi-host as the *default* case, with the local filesystem +treated as the constrained one: [IronClaw](./products/ironclaw/index.md), added +after the first freeze. Nothing in its design assumes a shared filesystem: +backends include libSQL and Postgres, and correctness rests on primitives +that work across processes: CAS with backend-minted versions, atomic +path-local sequence reservation, runner leases with lease tokens and +heartbeats ("Heartbeats only renew metadata for matching, unexpired runner ID/ +lease token"; liveness must use durable lease metadata rather than one event +per heartbeat), and a durable active-thread lock so crash detection is +lease-expiry reconciliation rather than a stale-PID heuristic. Its recovery +stance is the sharpest in the corpus, expiry being terminal rather than +resumable: +"Reborn does not automatically retry uncertain side-effecting work after a +lost lease — expiry is terminal, and the user resubmits explicitly." +**This axis is narrower than it was**: two products have now designed it +deliberately rather than assumed it away, and they chose different +mechanisms. OpenCode chose an ownership-claim replication protocol over an +append log, IronClaw a lease-plus-lock coordination plane over a CAS store; +everyone else either assumes single-host or punts the problem to the storage +substrate. **G. What "the store" persists vs what it treats as opaque.** Fully opaque entries, store is a pure byte-transport: [Claude Agent SDK](./products/claude-agent-sdk/index.md) @@ -292,7 +423,16 @@ through Effect Schema on both append and read"), [T3 Code](./products/t3code/ind (same, via Effect Schema, plus derived `actor_kind`). Partially parsed, targeted introspection: [Goose](./products/goose/index.md) ("neither fully opaque nor a normalized schema, JSON columns with targeted introspection" -via `json_extract`/`json_each`). **Divergence: does the Session Store +via `json_extract`/`json_each`). Typed at the domain layer, opaque at the +substrate layer, with a declared projection between them: +[IronClaw](./products/ironclaw/index.md), where the message is a strongly typed +record (`sequence`, `kind`, `status`, `turn_run_id`, `redaction_ref`) that the +domain store interprets for every query, while the storage backend is +forbidden to parse it: "Backends never look inside `body` for indexing; +everything queryable lives in `indexed`", which is what lets one contract be +served portably by libSQL, Postgres, local files, and HSM-backed mounts. That +split is worth stealing independently of the rest of its design. +**Divergence: does the Session Store validate event payloads against a schema at the storage boundary, or does it store bytes and leave validation to the caller?** The schema-validating designs (T3 Code, OpenCode) get validated event shapes at the storage @@ -312,6 +452,7 @@ application. | session-as-log-of-immutable-snapshots (event-sourcing-adjacent, unit is a state chain not a message stream) | LangGraph | | session-as-log with looser discipline (append-only JSONL, no formal OCC contract; projector-contract rigor varies -- Codex CLI's SQLite projection has an explicit rebuild/read-repair cursor) | Claude Agent SDK / Claude Code, Codex CLI, Gemini CLI, Grok Build | | session-as-directory (path/filename encodes identity; scope encoding varies -- Codex CLI's path is time-sharded only, with cwd scope applied as a query-time filter, not a path segment) | Claude Agent SDK, Codex CLI, Gemini CLI, Grok Build, OpenCode (legacy) | +| session-as-record-set (one CAS-versioned record per message/summary under a scoped virtual path; mutable in place but never without an expected-version precondition) | IronClaw | | session-as-row (mutable, single record + child table) | Goose, Hermes | | session-as-mutable-relational-record (flag-mutation instead of events) | Hermes (Goose achieves a similar visibility effect only via a full delete+re-insert rewrite of the message table, not an in-place flag mutation) | | session-as-document (whole-file rewrite, last-write-wins per id) | Gemini CLI (legacy `.json`) | @@ -330,6 +471,7 @@ the log is the file, and the file's path is the addressing scheme). | [Goose](./products/goose/index.md) | mutable SQLite row + message table | the `sessions`/`messages` rows themselves | server date+counter `YYYYMMDD_N`; global DB, no path key | `replace_conversation`: DELETE all rows, re-INSERT with visibility flags | `truncate_conversation`, a destructive delete; fork = `copy_session`, full physical copy | low; explicitly "not session-as-log" | | [Grok Build](./products/grok-build/index.md) | append-only JSONL (`updates.jsonl`) + derived caches/index | `updates.jsonl`; cache/summary/FTS all rebuildable | server UUIDv7 session id; path = `sessions/{encoded_cwd}/{id}/` | append marker (`CompactionCheckpoint`) plus external snapshot file, fails closed if missing | `RewindMarker` + dead-branch-filter replay; fork = copy files + lineage fields | high; "event sourcing on the filesystem in all but name" | | [Hermes](./products/hermes-agent/index.md) | mutable SQLite row + message table | the `sessions`/`messages` rows | client `{timestamp}_{6hex}` id; global per-profile DB | `active=0, compacted=1` in-place flag flip, content-preserving | `rewind_to_message`: soft-delete via flag flip (reversible); fork = `/branch`, full row copy | low; "least event-sourced of the products studied" | +| [IronClaw](./products/ironclaw/index.md) *(added post-synthesis)* | set of CAS-versioned records under a scoped virtual path (thread.json + one file per message + one per summary) | the per-record JSON entries; thread/message/ordered indexes are declared rebuildable projections | caller-suppliable validated `ThreadId` (UUIDv4 when absent) + durable per-thread `u64` sequence; path = scope axes (tenant/agent/project/owner/mission) | sibling `SummaryArtifact` record over a `[start_sequence, end_sequence]` range with `ReplaceRangeWhenSelected`; messages untouched | none of either; only `redact_message`, a destructive in-place erase | low as a log, highest in corpus for write-boundary OCC (`CasExpectation::Version`, 8 retries) | | [LangGraph](./products/langgraph/index.md) | parent-linked chain of immutable state snapshots | the `checkpoints` table; channel values content-addressed by `(channel, version)` | caller-supplied `thread_id`; checkpoint id = UUID6 | no store-level compaction; `prune`/shallow-saver drop history, `DeltaChannel` snapshots for large channels | rewind = select an older checkpoint (nothing destroyed); fork = new checkpoint sharing ancestor blobs, `copy_thread` | very high; "materially closer to event-sourcing than the transcript products" | | [OpenCode](./products/opencode/index.md) | append-only SQLite event log (v2) / mutable JSON-per-path store (legacy) | the `event` table, keyed `(aggregate_id, seq)` | client-minted ULID-like `ses_`/`evt_` ids; per-aggregate `seq` | in-log `compaction.ended` event; model-visible view folds from latest compaction seq | `revert.commit` truncates only the projection, event rows kept; no first-class fork found | very high; "unambiguously session-as-log (event-sourced)" | | [T3 Code](./products/t3code/index.md) | append-only SQLite event log | the `orchestration_events` table, keyed `(aggregate_kind, stream_id, stream_version)` | client-supplied `threadId`; server UUIDv4 `eventId`; global `sequence` + per-stream `stream_version` | none; log is never compacted, only view-side caps | `thread.reverted` event filters the projection, log kept; fork = new stream via `ThreadForkService`, O(history) copy | highest in corpus; "the corpus's cleanest event-sourced example" | @@ -352,7 +494,17 @@ each with the industry's answer where one exists: (`replace_conversation`), Hermes via both a destructive DELETE+re-INSERT (`replace_messages`, used by /retry, /undo, /compress) and a non-destructive flag-flip (`archive_and_compact`) -- both flagged in - their own dossiers as crash-risk and history-loss hazards. + their own dossiers as crash-risk and history-loss hazards. The + requirement that most punishes this decision is redaction, and IronClaw + shows the price of answering it destructively: it erases content in place + and treats propagation to every derived copy as a hard obligation (the + cached sidebar title is cleared non-best-effort; summary content becomes + `[redacted]`). An append-only store must reach the same guarantee by + indirection, with content referenced rather than inlined so that one + payload delete or crypto-erase satisfies redaction across every event that + points at it, and must apply IronClaw's non-best-effort rule to + projections, since any projection holding a copy of content is a + redaction liability. 2. **Separate identity from order: an opaque event/session id for addressing, a strictly monotonic per-aggregate sequence for ordering.** @@ -364,14 +516,22 @@ each with the industry's answer where one exists: 3. **Require an expected-version precondition on every append (real optimistic concurrency), not just a single-writer assumption.** - Industry's answer: only OpenCode (explicit "Sequence mismatch" check on - replay) enforces a real caller-supplied precondition; T3 Code gets - OCC-like protection only implicitly, via a single-writer command queue - combined with a unique `(aggregate_kind, stream_id, stream_version)` - index, with no caller-supplied expected-version check at all; the rest - (Claude Agent SDK, Goose, Hermes, Gemini CLI) admit they have none and - rely on tolerated multi-writer interleaving or a social single-writer - convention. + Industry's answer, revised after IronClaw: IronClaw is the strongest + precedent, and notably not an event-sourced one: every transcript write + carries a `CasExpectation` against a backend-minted, unforgeable + `RecordVersion`, with CAS declared the portable floor beneath an optional + transaction API. OpenCode (explicit "Sequence mismatch" check on replay) + enforces a real caller-supplied precondition on an append log; T3 Code + gets OCC-like protection only implicitly, via a single-writer command + queue combined with a unique `(aggregate_kind, stream_id, + stream_version)` index, with no caller-supplied expected-version check at + all; the rest (Claude Agent SDK, Goose, Hermes, Gemini CLI) admit they + have none and rely on tolerated multi-writer interleaving or a social + single-writer convention. The lesson to carry: per-record CAS is not + equivalent to a per-aggregate expected-sequence append. IronClaw needs + *three* mechanisms (record CAS, atomic sequence reservation, active-run + lock) to get what one expected-version append on a decider aggregate + gives us, which is an argument for our shape rather than against it. 4. **Compaction is upstream: the store persists an event carrying the summary/replacement content, it does not trigger, understand, or @@ -396,27 +556,44 @@ each with the industry's answer where one exists: 6. **Subagents are sibling streams linked by a parent pointer, and cascade behavior on parent delete/rewind must be decided explicitly; the industry has not decided it.** Industry's answer: convergence on - sibling-stream-plus-pointer (Convergence #6) but zero consistent answer - on cascade (Convergence #7); every product either orphans children or - doesn't say. This is a genuine gap we get to close rather than copy. + sibling-stream-plus-pointer (Convergence #6) but almost no consistent + answer on cascade (Convergence #7); every product either orphans children + or doesn't say. Revised after IronClaw, which supplies the one partial + answer worth copying: link lineage on the *run*, bound the tree with an + atomic descendant reservation before anything is queued, and record a + discard as a durable `SubagentResultTombstone` naming the disposition + rather than dropping it. Transcript-level cascade on parent delete remains + a genuine gap we get to close rather than copy. 7. **Retention and log truncation are not solved by event-sourcing and - must be designed deliberately, not deferred.** Industry's answer: the - two purest event-sourced stores (T3 Code, OpenCode) have *no* retention - story at all and grow unbounded; the JSONL products that do have - retention treat it as an out-of-store caller responsibility (Claude + must be designed deliberately, not deferred. Dedup and idempotency state + must live inside the aggregate's key space so retention and deletion + cascade to it.** Industry's answer: the two purest event-sourced stores + (T3 Code, OpenCode) have *no* retention story at all and grow unbounded, + and IronClaw has none for transcripts either; the JSONL products that do + have retention treat it as an out-of-store caller responsibility (Claude Agent SDK's explicit "the SDK never deletes from your store on its own") with concrete but ad hoc defaults (30-90 days) enforced by a sweep, not - a store primitive. - -8. **Multi-host correctness needs an explicit ownership/claim protocol on - the write path, not an assumption of a shared filesystem.** Industry's - answer: only OpenCode has one (`events.claim`, `ownerID` + - `strictOwner` replay guard, per-aggregate high-water history sync); - everyone else either assumes single-host (Codex CLI, Gemini CLI, Goose, - T3 Code, Hermes) or pushes the problem to the adapter/storage substrate - (Claude Agent SDK's pluggable mirror, Grok Build's per-host SQLite - files, LangGraph's shared Postgres). + a store primitive. IronClaw supplies the concrete failure mode for the + second half: its SHA-256 inbound-dedup records sit outside the thread + root, survive `delete_thread`, and have no sweeper, so a redelivered + pre-deletion event replays as an accepted message that no longer exists. + Our dedup key belongs in the appended event, not in a sidecar. + +8. **Multi-host correctness needs an explicit coordination protocol on the + write path, not an assumption of a shared filesystem.** Industry's + answer, revised after IronClaw: two products have designed one, with + different mechanisms. OpenCode uses ownership-claim replication + (`events.claim`, `ownerID` + `strictOwner` replay guard, per-aggregate + high-water history sync). IronClaw uses a coordination plane instead: + record-level CAS, atomic sequence reservation, runner leases with tokens + and heartbeats, a durable active-run lock per thread scope, and terminal + (never auto-retried) lease expiry. Everyone else either assumes + single-host (Codex CLI, Gemini CLI, Goose, T3 Code, Hermes) or pushes the + problem to the adapter/storage substrate (Claude Agent SDK's pluggable + mirror, Grok Build's per-host SQLite files, LangGraph's shared Postgres). + Worth noting that IronClaw's two mechanisms are separable from its storage + model: the lease/lock plane would sit on top of an append log unchanged. 9. **Event payloads should be schema-validated at the storage boundary, not treated as opaque bytes.** Industry's answer: split. T3 Code and @@ -440,6 +617,23 @@ everywhere else with append-only JSONL, which means our job is not to invent the pattern but to close the two gaps nobody has closed yet: subagent cascade semantics and retention on an unbounded log. +Revised after IronClaw: those two gaps are now one and a half. IronClaw is +the one product that treats both as first-class design problems and gets +partway through each, from the opposite end of the spectrum. On cascade, it +bounds the subagent tree with an atomic descendant reservation taken before +any child is queued and records a discarded child result as a durable +`SubagentResultTombstone` naming the disposition, which is more than any +event-sourced product does, but it still leaves transcript-level cascade on +parent deletion unaddressed. On retention, it declares a typed redaction +boundary with a leak-scanning test rather than a growth policy, so +transcripts still grow without bound while lifecycle records get a bounded +terminal-retention window. IronClaw also demonstrates that the write-boundary +concurrency guarantee we want is achievable outside event sourcing, at the +cost of three coordinating mechanisms instead of one expected-version append. +That strengthens rather than weakens the case for our shape, and it means the +remaining novel work is transcript cascade on delete plus a real growth +policy. + ## Stage-two results, not yet absorbed above Everything above is frozen as decision-time input from nine dossiers. Sixteen @@ -514,4 +708,4 @@ side: `ProviderBlock` exists to absorb blocks the typed arms cannot model, and nothing in the corpus enumerates what would go through it, so whether seven arms are the right seven is still open. The queued stage three in the [backlog](./backlog.md) takes a provider rather than a product as its unit of -study for that reason. \ No newline at end of file +study for that reason. From 5560fc311a982a5b405bd918a064a8449040e72b Mon Sep 17 00:00:00 2001 From: Yordis Prieto Date: Thu, 6 Aug 2026 00:40:48 -0400 Subject: [PATCH 2/2] chore(docs): keep the corpus prose from drifting as dossiers land Signed-off-by: Yordis Prieto --- docs/research/agent-platform/synthesis.md | 4 +- docs/research/session-store/index.md | 22 +++++----- .../session-store/products/ironclaw/index.md | 5 ++- docs/research/session-store/synthesis.md | 41 +++++++++++-------- 4 files changed, 41 insertions(+), 31 deletions(-) diff --git a/docs/research/agent-platform/synthesis.md b/docs/research/agent-platform/synthesis.md index dfb0402be..29407975d 100644 --- a/docs/research/agent-platform/synthesis.md +++ b/docs/research/agent-platform/synthesis.md @@ -433,8 +433,8 @@ and the most successful designs are the ones that made those decisions explicit rather than inheriting them. Revised after IronClaw: the trio is a set of *roles*, not necessarily a set of -resources, and the eighteenth product is the one that proves it by shipping -without the first member. What that reframing buys us is a sharper test for +resources, and IronClaw is the one that proves it by shipping without the +first member. What that reframing buys us is a sharper test for our own design. Every property we are tempted to put on the agent definition should have to answer why it belongs to the agent rather than to the scope (authorization), the files (persona), or the run (resolved mechanism). The diff --git a/docs/research/session-store/index.md b/docs/research/session-store/index.md index d5da3c02f..86a330442 100644 --- a/docs/research/session-store/index.md +++ b/docs/research/session-store/index.md @@ -21,8 +21,8 @@ evidence rules behind every artifact remain reproducible. ## Status -Synthesis complete for the first nine products, and the decision record exists -as draft +Synthesis complete for the products it was frozen over, and the decision +record exists as draft [ADR#0035: Session Store as a Decider Aggregate on NATS JetStream](../../adr/0035-session-store-decider-aggregate.md). The fx artifacts (the dossier, the session detail JSON reference, and the comparison against our event catalog) were added after the synthesis and are @@ -30,13 +30,13 @@ not yet folded into it. The IronClaw dossier was also added after the synthesis was frozen, but its evidence is folded in and marked inline wherever it revised a frozen claim. -The corpus is being extended to twenty-eight products. The queue, its +The corpus is still being extended. The queue, its ordering, and the verification state of each candidate are in the [backlog](./backlog.md). fx was the reference implementation of the stage-two prompt; Cline, OpenHands, and Zed were the calibration batch that exercised it against fresh dossiers, and the remaining comparisons follow their shape. -Stage-two comparisons now exist for sixteen of those products and are listed +Stage-two comparisons exist for many of those products and are listed under their dossiers below. The synthesis's frozen decision-time text has not absorbed them, but its cross-corpus results are recorded in [a closing section](./synthesis.md#stage-two-results-not-yet-absorbed-above) @@ -55,13 +55,13 @@ made about it. The mechanical layer is only trustworthy when it is run over the whole corpus at once. Run per-artifact as each landed, it reported clean on files that were not: a sweep of every artifact with a local clone found flattened paths, wrong -crates, and bare basenames in six files that individual waves had already -passed. The current state of that sweep is 1912 citations, all resolved, no -missing file and no out-of-range line, across every product whose -source is checked out locally. The remaining dossiers (fx, Claude Agent SDK, -Codex CLI, Gemini CLI, Goose, Grok Build, Hermes, IronClaw, LangGraph, -OpenCode, T3 Code) predate that sweep and were verified by hand only, so they -carry weaker mechanical guarantees than the number above suggests. +crates, and bare basenames in files that individual waves had already passed. +That sweep now resolves every citation it covers, with no missing file and no +out-of-range line, across every product whose source is checked out locally. +It does not cover the dossiers that predate it, which were verified by hand +only and therefore carry weaker mechanical guarantees; a dossier is inside the +sweep only if its product's source is checked out locally, so the boundary is +the local clone rather than a list kept here by hand. The second layer is the one that earns its cost. It is what caught a migration ratchet described as content-hashed when the source compares stored text, a diff --git a/docs/research/session-store/products/ironclaw/index.md b/docs/research/session-store/products/ironclaw/index.md index e4d4ae1c0..cea052c49 100644 --- a/docs/research/session-store/products/ironclaw/index.md +++ b/docs/research/session-store/products/ironclaw/index.md @@ -298,8 +298,9 @@ pub trait SessionThreadService: Send + Sync { } ``` -Source: `crates/domains/ironclaw_threads/src/service.rs:22-373`. Thirty-two -methods, of which sixteen are required. Three properties are worth extracting: +Source: `crates/domains/ironclaw_threads/src/service.rs:22-373`. Most of the +trait ships fail-closed defaults, leaving a small required core. Three +properties are worth extracting: - **Fail-closed defaults over silent lies.** The default `list_threads_for_scope` returns `Backend(...)` deliberately, "so backends diff --git a/docs/research/session-store/synthesis.md b/docs/research/session-store/synthesis.md index 7f827b7ab..635238f9a 100644 --- a/docs/research/session-store/synthesis.md +++ b/docs/research/session-store/synthesis.md @@ -204,9 +204,11 @@ work with CAS (`put` + `CasExpectation::Version`) as the floor." Two caveats keep this from being a free win. First, per-record CAS protects each record, not an invariant across records, which is exactly why IronClaw still needs a separate durable per-thread active-run lock and an atomic sequence -reservation; a per-aggregate expected-sequence append gets all three -properties from one mechanism. Second, CAS is a lost-update defense on a -mutable store, not an immutability guarantee. +reservation; a per-aggregate expected-sequence append folds lost-update +safety and cross-record ordering into one mechanism, though run ownership +still has to be modeled as explicit lease transitions on the aggregate rather +than obtained for free. Second, CAS is a lost-update defense on a mutable +store, not an immutability guarantee. ## Divergence @@ -529,9 +531,13 @@ each with the industry's answer where one exists: have none and rely on tolerated multi-writer interleaving or a social single-writer convention. The lesson to carry: per-record CAS is not equivalent to a per-aggregate expected-sequence append. IronClaw needs - *three* mechanisms (record CAS, atomic sequence reservation, active-run - lock) to get what one expected-version append on a decider aggregate - gives us, which is an argument for our shape rather than against it. + record CAS plus an atomic sequence reservation to get the lost-update and + ordering guarantees one expected-version append on a decider aggregate + gives us, which is an argument for our shape rather than against it. Its + third mechanism, the active-run lock, is not something the append replaces: + lease ownership, heartbeat renewal, expiry, and admission before any + model or tool side effect have to become modeled transitions on the + aggregate, with the append enforcing them rather than supplying them. 4. **Compaction is upstream: the store persists an event carrying the summary/replacement content, it does not trigger, understand, or @@ -629,20 +635,23 @@ boundary with a leak-scanning test rather than a growth policy, so transcripts still grow without bound while lifecycle records get a bounded terminal-retention window. IronClaw also demonstrates that the write-boundary concurrency guarantee we want is achievable outside event sourcing, at the -cost of three coordinating mechanisms instead of one expected-version append. +cost of coordinating separate record, sequence, and lock mechanisms where an +expected-version append covers the lost-update and ordering half on its own. That strengthens rather than weakens the case for our shape, and it means the remaining novel work is transcript cascade on delete plus a real growth policy. ## Stage-two results, not yet absorbed above -Everything above is frozen as decision-time input from nine dossiers. Sixteen -stage-two comparisons have landed since, and this section records what they add -without rewriting the frozen text around it. Where the two disagree, the -comparisons are the newer reading and the ADR is authoritative over both. +Everything above is frozen as decision-time input from the dossiers that +existed when it was written. The stage-two comparisons landed after it, and +this section records what they add without rewriting the frozen text around +it. Where the two disagree, the comparisons are the newer reading and the ADR +is authoritative over both. -**The design mostly survives contact with the evidence.** Across the -comparisons' 55 numbered recommendations there are 45 blast-radius statements. +**The design mostly survives contact with the evidence.** Across the numbered +recommendations in the comparisons read for this pass, 55 of them, there are 45 +blast-radius statements. Eight mention breaking in any form: four are "do not do X later" guardrails against regressions we have not committed to (Cline on deterministic child ids, Letta on relaxing optimistic concurrency for an `At` transition, OpenHands on a @@ -676,7 +685,7 @@ into the best-supported precondition in the corpus: any second `trogon-decider` implementation must pass a shared conformance suite over every `WRITE_PRECONDITION` class before it ships. -**Convergence 8 above survives eighteen more products and gets sharper.** Even +**Convergence 8 above survives every product added since and gets sharper.** Even the two stores that do have optimistic concurrency put it in the wrong place. Letta version-checks exactly one ORM model, `Block`, which holds memory-configuration data, while the actual per-turn hot pointer @@ -701,8 +710,8 @@ product validates `SessionHidden`, `RedactionApplied`, or `ArtifactErased`, and it is a property of the sample rather than a weakness in the decision. **The message payload is documented per product and not at all per provider.** -Twenty-four of twenty-five dossiers carry an entry-structure section, and twelve -comparisons map the product's message type row by row against `CanonicalMessage` +All but one dossier carries an entry-structure section, and the comparisons that +go further map the product's message type row by row against `CanonicalMessage` and its seven-arm `ContentBlock` oneof. What no artifact covers is the provider side: `ProviderBlock` exists to absorb blocks the typed arms cannot model, and nothing in the corpus enumerates what would go through it, so whether seven arms