CL_EX: The Self-Architecting Distributed Agent System

[nshkrdotcom]

CL_EX Ecosystem Mapping

Component-by-Component Mapping to BEAM/Elixir + CL_EX Ecosystem Equivalents

Document 03 of the CL_EX Design Series
Date: 2026-02-02

---

Table of Contents

1. Component Mapping Table
2. Detailed Component Mappings
3. Dependency Graph
4. Heterogeneous Stack Diagrams
5. What's Missing
6. Generic Agentic Stack

---

1. Component Mapping Table

#	OpenClaw Component	Role	CL_EX Equivalent	Status	Hex Package
1	pi-agent-core	Agent execution, AgentTool, AgentMessage, streaming	cl_ex_agent (Jido.Agent-based runtime, OTP GenServer)	NEW	{:jido, "~> 1.0"}
2	pi-coding-agent (SessionManager, tools)	Session lifecycle, filesystem tools, skills	cl_ex_agent (sessions) + cl_ex_tools (Jido.Action-based tools)	NEW	--
3	pi-ai (Model API, streaming)	Model abstraction, API clients, completions	cl_ex_agent LLM layer via altar_ai + provider SDKs (claude_agent_sdk, codex_sdk, gemini_ex, ollixir)	ADAPT	{:altar_ai, "~> 0.1"}
4	pi-tui (Terminal UI)	Terminal UI components	owl + ratatouille + custom cl_ex_tui	ADAPT	{:owl, "~> 0.1"}, {:ratatouille, "~> 0.5"}
5	grammy (Telegram)	Telegram Bot API	nadia / telegex / ex_gram	ADAPT	{:ex_gram, "~> 0.52"}
6	@buape/carbon (Discord)	Discord framework	nostrum	ADAPT	{:nostrum, "~> 0.10"}
7	@slack/bolt (Slack)	Slack integration	slack_elixir / custom	ADAPT	{:slack_elixir, "~> 1.0"}
8	@whiskeysockets/baileys (WhatsApp)	WhatsApp Web protocol	None -- must wrap or port	NEW	--
9	playwright-core (Browser)	Browser automation	wallaby + playwright_elixir	ADAPT	{:wallaby, "~> 0.30"}
10	sqlite-vec (Vector search)	Vector search, embeddings	cl_ex_memory via Bedrock (primary), portfolio_core/portfolio_index for RAG, pgvector + ecto (fallback)	ADAPT	{:bedrock, "~> 0.1"}, {:pgvector, "~> 0.3"}
11	commander (CLI)	CLI argument parsing	Built-in OptionParser + owl	EXISTS	(stdlib)
12	hono (HTTP server)	HTTP server, webhooks	bandit + plug / phoenix	EXISTS	{:bandit, "~> 1.0"}, {:plug, "~> 1.16"}
13	croner (Cron)	Cron scheduling	quantum	EXISTS	{:quantum, "~> 3.5"}
14	dotenv (Env)	Environment variable loading	dotenvy	EXISTS	{:dotenvy, "~> 0.8"}
15	JSONL session files	Session transcript persistence	Bedrock (primary state store) with ETS cache; session management in cl_ex_agent	NEW	{:bedrock, "~> 0.1"}
16	Docker sandbox	Sandboxed code execution	cl_ex_sandbox (Docker via :erlexec + NIF, or Firecracker)	NEW	--
17	Plugin loader (jiti)	Dynamic plugin loading	Native BEAM code loading + cl_ex_plugins	EXISTS	(OTP native)
18	Config (JSON5 + Zod)	Configuration + validation	NimbleOptions + Application config (part of cl_ex core)	EXISTS	{:nimble_options, "~> 1.1"}

---

2. Detailed Component Mappings

2.1 pi-agent-core --> cl_ex_agent

OpenClaw Role: Core agent execution runtime. Defines AgentTool, AgentMessage, StreamFn, and event primitives. Manages the run/attempt lifecycle, streaming token delivery, and tool-call dispatch loop.

CL_EX Equivalent: cl_ex_agent -- a NEW OTP application built on Jido.Agent + Jido.AgentServer (GenServer-based agent runtime). See Doc 01 Section 3.4 and Doc 02 Section 3 for the canonical agent architecture.

Status: NEW

Integration Notes:

• Each agent instance is a Jido.AgentServer process (GenServer) wrapping a Jido.Agent struct. Agent state transitions (idle, thinking, tool_calling, streaming, error) are managed via Jido's directive-based effect model rather than explicit state machine states.
• Agent processes are registered in a Registry namespaced by {agent_id, session_key}. Session keys follow the canonical format: session:{identity_id}:{channel}:{session_id}.
• The run/attempt loop uses Jido's cmd/2 pipeline: each tool call is a Jido.Action dispatched via directives, each LLM response updates agent state immutably.
• AgentTool maps to Jido.Action with @callback run(params, context) :: {:ok, result} | {:error, %ClEx.Error{}}. Higher-level tool compositions use Jido.Skill.
• AgentMessage maps to a typed struct: %ClEx.Signal{role: :user | :assistant | :tool, content: term(), metadata: map()} -- the canonical internal envelope (see Doc 01).
• Streaming is handled via Process.send/2 -- the BEAM's message passing IS the streaming primitive. No need for StreamFn callbacks; subscribers simply receive messages.
• Event bus: Phoenix.PubSub (or :pg for pure OTP) for lifecycle events. Synapse workflow integration provides step-level retry and signal registry.
• Supervision tree: one DynamicSupervisor per node manages all agent processes. Crash recovery is automatic via OTP. State is persisted to Bedrock and restored on restart.

Key Elixir Advantage: The BEAM makes every agent a lightweight process (~2KB), enabling thousands of concurrent agents per node with preemptive scheduling. See Doc 04 for detailed comparison with OpenClaw's concurrency model.

defmodule MyApp.Agent do
  use Jido.Agent,
    name: "my_agent",
    description: "A CL_EX agent",
    actions: [ClEx.Tools.Read, ClEx.Tools.Write, ClEx.Tools.Search],
    schema: [
      model: [type: :string, default: "claude-sonnet-4-20250514"],
      workspace: [type: :string]
    ]

  # Jido agents are pure functional -- state transitions return
  # directives (effects) rather than performing side effects directly.
  # The AgentServer (GenServer) executes directives on behalf of the agent.
end

# Starting an agent (managed by cl_ex_agent supervisor):
{:ok, pid} = ClEx.Agent.Runtime.start_agent(%{
  id: "helper",
  module: MyApp.Agent,
  model: "claude-sonnet-4-20250514"
})

---

2.2 pi-coding-agent --> cl_ex_agent (sessions) + cl_ex_tools

OpenClaw Role: SessionManager handles JSONL transcript persistence, session keying, reset policies. Coding tools provide read, write, edit filesystem operations with policy enforcement.

CL_EX Equivalent:

• cl_ex_agent -- Session lifecycle management is embedded within the agent component (as ClEx.Agent.Session / ClEx.Agent.Session.Store). See Doc 02 Section 3.
• cl_ex_tools -- Tool behaviour and standard tool library, using Jido.Action as the tool primitive (NEW)

Status: NEW

Integration Notes:

• Session storage: Bedrock is the primary state store for all session data. Sessions are keyed by session:{identity_id}:{channel}:{session_id} (matching Doc 01's canonical key format). ETS serves as a hot cache with Bedrock as the durable backend.
• For single-node/embedded deployments without Bedrock: ETS + DETS provides a graceful degradation path.
• Reset policies: Implemented as configurable strategies via NimbleOptions: :manual, :age_based, :token_count, :message_count.
• Coding tools: Implemented as Jido.Action modules (e.g., ClEx.Tools.Read, ClEx.Tools.Write, ClEx.Tools.Edit). Elixir's File module provides all filesystem ops. The edit tool can use :binary.match/2 for fast string replacement. Sandbox policies enforced via cl_ex_security (GUARDRAIL) by checking paths against allowed directories before any operation.
• Session compaction: Transcript summarization via LLM call when token count exceeds threshold -- same pattern as OpenClaw but triggered by a GenServer timer. FlowStone can orchestrate the compaction pipeline with lineage tracking.

defmodule ClEx.Agent.Session.Store do
  @moduledoc "Bedrock-backed session store with ETS cache"

  def append(session_key, message) do
    # Write-through: ETS cache + Bedrock persistence
    :ets.insert(:cl_ex_sessions, {session_key, message})
    Bedrock.put("session:#{session_key}", message)
  end

  def load(session_key) do
    case :ets.lookup(:cl_ex_sessions, session_key) do
      [] -> Bedrock.get("session:#{session_key}")
      [{_, messages}] -> {:ok, messages}
    end
  end
end

---

2.3 pi-ai --> cl_ex_agent LLM layer (ADAPT ecosystem SDKs)

OpenClaw Role: Model abstraction layer. Provides unified API across OpenAI, Anthropic, Google, Bedrock, Ollama. Handles streaming completions, tool-call formatting, token counting, and fallback chains.

CL_EX Equivalent: The LLM abstraction lives within cl_ex_agent as ClEx.Agent.LLM (see Doc 02 Section 3). It is built on the ecosystem's own provider SDKs:

• altar_ai -- Multi-provider LLM abstraction with unified API
• claude_agent_sdk -- Anthropic Claude provider adapter
• codex_sdk -- OpenAI / Codex provider adapter
• gemini_ex -- Google Gemini provider adapter
• ollixir -- Ollama / local model provider adapter
• langchain (Elixir) -- Optional fallback for providers not yet covered by ecosystem SDKs

Status: ADAPT

Integration Notes:

• altar_ai provides the unified LLM interface with streaming, tool calls, and structured output. Provider-specific adapters (ClEx.Agent.LLM.Anthropic, ClEx.Agent.LLM.OpenAI, ClEx.Agent.LLM.Gemini, ClEx.Agent.LLM.Ollama) wrap the ecosystem SDKs.
• ClEx.LLM.Router handles ensemble routing via crucible_* modules for model selection, fallback chains, and load balancing across providers.
• Structured output validation uses Exdantic schemas (replacing Zod-based output parsing in OpenClaw).
• Rate limiting via Foundation's rate limiter. Token counting via provider SDK utilities.
• Telemetry integration via :telemetry for cost tracking; AITrace for LLM call observability.
• Streaming: Provider SDKs emit BEAM messages natively -- the subscribing agent process receives tokens directly. No callback adapter layer needed.
• langchain can be retained as an optional dependency for providers not yet supported by the ecosystem SDKs, but it is not the primary abstraction.

defmodule ClEx.Agent.LLM do
  @moduledoc "Unified LLM interface with provider routing and fallback chains"

  def chat(messages, opts \\ []) do
    chain = Keyword.get(opts, :fallback_chain, [opts[:model] || default_model()])

    Enum.reduce_while(chain, {:error, %ClEx.Error{type: :llm, reason: :all_models_failed}}, fn model, _acc ->
      adapter = adapter_for(model)
      case adapter.chat(messages, opts) do
        {:ok, response} -> {:halt, {:ok, response}}
        {:error, _reason} -> {:cont, {:error, %ClEx.Error{type: :llm, reason: :all_models_failed}}}
      end
    end)
  end

  defp adapter_for("claude-" <> _), do: ClEx.Agent.LLM.Anthropic
  defp adapter_for("gpt-" <> _), do: ClEx.Agent.LLM.OpenAI
  defp adapter_for("gemini-" <> _), do: ClEx.Agent.LLM.Gemini
  defp adapter_for(_), do: ClEx.Agent.LLM.Ollama
end

Hex Dependencies:

{:altar_ai, "~> 0.1"},            # Multi-provider LLM abstraction
{:claude_agent_sdk, "~> 0.1"},    # Anthropic provider
{:codex_sdk, "~> 0.1"},           # OpenAI provider
{:gemini_ex, "~> 0.1"},           # Google Gemini provider
{:ollixir, "~> 0.1"},             # Ollama / local models
{:langchain, "~> 0.3", optional: true},  # Optional fallback

---

2.4 pi-tui --> cl_ex_tui (ADAPT)

OpenClaw Role: Terminal UI for interactive agent sessions. Renders streaming output, tool call progress, thinking indicators, and markdown-formatted responses.

CL_EX Equivalent:

• ratatouille -- Termbox-based terminal UI framework for Elixir
• owl -- Terminal UI component library (progress bars, spinners, tables)
• cl_ex_tui -- NEW composition layer

Status: ADAPT

Integration Notes:

• ratatouille provides the Elm-architecture terminal UI framework (model-update-view). It handles raw terminal rendering, keyboard input, and layout.
• owl provides higher-level components: spinners, progress bars, tables, and live-updating output. Good for CLI-mode rendering.
• cl_ex_tui composes these into agent-specific views:
  • Streaming text panel (assistant output arriving token-by-token)
  • Tool call sidebar (shows active tool executions)
  • Session info bar (model, token count, session key)
  • Input area with history
• Alternative: ExTermbox for lower-level control, or simply use IO.ANSI + owl for a lighter approach.
• The BEAM's process model means the TUI is a separate process receiving messages from the agent -- no callback spaghetti.

---

2.5 grammy --> ExGram / Telegex (ADAPT)

OpenClaw Role: Telegram Bot API client. Handles long polling, webhook mode, inline keyboards, message formatting, media upload/download, and bot commands.

CL_EX Equivalent: ex_gram -- Telegram Bot API framework for Elixir.

Status: ADAPT

Integration Notes:

• ex_gram provides a full Telegram Bot API implementation with both long-polling and webhook modes.
• It includes middleware, command handling, and inline keyboard support.
• Adaptation needed: Wrap ex_gram in a ClEx.Transport.Telegram adapter that implements the ClEx.Transport.Channel behaviour (see Doc 02 Section 2):
  • Normalize inbound messages to %ClEx.Signal{}
  • Translate outbound %ClEx.Signal{} to Telegram API calls
  • Map Telegram threading (reply_to_message_id) to CL_EX session threading
  • Handle media attachments (photos, documents, voice)
• Alternative: telegex is another option with a more modern API but smaller community.
• nadia exists but is older and less maintained.

defmodule ClEx.Transport.Telegram do
  @behaviour ClEx.Transport.Channel

  @impl true
  def normalize(%ExGram.Model.Message{} = msg) do
    %ClEx.Signal{
      channel: :telegram,
      sender_id: "tg:#{msg.from.id}",
      group_id: if(msg.chat.type != "private", do: "tg:#{msg.chat.id}"),
      text: msg.text,
      thread_id: msg.message_thread_id,
      metadata: %{chat_type: msg.chat.type}
    }
  end

  @impl true
  def deliver(signal, opts) do
    ExGram.send_message(signal.metadata.chat_id, signal.content, opts)
  end
end

---

2.6 @buape/carbon --> Nostrum (ADAPT)

OpenClaw Role: Discord bot framework. Handles gateway connection, slash commands, message events, embeds, reactions, threads, and voice channel awareness.

CL_EX Equivalent: nostrum -- Discord API library for Elixir.

Status: ADAPT

Integration Notes:

• nostrum is the mature Discord library in Elixir. It provides:
  • Gateway WebSocket connection with automatic reconnection
  • Full REST API coverage
  • Built-in caching (ETS-backed) for guilds, channels, users
  • Consumer-based event handling
• Adaptation needed: Same pattern as Telegram -- wrap in ClEx.Transport.Discord:
  • Normalize Discord events to %ClEx.Signal{}
  • Handle Discord-specific features: embeds, slash commands, components (buttons, selects)
  • Map Discord threads to CL_EX sessions
  • Handle rate limiting (Nostrum does this internally)
• nostrum uses its own supervision tree and process structure -- fits naturally into the BEAM.

---

2.7 @slack/bolt --> Slack Elixir (ADAPT)

OpenClaw Role: Slack integration. Handles Events API, slash commands, interactive messages, modal dialogs, threading, and workspace management.

CL_EX Equivalent: slack_elixir or custom Plug-based integration.

Status: ADAPT

Integration Notes:

• The Elixir Slack ecosystem is less mature than Telegram/Discord. Options:
  • slack_elixir -- Basic Slack Web API + Events API
  • slack_web -- Auto-generated API client from Slack's OpenAPI spec
  • Custom: Use Plug to receive Events API webhooks + Req/Finch for API calls
• Recommendation: Build ClEx.Transport.Slack with Req for API calls and a Plug.Router for incoming webhooks. This gives full control and avoids dependency on under-maintained wrappers.
• Socket Mode support may need to be built (WebSocket connection to Slack instead of HTTP webhooks).

---

2.8 @whiskeysockets/baileys --> cl_ex_whatsapp (NEW)

OpenClaw Role: WhatsApp Web multi-device protocol implementation. Handles QR code authentication, message encryption (Signal protocol), media upload/download, group management, and presence updates.

CL_EX Equivalent: cl_ex_whatsapp -- Must be built or wrapped.

Status: NEW

Integration Notes:

• This is the biggest gap. Baileys is a complex reverse-engineered protocol implementation (~20k lines of TypeScript). There is no Elixir equivalent.
• Options:
  1. NIF/Port wrapper: Run a minimal Node.js Baileys instance as a Port or use Rustler to wrap a Rust WhatsApp library (e.g., whatsmeow in Go can be called via Port).
  2. Go port: whatsmeow is a mature Go implementation. Call it via Erlang ports or use go_erlang for tighter integration.
  3. Direct port: Long-term, port the Signal protocol implementation and WhatsApp Web protocol to Elixir. Massive effort.
  4. Cloud API: Use WhatsApp Business Cloud API (official REST API) instead of Web protocol. Much simpler but requires Business account and has message template restrictions.
• Recommendation for v1: Use WhatsApp Business Cloud API via Req HTTP client. For full feature parity with OpenClaw, wrap whatsmeow (Go) as an Erlang port.

---

2.9 playwright-core --> Wallaby / Playwright Elixir (ADAPT)

OpenClaw Role: Browser automation for web scraping, screenshot capture, and interactive browsing by agents.

CL_EX Equivalent:

• wallaby -- Browser automation testing framework (wraps ChromeDriver/Selenium)
• playwright_elixir -- Elixir bindings for Playwright (if available)
• Custom: Use Req + Floki for HTML scraping without a browser

Status: ADAPT

Integration Notes:

• wallaby is mature but oriented toward testing. Needs adaptation for agent-driven browsing.
• For headless browsing: Use Chrome DevTools Protocol (CDP) directly via WebSocket. Libraries like chrome_remote_interface exist.
• For simple scraping: Req + Floki (HTML parser) handles most cases without browser overhead.
• Recommendation: Layer approach:
  1. Light scraping: Req + Floki (no browser needed)
  2. JavaScript-rendered pages: CDP via WebSocket
  3. Full interaction: Playwright via Node.js port (same pattern as WhatsApp)

---

2.10 sqlite-vec --> cl_ex_memory (Bedrock + portfolio_core) (ADAPT)

OpenClaw Role: SQLite with vector search extension for semantic memory. Stores embeddings, enables cosine similarity search, and provides the memory/RAG backend.

CL_EX Equivalent:

• Bedrock -- Primary distributed state store for all memory data (see Doc 01)
• portfolio_core / portfolio_index -- RAG pipeline: chunking, indexing, and retrieval
• embed_ex -- Embedding generation (local or API-based)
• pgvector + ecto -- PostgreSQL vector extension (fallback for deployments without Bedrock)
• exqlite -- SQLite3 NIF bindings (single-node/embedded fallback)

Status: ADAPT

Integration Notes:

• Primary path (Bedrock): Bedrock provides the distributed KV store for memory entries with consistent reads across the cluster. Vector indices are managed via portfolio_index, which provides chunking, embedding, and semantic search over Bedrock-stored documents.
• RAG pipeline: portfolio_core handles the end-to-end RAG flow: document ingestion, chunking strategies, embedding via embed_ex, storage in Bedrock, and retrieval with reranking. This replaces the need to manually wire pgvector + Ecto for RAG.
• Fallback Option A (PostgreSQL): Use pgvector with Ecto. Better for teams with existing Postgres infrastructure who do not want the Bedrock dependency.
• Fallback Option B (SQLite): Use exqlite and load the sqlite-vec extension at runtime. Maintains parity with OpenClaw's approach for single-node embedded deployments.
• Embedding generation: Use embed_ex which wraps Bumblebee + EXLA for local embedding models, or calls OpenAI/Voyage embeddings API via Req.
• Recommendation: Default to Bedrock + portfolio_core for production. Fall back to pgvector + Ecto when Bedrock is unavailable. Use exqlite + sqlite-vec only for single-node/embedded deployments. Abstract behind ClEx.Memory.VectorStore behaviour.

defmodule ClEx.Memory.VectorStore do
  @callback store(collection :: String.t(), text :: String.t(), embedding :: [float()], metadata :: map()) :: :ok
  @callback search(collection :: String.t(), query_embedding :: [float()], opts :: keyword()) :: [result()]
  @callback delete(collection :: String.t(), id :: String.t()) :: :ok
end

---

2.11 commander --> OptionParser (EXISTS)

OpenClaw Role: CLI argument parsing, subcommand routing, help generation.

CL_EX Equivalent: Elixir's built-in OptionParser + Mix tasks + Owl for rich output.

Status: EXISTS

Integration Notes:

• Elixir's OptionParser handles flag/option parsing natively. No external dependency needed.
• For subcommand routing: Use Mix tasks (mix cl_ex.start, mix cl_ex.status) during development. For production CLI, use escript or Burrito for self-contained binaries.
• Owl provides rich terminal output (tables, trees, spinners) for CLI responses.
• For interactive prompts: IO.gets/1 + custom wrapper, or ex_prompt.

---

2.12 hono --> Bandit + Plug / Phoenix (EXISTS)

OpenClaw Role: Lightweight HTTP server for webhook endpoints, gateway API, and health checks.

CL_EX Equivalent: Bandit (HTTP server) + Plug (middleware) or full Phoenix framework.

Status: EXISTS

Integration Notes:

• Bandit is a modern, pure-Elixir HTTP/1.1 and HTTP/2 server. Drop-in replacement for Cowboy.
• Plug provides composable middleware (routing, parsing, CORS, etc.).
• For WebSocket support: Phoenix.Channel or WebSockAdapter with Bandit.
• Decision: Use Plug + Bandit for the lightweight gateway. If users want to embed CL_EX into a Phoenix app, it plugs directly into the Phoenix router.
• Phoenix is NOT required -- Plug + Bandit gives a full HTTP stack in ~50 lines of config.

defmodule ClEx.Gateway.Router do
  use Plug.Router

  plug :match
  plug Plug.Parsers, parsers: [:json], json_decoder: Jason
  plug :dispatch

  post "/webhook/:channel" do
    ClEx.Transport.dispatch(channel, conn.body_params)
    send_resp(conn, 200, "ok")
  end

  get "/health" do
    send_resp(conn, 200, Jason.encode!(%{status: "ok", agents: ClEx.Agent.count()}))
  end
end

---

2.13 croner --> Quantum (EXISTS)

OpenClaw Role: Cron-style job scheduling for periodic agent tasks, health checks, and automation.

CL_EX Equivalent: Quantum -- Cron-like job scheduler for Elixir.

Status: EXISTS

Integration Notes:

• Quantum supports cron expressions, runtime job management, and clustered execution (only one node runs each job).
• Direct replacement -- no adaptation needed.
• Alternative: Oban for persistent job queues with retry/backoff (heavier but more robust for critical tasks).

---

2.14 dotenv --> Dotenvy (EXISTS)

OpenClaw Role: Load environment variables from .env files.

CL_EX Equivalent: Dotenvy -- Dotenv file loading for Elixir.

Status: EXISTS

Integration Notes:

• Dotenvy loads .env files and integrates with Elixir's Config system.
• Elixir's config/runtime.exs already provides environment-based configuration.
• No adaptation needed. Use Dotenvy for development convenience; rely on system env vars or Config.Provider for production.

---

2.15 JSONL Session Files --> Bedrock (primary) + ETS cache (NEW)

OpenClaw Role: Persist session transcripts as JSONL (one JSON object per line) files on the filesystem. Each session has a file; messages are appended.

CL_EX Equivalent: Session data is managed within cl_ex_agent (as ClEx.Agent.Session.Store). Bedrock is the primary durable store; ETS serves as a hot cache.

Status: NEW

Integration Notes:

• Bedrock is the primary state store for session transcripts. Session keys follow the canonical format session:{identity_id}:{channel}:{session_id} (see Doc 01 Section 3.3.2). Bedrock provides distributed durability with Raft consensus, so sessions survive node failures without Mnesia.
• ETS serves as the hot cache for active sessions -- O(1) lookup, ordered_set for message ordering, ~microsecond access. Write-through to Bedrock on every append.
• Graceful degradation: When Bedrock is unavailable (e.g., single-node dev mode), ETS + DETS provides a local-only fallback (see Doc 02 Section 3).
• JSONL format can be maintained for export/import compatibility, but internal representation should be native Erlang terms for performance.
• Session compaction and memory consolidation can be orchestrated via FlowStone with lineage tracking.

---

2.16 Docker Sandbox --> cl_ex_sandbox (NEW)

OpenClaw Role: Run agent-generated code in a Docker container for isolation. Mounts workspace directory, sets resource limits, captures stdout/stderr.

CL_EX Equivalent: cl_ex_sandbox -- NEW module for sandboxed execution.

Status: NEW

Integration Notes:

• Option A (Docker): Use Erlang :os.cmd/1 or System.cmd/2 to invoke docker run. Wrap in a GenServer for lifecycle management.
• Option B (erlexec): Use erlexec for advanced OS process management with resource limits, signal handling, and stdout/stderr capture. Can run Docker or direct processes.
• Option C (Firecracker): MicroVMs for stronger isolation. More complex but better security properties.
• Option D (BEAM sandbox): For Elixir code execution, use Code.eval_string/3 in a restricted environment with a separate node (Node.spawn_link/2) and resource limits via cgroups.
• Recommendation: Start with Docker via System.cmd/2 wrapped in a GenServer. Add Firecracker support later for enterprise deployments.

defmodule ClEx.Sandbox do
  use GenServer

  def execute(code, opts \\ []) do
    timeout = Keyword.get(opts, :timeout, 30_000)
    workspace = Keyword.get(opts, :workspace, "/tmp/cl_ex_sandbox")

    GenServer.call(__MODULE__, {:execute, code, workspace, timeout}, timeout + 5_000)
  end
end

---

2.17 Plugin Loader (jiti) --> OTP Code Loading (EXISTS)

OpenClaw Role: Dynamic plugin loading at runtime. jiti transpiles and loads TypeScript/JavaScript modules on-the-fly.

CL_EX Equivalent: Native BEAM code loading, managed by cl_ex_plugins (see Doc 02 Section 10).

Status: EXISTS

Integration Notes:

• The BEAM has first-class hot code loading. Plugins can be:
  1. Compiled .beam files: Loaded via :code.load_file/1 or :code.load_binary/3
  2. Mix dependencies: Standard Hex packages added to mix.exs
  3. Script plugins: .exs files evaluated via Code.eval_file/1 (similar to jiti's dynamic loading)
  4. OTP releases with plugins: Use :release_handler for hot upgrades
• Plugin discovery: Scan a plugins/ directory for .beam files or .exs scripts.
• Plugin contract: Elixir behaviours enforce the interface at compile time. Runtime plugin installation is gated by allow_runtime_install: false by default (security consideration -- see Doc 02).
• This is a massive BEAM advantage. No transpilation, no module resolution hacks, no bundler. Plugins are just modules that implement a behaviour.

defmodule ClEx.Plugins.Plugin do
  @callback init(config :: map()) :: {:ok, state :: term()} | {:error, %ClEx.Error{}}
  @callback handle_event(event :: ClEx.Signal.t(), state :: term()) :: {:ok, state :: term()}
  @callback tools() :: [Jido.Action.t()]
  @optional_callbacks [tools: 0]
end

---

2.18 Config (JSON5 + Zod) --> NimbleOptions + Application Config (EXISTS)

OpenClaw Role: Configuration loading (JSON5 format) with runtime validation (Zod schemas). Supports nested config, defaults, and type coercion.

CL_EX Equivalent: NimbleOptions + Elixir's Config system.

Status: EXISTS

Integration Notes:

• NimbleOptions provides declarative option schemas with types, defaults, validation, and documentation generation. It is the Zod equivalent for Elixir.
• Elixir's Config system (config/config.exs, config/runtime.exs) replaces JSON5 files.
• For user-facing config files: Support TOML via toml hex package (more user-friendly than Elixir syntax for non-developers).
• Config validation happens at application startup -- fail fast with clear error messages.

defmodule ClEx.Config do
  @schema [
    agents: [
      type: {:list, :keyword_list},
      default: [],
      doc: "List of agent configurations",
      keys: [
        id: [type: :string, required: true],
        model: [type: :string, default: "claude-sonnet-4-20250514"],
        workspace: [type: :string],
        tools: [type: {:list, :atom}, default: [:read, :write, :edit, :search]]
      ]
    ],
    channels: [
      type: :keyword_list,
      default: [],
      doc: "Channel configurations"
    ]
  ]

  def validate!(opts), do: NimbleOptions.validate!(opts, @schema)
end

---

3. Dependency Graph

                         ┌─────────────────────────────────────────────────┐
                         │                  cl_ex (core)                   │
                         │       Top-level application + public API        │
                         └──────────────────────┬──────────────────────────┘
                                                │
          ┌─────────────────┬───────────────────┼───────────────────┬─────────────────┐
          │                 │                   │                   │                 │
          ▼                 ▼                   ▼                   ▼                 ▼
┌──────────────────┐ ┌──────────────────┐ ┌──────────────────┐ ┌──────────────┐ ┌──────────────┐
│  cl_ex_transport │ │   cl_ex_agent    │ │   cl_ex_tools    │ │ cl_ex_memory │ │ cl_ex_observe│
│  (Transport)     │ │  (Agent Runtime) │ │  (Tool Framework)│ │ (Memory/RAG) │ │ (Telemetry)  │
│                  │ │                  │ │                  │ │              │ │              │
│ Channel behaviour│ │ Jido.AgentServer │ │ Jido.Action-based│ │ Bedrock store│ │ AITrace      │
│ Bandit + Plug    │ │ Session mgmt     │ │ Coding tools     │ │ portfolio_*  │ │ OpenTelemetry│
│ Webhook ingress  │ │ LLM adapter layer│ │ Policy enforce   │ │ embed_ex     │ │ Metrics      │
└────────┬─────────┘ └────────┬─────────┘ └────────┬─────────┘ └──────────────┘ └──────────────┘
         │                    │                     │
         ▼                    │                     ▼
  ┌──────────────┐            │            ┌──────────────────┐
  │ Adapters:    │            │            │  cl_ex_sandbox   │
  │ Telegram     │            │            │ (Sandboxed Exec) │
  │ Discord      │            │            │                  │
  │ Slack        │            │            │ Docker/Firecracker│
  │ WhatsApp     │            │            │ Resource limits   │
  │ Signal       │            │            └──────────────────┘
  │ HTTP/WS      │            │
  └──────────────┘            │
                              │
          ┌───────────────────┼───────────────────┐
          │                   │                   │
          ▼                   ▼                   ▼
┌──────────────────┐ ┌──────────────────┐ ┌──────────────────┐
│  cl_ex_security  │ │   cl_ex_mesh     │ │  cl_ex_plugins   │
│  (Security)      │ │  (Distribution)  │ │  (Plugin System) │
│                  │ │                  │ │                  │
│ GUARDRAIL engine │ │ libcluster       │ │ Behaviour-based  │
│ Shield zero-trust│ │ Bedrock distrib. │ │ Hot code loading │
│ Capability tokens│ │ :pg process grps │ │ Plugin discovery │
└──────────────────┘ └──────────────────┘ └──────────────────┘

    ════════════════════════════════════════════════════════════════════════════
                     ECOSYSTEM + EXTERNAL HEX DEPENDENCIES
    ════════════════════════════════════════════════════════════════════════════

    Ecosystem (custom)      LLM/AI SDKs         Channels            Infrastructure
    ──────────────────      ───────────          ────────            ──────────────
    jido                    altar_ai             ex_gram             bandit
    bedrock                 claude_agent_sdk     nostrum             plug
    foundation              codex_sdk            req                 phoenix_pubsub
    synapse                 gemini_ex            floki               quantum
    flowstone               ollixir              wallaby             oban
    guardrail               langchain (optional)                     libcluster
    shield                  bumblebee                                telemetry
    portfolio_core          exla
    portfolio_index
    embed_ex                Storage              Utilities
    exdantic                ───────              ─────────
    aitrace                 ecto                 jason
                            pgvector             nimble_options
                            exqlite (fallback)   dotenvy
                                                 owl
                                                 ratatouille

---

4. Heterogeneous Stack Diagrams

4.1 Minimal: Single BEAM Node, Embedded in Phoenix App

For developers who want to add AI agent capability to an existing Phoenix application.

┌─────────────────────────────────────────────────────────────────┐
│                    Single BEAM Node (fly.io / VPS)              │
│                                                                 │
│  ┌───────────────────────────────────────────────────────────┐  │
│  │                   Phoenix Application                      │  │
│  │                                                           │  │
│  │  ┌─────────────┐  ┌──────────────┐  ┌────────────────┐  │  │
│  │  │ Phoenix      │  │ Phoenix      │  │ Your existing  │  │  │
│  │  │ Router       │  │ LiveView     │  │ business logic │  │  │
│  │  │              │  │              │  │                │  │  │
│  │  │ /webhook/*   │  │ /agent/chat  │  │                │  │  │
│  │  └──────┬───────┘  └──────┬───────┘  └────────────────┘  │  │
│  │         │                  │                               │  │
│  │  ┌──────▼──────────────────▼──────────────────────────┐   │  │
│  │  │              ClEx (embedded)                         │   │  │
│  │  │                                                     │   │  │
│  │  │  ┌──────────┐  ┌──────────┐  ┌──────────────────┐ │   │  │
│  │  │  │ 1 Agent  │  │ Session  │  │ LLM (altar_ai)   │ │   │  │
│  │  │  │ (Jido)   │  │ (ETS+BD) │  │ Anthropic/OpenAI │ │   │  │
│  │  │  └──────────┘  └──────────┘  └──────────────────┘ │   │  │
│  │  │  ┌──────────┐  ┌──────────┐                       │   │  │
│  │  │  │ Telegram  │  │ Memory   │                       │   │  │
│  │  │  │ Channel   │  │ (SQLite) │                       │   │  │
│  │  │  └──────────┘  └──────────┘                       │   │  │
│  │  └─────────────────────────────────────────────────────┘   │  │
│  └───────────────────────────────────────────────────────────┘  │
│                                                                 │
│  Resources: 1 CPU, 512MB RAM, 1GB disk                         │
│  Agents: 1-10 concurrent                                       │
│  Channels: 1-2                                                  │
└─────────────────────────────────────────────────────────────────┘

mix.exs:

defp deps do
  [
    {:phoenix, "~> 1.7"},
    {:cl_ex, "~> 0.1"},          # <-- Just add this
    # ... your existing deps
  ]
end

application.ex:

children = [
  # ... your existing children
  {ClEx, config: Application.get_env(:my_app, :cl_ex)}
]

---

4.2 Standard: Multi-Node BEAM Cluster

For teams running multiple agents across channels with high availability.

                        ┌──────────────────────┐
                        │    Load Balancer      │
                        │  (Fly.io / AWS ALB)   │
                        └──────────┬───────────┘
                                   │
              ┌────────────────────┼────────────────────┐
              │                    │                     │
              ▼                    ▼                     ▼
┌──────────────────┐ ┌──────────────────┐ ┌──────────────────┐
│  BEAM Node A     │ │  BEAM Node B     │ │  BEAM Node C     │
│  (us-east-1)     │ │  (eu-west-1)     │ │  (ap-southeast-1)│
│                  │ │                  │ │                  │
│ ┌──────────────┐ │ │ ┌──────────────┐ │ │ ┌──────────────┐ │
│ │  ClEx Core   │ │ │ │  ClEx Core   │ │ │ │  ClEx Core   │ │
│ │              │ │ │ │              │ │ │ │              │ │
│ │ Agents: 500  │ │ │ │ Agents: 500  │ │ │ │ Agents: 500  │ │
│ │ Telegram ch. │ │ │ │ Discord ch.  │ │ │ │ Slack ch.    │ │
│ │ Slack ch.    │ │ │ │ WhatsApp ch. │ │ │ │ Web ch.      │ │
│ └──────────────┘ │ │ └──────────────┘ │ │ └──────────────┘ │
│                  │ │                  │ │                  │
│ ┌──────────────┐ │ │ ┌──────────────┐ │ │ ┌──────────────┐ │
│ │  Gateway     │ │ │ │  Gateway     │ │ │ │  Gateway     │ │
│ │  (Bandit)    │ │ │ │  (Bandit)    │ │ │ │  (Bandit)    │ │
│ └──────────────┘ │ │ └──────────────┘ │ │ └──────────────┘ │
│                  │ │                  │ │                  │
│ ┌──────────────┐ │ │ ┌──────────────┐ │ │ ┌──────────────┐ │
│ │ Bedrock      │◄├─┤►│ Bedrock      │◄├─┤►│ Bedrock      │ │
│ │ (Distributed │ │ │ │ (Distributed │ │ │ │ (Distributed │ │
│ │  State/Raft) │ │ │ │  State/Raft) │ │ │ │  State/Raft) │ │
│ └──────────────┘ │ │ └──────────────┘ │ │ └──────────────┘ │
└────────┬─────────┘ └────────┬─────────┘ └────────┬─────────┘
         │                     │                     │
         │    ┌────────────────┤                     │
         │    │    libcluster  │  (Gossip / DNS)     │
         │    │    auto-discovery                    │
         └────┴─────────────┬──┴─────────────────────┘
                            │
                   ┌────────▼────────┐
                   │   PostgreSQL    │
                   │   + pgvector    │
                   │   (fallback DB) │
                   │                 │
                   │ Vectors/search  │
                   │ Analytics       │
                   └─────────────────┘

  Total capacity: ~1500 concurrent agents
  Channels: All supported
  State: Bedrock (Raft consensus) for sessions, config, and distributed KV
  Failover: Automatic (Bedrock replicates state; OTP restarts agents on surviving nodes)
  Session affinity: Not needed (Bedrock provides consistent reads from any node)

---

4.3 Full: BEAM Cluster + Sprites + Cloud VPS Auto-Provisioning

For power users and organizations running autonomous agents that need compute resources.

┌─────────────────────────────────────────────────────────────────────────────┐
│                         CONTROL PLANE (BEAM Cluster)                        │
│                                                                             │
│  ┌──────────────┐  ┌──────────────┐  ┌──────────────┐  ┌──────────────┐  │
│  │  Node A       │  │  Node B       │  │  Node C       │  │  Node D       │  │
│  │  Agent Mesh   │  │  Agent Mesh   │  │  Channel Hub  │  │  Scheduler   │  │
│  │  100 agents   │  │  100 agents   │  │  All channels │  │  Quantum     │  │
│  └──────┬───────┘  └──────┬───────┘  └──────┬───────┘  └──────┬───────┘  │
│         │    libcluster    │                  │                  │          │
│         └──────────────────┴──────────────────┴──────────────────┘          │
└────────────────────────────────┬────────────────────────────────────────────┘
                                 │
                    ┌────────────┴────────────┐
                    │   Sprite Orchestrator   │
                    │   (cl_ex_sprites)       │
                    │                         │
                    │   Provisions, monitors, │
                    │   and reclaims compute  │
                    └────┬──────────┬─────────┘
                         │          │
            ┌────────────┘          └────────────┐
            ▼                                     ▼
┌───────────────────────┐            ┌───────────────────────┐
│  Sprite Pool A        │            │  Sprite Pool B        │
│  (Hetzner ARM VPS)    │            │  (AWS Spot Instances) │
│                       │            │                       │
│  ┌─────────────────┐  │            │  ┌─────────────────┐  │
│  │ Sprite 1        │  │            │  │ Sprite 5        │  │
│  │ Docker sandbox  │  │            │  │ GPU instance    │  │
│  │ Agent workspace │  │            │  │ Bumblebee       │  │
│  │ 2 CPU / 4GB     │  │            │  │ Local embeddings│  │
│  ├─────────────────┤  │            │  │ Local LLM       │  │
│  │ Sprite 2        │  │            │  │ 8 CPU / 32GB    │  │
│  │ Docker sandbox  │  │            │  ├─────────────────┤  │
│  │ Agent workspace │  │            │  │ Sprite 6        │  │
│  │ 2 CPU / 4GB     │  │            │  │ Browser farm    │  │
│  ├─────────────────┤  │            │  │ Playwright      │  │
│  │ Sprite 3        │  │            │  │ 4 CPU / 8GB     │  │
│  │ Docker sandbox  │  │            │  └─────────────────┘  │
│  │ 2 CPU / 4GB     │  │            │                       │
│  ├─────────────────┤  │            └───────────────────────┘
│  │ Sprite 4        │  │
│  │ Docker sandbox  │  │
│  │ 2 CPU / 4GB     │  │
│  └─────────────────┘  │
└───────────────────────┘

  Sprite lifecycle:
    1. Agent needs compute → Orchestrator provisions VPS via cloud API
    2. Sprite boots → Connects back to BEAM cluster as hidden node
    3. Agent dispatches work → Sprite executes in Docker sandbox
    4. Work complete → Sprite idles → Auto-reclaimed after timeout
    5. Crash → Orchestrator detects via BEAM monitor → Reprovisions

  Cloud providers supported:
    - Hetzner (ARM, cheapest)
    - AWS (Spot instances, GPU)
    - GCP, Azure, DigitalOcean, Vultr, Linode
    - Fly.io (BEAM-native, fastest provision)

---

4.4 Enterprise: Full + ASKA Security + Shield Zero-Trust

For regulated industries, government, and high-security deployments.

┌─────────────────────────────────────────────────────────────────────────────────┐
│                            ASKA SECURITY PERIMETER                              │
│                     (Adaptive Security Kernel Architecture)                      │
│                                                                                 │
│  ┌───────────────────────────────────────────────────────────────────────────┐  │
│  │                         Shield Zero-Trust Layer                           │  │
│  │                                                                           │  │
│  │  ┌─────────────┐  ┌─────────────┐  ┌─────────────┐  ┌────────────────┐  │  │
│  │  │ Identity    │  │ Policy      │  │ Audit       │  │ Secrets        │  │  │
│  │  │ Provider    │  │ Engine      │  │ Logger      │  │ Manager        │  │  │
│  │  │             │  │             │  │             │  │                │  │  │
│  │  │ mTLS certs  │  │ OPA/Rego   │  │ Immutable   │  │ Vault / SOPS  │  │  │
│  │  │ SPIFFE IDs  │  │ ABAC rules │  │ append-only │  │ Rotate on use │  │  │
│  │  │ JWT + RBAC  │  │ Per-agent   │  │ Signed log  │  │ Env isolation │  │  │
│  │  └──────┬──────┘  └──────┬──────┘  └──────┬──────┘  └───────┬────────┘  │  │
│  │         └────────────────┴────────────────┴──────────────────┘            │  │
│  └──────────────────────────────┬────────────────────────────────────────────┘  │
│                                  │                                               │
│  ┌──────────────────────────────▼────────────────────────────────────────────┐  │
│  │                      BEAM CLUSTER (Encrypted Mesh)                        │  │
│  │                   All inter-node traffic: TLS 1.3                         │  │
│  │                   Cookie: Derived from HSM / TPM                          │  │
│  │                                                                           │  │
│  │   ┌──────────────┐  ┌──────────────┐  ┌──────────────┐                   │  │
│  │   │  Agent Node  │  │  Agent Node  │  │  Gateway     │                   │  │
│  │   │  (Enclave A) │  │  (Enclave B) │  │  (DMZ Node)  │                   │  │
│  │   │              │  │              │  │              │                   │  │
│  │   │ Agents run   │  │ Agents run   │  │ TLS termn.  │                   │  │
│  │   │ in isolated  │  │ in isolated  │  │ WAF rules   │                   │  │
│  │   │ schedulers   │  │ schedulers   │  │ Rate limit  │                   │  │
│  │   │              │  │              │  │ DDoS prot.  │                   │  │
│  │   │ Memory:      │  │ Memory:      │  │              │                   │  │
│  │   │  encrypted   │  │  encrypted   │  │ Channels:   │                   │  │
│  │   │  at rest     │  │  at rest     │  │  All, with  │                   │  │
│  │   │              │  │              │  │  E2E proxy  │                   │  │
│  │   └──────────────┘  └──────────────┘  └──────────────┘                   │  │
│  │                                                                           │  │
│  │   ┌──────────────────────────────────────────────────────────────────┐   │  │
│  │   │  Tool Execution: Sandboxed Sprites (Firecracker microVMs)       │   │  │
│  │   │                                                                  │   │  │
│  │   │  - Each agent gets dedicated microVM                             │   │  │
│  │   │  - Network: Isolated VPC, egress whitelist only                  │   │  │
│  │   │  - Filesystem: Encrypted, ephemeral, no persistence by default   │   │  │
│  │   │  - Resource: Hard cgroup limits (CPU, RAM, I/O, network)         │   │  │
│  │   │  - Lifetime: Max 1 hour, then force-terminated and shredded      │   │  │
│  │   └──────────────────────────────────────────────────────────────────┘   │  │
│  └───────────────────────────────────────────────────────────────────────────┘  │
│                                                                                 │
│  ┌───────────────────────────────────────────────────────────────────────────┐  │
│  │                         Data Layer (Encrypted)                            │  │
│  │                                                                           │  │
│  │  ┌──────────────────┐  ┌──────────────────┐  ┌────────────────────────┐  │  │
│  │  │ PostgreSQL       │  │ Object Storage   │  │ Audit Store            │  │  │
│  │  │ (TDE enabled)    │  │ (S3 + SSE-KMS)   │  │ (Immutable, signed)    │  │  │
│  │  │                  │  │                  │  │                        │  │  │
│  │  │ Sessions         │  │ Agent artifacts  │  │ Every agent action     │  │  │
│  │  │ Vectors          │  │ Media files      │  │ Every tool invocation  │  │  │
│  │  │ Config           │  │ Exports          │  │ Every LLM call         │  │  │
│  │  │ (Row-level       │  │                  │  │ Tamper-evident chain   │  │  │
│  │  │  encryption)     │  │                  │  │                        │  │  │
│  │  └──────────────────┘  └──────────────────┘  └────────────────────────┘  │  │
│  └───────────────────────────────────────────────────────────────────────────┘  │
│                                                                                 │
│  Compliance: SOC2 Type II, HIPAA, FedRAMP (with appropriate CSP)               │
│  Key management: AWS KMS / GCP KMS / HashiCorp Vault                           │
│  Network: Private VPC, no public IPs on agent/data nodes                       │
│  Observability: OpenTelemetry → encrypted Grafana stack                        │
└─────────────────────────────────────────────────────────────────────────────────┘

---

5. What's Missing

5.1 Critical Gaps (Must Build for v0.1)

Gap	Severity	Description	Estimated Effort
Agent Runtime (cl_ex_agent)	CRITICAL	Jido provides the core agent model (Jido.Agent, Jido.AgentServer), so the work is integration rather than greenfield. Must build: LLM adapter layer (over altar_ai + provider SDKs), session management (ClEx.Agent.Session), and the ClEx.Agent.Runtime orchestration wrapper.	3-4 weeks
Transport Layer (cl_ex_transport)	CRITICAL	No unified channel behaviour exists. Each platform library (ExGram, Nostrum, etc.) has its own API. Need a ClEx.Transport.Channel behaviour, normalization pipeline (to ClEx.Signal), and per-platform adapters.	2-3 weeks
Session Management (within cl_ex_agent)	HIGH	Session lifecycle is part of cl_ex_agent (not a separate package). Must integrate with Bedrock for persistence, implement compaction and reset policies. Effort is lower since Bedrock provides the state backend.	1-2 weeks
WhatsApp Adapter	HIGH	No Elixir WhatsApp library exists. Must wrap Go's whatsmeow via Port or use Business Cloud API.	3-4 weeks (Port) or 1 week (Cloud API)
Tool Framework (cl_ex_tools)	HIGH	Jido provides Jido.Action as the tool primitive, so the behaviour contract exists. Must build: standard tool library (read, write, edit, search, bash), policy enforcement via cl_ex_security (GUARDRAIL), and sandboxing hooks.	2-3 weeks
Sandbox Execution (cl_ex_sandbox)	MEDIUM	No existing package provides agent-oriented sandboxed code execution. Must build Docker/Firecracker integration.	2-3 weeks

5.2 Channel Adapter Libraries -- Detailed Gap Analysis

Channel        Elixir Library    Quality    Gap / Work Needed
─────────────  ────────────────  ─────────  ──────────────────────────────────
Telegram       ex_gram           Good       Wrap in ClEx.Transport.Channel behaviour.
                                            Needs: media handling adapter,
                                            inline keyboard → CL_EX action map.
                                            Effort: 3-5 days.

Discord        nostrum           Excellent  Wrap in ClEx.Transport.Channel behaviour.
                                            Needs: slash command registration,
                                            embed builder, component mapping.
                                            Effort: 3-5 days.

Slack          slack_elixir      Fair       Thin wrapper, may need custom
                                            Events API webhook handler.
                                            Socket Mode: must build.
                                            Effort: 1-2 weeks.

WhatsApp       NONE              N/A        Biggest gap. Options:
                                            A) Cloud API (Req) - 1 week
                                            B) whatsmeow Port - 3-4 weeks
                                            C) Baileys Port - 2-3 weeks
                                            Effort: 1-4 weeks.

Signal         NONE              N/A        signal-cli exists (Java). Wrap as
                                            Port or use Signal REST API.
                                            Effort: 2-3 weeks.

iMessage       NONE              N/A        macOS-only. Requires AppleScript
                                            or BlueBubbles server integration.
                                            Effort: 2-3 weeks.

LINE           NONE              N/A        REST API is straightforward.
                                            Build from scratch with Req.
                                            Effort: 1 week.

Google Chat    NONE              N/A        REST API + PubSub. Build with
                                            Req + goth (Google auth).
                                            Effort: 1 week.

Matrix         NONE partial      Poor       matrix_sdk_ex exists but
                                            unmaintained. Likely rebuild.
                                            Effort: 2 weeks.

MS Teams       NONE              N/A        Bot Framework REST API.
                                            Build with Req.
                                            Effort: 1-2 weeks.

5.3 CLI Framework Gap

Elixir has no single "commander.js equivalent" that provides:

• Subcommand trees with help generation
• Interactive prompts
• Rich terminal output (tables, colors, spinners)
• Shell completions

Solution: Compose from existing pieces:

• OptionParser (stdlib) -- argument parsing
• Owl -- rich output, tables, spinners, progress bars
• IO.ANSI (stdlib) -- colors
• Custom subcommand router (~100 lines)
• Burrito -- package as standalone binary (replaces npm global install)

This is a minor gap -- the pieces exist, they just need composition.

5.4 Other Gaps

Gap	Description	Solution
Sprite Orchestrator	Cloud VPS provisioning and management. No existing package.	Build within cl_ex_mesh using cloud provider APIs via Req. 4-6 weeks.
Security Layer	Zero-trust security kernel.	Build cl_ex_security. Integrate GUARDRAIL (policy engine) and Shield (zero-trust auth). Leverage OTP distribution security (TLS), add SPIFFE/mTLS. 6-8 weeks.
Skill System	Reusable prompt fragments with discovery.	Leverage Jido.Skill for higher-level compositions of Jido.Action tools. Build skill registry and filesystem discovery. 1-2 weeks.
Memory/RAG Pipeline	End-to-end: chunk, embed, store, search, inject.	Build cl_ex_memory. Integrate portfolio_core/portfolio_index for RAG, embed_ex for embeddings, Bedrock for storage. 2-3 weeks.
Config File Format	User-friendly config for non-Elixir developers.	Support TOML via toml package + NimbleOptions validation. 3-5 days.
Hot Code Upgrade	Seamless agent updates without dropping sessions.	Leverage OTP releases + :release_handler. Built-in but needs orchestration. 1-2 weeks.

---

6. Generic Agentic Stack

The Vision: One Dependency, Full Agent Infrastructure

Any Elixir developer adds one line to their mix.exs and gets a complete, production-grade AI agent platform:

# mix.exs
defp deps do
  [
    {:cl_ex, "~> 0.1"}
  ]
end

This single dependency brings the entire stack. Here is what activates and how.

6.1 What You Get

┌─────────────────────────────────────────────────────────────────────────────┐
│                                                                             │
│   {:cl_ex, "~> 0.1"}                                                       │
│                                                                             │
│   ┌─────────────────────────────────────────────────────────────────────┐   │
│   │  AUTO-DISCOVERED BEAM CLUSTERING                                    │   │
│   │                                                                     │   │
│   │  ClEx starts libcluster automatically.                              │   │
│   │  Strategy auto-detected:                                            │   │
│   │    - Fly.io  → Fly.io DNS strategy (zero config)                    │   │
│   │    - K8s     → Kubernetes DNS strategy (zero config)                │   │
│   │    - Local   → Epmd strategy (zero config)                          │   │
│   │    - Custom  → Configure in application env                         │   │
│   │                                                                     │   │
│   │  Result: Add more nodes → agents auto-distribute.                   │   │
│   └─────────────────────────────────────────────────────────────────────┘   │
│                                                                             │
│   ┌─────────────────────────────────────────────────────────────────────┐   │
│   │  SELF-HEALING AGENT MESH                                            │   │
│   │                                                                     │   │
│   │  Every agent is a supervised Jido.AgentServer (GenServer) process.  │   │
│   │  Bedrock + cl_ex_mesh distribute agents across cluster.             │   │
│   │                                                                     │   │
│   │  Node dies → cl_ex_mesh restarts agents on surviving nodes.          │   │
│   │  Agent crashes → DynamicSupervisor restarts it.                     │   │
│   │  Session restored from persistent storage automatically.            │   │
│   │                                                                     │   │
│   │  You write:                                                         │   │
│   │    ClEx.chat("my-agent", "Hello!", model: "claude-sonnet-4-20250514")│   │
│   │  That's it. Crash recovery, distribution, and monitoring are free.  │   │
│   └─────────────────────────────────────────────────────────────────────┘   │
│                                                                             │
│   ┌─────────────────────────────────────────────────────────────────────┐   │
│   │  ZERO-CONFIG CHANNEL CONNECTIVITY                                   │   │
│   │                                                                     │   │
│   │  Set environment variables. Channels auto-activate:                 │   │
│   │                                                                     │   │
│   │    CLEX_TELEGRAM_BOT_TOKEN=xxx  → Telegram channel starts            │   │
│   │    CLEX_DISCORD_BOT_TOKEN=xxx  → Discord channel starts             │   │
│   │    CLEX_SLACK_BOT_TOKEN=xxx    → Slack channel starts               │   │
│   │    CLEX_WHATSAPP_API_TOKEN=xxx → WhatsApp channel starts            │   │
│   │                                                                     │   │
│   │  No channel configured? HTTP webhook gateway starts by default.     │   │
│   │  All channels normalize to %ClEx.Signal{} automatically.             │   │
│   └─────────────────────────────────────────────────────────────────────┘   │
│                                                                             │
│   ┌─────────────────────────────────────────────────────────────────────┐   │
│   │  EMBEDDED AI AGENT WITH TOOLS                                       │   │
│   │                                                                     │   │
│   │  Built-in tools (opt-in via config):                                │   │
│   │    :read, :write, :edit       → Filesystem operations               │   │
│   │    :search                    → Code/text search                    │   │
│   │    :bash                      → Shell execution (sandboxed)         │   │
│   │    :web_search, :web_fetch    → Internet access                     │   │
│   │    :message                   → Cross-channel messaging             │   │
│   │    :memory_search             → Semantic memory recall              │   │
│   │    :browser                   → Headless browser automation         │   │
│   │                                                                     │   │
│   │  Custom tools in 5 lines:                                           │   │
│   │    defmodule MyTool do                                              │   │
│   │      use Jido.Action,                                               │   │
│   │        name: "my_tool",                                             │   │
│   │        description: "Does something useful",                        │   │
│   │        schema: [input: [type: :string, required: true]]             │   │
│   │      def run(%{input: val}, _ctx), do: {:ok, process(val)}          │   │
│   │    end                                                              │   │
│   └─────────────────────────────────────────────────────────────────────┘   │
│                                                                             │
│   ┌─────────────────────────────────────────────────────────────────────┐   │
│   │  MEMORY / RAG CAPABILITY                                            │   │
│   │                                                                     │   │
│   │  Three memory tiers, all automatic:                                 │   │
│   │                                                                     │   │
│   │  1. Session Memory (Bedrock + ETS cache)                             │   │
│   │     - Current conversation transcript                               │   │
│   │     - Auto-compaction when token limit approached                   │   │
│   │     - Distributed across cluster via Bedrock (Raft consensus)       │   │
│   │                                                                     │   │
│   │  2. Semantic Memory (portfolio_core + portfolio_index)               │   │
│   │     - Long-term knowledge base with RAG pipeline                    │   │
│   │     - Auto-embedded via embed_ex (local Bumblebee or API)           │   │
│   │     - Searched automatically when agent needs context               │   │
│   │     - Backend: Bedrock (primary) or pgvector/SQLite (fallback)      │   │
│   │                                                                     │   │
│   │  3. Episodic Memory (Event Log)                                     │   │
│   │     - What happened, when, across which channels                    │   │
│   │     - Queryable timeline of agent actions                           │   │
│   │     - Powers "what did I do yesterday?" queries                     │   │
│   │                                                                     │   │
│   │  Config:                                                            │   │
│   │    config :cl_ex_memory,                                            │   │
│   │      backend: :bedrock,      # or :pgvector, :sqlite, :in_memory   │   │
│   │      embedding: :local,      # or :openai, :voyage (via embed_ex)  │   │
│   │      auto_embed: true                                               │   │
│   └─────────────────────────────────────────────────────────────────────┘   │
│                                                                             │
└─────────────────────────────────────────────────────────────────────────────┘

6.2 Progressive Disclosure: From Simple to Full

LEVEL 0: Library Mode (10 seconds)
══════════════════════════════════

    ClEx.chat("What is 2+2?")
    #=> "2 + 2 equals 4."

    No config. No processes. Just a function call.
    Uses default model, no tools, no memory.


LEVEL 1: Agent Mode (1 minute)
══════════════════════════════

    # In your application.ex:
    children = [{ClEx, agents: [%{id: "helper", model: "claude-sonnet-4-20250514"}]}]

    # Anywhere in your code:
    ClEx.chat("helper", "Summarize this document", context: doc_text)
    #=> "The document discusses..."

    One supervised agent. Tools optional. Memory optional.


LEVEL 2: Channel Mode (5 minutes)
═════════════════════════════════

    # config/runtime.exs:
    config :cl_ex,
      agents: [%{id: "bot", model: "claude-sonnet-4-20250514", tools: [:search, :memory_search]}],
      channels: [
        telegram: [token: System.get_env("CLEX_TELEGRAM_BOT_TOKEN")],
        discord: [token: System.get_env("CLEX_DISCORD_BOT_TOKEN")]
      ]

    # That's it. Bot is now live on Telegram and Discord.
    # Messages auto-route to the "bot" agent.
    # Sessions auto-managed per user per channel.


LEVEL 3: Multi-Agent Mode (15 minutes)
══════════════════════════════════════

    config :cl_ex,
      agents: [
        %{id: "triage",   model: "claude-haiku", tools: [:message],
          system: "Route questions to the right specialist agent."},
        %{id: "coder",    model: "claude-sonnet-4-20250514", tools: [:read, :write, :edit, :bash],
          workspace: "/app/code"},
        %{id: "research", model: "claude-sonnet-4-20250514", tools: [:web_search, :web_fetch, :memory_search]},
        %{id: "ops",      model: "claude-haiku", tools: [:bash, :message],
          system: "You manage infrastructure."}
      ],
      channels: [
        telegram: [token: "..."],
        slack: [token: "...", channel_routing: %{"#dev" => "coder", "#research" => "research"}]
      ]

    # Triage agent receives all unrouted messages.
    # It uses the :message tool to hand off to specialists.
    # Each specialist has its own session, tools, and workspace.


LEVEL 4: Distributed + Sprites (30 minutes)
═══════════════════════════════════════════

    config :cl_ex,
      cluster: [strategy: :fly_io],           # Auto-discover peers
      sprites: [
        provider: :hetzner,                    # Auto-provision VPS
        pool_size: {2, 10},                    # Min 2, max 10 sprites
        instance_type: "cax11"                 # ARM, 2 CPU, 4GB
      ],
      # ... agents and channels as before

[nshkrdotcom]

CL_EX: The Self-Architecting Distributed Agent System

Continues from ../20260202/ — assumes familiarity with Plan/Gate/Commit model, security architecture (1000, 1001), and core vision (100.md).

---

One-Sentence Vision

A distributed planning and building system for a distributed self-healing, self-building, self-improving system — with a meta-planner managing plan/build/test/sandbox→prod cycles, secured through granular but wide segmentation boundaries, running on heterogeneous actor-standardized nodes as a decentralized self-organizing emergent swarm.

---

The Two-Layer Architecture

CL_EX operates at two levels simultaneously:

┌──────────────────────────────────────────────────────────────────────────┐
│  LAYER 2: META-SYSTEM                                                    │
│  Plans and commits changes to the system's own architecture              │
│                                                                          │
│  observe → plan architectural change → gate → sandbox → commit → observe │
└──────────────────────────────────────────────────────────────────────────┘
                                    │
                    ┌───────────────┼───────────────┐
                    ▼               ▼               ▼
┌─────────────────────────────────────────────────────────────────────────┐
│  LAYER 1: AGENT EXECUTION                                               │
│  Individual agents plan and commit task-level changes                   │
│                                                                         │
│  ┌─────────┐      ┌─────────┐      ┌─────────┐      ┌─────────┐         │
│  │ Jido    │      │ Claude  │      │ Codex   │      │ Custom  │         │
│  │ Agent   │      │ Code    │      │ CLI     │      │ Agent   │         │
│  └────┬────┘      └────┬────┘      └────┬────┘      └────┬────┘         │
│       │                │                │                │              │
│       └────────────────┴────────────────┴────────────────┘              │
│                    All speak cmd/2 → Plan                               │
│                    All gated by same security stack                     │
└─────────────────────────────────────────────────────────────────────────┘

Layer 1 (from 20260202 docs): Agents propose Plans, Gate approves, Executor runs, Commit applies. Audit everything.

Layer 2 (this doc): The system itself is an agent that proposes Plans for its own evolution. Same pattern, different executor.

---

Key Properties

1. Self-Healing

The system detects failures and plans fixes without human intervention.

Node C stops responding
    → Layer 2 observes: health check failure
    → Layer 2 plans: "redistribute C's workload to A, B; spawn replacement C'"
    → Gate: policy allows auto-recovery for this failure class
    → Sandbox: simulate topology change
    → Commit: apply new topology
    → System continues operating

2. Self-Building

The system can extend itself with new capabilities.

User requests: "add support for Gemini models"
    → Layer 2 plans: "add GeminiProvider node, register tools, update routing"
    → Gate: requires human approval (new capability class)
    → Sandbox: test provider integration
    → Commit: deploy to production topology
    → System now supports Gemini

3. Self-Improving

The system observes its own performance and optimizes.

Observation: tool X takes 5s average, tool Y (equivalent) takes 0.5s
    → Layer 2 plans: "route X-like requests to Y when applicable"
    → Gate: policy allows optimization within bounds
    → Sandbox: A/B test shows 10x improvement
    → Commit: update routing rules
    → System is now faster

---

The Meta-Planner

Layer 2 needs a coordinator that isn't a single point of failure. The meta-planner is itself distributed:

┌─────────────────────────────────────────────────────────────────┐
│                    META-PLANNER (Distributed)                   │
│                                                                 │
│   ┌──────────┐    ┌──────────┐    ┌──────────┐                  │
│   │ Planner  │    │ Planner  │    │ Planner  │   (consensus)    │
│   │ Node α   │◄──►│ Node β   │◄──►│ Node γ   │                  │
│   └──────────┘    └──────────┘    └──────────┘                  │
│        │                                                        │
│        ▼                                                        │
│   ┌──────────────────────────────────────────┐                  │
│   │ Shared Ledger (source of truth)          │                  │
│   │ - system topology                        │                  │
│   │ - architectural plans proposed/committed │                  │
│   │ - health observations                    │                  │
│   └──────────────────────────────────────────┘                  │
└─────────────────────────────────────────────────────────────────┘

Properties:

• Multiple planner nodes, consensus on proposals
• Shared ledger (same as Layer 1 ledger, different event types)
• Any planner can propose; majority must agree to commit
• Planners themselves are replaceable (self-healing applies to meta-layer)

---

Heterogeneous Nodes, Standardized Interface

Nodes can be wildly different implementations:

Node Type	Implementation	Capabilities
Jido Native	Elixir, runs in BEAM cluster	Full capability, trusted
Foreign CLI	Claude Code, Codex in containers	Patch producer, sandboxed
Specialist	Fine-tuned model for specific domain	Limited scope, high skill
Legacy	Wrapped existing service	Adapter pattern
Remote	External API (OpenAI, Anthropic)	Provider, rate-limited

All speak the same protocol:

# Every node, regardless of implementation, produces:
@callback cmd(state, context) :: {:ok, Plan.t()} | {:error, Error.t()}

# Plan contains Directives that go through Gate
# No node can execute directly — all effects mediated

This is "actor-standardized" — the actor model with a universal contract.

---

Decentralized Self-Organizing

No master node. Coordination emerges from:

1. Shared Ledger as Ground Truth

All nodes read/write to the same append-only ledger. Consistency from convergence, not coordination.

2. Local Decision Rules

Each node follows local rules that produce global coherence:

• "If I see work I can do, claim it"
• "If I see a failure I can fix, propose fix"
• "If I'm overloaded, shed load"
• "If I'm idle, request work"

3. Capability-Based Routing

Work flows to nodes that have capability tokens for it. No central router decides — work finds its executor.

4. Gossip for Ephemeral State

Health, load, availability gossipped between nodes. Ledger for durable state, gossip for real-time.

---

Emergent Swarm Behavior

With many simple agents following protocols, complex behavior emerges:

                    ┌─────────────────────────────────┐
                    │     EMERGENT PROPERTIES         │
                    │                                 │
                    │  - Load balancing (no balancer) │
                    │  - Fault tolerance (no master)  │
                    │  - Scaling (add nodes, done)    │
                    │  - Optimization (observe+adapt) │
                    └─────────────────────────────────┘
                                   ▲
                                   │ emerges from
                    ┌──────────────┴──────────────────┐
                    │        LOCAL RULES              │
                    │                                 │
                    │  - Claim work you can do        │
                    │  - Shed load when overloaded    │
                    │  - Propose fixes for failures   │
                    │  - Report observations          │
                    └─────────────────────────────────┘
                                   ▲
                                   │ applied by
         ┌─────────────────────────┼─────────────────────────┐
         │                         │                         │
    ┌────┴────┐               ┌────┴────┐               ┌────┴────┐
    │ Agent 1 │               │ Agent 2 │               │ Agent N │
    └─────────┘               └─────────┘               └─────────┘

No agent "knows" the global optimal. The swarm finds it.

---

Security: Granular but Wide Segmentation

Granular: Every boundary has fine-grained capability controls.

Wide: Boundaries exist everywhere, not just at the perimeter.

┌─────────────────────────────────────────────────────────────────────────────┐
│                          SEGMENTATION BOUNDARIES                            │
│                                                                             │
│            ┌─────────┐ ──cap──► ┌─────────┐ ──cap──► ┌─────────┐            │
│            │ Agent A │          │ Tool X  │          │ Store   │            │
│            └─────────┘ ◄──cap── └─────────┘ ◄──cap── └─────────┘            │
│                 │                    │                    │                 │
│                cap                  cap                  cap                │
│                 │                    │                    │                 │
│                 ▼                    ▼                    ▼                 │
│            ┌─────────┐          ┌─────────┐          ┌─────────┐            │
│            │ Agent B │          │ Network │          │ Commit  │            │
│            └─────────┘          └─────────┘          └─────────┘            │
│                                                                             │
│          Every arrow is a capability check. Every box is isolated.          │
└─────────────────────────────────────────────────────────────────────────────┘

Boundary Types

Boundary	Mechanism	Granularity
Agent ↔ Agent	Shield delegation tokens	Per-action, scoped resources
Agent ↔ Tool	Gate capability check	Per-call, taint-aware
Agent ↔ Store	Namespace isolation	Per-session, per-artifact
Node ↔ Node	mTLS + capability tokens	Per-connection, time-limited
Layer 1 ↔ Layer 2	Architectural capability	High-trust, approval-required
Sandbox ↔ Prod	Commit fence	Lease-gated, verified

The Security Invariant Across Layers

Layer 1 invariants (from 1001_revised.md) apply identically to Layer 2:

1. No unmediated effects (architectural changes need gate approval)
2. Capability attenuation (spawned subsystems can't exceed parent capabilities)
3. Taint non-erasure (external input to architectural plans stays tainted)
4. Audit completeness (every architectural commit recorded with policy rev)
5. No TOCTOU (architectural plan executed exactly as gated)
6. Commit requires fence (topology changes need lease)
7. Artifact integrity (system images verified by hash)
8. Ledger hash chain (architectural history tamper-evident)

---

Plan Types

Layer 1 has Plan (task-level directives). Layer 2 adds SystemPlan:

defmodule ClEx.SystemPlan do
  @moduledoc """
  Architectural change proposal. Same structure as Plan,
  different directive types, different executor.
  """

  defstruct [
    :id,
    :proposed_by,     # which meta-planner node
    :directives,      # SystemDirective list
    :rationale,       # why this change
    :rollback_plan,   # how to undo if needed
    :consensus_votes, # votes from planner nodes
    :sandbox_results  # test results before commit
  ]
end

defmodule ClEx.SystemDirective do
  @type action ::
    :node_add |
    :node_remove |
    :node_replace |
    :capability_grant |
    :capability_revoke |
    :route_update |
    :policy_update |
    :topology_rebalance |
    :provider_register |
    :tool_register

  defstruct [
    :id,
    :action,
    :params,
    :required_caps,   # what capabilities needed to execute
    :risk_level,      # :low | :medium | :high | :critical
    :approval_required
  ]
end

---

The Sandbox → Prod Cycle

Architectural changes are tested before commit:

1. PROPOSE
   Meta-planner creates SystemPlan with directives

2. CONSENSUS
   Other planner nodes vote (majority required)

3. GATE
   Same gate pipeline: capabilities, policies, budgets
   High-risk changes require human approval

4. SANDBOX
   Spin up isolated test topology
   Apply proposed changes
   Run verification suite
   Measure against success criteria

5. EVALUATE
   If sandbox fails: reject plan, log, done
   If sandbox succeeds: proceed to commit

6. COMMIT
   Acquire topology lease (fence token)
   Apply changes to production
   Verify health
   Release lease

7. OBSERVE
   Monitor for regressions
   If problems detected: trigger rollback plan

---

Implementation Path

Phase 1: Layer 1 Complete (from 20260202)

• Plan/Gate/Commit for single agent
• Security stack (Shield, GUARDRAIL, LlmGuard, Foundation)
• Ledger with audit events
• Foreign agent containment

Phase 2: Multi-Node Layer 1

• Distributed KV (Bedrock)
• Session leasing and fencing
• Node-to-node mTLS + capability delegation
• Gossip for health/load

Phase 3: Layer 2 Bootstrap

• SystemPlan and SystemDirective types
• Meta-planner (single node first)
• Sandbox environment provisioning
• Basic self-healing (node replacement)

Phase 4: Distributed Meta-Planner

• Multiple planner nodes with consensus
• Decentralized proposal/voting
• Self-healing applies to planners themselves

Phase 5: Emergent Optimization

• Observation → improvement loop
• A/B testing infrastructure
• Automatic capability routing
• Swarm load balancing

---

What This Enables

With both layers operational:

Capability	How
Zero-downtime evolution	Sandbox test, gradual rollout, instant rollback
Automatic scaling	Observe load, plan more nodes, commit topology
Self-repair	Detect failure, plan replacement, execute
Continuous optimization	Observe performance, plan improvements, A/B test, commit winners
Heterogeneous integration	Any agent type that speaks cmd/2 can join
Security at every level	Same invariants, same gate, different executors
Complete auditability	Every change (task and architectural) in the ledger

---

The Core Insight

CL_EX's Plan/Gate/Commit pattern is scale-invariant.

• At Layer 1: agent proposes file edit → gate checks → executor applies → commit to workspace
• At Layer 2: system proposes topology change → gate checks → executor applies → commit to cluster

Same pattern. Same security. Same audit. Different scope.

The system that runs agents is itself an agent running the same protocol.

This is how you get a self-improving system that doesn't spiral out of control — every improvement proposal goes through the same gate that governs everything else.

---

Next: 0002.md — Concrete SystemPlan examples and meta-planner consensus protocol.

[nshkrdotcom]

CL_EX Architecture Overview

Document Metadata

Field	Value
Version	0.1.0-draft
Date	2026-02-02
Status	Architectural Design - Pre-Implementation
Audience	Core engineering team, ecosystem maintainers

---

1. Executive Summary & Vision

CL_EX is a BEAM-native distributed AI agent orchestration platform that unifies
multi-channel communication (Telegram, Discord, Slack, Signal, iMessage, WhatsApp,
Matrix, IRC, SMS, email, web) with a first-class agent execution model built on
OTP supervision trees, Jido's pure-functional agent contract, and heterogeneous
distributed computing.

What CL_EX is NOT: a port of OpenClaw to Elixir. OpenClaw is a TypeScript
monolith that wraps a single-process coding agent runtime (pi-agent-core) behind
channel adapters and tool policies. It stores sessions as JSONL files, runs tools
through a 9-layer policy stack evaluated synchronously, sandboxes via Docker, and
assumes single-machine execution.

What CL_EX IS: a distributed runtime where every conversation is a supervised
process, every agent is a Jido worker with crash-recovery semantics, every session
is a replicated Bedrock entry with causal consistency, and sandboxed execution
happens in Sprites (disposable Fly.io VMs) or local Sandbox OTP applications. The
platform self-heals, self-networks across BEAM clusters, and can be dropped into
any Elixir application with zero configuration to gain full multi-channel AI agent
capability.

Key Ecosystem Dependencies

CL_EX builds on a set of custom ecosystem packages. Their roles and status:

Package	Role	Status
Jido	Agent contract, workers, actions	Internal / pre-1.0
Bedrock	Distributed KV, session state	Internal / pre-1.0
Foundation	Circuit breakers, rate limiting	Internal / pre-1.0
GUARDRAIL	MCP security gateway, policies	Internal / pre-1.0
Shield	Inter-agent auth, capabilities	Internal / pre-1.0
AITrace	Observability, telemetry	Internal / pre-1.0
Citadel	Command & control, dashboards	Internal / planned
FlowStone	Data orchestration, lineage	Internal / planned
Synapse	Workflow engine, signal registry	Internal / pre-1.0
DSPEx	Self-improving prompt chains	Internal / planned
exdantic	Schema validation	Published hex

Fallback strategy: If Bedrock is unavailable, session state falls back to
node-local ETS (with degraded consistency guarantees). See Doc 02 Section 3
for the ETS fallback path. All other ecosystem dependencies degrade gracefully
when absent -- features they provide are simply disabled.

Design Principles

1. Let it crash, let it recover. Agent processes die. OTP restarts them.
   Bedrock preserves state. The user never notices.
2. Distribution by default. A single node works. Two nodes auto-cluster.
   Twenty nodes across three continents just work.
3. Functional core, process shell. Business logic is pure Jido cmd/2 with
   deterministic testing. Side effects live in supervised GenServer boundaries.
4. Zero-trust at every boundary. GUARDRAIL mediates all tool access. Shield
   authenticates inter-agent communication. ASKA principles root trust in
   hardware where available.
5. Heterogeneous compute. BEAM nodes for orchestration, Sprites for
   sandboxed code execution, GPU VPS for inference, cloud functions for burst.
6. Observability is not optional. Every agent action is an AITrace span.
   Every tool call is a traced event. Every LLM interaction records tokens,
   latency, and cost.
7. Testability by design. Pure-functional agent cores (cmd/2) are
   deterministically testable without mocks. Transport and infrastructure
   layers use behaviours with Mox-compatible contracts. See Doc 02
   "Cross-Cutting Concerns" for the testing strategy.

---

2. High-Level Architecture

                           +-----------------------------------------+
                           |            EXTERNAL WORLD               |
                           |  Telegram  Discord  Slack  Signal  Web  |
                           |  iMessage  WhatsApp  Matrix  IRC  SMS   |
                           +---+------+------+------+------+----+----+
                               |      |      |      |      |    |
                    +==========V======V======V======V======V====V==========+
                    |              LAYER 1: TRANSPORT                       |
                    |                                                       |
                    |  ClEx.Transport.Telegram    ClEx.Transport.Discord    |
                    |  ClEx.Transport.Slack       ClEx.Transport.Signal     |
                    |  ClEx.Transport.IMessage    ClEx.Transport.WhatsApp   |
                    |  ClEx.Transport.Matrix      ClEx.Transport.Web        |
                    |                                                       |
                    |  Each: GenServer + WebSocket/Polling + Reconnection   |
                    |  Supervised under ClEx.Transport.Supervisor           |
                    +=========================+============================+
                                              |
                           ClEx.Message (unified envelope)
                                              |
                    +=========================V============================+
                    |            LAYER 2: NORMALIZATION                     |
                    |                                                       |
                    |  ClEx.Normalizer.Pipeline                             |
                    |    -> ClEx.Normalizer.Identity     (user resolution)  |
                    |    -> ClEx.Normalizer.Content      (media handling)   |
                    |    -> ClEx.Normalizer.Context      (thread/channel)   |
                    |    -> ClEx.Normalizer.RateLimit    (Foundation)       |
                    |    -> ClEx.Normalizer.Policy       (GUARDRAIL check)  |
                    |                                                       |
                    |  exdantic schemas for every message type              |
                    +=========================+============================+
                                              |
                           ClEx.Message (validated, enriched)
                                              |
                    +=========================V============================+
                    |           LAYER 3: ORCHESTRATION                      |
                    |                                                       |
                    |  ClEx.Orchestrator        (Synapse workflow engine)   |
                    |    -> ClEx.Router          (channel -> agent mapping) |
                    |    -> ClEx.SessionManager  (Bedrock-backed state)    |
                    |    -> ClEx.ConversationSup (DynamicSupervisor)       |
                    |    -> ClEx.SkillRegistry   (hot-loadable skills)     |
                    |    -> ClEx.MemoryManager   (portfolio_core RAG)      |
                    |                                                       |
                    |  Synapse signal registry + step-level retry           |
                    +=========================+============================+
                                              |
                              Jido.Signal (agent directives)
                                              |
                    +=========================V============================+
                    |          LAYER 4: AGENT RUNTIME                       |
                    |                                                       |
                    |  Jido.Agent (pure functional core)                    |
                    |    -> cmd/2 contract execution                        |
                    |    -> Skill dispatch (tool use, code gen, search)     |
                    |    -> FSM strategy (idle->thinking->acting->waiting)  |
                    |    -> DSPEx self-improving prompt chains              |
                    |                                                       |
                    |  Jido.Agent.Worker (OTP process shell)                |
                    |    -> Supervised lifecycle                            |
                    |    -> Mailbox-based message passing                   |
                    |    -> Crash recovery with Bedrock state reload        |
                    |                                                       |
                    |  LLM Providers:                                       |
                    |    altar_ai | gemini_ex | claude_agent_sdk |          |
                    |    codex_sdk | ollixir                                |
                    |                                                       |
                    |  crucible_* ensemble for reliability                  |
                    +=========================+============================+
                                              |
                              Tool calls / Code execution
                                              |
                    +=========================V============================+
                    |         LAYER 5: INFRASTRUCTURE                       |
                    |                                                       |
                    |  +-------------+  +-----------+  +----------------+  |
                    |  |  Bedrock    |  | Sandbox   |  |   Sprites      |  |
                    |  |  (state)    |  | (local)   |  |   (remote VM)  |  |
                    |  +-------------+  +-----------+  +----------------+  |
                    |  +-------------+  +-----------+  +----------------+  |
                    |  | Foundation  |  | AITrace   |  |   Shield       |  |
                    |  | (resilience)|  | (observe) |  |   (auth)       |  |
                    |  +-------------+  +-----------+  +----------------+  |
                    |  +-------------+  +-----------+  +----------------+  |
                    |  | GUARDRAIL   |  | Citadel   |  |   ASKA         |  |
                    |  | (MCP gate)  |  | (C2)      |  |   (hw trust)   |  |
                    |  +-------------+  +-----------+  +----------------+  |
                    |  +-------------+  +-----------+                      |
                    |  | FlowStone   |  | mcp_client|                      |
                    |  | (data orch) |  | (MCP)     |                      |
                    |  +-------------+  +-----------+                      |
                    +======================================================+

ClEx.Message is the unified internal envelope. Key fields include id,
channel, sender (a ClEx.Identity reference), content, attachments,
thread_id, metadata, and timestamp. See Doc 02 Section 2 (lines 262-315)
for the full struct definition.

---

3. The Five-Layer Architecture in Detail

3.1 Layer 1: Transport

The transport layer owns exactly one responsibility: maintaining persistent
connections to external messaging platforms and converting their wire protocols
into ClEx.Message structs. Transport processes do not interpret content; they
are protocol translators.

ClEx.Transport.Supervisor (one_for_one)
  |
  +-- ClEx.Transport.Telegram       (GenServer, long-poll or webhook)
  +-- ClEx.Transport.Discord        (GenServer, WebSocket via gateway)
  +-- ClEx.Transport.Slack          (GenServer, Socket Mode / Events API)
  +-- ClEx.Transport.Signal         (GenServer, signal-cli bridge)
  +-- ClEx.Transport.IMessage       (GenServer, AppleScript / BlueBubbles)
  +-- ClEx.Transport.WhatsApp       (GenServer, Baileys bridge / Cloud API)
  +-- ClEx.Transport.Matrix         (GenServer, Matrix CS API)
  +-- ClEx.Transport.Web            (Phoenix.Socket / LiveView channel)
  +-- ClEx.Transport.IRC            (GenServer, RFC 2812)
  +-- ClEx.Transport.SMS            (GenServer, Twilio / provider adapter)
  +-- ClEx.Transport.Email          (GenServer, IMAP idle + SMTP)

Key design decisions:

• Each transport is a GenServer that implements the ClEx.Transport behaviour
  (simplified here; see Doc 02 Section 2 for the full ClEx.Transport.Channel
  behaviour with streaming, capabilities, and normalization callbacks):

defmodule ClEx.Transport do
  @type config :: map()
  @type signal :: ClEx.Message.t()

  @callback connect(config) :: {:ok, state} | {:error, reason}
  @callback disconnect(state) :: :ok
  @callback send_message(signal, state) :: {:ok, state} | {:error, reason, state}
  @callback handle_platform_event(raw_event, state) :: {:emit, signal, state} | {:noop, state}
end

• Transports emit ClEx.Message structs into the normalization pipeline via
  Synapse.SignalRegistry.emit/2. This decouples transport from processing --
  the transport process never blocks waiting for a response.

• Reconnection is handled by Foundation circuit breakers. Each transport wraps
  its connection logic in Foundation.CircuitBreaker, with exponential backoff.
  A transport that fails 5 times in 60 seconds trips the breaker; Citadel is
  notified; the supervisor does not restart until the breaker half-opens.

• Outbound path: Responses flow back through ClEx.Transport.Dispatcher, a
  GenServer that receives ClEx.Message structs tagged with a transport_id and
  routes them to the correct transport process. The dispatcher handles
  platform-specific formatting (markdown to Telegram HTML, Discord embeds, Slack
  blocks, etc.) via ClEx.Transport.Formatter protocol implementations.

defprotocol ClEx.Transport.Formatter do
  @spec format(signal :: ClEx.Message.t(), platform :: atom()) :: iodata()
  def format(signal, platform)
end

Contrast with OpenClaw: CL_EX's transport layer is pure plumbing with a
unified envelope; all semantics live above. See Doc 04 Section 1 for a
detailed comparison with OpenClaw's channel plugin model.

---

3.2 Layer 2: Normalization

Raw platform signals are structurally valid but semantically raw. The
normalization layer enriches, validates, rate-limits, and policy-checks every
inbound signal before it reaches orchestration.

The pipeline is implemented as a Broadway-style processing chain (but using
Synapse's step-level orchestration rather than Broadway itself, since we need
step-level retry and compensation):

defmodule ClEx.Normalizer.Pipeline do
  use Synapse.Workflow

  step :resolve_identity,   handler: ClEx.Normalizer.Identity
  step :process_content,    handler: ClEx.Normalizer.Content
  step :resolve_context,    handler: ClEx.Normalizer.Context
  step :check_rate_limit,   handler: ClEx.Normalizer.RateLimit,  compensate: :noop
  step :check_policy,       handler: ClEx.Normalizer.Policy,     compensate: :reject
end

Identity resolution (ClEx.Normalizer.Identity): Maps platform-specific
user identifiers (Telegram chat_id, Discord user snowflake, Slack member_id) to
a unified ClEx.Identity struct stored in Bedrock. A single human across
Telegram and Discord resolves to one identity. Identity linking is explicit
(user-initiated via a linking flow) or implicit (same phone number on Signal and
WhatsApp). Bedrock's consistent reads ensure identity resolution never returns
stale data, even across nodes.

defmodule ClEx.Identity do
  use Exdantic.Schema

  field :id,              :binary_id
  field :display_name,    :string
  field :platform_ids,    {:map, :string, :string}  # %{"telegram" => "12345", ...}
  field :permissions,     {:list, :atom}
  field :preferences,     ClEx.Identity.Preferences
  field :created_at,      :utc_datetime
  field :last_seen_at,    :utc_datetime
end

Content processing (ClEx.Normalizer.Content): Handles media attachments
(images, audio, video, documents). Media is downloaded, checksummed, stored via
FlowStone with lineage tracking, and a reference is attached to the signal. For
images, a thumbnail and optional OCR/description (via LLM vision) is generated.
For audio, transcription is triggered asynchronously. All media processing is
supervised under ClEx.Media.Supervisor with Foundation semaphores controlling
concurrency.

Context resolution (ClEx.Normalizer.Context): Determines thread context.
Is this a reply? Part of an ongoing conversation? A new topic? The context
resolver queries Bedrock for the active session associated with this identity +
channel combination and attaches ClEx.Context metadata.

Rate limiting (ClEx.Normalizer.RateLimit): Per-identity, per-channel, and
global rate limits via Foundation.RateLimiter. Exceeded limits produce a
polite rejection signal routed back through the transport layer. Rate limit
configuration is per-identity (stored in Bedrock), allowing trusted users
higher throughput.

Policy checking (ClEx.Normalizer.Policy): First security gate. GUARDRAIL
evaluates the inbound signal against channel-level policies: Is this user
allowed to interact on this channel? Is the content type permitted? Are there
content-safety flags? Policy evaluation is synchronous but fast (in-memory
policy cache with Bedrock watch-based invalidation).

---

3.3 Layer 3: Orchestration

This is the brain of CL_EX. The orchestration layer decides what happens with
each normalized signal: which agent handles it, how conversation state is
managed, when to fork/join agent work, and how to coordinate multi-step
workflows.

3.3.1 The Router (ClEx.Router)

The router maps incoming messages to agent configurations. Routing is a pure
function over a loaded config snapshot: (message, config) -> {agent_spec, session_id}. The config itself is loaded from Bedrock and cached locally with
watch-based invalidation, but the routing decision involves no I/O.

defmodule ClEx.Router do
  @spec route(ClEx.Message.t(), ClEx.Router.Config.t()) ::
    {:ok, ClEx.Router.Decision.t()} | {:error, reason}

  def route(signal, config) do
    signal
    |> match_channel_rules(config.channel_rules)
    |> match_identity_rules(config.identity_rules)
    |> match_content_rules(config.content_rules)
    |> resolve_agent_spec()
    |> resolve_session()
  end
end

Routing rules are declarative and stored in Bedrock:

# Example: Route all Telegram messages from admin users to a power-agent,
# all others to a standard conversational agent.
%ClEx.Router.Config{
  channel_rules: [
    %{channel: :telegram, identity_tag: :admin, agent: :power_agent},
    %{channel: :telegram, agent: :conversational_agent},
    %{channel: :discord, guild: "12345", agent: :discord_community_agent},
    %{channel: :slack, workspace: "myteam", agent: :work_agent},
    %{channel: :_, agent: :default_agent}
  ]
}

3.3.2 Session Management (ClEx.SessionManager)

Every conversation is a session: a sequence of signals and agent responses
with associated state. Sessions are the unit of persistence, recovery, and
cost tracking (token usage is recorded per session via ClEx.Session.TokenUsage
for cost attribution; billing integration is out of scope for v0.1).

defmodule ClEx.Session do
  use Exdantic.Schema

  field :id,              :binary_id
  field :identity_id,     :binary_id
  field :channel,         :atom
  field :agent_spec,      :map
  field :state,           :map                   # Agent-specific state
  field :message_count,   :integer, default: 0
  field :token_usage,     ClEx.Session.TokenUsage
  field :started_at,      :utc_datetime
  field :last_active_at,  :utc_datetime
  field :status,          Ecto.Enum, values: [:active, :idle, :suspended, :closed]
end

Storage: Sessions are stored in Bedrock keyed by the tuple
{agent_id, conversation_id} (see Doc 02 for the canonical key schema).
Bedrock's ACID++ guarantees
mean session state is never lost, even during node failures. The strict
serialization guarantee means concurrent messages from the same user are
processed in order.

Contrast with OpenClaw: CL_EX's Bedrock-backed sessions survive node
failures, replicate across the cluster, and support consistent reads from any
node -- unlike OpenClaw's local JSONL file storage. See Doc 04 Section 2 for
a detailed comparison.

3.3.3 Conversation Supervisor (ClEx.ConversationSup)

Each active session has a corresponding conversation process tree:

ClEx.ConversationSup (DynamicSupervisor)
  |
  +-- ClEx.Conversation (session_id: "abc123")
  |     |
  |     +-- Jido.Agent.Worker (the agent for this conversation)
  |     +-- ClEx.Conversation.History (ETS-cached message history)
  |     +-- ClEx.Conversation.ToolSandbox (Sandbox or Sprite ref)
  |
  +-- ClEx.Conversation (session_id: "def456")
  |     |
  |     ...

ClEx.Conversation is a GenServer that mediates between the orchestration
layer and the agent runtime. It:

1. Receives normalized signals from the router
2. Loads/creates session state from Bedrock
3. Constructs a Jido directive (signal + context + history + available skills)
4. Dispatches to the Jido.Agent.Worker
5. Receives the agent's response
6. Persists updated state to Bedrock
7. Emits response signals back to the transport layer via Dispatcher

Lifecycle: Conversations start when a signal arrives for a new or idle
session. They remain alive for a configurable idle timeout (default 15 min),
during which they hold cached state in memory for fast response. On timeout,
they serialize state to Bedrock and terminate. On the next message, the
supervisor starts a new process that rehydrates from Bedrock. The user
experiences seamless continuity.

defmodule ClEx.Conversation do
  use GenServer

  # Crash recovery: on init, always rehydrate from Bedrock
  def init(%{session_id: sid} = args) do
    case Bedrock.get("session:#{sid}") do
      {:ok, session} -> {:ok, %{session: session, agent_pid: nil}, {:continue, :start_agent}}
      {:error, :not_found} -> {:ok, %{session: new_session(args), agent_pid: nil}, {:continue, :start_agent}}
    end
  end

  # If the agent worker crashes, the conversation process survives
  # and can restart it with preserved session state
  def handle_info({:DOWN, _ref, :process, pid, reason}, %{agent_pid: pid} = state) do
    Logger.warning("Agent crashed: #{inspect(reason)}, restarting")
    AITrace.event(:agent_crash, %{session: state.session.id, reason: reason})
    {:noreply, state, {:continue, :start_agent}}
  end
end

3.3.4 Skill Registry (ClEx.SkillRegistry)

Skills are the CL_EX equivalent of OpenClaw's tools, but implemented as Jido
skills with hot-loading capability:

defmodule ClEx.Skills.WebSearch do
  use Jido.Skill

  @impl true
  def cmd(%{query: query}, ctx) do
    # ... perform web search ...
    {:ok, %{results: results}}
  end
end

The registry is a Synapse-managed signal registry that maps skill names to
modules and tracks availability:

ClEx.SkillRegistry.register(:web_search, ClEx.Skills.WebSearch, %{
  description: "Search the web",
  parameters: [%{name: :query, type: :string, required: true}],
  policy: :standard,          # GUARDRAIL policy tier
  sandbox: :none,             # No sandboxing needed
  cost: :low                  # For billing/rate-limiting
})

Skills can be loaded, unloaded, and updated at runtime without restarting any
process. New skills can be deployed as hot-code-loaded modules or as MCP tool
servers mediated by GUARDRAIL.

3.3.5 Memory Manager (ClEx.MemoryManager)

Long-term memory and semantic search powered by portfolio_core (RAG):

defmodule ClEx.MemoryManager do
  @doc """
  Store a memory fragment with embedding for semantic retrieval.
  """
  def remember(identity_id, content, metadata \\ %{}) do
    embedding = ClEx.Embeddings.embed(content)
    Portfolio.Core.index(
      collection: "memory:#{identity_id}",
      document: %{content: content, metadata: metadata, embedding: embedding}
    )
  end

  @doc """
  Semantic search across an identity's memory.
  """
  def recall(identity_id, query, opts \\ []) do
    embedding = ClEx.Embeddings.embed(query)
    Portfolio.Core.search(
      collection: "memory:#{identity_id}",
      query_embedding: embedding,
      limit: Keyword.get(opts, :limit, 10),
      threshold: Keyword.get(opts, :threshold, 0.7)
    )
  end
end

CL_EX uses portfolio_core's distributed RAG with Bedrock-backed index storage,
meaning memories are available from any node in the cluster with consistent
reads. See Doc 04 Section 2 for a comparison with OpenClaw's SQLite-based
approach.

---

3.4 Layer 4: Agent Runtime

The agent runtime is where intelligence happens. This layer is built entirely
on Jido's agent model.

3.4.1 The Jido Agent Contract

Every CL_EX agent is a Jido agent -- a pure functional module implementing
cmd/2:

defmodule ClEx.Agents.Conversational do
  use Jido.Agent

  @impl true
  def cmd(:respond, %{signal: signal, session: session, history: history}) do
    prompt = build_prompt(signal, session, history)
    skills = ClEx.SkillRegistry.available_for(session.agent_spec)

    case invoke_llm(prompt, skills, session) do
      {:ok, %{response: text, tool_calls: []}} ->
        {:ok, %{response: text, state_updates: %{}}}

      {:ok, %{response: _text, tool_calls: calls}} ->
        execute_tool_chain(calls, session)

      {:error, reason} ->
        {:error, reason}
    end
  end

  # Pure function: deterministic for the same inputs
  defp build_prompt(signal, session, history) do
    %ClEx.Prompt{
      system: session.agent_spec.system_prompt,
      history: Enum.take(history, -session.agent_spec.context_window),
      user_message: signal.content,
      available_tools: ClEx.SkillRegistry.tool_specs_for(session.agent_spec),
      memory_context: ClEx.MemoryManager.recall(session.identity_id, signal.content)
    }
  end
end

The key insight: cmd/2 is a pure function. It takes data, returns data.
No side effects. This means agents are trivially testable:

test "conversational agent responds to greeting" do
  signal = %ClEx.Message{content: "Hello!"}
  session = %ClEx.Session{agent_spec: %{system_prompt: "You are helpful."}}
  history = []

  assert {:ok, %{response: response}} =
    ClEx.Agents.Conversational.cmd(:respond, %{
      signal: signal, session: session, history: history
    })

  assert is_binary(response)
end

3.4.2 Agent FSM Strategy

Agents follow a finite state machine for their lifecycle:

                     +-------+
           +-------->| idle  |<--------+
           |         +---+---+         |
           |             |             |
      (timeout)    (signal arrives)  (error + max retries)
           |             |             |
           |         +---V---+         |
           |         |receive|         |
           |         +---+---+         |
           |             |             |
           |      (build context)      |
           |             |             |
           |         +---V----+        |
           +---------|thinking|--------+
                     +---+----+
                         |
                   (LLM response)
                         |
                  +------V------+
                  | tool_use?   |
                  +--+-------+--+
                     |       |
                  (yes)    (no)
                     |       |
               +-----V--+ +-V--------+
               | acting  | |responding|
               +-----+--+ +----+-----+
                     |          |
              (tool result)  (emit response)
                     |          |
                     +----+-----+
                          |
                      +---V---+
                      | idle  |
                      +-------+

The FSM is managed within Jido.Agent.Worker (a GenServer wrapping
Jido.AgentServer) with state transitions logged as AITrace spans.
The agent process model is GenServer-based, not :gen_statem; the FSM
states above are logical states tracked as data within the GenServer state. Each state has a configurable timeout; if the LLM takes too long
in thinking, the agent transitions to idle and emits an apology signal.

3.4.3 LLM Provider Abstraction

CL_EX supports multiple LLM providers through a unified interface:

defmodule ClEx.LLM do
  @callback chat(messages :: list(), opts :: keyword()) ::
    {:ok, ClEx.LLM.Response.t()} | {:error, reason}

  @callback stream(messages :: list(), opts :: keyword()) ::
    {:ok, Enumerable.t()} | {:error, reason}
end

defmodule ClEx.LLM.Router do
  @doc """
  Route LLM calls through crucible_* ensemble for reliability.
  Primary provider with automatic fallback.
  """
  def chat(messages, opts \\ []) do
    provider = Keyword.get(opts, :provider, :auto)

    Crucible.Ensemble.call(
      primary: resolve_provider(provider),
      fallbacks: configured_fallbacks(),
      hedging: Keyword.get(opts, :hedge, false),
      timeout: Keyword.get(opts, :timeout, 30_000),
      fn provider_mod ->
        provider_mod.chat(messages, opts)
      end
    )
  end
end

Provider mapping:

Provider	SDK Module	Notes
Anthropic	claude_agent_sdk	Claude models, tool use, streaming
Google	gemini_ex	Gemini models, multimodal
OpenAI	altar_ai	GPT models, function calling
OpenAI Codex	codex_sdk	Code-specialized models
Local/Ollama	ollixir	Self-hosted models, privacy mode

crucible_* provides ensemble strategies: primary with fallback, hedged
requests (fire at two providers, take first response), adversarial validation
(ask a second model to verify the first), and cost-optimized routing (use
cheaper models for simple queries, expensive ones for complex reasoning).

3.4.4 DSPEx Self-Improving Chains

For agents that need to improve over time, DSPEx provides declarative
self-improving prompt pipelines. Note: DSPEx integration is optional and
planned for Phase 6. The agent runtime does not depend on DSPEx; agents
function fully without it.

defmodule ClEx.Agents.CodeReview do
  use DSPEx.Program

  signature "code_diff, project_context -> review_comments, severity"

  chain do
    step :analyze,  DSPEx.ChainOfThought, signature: "code_diff -> analysis"
    step :context,  DSPEx.Retrieve,       k: 5, index: :project_docs
    step :review,   DSPEx.ChainOfThought, signature: "analysis, context -> review_comments, severity"
  end

  # DSPEx automatically optimizes prompts based on feedback signals
  optimize_with DSPEx.BootstrapFewShot, metric: :review_quality
end

---

3.5 Layer 5: Infrastructure

3.5.1 Bedrock (Distributed State)

Bedrock is the persistence backbone. Every piece of CL_EX state that must
survive process crashes, node failures, or cluster splits lives in Bedrock.

Key data stored in Bedrock:

Key Pattern	Data	Access Pattern
identity:{id}	User identity record	Consistent read
session:{agent_id}:{conversation_id}	Session state	Serialized writes
config:router	Routing configuration	Watch + cache
config:skills	Skill registry state	Watch + cache
config:transport:{channel}	Transport configuration	Watch + cache
rate_limit:{identity}:{window}	Rate limit counters	Atomic increment
agent_spec:{name}	Agent specifications	Read-heavy

Bedrock's ACID++ guarantees mean:

• Session state is never partially written (atomicity)
• Concurrent messages from the same user see a consistent session (isolation)
• Written state survives node crashes (durability)
• Any node can read the latest state (consistent reads)

3.5.2 Foundation (Resilience)

Foundation provides the resilience primitives that keep CL_EX stable under load
and failure:

# Circuit breaker around external API calls
Foundation.CircuitBreaker.call(:telegram_api, fn ->
  Telegram.Bot.API.send_message(chat_id, text)
end, threshold: 5, timeout: 60_000)

# Rate limiter for per-user message processing
Foundation.RateLimiter.check(:user_messages, identity_id,
  limit: 30, window: :timer.minutes(1))

# Semaphore for concurrent LLM calls (control cost)
Foundation.Semaphore.acquire(:llm_calls, fn ->
  ClEx.LLM.Router.chat(messages)
end, max: 10)

# Retry with exponential backoff for transient failures
Foundation.Retry.with_retry(fn ->
  ClEx.LLM.Router.chat(messages)
end, max_attempts: 3, backoff: :exponential, base: 1_000)

3.5.3 AITrace (Observability)

Every significant operation in CL_EX produces AITrace telemetry:

defmodule ClEx.Conversation do
  def handle_signal(signal, state) do
    AITrace.span(:conversation_turn, %{session_id: state.session.id}) do
      AITrace.span(:routing, %{channel: signal.channel}) do
        decision = ClEx.Router.route(signal, config())
      end

      AITrace.span(:agent_execution, %{agent: decision.agent_spec.name}) do
        result = Jido.Agent.Worker.call(state.agent_pid, {:respond, signal})
      end

      AITrace.event(:token_usage, %{
        input_tokens: result.usage.input,
        output_tokens: result.usage.output,
        model: result.model,
        cost_usd: calculate_cost(result.usage, result.model)
      })

      emit_response(result, state)
    end
  end
end

Traces export to OpenTelemetry-compatible backends. Citadel aggregates traces
for real-time dashboards showing:

• Messages per second by channel
• Agent response latency (p50, p95, p99)
• LLM token usage and cost by model
• Error rates and circuit breaker states
• Active sessions and memory usage

3.5.4 Sandbox & Sprites (Execution Isolation)

When agents need to execute code, two isolation modes are available:

Sandbox (local): Uses the Sandbox library to run code in an isolated OTP
application within the same BEAM node. Fast startup (~10ms), limited isolation
(same OS), suitable for trusted Elixir/Erlang code execution.

Sandbox.run(fn ->
  # This runs in an isolated OTP application
  # with its own supervision tree and application env
  Code.eval_string(user_code)
end, timeout: 30_000, memory_limit: 100_000_000)

Sprites (remote VM): For untrusted code, arbitrary languages, or
resource-intensive computation, CL_EX provisions Sprites -- disposable Fly.io
VMs with:

• Instant creation (~300ms boot)
• Persistent storage (object-storage-backed volumes)
• Auto-sleep when idle (cost control)
• Inside-out orchestration (the Sprite calls back to CL_EX, not the other way)
• Full OS-level isolation

defmodule ClEx.Skills.CodeExecution do
  use Jido.Skill

  def cmd(%{code: code, language: lang}, ctx) do
    sprite = ClEx.Sprites.provision(%{
      image: image_for_language(lang),
      memory: 512,
      timeout: 60_000
    })

    case ClEx.Sprites.execute(sprite, code) do
      {:ok, output} ->
        ClEx.Sprites.destroy(sprite)
        {:ok, %{output: output}}
      {:error, reason} ->
        ClEx.Sprites.destroy(sprite)
        {:error, reason}
    end
  end
end

---

4. Layer-to-Ecosystem Component Mapping

See Doc 03 (Ecosystem Mapping) for a detailed breakdown of each ecosystem
package, including hex alternatives and integration notes.

+-------------------+--------------------------------------------------+
| Layer             | Ecosystem Components                             |
+-------------------+--------------------------------------------------+
| Transport         | Custom GenServers per platform                   |
|                   | Foundation (circuit breakers, reconnection)       |
|                   | exdantic (signal schema validation)              |
+-------------------+--------------------------------------------------+
| Normalization     | Synapse (pipeline orchestration)                 |
|                   | GUARDRAIL (policy evaluation)                    |
|                   | Foundation (rate limiting)                        |
|                   | FlowStone (media asset tracking)                 |
|                   | Bedrock (identity storage)                       |
|                   | exdantic (schema validation)                     |
+-------------------+--------------------------------------------------+
| Orchestration     | Synapse (signal registry, workflow engine)        |
|                   | Bedrock (session state, routing config)           |
|                   | portfolio_core (RAG / memory)                    |
|                   | OTP DynamicSupervisor (conversation processes)   |
+-------------------+--------------------------------------------------+
| Agent Runtime     | Jido (agent contract, skills, FSM, workers)      |
|                   | altar_ai / gemini_ex / claude_agent_sdk /        |
|                   |   codex_sdk / ollixir (LLM providers)            |
|                   | crucible_* (ensemble, hedging, adversarial)      |
|                   | DSPEx (self-improving programs)                  |
|                   | mcp_client (MCP tool servers)                    |
+-------------------+--------------------------------------------------+
| Infrastructure    | Bedrock (distributed KV)                         |
|                   | Foundation (resilience primitives)                |
|                   | AITrace (observability)                          |
|                   | Citadel (command and control)                    |
|                   | Sandbox (local isolation)                        |
|                   | Sprites (remote VM isolation)                    |
|                   | GUARDRAIL (MCP security gateway)                 |
|                   | Shield (inter-agent authentication)              |
|                   | ASKA (hardware-rooted trust)                     |
|                   | FlowStone (data orchestration, lineage)          |
+-------------------+--------------------------------------------------+

---

5. Key Innovations Over OpenClaw (Summary)

CL_EX's architecture differs from OpenClaw in six fundamental ways. Each is
covered in detail in Doc 04 (Innovations and Advantages); this section
provides a brief summary for architectural context.

1. Process-per-conversation vs. single event loop. Each conversation is a
   supervised OTP process tree with full isolation and preemptive scheduling.
2. Distributed state vs. local files. All state lives in Bedrock with
   cluster-wide replication and consistent reads, replacing JSONL files and
   node-local SQLite.
3. Zero-trust security vs. static allowlists. GUARDRAIL, Shield, and ASKA
   provide dynamic, context-aware, hierarchical security at every boundary.
4. Heterogeneous compute vs. Docker-only sandboxing. Local BEAM, Sandbox
   OTP apps, Sprites (remote VMs), cloud functions, and GPU VPS pools.
5. Self-improving agents vs. static prompts. DSPEx prompt optimization with
   automatic feedback-driven improvement (Phase 6).
6. Native observability vs. bolted-on logging. AITrace structured telemetry
   for every operation, with Citadel dashboards and alerting.

---

6. The Zero-Config Magic

When a developer adds cl_ex to their mix.exs and provides minimal
configuration (an LLM API key and at least one transport credential), the
following happens automatically:

6.1 Application Startup Sequence

# mix.exs
defp deps do
  [{:cl_ex, "~> 0.1"}]
end

# config/config.exs
config :cl_ex,
  llm: [provider: :anthropic, api_key: System.get_env("CLEX_ANTHROPIC_API_KEY")],
  transports: [
    telegram: [token: System.get_env("CLEX_TELEGRAM_BOT_TOKEN")]
  ]

On Application.start/2:

1. ClEx.Application.start/2
   |
   2. Start ClEx.Supervisor (rest_for_one strategy)
      |
      3. Start Bedrock node (joins existing cluster if libcluster discovers peers)
      |  -> Local state store ready
      |  -> Cluster state syncing in background
      |
      4. Start Foundation services
      |  -> Circuit breakers registered
      |  -> Rate limiters initialized
      |  -> Semaphores configured
      |
      5. Start AITrace collector
      |  -> Telemetry handlers attached
      |  -> Export pipeline started (stdout by default, OTLP if configured)
      |
      6. Start ClEx.SkillRegistry
      |  -> Built-in skills registered (web_search, code_exec, memory, etc.)
      |  -> MCP tool discovery via mcp_client (if MCP servers configured)
      |
      7. Start GUARDRAIL policy engine
      |  -> Default policies loaded (safe defaults, no tool access without explicit grant)
      |  -> Policy watch on Bedrock for live updates
      |
      8. Start ClEx.MemoryManager
      |  -> portfolio_core index initialized
      |  -> Embedding model loaded (or API configured)
      |
      9. Start ClEx.Router
      |  -> Default routing config (all channels -> default_agent)
      |
      10. Start ClEx.ConversationSup (DynamicSupervisor, initially empty)
      |
      11. Start ClEx.Transport.Supervisor
      |   -> For each configured transport, start the transport GenServer
      |   -> Each transport connects to its platform
      |   -> Connection success/failure logged via AITrace
      |
      12. Ready. Accepting messages.

Elapsed time from Application.start to accepting messages: < 2 seconds.

6.2 Auto-Clustering

If libcluster is configured (or CL_EX detects a Kubernetes/Fly.io environment),
nodes auto-discover each other. Bedrock syncs state across nodes. New nodes
immediately participate in handling conversations. No manual sharding, no
external coordination service.

# In a Fly.io environment, this happens automatically:
# Node A starts, joins cluster
# Node B starts, discovers Node A via DNS
# Bedrock on B replicates from A
# Transport connections are distributed: Telegram on A, Discord on B
# Conversations route to whichever node has capacity

6.3 Self-Healing Behaviors

• Transport disconnection: Circuit breaker trips, reconnect with backoff,
  Citadel alert. Conversations on that channel queue; messages delivered when
  connection restores.

• Agent crash: Conversation GenServer catches the DOWN signal, restarts the
  agent worker, rehydrates state from Bedrock, retries the last operation.
  User sees at most a brief delay.

• Node crash: Other nodes detect via BEAM distribution heartbeat. Bedrock
  state is already replicated. Conversations that were on the dead node are
  restarted on surviving nodes within seconds (triggered by the next incoming
  message for that session, or proactively by Citadel).

• LLM provider outage: crucible_* ensemble automatically fails over to the
  configured fallback provider. If all providers are down, agent transitions to
  waiting state and queues the request for retry.

• Memory pressure: Idle conversation processes are terminated first (state is
  safe in Bedrock). If pressure continues, Foundation back-pressure signals
  propagate through the normalization layer, slowing intake.

---

7. Heterogeneous Compute Model

CL_EX operates across a heterogeneous compute fabric:

+------------------------------------------------------------------+
|                    CL_EX COMPUTE FABRIC                          |
|                                                                  |
|  +------------------+  +------------------+  +----------------+  |
|  | BEAM Node A      |  | BEAM Node B      |  | BEAM Node C    |  |
|  | (orchestration)  |  | (orchestration)  |  | (orchestration)|  |
|  | Bedrock shard    |  | Bedrock shard    |  | Bedrock shard  |  |
|  | Telegram txport  |  | Discord txport   |  | Slack txport   |  |
|  | 50 conversations |  | 80 conversations |  | 30 convos      |  |
|  +--------+---------+  +--------+---------+  +-------+--------+  |
|           |                      |                    |           |
|           +----------+-----------+--------------------+           |
|                      |                                           |
|  +-------------------V-------------------------------------------+
|  |              BEAM Distribution Layer (Erlang mesh)            |
|  +-------------------+-------------------------------------------+
|                      |
|  +-------------------V-------------------+  +--------------------+
|  |          Sprite Pool                  |  |   GPU VPS Pool     |
|  |  +--------+  +--------+  +--------+  |  |  +--------+        |
|  |  |Sprite 1|  |Sprite 2|  |Sprite 3|  |  |  |GPU VM 1|        |
|  |  |Python  |  |Node.js |  |Rust    |  |  |  |Ollama  |        |
|  |  |sandbox |  |sandbox |  |sandbox |  |  |  |local   |        |
|  |  +--------+  +--------+  +--------+  |  |  |models  |        |
|  |  (Fly.io Machines, auto-sleep)        |  |  +--------+        |
|  +---------------------------------------+  +--------------------+

7.1 BEAM Nodes (Orchestration Tier)

BEAM nodes are the primary compute tier. They run all orchestration logic,
maintain transport connections, host Bedrock shards, and execute conversation
processes. BEAM nodes are stateless in the traditional sense -- all persistent
state is in Bedrock. This means any BEAM node can be replaced at any time.

BEAM nodes communicate via Erlang distribution (encrypted with TLS). Process
migration is not needed because state lives in Bedrock; instead, a conversation
process simply starts on whichever node receives the next message for that
session.

7.2 Sprites (Sandboxed Execution Tier)

Sprites are Fly.io Machines (microVMs) managed by ClEx.Sprites.Manager:

defmodule ClEx.Sprites.Manager do
  use GenServer

  @doc "Provision a Sprite for code execution"
  def provision(spec) do
    GenServer.call(__MODULE__, {:provision, spec})
  end

  def handle_call({:provision, spec}, _from, state) do
    # 1. Check pool for idle Sprite matching spec
    case find_idle_sprite(state.pool, spec) do
      {:ok, sprite} ->
        {:reply, {:ok, wake_sprite(sprite)}, state}
      :none ->
        # 2. Create new Sprite via Fly.io Machines API
        {:ok, sprite} = Fly.Machines.create(%{
          image: spec.image,
          guest: %{memory_mb: spec.memory, cpus: spec.cpus},
          auto_destroy: true,
          restart: %{policy: "no"},
          metadata: %{cl_ex_session: spec.session_id}
        })
        {:reply, {:ok, sprite}, %{state | pool: [sprite | state.pool]}}
    end
  end
end

Inside-out orchestration: Sprites do not expose SSH or direct shell access
from CL_EX. Instead, each Sprite runs a lightweight agent process that connects
back to CL_EX via WebSocket. This inverts the control flow:

  CL_EX BEAM Node                    Sprite VM
  +--------------+                   +------------------+
  |              |<---WebSocket------|  sprite_agent    |
  | Sprites.Hub  |                   |    |             |
  |              |---task payload--->|    +-> executor  |
  |              |<---result---------|    +-> sandbox   |
  +--------------+                   +------------------+

This inside-out model means:

• No inbound firewall rules on CL_EX nodes
• Sprites can be in any network/region
• Connection is authenticated via Shield tokens
• If a Sprite disconnects, the task is retried on a new Sprite

7.3 GPU VPS (Inference Tier)

For self-hosted model inference (via ollixir), CL_EX can provision and manage
GPU VPS instances:

config :cl_ex, :gpu_pool,
  provider: :lambda_labs,  # or :vast_ai, :runpod, etc.
  instances: [
    %{model: "llama-3.3-70b", gpu: "A100", count: 1, auto_scale: true},
    %{model: "codellama-34b", gpu: "A10G", count: 2, auto_scale: false}
  ]

GPU instances are managed as a resource pool with Foundation semaphores
controlling access and Foundation circuit breakers handling failures.

7.4 Compute Routing

ClEx.ComputeRouter decides where to execute each task:

defmodule ClEx.ComputeRouter do
  def route(%{type: :elixir_code, trusted: true}),   do: {:sandbox, :local}
  def route(%{type: :elixir_code, trusted: false}),   do: {:sprite, :elixir}
  def route(%{type: :python_code}),                    do: {:sprite, :python}
  def route(%{type: :shell_command}),                  do: {:sprite, :ubuntu}
  def route(%{type: :llm_inference, model: model}),    do: {:gpu_vps, model}
  def route(%{type: :web_search}),                     do: {:local, :beam}
  def route(%{type: :file_operation, size: s}) when s > 1_000_000_000,
                                                       do: {:sprite, :storage}
  def route(_),                                        do: {:local, :beam}
end

---

8. Security Architecture

8.1 Threat Model

CL_EX faces threats at every layer:

Threat	Layer	Mitigation
Malicious user input	Transport	Rate limiting, content policy
Prompt injection	Normalizer	Input sanitization, GUARDRAIL
Tool abuse (data exfil)	Orchestrate	GUARDRAIL policy, Shield scope
Agent privilege escalation	Runtime	Shield hierarchy, capability tokens
Untrusted code execution	Infra	Sprites isolation, Sandbox limits
Inter-node eavesdropping	Infra	TLS distribution, ASKA attestation
State tampering	Infra	Bedrock integrity checks

8.2 GUARDRAIL: MCP Security Gateway

GUARDRAIL sits between agents and all external tool access (MCP servers, APIs,
file systems, network resources). Every tool call passes through GUARDRAIL
before execution.

  Agent -> GUARDRAIL -> Tool Server
              |
              +-> Policy Engine
              |     +-> Identity context (who is the user?)
              |     +-> Agent context (what agent is calling?)
              |     +-> Conversation context (what is the topic?)
              |     +-> Tool risk level (read-only? destructive? network?)
              |     +-> Historical context (what has this session done?)
              |
              +-> Decision: ALLOW | DENY | ASK_USER | MODIFY

GUARDRAIL policies are expressed declaratively:

%GUARDRAIL.Policy{
  name: :file_system_access,
  rules: [
    # Read-only access to project files is always allowed
    %{tool: "read_file", scope: ~r{^/project/}, action: :allow},
    # Writing requires user confirmation
    %{tool: "write_file", scope: ~r{^/project/}, action: :ask_user},
    # System files are never accessible
    %{tool: :any, scope: ~r{^/(etc|usr|sys)/}, action: :deny},
    # Network access is logged and rate-limited
    %{tool: "http_request", action: :allow, audit: true,
      rate_limit: {10, :per_minute}}
  ]
}

GUARDRAIL policies are dynamic, context-aware, and support runtime
modification via Bedrock-watched configuration. Policies can be updated live
without restart. See Doc 04 Section 4 for a comparison with OpenClaw's
static policy chain.

8.3 Shield: Hierarchical Inter-Agent Trust

When agents spawn sub-agents or communicate across the cluster, Shield
authenticates and authorizes every message:

  Parent Agent (permission set P)
       |
       +--- spawns ---> Child Agent (permission set C, where C ⊆ P)
                              |
                              +--- cannot escalate beyond P
                              +--- all tool calls filtered by C
                              +--- all inter-agent messages signed

Shield uses capability tokens: unforgeable, scopeable, time-limited credentials
that encode exactly what an agent is allowed to do.

# Parent agent creates a scoped capability for child
capability = Shield.mint_capability(%{
  parent: parent_agent_id,
  child: child_agent_id,
  permissions: [:read_files, :web_search],  # Subset of parent's permissions
  ttl: :timer.minutes(5),
  max_tool_calls: 20
})

# Child agent presents capability with every tool call
Shield.authorize(capability, %{tool: "read_file", args: %{path: "/project/foo"}})
# => :ok

Shield.authorize(capability, %{tool: "write_file", args: %{path: "/etc/passwd"}})
# => {:error, :permission_denied}

8.4 ASKA Principles: Hardware-Rooted Trust

Where available (SGX/TDX-capable hardware), ASKA provides:

• Sealed storage: LLM API keys and user credentials stored in hardware
  enclaves, inaccessible even to the OS.
• Remote attestation: BEAM nodes prove their integrity to each other before
  joining the cluster. A compromised node cannot participate.
• Confidential computing: Sensitive conversation processing (medical,
  financial, legal) runs in encrypted enclaves where even the host operator
  cannot inspect memory.

ASKA integration is optional and gracefully degrades. Without compatible
hardware, CL_EX uses software-based encryption and standard TLS.

8.5 Security Pipeline (Full Path)

  User Message
       |
       V
  [Transport] --- TLS/platform encryption
       |
       V
  [Rate Limiter] --- Foundation.RateLimiter (identity + global)
       |
       V
  [Content Policy] --- GUARDRAIL inbound policy (content safety, input length)
       |
       V
  [Identity Resolution] --- Bedrock lookup, permission check
       |
       V
  [Agent Dispatch] --- Shield capability scoping
       |
       V
  [LLM Call] --- GUARDRAIL prompt injection filters
       |
       V
  [Tool Execution] --- GUARDRAIL tool policy + Shield capability check
       |                     |
       |               (if code execution)
       |                     V
       |               [Sprite/Sandbox] --- OS-level isolation
       |
       V
  [Response] --- Output filtering (PII, secrets, content safety)
       |
       V
  [Transport] --- TLS/platform encryption
       |
       V
  User

---

9. Error Handling

CL_EX uses a structured error contract across all modules. Every public
function returns {:ok, result} or {:error, %ClEx.Error{}}. The
ClEx.Error struct provides machine-readable error categorization:

defmodule ClEx.Error do
  defstruct [
    :code,        # atom, e.g. :transport_disconnected, :llm_timeout, :policy_denied
    :message,     # human-readable string
    :details,     # arbitrary map with context
    :recoverable  # boolean -- can the caller retry?
  ]

  @type t :: %__MODULE__{
    code: atom(),
    message: String.t(),
    details: map(),
    recoverable: boolean()
  }
end

Error propagation rules:

1. Transport errors (connection failures, API errors) are wrapped in
   %ClEx.Error{code: :transport_*} and handled by Foundation circuit
   breakers. The user is not exposed to transport errors unless all retries
   are exhausted.
2. LLM errors (timeouts, rate limits, content filters) are wrapped in
   %ClEx.Error{code: :llm_*} and handled by crucible_* ensemble fallback.
3. Policy errors (GUARDRAIL denials, Shield auth failures) are
   non-recoverable and produce a user-facing rejection message.
4. Agent errors (crashes, unexpected states) trigger OTP restart via the
   supervision tree. The conversation GenServer catches :DOWN signals and
   rehydrates from Bedrock.

All error paths produce AITrace events for observability. See Doc 02
"Cross-Cutting Concerns" for the full error contract specification.

---

10. The Self-Building / Self-Healing Paradigm

CL_EX is designed to be a platform that improves itself over time and recovers
from failures without human intervention.

10.1 Self-Healing (Reactive Recovery)

OTP supervision trees are the foundation. The supervision strategy is
deliberately designed so that failures propagate only as far as necessary:

ClEx.Application (one_for_one)
  |
  +-- ClEx.Supervisor (rest_for_one)
  |     |
  |     +-- Bedrock.Node           <- If this dies, everything above restarts
  |     +-- Foundation.Supervisor  <- Resilience primitives
  |     +-- AITrace.Collector      <- Observability
  |     +-- ClEx.SkillRegistry     <- Skills
  |     +-- GUARDRAIL.Engine       <- Security (must be up before conversations)
  |     +-- ClEx.MemoryManager     <- RAG
  |     +-- ClEx.Router            <- Routing
  |     +-- ClEx.ConversationSup   <- DynamicSupervisor for conversations
  |     +-- ClEx.Transport.Supervisor <- Transport connections
  |
  +-- Citadel.Agent                <- C2 (independent, survives core restarts)

The rest_for_one strategy means: if Bedrock crashes, everything above it
restarts in order (Foundation, then AITrace, then skills, etc.). But if a
single transport crashes, only that transport restarts -- conversations and
other transports are unaffected.

Individual conversation processes are temporary children of the
DynamicSupervisor. If a conversation crashes, it is NOT automatically
restarted by the supervisor. Instead, it is restarted on demand when the next
message arrives for that session. This prevents thundering-herd restarts and
ensures resources are only used for active conversations.

10.2 Self-Networking (Cluster Formation)

defmodule ClEx.Cluster do
  @moduledoc """
  Auto-clustering via multiple discovery strategies.
  Tries each strategy in order until one succeeds.
  """

  def child_spec(_opts) do
    topologies = [
      cl_ex: [
        strategy: determine_strategy(),
        config: strategy_config()
      ]
    ]
    {Cluster.Supervisor, [topologies, [name: ClEx.ClusterSupervisor]]}
  end

  defp determine_strategy do
    cond do
      System.get_env("FLY_APP_NAME")     -> Cluster.Strategy.DNSPoll
      System.get_env("KUBERNETES_SERVICE_HOST") -> Cluster.Strategy.Kubernetes.DNS
      System.get_env("CLEX_CLUSTER_NODES")      -> Cluster.Strategy.Epmd
      true                                       -> Cluster.Strategy.Gossip
    end
  end
end

When a new node joins:

1. libcluster discovers peers and connects via Erlang distribution
2. Bedrock syncs state from existing nodes
3. Transport connections are rebalanced (Citadel coordinates which node owns
   which transport)
4. The new node immediately begins accepting conversations

When a node leaves (graceful or crash):

1. Remaining nodes detect the departure via Node.monitor/1
2. Bedrock detects the lost shard and promotes replicas
3. Citadel reassigns orphaned transport connections to surviving nodes
4. Active conversations on the lost node are considered idle; they restart on
   demand on any surviving node with full state from Bedrock

10.3 Self-Improving (Proactive Enhancement)

CL_EX agents improve over time through multiple mechanisms:

DSPEx prompt optimization: Agents using DSPEx programs collect feedback
signals (explicit user ratings, implicit signals like conversation length,
retry rates, tool call success rates). DSPEx's bootstrap-few-shot and MIPRO
optimizers periodically generate improved prompts, which are stored in Bedrock
and hot-swapped into running agents.

Skill discovery: When an agent encounters a task it cannot complete with
existing skills, it can:

1. Search the MCP registry for relevant tool servers
2. Propose a new skill implementation (code generated by the agent itself)
3. Submit the proposed skill for review (GUARDRAIL policy check + optional
   human approval)
4. If approved, the skill is compiled, registered, and immediately available

Memory consolidation: ClEx.MemoryManager periodically consolidates
conversation memories:

1. Summarize long conversations into key facts
2. Merge duplicate or redundant memories
3. Promote frequently-accessed memories to higher priority
4. Decay stale memories that are never accessed

defmodule ClEx.MemoryManager.Consolidation do
  use GenServer

  # Runs every 6 hours
  @consolidation_interval :timer.hours(6)

  def handle_info(:consolidate, state) do
    Enum.each(active_identities(), fn identity_id ->
      memories = ClEx.MemoryManager.all(identity_id)

      if length(memories) > 1000 do
        summarized = ClEx.LLM.Router.chat([
          %{role: :system, content: "Summarize these memory fragments into key facts."},
          %{role: :user, content: format_memories(memories)}
        ])

        ClEx.MemoryManager.replace_all(identity_id, summarized)
      end
    end)

    Process.send_after(self(), :consolidate, @consolidation_interval)
    {:noreply, state}
  end
end

Cost note: Memory consolidation invokes an LLM call per identity that
exceeds the memory threshold. To control costs, consolidation should use the
cheapest available model (configured via config :cl_ex, :memory, consolidation_model: :cheapest), and the consolidation schedule and threshold
are configurable. Consolidation can be disabled entirely with
consolidation: false.

Adaptive routing: The router learns from conversation outcomes. If a
particular agent consistently performs poorly on certain types of queries
(measured by user feedback and task completion), the router can adjust to
prefer a different agent for those query patterns.

---

11. Deployment Topologies

11.1 Single Developer (Minimal)

+----------------------------------+
|  Developer Laptop                |
|  1 BEAM node                     |
|  Bedrock (single-node mode)      |
|  1-2 transports (Telegram+Web)   |
|  Local Sandbox for code exec     |
|  ollixir for local LLM           |
+----------------------------------+

mix.exs dependencies + a few env vars. That is it.

11.2 Small Team (Production)

+------------------+  +------------------+
|  Fly.io Node A   |  |  Fly.io Node B   |
|  BEAM + Bedrock  |  |  BEAM + Bedrock  |
|  Telegram, Slack |  |  Discord, Web    |
+--------+---------+  +--------+---------+
         |                      |
         +----------+-----------+
                    |
         +----------V-----------+
         |  Sprite Pool (Fly)   |
         |  3-5 auto-sleeping   |
         |  microVMs             |
         +----------------------+

Two Fly.io machines with auto-clustering. Sprites for sandboxed execution.
Bedrock replicates between both nodes. Total cost: ~$20/month.

11.3 Enterprise (Multi-Region)

+------ US-EAST ------+  +------ EU-WEST ------+  +------ AP-SOUTH -----+
| 3 BEAM nodes         |  | 3 BEAM nodes         |  | 2 BEAM nodes        |
| Bedrock (3 replicas) |  | Bedrock (3 replicas) |  | Bedrock (2 replicas)|
| All transports       |  | All transports       |  | All transports      |
| GPU VPS (2x A100)    |  | GPU VPS (1x A100)    |  | (API-only LLM)      |
| 10 Sprites           |  | 5 Sprites            |  | 3 Sprites           |
+----------------------+  +----------------------+  +---------------------+
         |                          |                         |
         +------------- Erlang Distribution (TLS) -----------+
         +------------- Bedrock Cross-Region Sync ------------+

Conversations route to the nearest region. Bedrock provides eventual
consistency across regions with strong consistency within a region. ASKA
attestation validates node integrity before cluster admission.

Cross-region consistency note: "Consistent reads from any node" (Section
3.5.1) applies within a single region. Across regions, Bedrock uses eventual
consistency. If a user messages from EU-WEST and then immediately from
AP-SOUTH, the AP-SOUTH node may serve slightly stale session state. In
practice, this is mitigated by sticky routing (conversations prefer the region
where they were started) and Bedrock's low replication lag (sub-second under
normal conditions).

---

12. Configuration Reference

All configuration lives under the :cl_ex application namespace. When CL_EX
sub-packages are used independently, they share this namespace with component
sub-keys rather than using separate config namespaces.

Environment variable convention: All CL_EX-specific environment variables
use the CLEX_ prefix (e.g., CLEX_ANTHROPIC_API_KEY,
CLEX_TELEGRAM_BOT_TOKEN, CLEX_CLUSTER_NODES). This avoids collisions with
other applications that may use the same provider SDKs.

The full configuration surface with defaults:

config :cl_ex,
  # LLM configuration
  llm: [
    primary: [provider: :anthropic, model: "claude-sonnet-4-20250514", api_key: nil],
    fallbacks: [
      [provider: :google, model: "gemini-2.0-flash", api_key: nil],
      [provider: :local, model: "llama-3.3-70b", endpoint: "http://localhost:11434"]
    ],
    ensemble: :primary_with_fallback,  # :hedged, :adversarial, :cost_optimized
    max_concurrent: 10,
    default_timeout: 30_000
  ],

  # Transport configuration
  transports: [
    telegram: [token: nil, webhook_url: nil, mode: :polling],
    discord: [token: nil, intents: [:guilds, :guild_messages, :direct_messages]],
    slack: [app_token: nil, bot_token: nil, mode: :socket],
    signal: [phone: nil, bridge: :signal_cli],
    imessage: [bridge: :bluebubbles, url: nil, password: nil],
    whatsapp: [mode: :cloud_api, token: nil, phone_number_id: nil],
    matrix: [homeserver: nil, access_token: nil],
    web: [enabled: true, port: 4000],
    irc: [server: nil, nick: nil, channels: []],
    sms: [provider: :twilio, account_sid: nil, auth_token: nil],
    email: [imap: nil, smtp: nil]
  ],

  # Agent configuration
  agents: [
    default: [
      module: ClEx.Agents.Conversational,
      system_prompt: "You are a helpful AI assistant.",
      context_window: 50,
      skills: [:web_search, :memory, :calculator]
    ]
  ],

  # Session configuration
  sessions: [
    idle_timeout: :timer.minutes(15),
    max_history: 200,
    persist_interval: :timer.seconds(30)
  ],

  # Security configuration
  security: [
    guardrail: [default_policy: :restrictive],
    shield: [enabled: true, capability_ttl: :timer.minutes(30)],
    aska: [enabled: false]  # Requires compatible hardware
  ],

  # Compute configuration
  compute: [
    sandbox: [enabled: true, timeout: 30_000, memory_limit: 100_000_000],
    sprites: [enabled: false, fly_api_token: nil, region: "iad"],
    gpu: [enabled: false, provider: nil]
  ],

  # Observability
  observability: [
    aitrace: [export: :stdout],  # :otlp, :datadog, :honeycomb
    citadel: [dashboard: true, port: 4001]
  ],

  # Clustering
  cluster: [
    strategy: :auto,  # :dns, :kubernetes, :epmd, :gossip, :none
    cookie: nil        # Auto-generated if not set
  ]

---

13. What Comes Next

This document establishes the architectural foundation. The implementation
roadmap proceeds in phases:

1. Phase 1 - Core Loop: Transport (Telegram only) -> Normalization ->
   Orchestration -> Agent (single Jido agent, single LLM provider) ->
   Bedrock session storage. Minimum viable conversation.

2. Phase 2 - Multi-Channel: Add Discord, Slack, Web transports.
   Identity resolution across channels. Router with channel-specific agents.

3. Phase 3 - Tools & Security: Skill registry, GUARDRAIL policy engine,
   Shield inter-agent auth. MCP tool server integration.

4. Phase 4 - Distribution: Multi-node clustering, Bedrock replication,
   transport rebalancing, Citadel C2.

5. Phase 5 - Heterogeneous Compute: Sprites integration, GPU VPS pools,
   FlowStone data orchestration.

6. Phase 6 - Self-Improvement: DSPEx prompt optimization, adaptive routing,
   memory consolidation, skill discovery.

---

This document is a living artifact. As implementation proceeds, sections will
be expanded with concrete module interfaces, protocol specifications, and
deployment guides.

[nshkrdotcom]

The Three Execution Modes:

1. DETERMINISTIC  - Pure state machine, no LLM
2. BOUNDED AGENT  - LLM in loop, but constrained action space
3. EVOLUTION      - LLM changes the system itself (out of band)

The mistake everyone makes is treating these as a spectrum. They're not. They're three distinct execution contexts that compose.

The Core Abstraction: Workflows as Typed Effect Systems

┌─────────────────────────────────────────────────────────────────┐
│                        WORKFLOW DEFINITION                       │
│                                                                  │
│  workflow :process_document do                                   │
│    step :classify,    type: :deterministic, handler: Classifier │
│    step :extract,     type: :bounded_agent, handler: Extractor  │
│    step :validate,    type: :deterministic, handler: Validator  │
│    step :remediate,   type: :bounded_agent, handler: Remediator │
│    step :persist,     type: :deterministic, handler: Storage    │
│  end                                                             │
│                                                                  │
└─────────────────────────────────────────────────────────────────┘

Each step declares its execution type. The runtime enforces different guarantees per type.

The Runtime Contract Per Type:

DETERMINISTIC:
  - Same input → same output (referentially transparent)
  - No side effects except declared outputs
  - Fails fast, no retries with variation
  - Fully replayable from logs

BOUNDED_AGENT:
  - Declared action space (what it CAN do)
  - Declared resource limits (tokens, time, actions)
  - Each action is validated before execution
  - Can be interrupted/rolled back at any action boundary
  - Outcome is non-deterministic but VERIFIABLE

EVOLUTION (out of band):
  - Changes workflow definitions, prompts, rules
  - Gated by test suite
  - Atomic deployment
  - Never runs during workflow execution

The Architecture:

┌─────────────────────────────────────────────────────────────────┐
│                         EVOLUTION PLANE                          │
│  ┌──────────┐  ┌──────────┐  ┌──────────┐  ┌──────────┐        │
│  │ Observer │─▶│ Analyzer │─▶│ Proposer │─▶│   Gate   │        │
│  └──────────┘  └──────────┘  └──────────┘  └────┬─────┘        │
│       ▲                                         │ deploy        │
│       │ telemetry                               ▼               │
├───────┴─────────────────────────────────────────────────────────┤
│                         ARTIFACT PLANE                           │
│  ┌─────────────────────────────────────────────────────────┐    │
│  │  Workflows │ Prompts │ Actions │ Schemas │ Rules │ Code │    │
│  │  (versioned, immutable, content-addressed)              │    │
│  └─────────────────────────────────────────────────────────┘    │
├─────────────────────────────────────────────────────────────────┤
│                        EXECUTION PLANE                           │
│                                                                  │
│  ┌─────────────────────────────────────────────────────────┐    │
│  │                   WORKFLOW RUNTIME                       │    │
│  │  ┌───────────┐  ┌───────────┐  ┌───────────┐           │    │
│  │  │DETERMINIST│  │  BOUNDED  │  │ VALIDATOR │           │    │
│  │  │ EXECUTOR  │  │   AGENT   │  │           │           │    │
│  │  │           │  │  SANDBOX  │  │           │           │    │
│  │  └───────────┘  └───────────┘  └───────────┘           │    │
│  └─────────────────────────────────────────────────────────┘    │
│                              │                                   │
│                              ▼                                   │
│  ┌─────────────────────────────────────────────────────────┐    │
│  │                    EFFECT HANDLERS                       │    │
│  │   IO │ HTTP │ DB │ LLM │ FileSystem │ External APIs     │    │
│  └─────────────────────────────────────────────────────────┘    │
└─────────────────────────────────────────────────────────────────┘

The Bounded Agent Sandbox - This Is The Key:

The agent doesn't have "tools." It has a declared effect space:

@bounded_agent(
    max_steps=10,
    max_tokens=50000,
    timeout_seconds=120,
    
    # The action space - agent can ONLY do these things
    effects=[
        Effect.read("document", schema=DocumentSchema),
        Effect.write("extracted", schema=ExtractedSchema),
        Effect.call("validate_entity", handler=validate_entity),
        Effect.call("lookup_reference", handler=lookup_reference),
    ],
    
    # Hard boundaries
    cannot=[
        Effect.write("document"),  # Can't mutate input
        Effect.call("delete_*"),   # Can't delete anything
        Effect.http("*"),          # No arbitrary HTTP
    ],
    
    # Verification after each action
    invariants=[
        lambda state: len(state.extracted) <= len(state.document.entities),
        lambda state: state.step_count < 10,
    ],
    
    # What happens on failure
    on_failure=Fallback.return_partial | Fallback.escalate_human,
)
def extract_entities(document: Document) -> Extracted:
    """Agent figures out HOW, but within these bounds."""
    ...

The Execution Model:

AGENT STEP LOOP:
  1. Agent proposes action (LLM call)
  2. Runtime checks: is action in declared effect space?
  3. Runtime checks: do invariants still hold?
  4. Runtime checks: resource budget remaining?
  5. If all pass: execute action, record in trace
  6. If any fail: halt, trigger on_failure

This gives you:
  - Agentic flexibility (agent chooses actions)
  - Deterministic BOUNDARIES (action space is fixed)
  - Full auditability (trace of all actions)
  - Rollback capability (at action boundaries)

The Composition Model:

Workflows compose the three types:

workflow :intelligence_pipeline do
  # Deterministic: route based on rules
  step :route do
    type :deterministic
    handler fn doc -> 
      Rules.match(doc, artifacts().routing_rules)
    end
  end
  
  # Bounded agent: complex extraction
  step :extract do
    type :bounded_agent
    effects [:read_doc, :write_entities, :call_validator]
    limits [steps: 15, tokens: 100_000]
    verify fn result -> Schema.valid?(result, EntitySchema) end
  end
  
  # Deterministic: always runs same way
  step :normalize do
    type :deterministic
    handler &Normalizer.run/1
  end
  
  # Bounded agent: only if extraction confidence low
  step :remediate do
    type :bounded_agent
    when fn ctx -> ctx.extract.confidence < 0.8 end
    effects [:read_entities, :write_entities, :call_llm_verify]
    limits [steps: 5, tokens: 20_000]
  end
  
  # Deterministic: persist
  step :persist do
    type :deterministic
    handler &Storage.write/1
  end
end

The Evolution Loop Drives Everything:

WHAT EVOLVES:
  - Routing rules (which path through workflow)
  - Prompts (how agent reasons)
  - Action space definitions (what agent CAN do)
  - Verification schemas (what's valid output)
  - Workflow structure itself (add/remove/reorder steps)

WHAT DOESN'T EVOLVE AT RUNTIME:
  - Core execution semantics
  - Security boundaries
  - Effect handler implementations

EVOLUTION TRIGGERS:
  - Failure rate exceeds threshold
  - Confidence distribution shifts
  - New input patterns detected
  - Scheduled (nightly optimization)
  - Manual (human says "improve this")

The Data Model:

TRACE (immutable log of execution):
  workflow_id: uuid
  version: content_hash of artifacts used
  steps: [
    {name: :route, type: :deterministic, input_hash: x, output_hash: y, duration_ms: 5},
    {name: :extract, type: :bounded_agent, actions: [
      {action: :read_doc, result_hash: a},
      {action: :call_validator, input_hash: b, result_hash: c},
      {action: :write_entities, value_hash: d},
    ], total_tokens: 12000, duration_ms: 3400},
    ...
  ]
  outcome: :success | {:failure, reason}

ARTIFACT (versioned, content-addressed):
  type: :prompt | :rule | :schema | :workflow | :action_space
  content_hash: sha256
  content: ...
  valid_from: timestamp
  valid_to: timestamp | nil
  parent: content_hash | nil  # evolution lineage

The Key Insight:

"Deterministic" doesn't mean "no LLM." It means the boundaries are deterministic:

FULLY DETERMINISTIC:
  input → [pure function] → output
  (same input always same output)

BOUNDED STOCHASTIC:
  input → [agent in sandbox] → output ∈ ValidOutputSpace
  (output varies, but always valid, always auditable)
  
The BOUNDARY is deterministic even if the PATH isn't.

For Your Elixir/Python Split:

ELIXIR (execution plane):
  - Workflow runtime
  - Effect handler dispatch  
  - Sandbox enforcement
  - Trace recording
  - Artifact caching

PYTHON (workers):
  - LLM calls
  - Bounded agent "brains"
  - Evolution proposals
  - ML inference

SHARED (artifact plane):
  - Content-addressed store
  - Schema definitions
  - Could be SQLite, could be git, could be S3

What Would You Call This?

The pattern is: Typed Effects with Bounded Agency

Or maybe: Deterministic Envelope, Stochastic Core

The system is deterministic in its structure and boundaries. The agency happens inside verified sandboxes. Evolution happens out-of-band and is gated.

Does this framing work? Want me to sketch the actual interfaces?

[nshkrdotcom]

My bad on putting this here. None of this is built. It's just brainstorms that anyone can use, copy, change, for any jido project or their own. Just keeping them in the group.