Skip to content

SUBSTRATE_NATIVE_AGENT.md

Latest commit

 

History

History
216 lines (171 loc) · 8.14 KB

SUBSTRATE_NATIVE_AGENT.md

File metadata and controls

216 lines (171 loc) · 8.14 KB

The substrate-native agent — every primitive composed

This is the demonstrable end of the week's substrate-native AI work. Every primitive shipped earlier (kernel, codec, fibtier, OMC-PROTOCOL, Prometheus, content-addressed checkpoints) is load-bearing in this single demo. Each piece's value is visible because the others are present.

The demo

omnimcode-standalone examples/substrate_agent_demo.omc

Two agents — Curio (questioner) and Sage (responder) — hold a 15-turn conversation across a simulated process restart. Each agent runs the full substrate-native AI stack:

Layer Primitive
identity fnv1a_hash(name) → sender_id (no shared key needed)
memory Persistent fibtier (~/.omc/fibtier/<name>/)
wire format OMC-PROTOCOL substrate-signed messages
persistence Manifest JSON journaled per push; reload reconstructs full state
responder Knowledge-dispatch (could be Prometheus LM via one swap)

What happens, scene by scene

Phase A — fresh start, 12-turn conversation

Both agents are constructed from scratch. Each push to memory triggers:

  1. fibtier cascade — overflow folds upward through Fibonacci tiers
  2. manifest journal — current state written to disk
  3. content-addressed entry IDs — every entry has its canonical hash

Sample turn:

[Curio → Sage] "What is CRT-PE?"
[Sage → Curio] "CRT-PE is positional encoding using sin/cos pairs
                over Fibonacci moduli {5,8,13,21,...}. It won -5.4%
                val loss on TinyShakespeare in 3/3 seeds."

Behind that two-line exchange:

  • Curio signs a 1-line wire message (~200 bytes JSON, substrate-signed)
  • Sage receives, verifies signature (omc_msg_verify returns valid: 1)
  • Sage's responder looks up CRT-PE in its knowledge dict
  • Sage signs the reply, ships it
  • Curio verifies the reply, pushes Q+A into its fibtier
  • Sage also pushes the Q+A into its fibtier
  • Both manifests update on disk

Phase B — memory snapshot after 12 turns

[curio_agent | role=questioner | sender_id=410668497]
  memory: 12 pushes, 6 folds, 6 entries
  tier occupancy: [1, 1, 3, 1, 0, 0, 0]
[sage_agent  | role=responder  | sender_id=144951395]
  memory: 12 pushes, 6 folds, 6 entries
  tier occupancy: [1, 1, 3, 1, 0, 0, 0]

12 conversation turns → 6 stored entries per agent. Memory is bounded by the Fibonacci tier capacities, not by conversation length.

Phase C — simulated process restart

(discarding in-memory state; reloading both agents from disk)

Reloaded state:
[curio_agent | ...]
  memory: 12 pushes, 6 folds, 6 entries  ← identical to pre-restart
  tier occupancy: [1, 1, 3, 1, 0, 0, 0]  ← identical

The fibtier_persistent_load reads the manifest JSON, rebuilds the in-memory representation, and the agent picks up exactly where it left off. No state lost; no shared key needed for verification.

Phase D — resume conversation

Three more turns. Curio asks a question Sage has no direct knowledge match for ("Out of all those, which gave the biggest win?"). Sage's responder falls back to querying its own fibtier memory by substrate distance, retrieves the most relevant past entry, and uses it as the response:

[Sage → Curio] "That reminds me of: Q: What is L1 substrate-K? | 
                A: L1 replaces attention's learned K matrix with the
                CRT-PE positional table. On TinyShakespeare with proper
                train/val split it wins -8.0% with ~9% fewer params,
                3/3 seeds."

This is the moment all the pieces compose: the agent's memory of past turns becomes its fallback knowledge because fibtier stored the Q+A as a substrate-addressable entry, the query found it by substrate distance, and the responder used the stored content directly.

Final state

[curio_agent]
  memory: 15 pushes, 8 folds, 7 entries
  tier occupancy: [1, 2, 2, 2, 0, 0, 0]
[sage_agent]
  memory: 15 pushes, 8 folds, 7 entries
  tier occupancy: [1, 2, 2, 2, 0, 0, 0]

15 conversation turns → 7 entries. Still bounded. Disk artifacts under ~/.omc/fibtier/{curio_agent, sage_agent}/manifest.json.

What each primitive contributed

Primitive Role in the demo Without it, what fails
fnv1a_hash Stable sender_id from agent name Identity coordination requires shared keys
omc_msg_sign / omc_msg_verify Substrate-signed wire format No integrity guarantee on inter-agent messages
fibtier_push / _cascade_overflow Bounded memory with Fibonacci tiering Context grows linearly forever
fibtier_query Substrate-distance memory retrieval Agent has no fallback for unknown queries
fibtier_persistent_* Manifest journaling Memory dies with the process
Canonical hash addressing Per-entry content identity No dedup, no integrity, no cross-agent reference
py_exec/py_eval OS path management (mkdir, env vars) Persistence layer can't bootstrap its own paths

Remove any one and the demo breaks at a specific point. They're not independent features — they're a system.

What it would take to make Sage's responder a Prometheus LM

Replace _agent_respond in examples/lib/agent.omc:

fn _agent_respond(agent, input_text) {
    h ctx = fibtier_query(dict_get(agent, "memory"), input_text, 3);
    h ctx_text = render_context(ctx);
    h prompt = concat_many(ctx_text, "\n\nQ: ", input_text, "\nA:");
    h tokens = prom_generate_greedy(
        agent_model_forward,
        dict_get(agent, "prom_model"),
        encode_chars(prompt),
        50,
        VOCAB_SIZE
    );
    return decode(tokens);
}

Plug in any trained Prometheus model (built with our L1 substrate-K attention as the default), and the agent generates substrate-native LM responses while keeping every other layer of the stack the same.

What this proves

The substrate-native AI stack OMC built this week is composable in practice, not just on a diagram. Two agents share an OMC-PROTOCOL channel; each maintains a persistent fibtier; both survive process restart; the memory layer surfaces as a fallback knowledge source when the direct responder runs out of answers.

Six primitives (codec, kernel, fibtier, protocol, prometheus, checkpoints) → one working agent demo → ~250 lines of OMC + ~150 lines of agent.omc + ~200 lines of fibtier_persistent.omc.

The architecture is the substrate. The substrate is the architecture.

Files

Path What
examples/lib/fibtier.omc In-memory Fibonacci-tier core
examples/lib/fibtier_persistent.omc Manifest-journaled persistence layer
examples/lib/agent.omc Agent abstraction (identity + memory + send/receive)
examples/substrate_agent_demo.omc The end-to-end demo script
examples/tests/test_fibtier.omc 8/8 tests
examples/tests/test_fibtier_persistent.omc 4/4 persistence tests
docs/SUBSTRATE_NATIVE_AGENT.md This file

How to reproduce

PYO3_USE_ABI3_FORWARD_COMPATIBILITY=1 cargo build --release --bin omnimcode-standalone
./target/release/omnimcode-standalone examples/substrate_agent_demo.omc

Memory artifacts persist under ~/.omc/fibtier/curio_agent/ and ~/.omc/fibtier/sage_agent/. Re-running the demo from a clean state:

rm -rf ~/.omc/fibtier/curio_agent ~/.omc/fibtier/sage_agent
./target/release/omnimcode-standalone examples/substrate_agent_demo.omc

Tests:

./target/release/omnimcode-standalone --test examples/tests/test_fibtier.omc
./target/release/omnimcode-standalone --test examples/tests/test_fibtier_persistent.omc

What's next (not part of this demo)

  • LLM-summarization fold — replace concat-fold with a py_callback to Claude/GPT for true semantic compression. Substrate captures structure; LLM captures meaning.
  • MCP exposure — wrap fibtier as MCP tools so any Claude Desktop / Cursor session gets the bounded-memory architecture natively.
  • Substrate transformer integration — wire Prometheus' L1 substrate-K transformer as the agent's response generator.
  • N-agent mesh — extend from 2 agents to a network. OMC-PROTOCOL handles arbitrary peers; fibtier handles arbitrary message volume.

Each is a natural extension of what already works.