ERL Heuristic Memory

Minimal Python implementation of ERL (Experiential Reflective Learning), a memory system that learns reusable heuristics from task trajectories and applies them to future tasks.

MVP features

• trajectory and heuristic data models
• local SQLite heuristic store
• reflection pipeline that turns trajectories into structured heuristics
• retrieval and reranking pipeline for new tasks
• pluggable LLM provider with Anthropic Messages support
• pluggable embedding interface for retrieval
• seeded benchmark and evaluation harness
• offline tests and optional live smoke tests

Planned flow

1. complete a task and record its trajectory
2. reflect on the trajectory with an LLM
3. store the extracted heuristic
4. retrieve relevant heuristics for a new task
5. inject top heuristics into the next prompt

Development

The codebase is designed to work offline by default. Live provider tests are optional and controlled through environment variables.

Setup

uv sync --extra dev

Copy the environment template and fill in your provider-specific values:

cp .env.example .env

Test

Offline tests:

uv run pytest -q

Live provider smoke test:

ERL_LLM_API_KEY=...
ERL_LLM_BASE_URL=https://your-provider.example/v1/messages
ERL_LLM_MODEL=your-model-id
uv run pytest tests/test_live_provider.py -q -rX

The live test is tolerant to backend instability and may report xfailed when the external provider returns transient errors or an empty content payload.

Production baseline

• configuration is loaded from environment variables via Settings
• logging is structured JSON on stderr
• external LLM calls use retries and backoff
• embeddings are pluggable and can be backed by an external HTTP service
• SQLite schema is managed through explicit migrations
• SQLite connections apply bounded busy timeout and explicit journal/synchronous pragmas

CLI

Smoke the configured LLM:

erl-memory smoke-llm

Reflect on a saved trajectory and optionally store the heuristic:

erl-memory --db heuristics.db reflect trajectory.json --store

Retrieve heuristics for a new task:

erl-memory --db heuristics.db retrieve "debug a failing SQL insert"

Run the seeded benchmark:

erl-memory --db heuristics.db evaluate

Run the local JSON API:

erl-memory serve

Override bind address or use environment-backed runtime settings:

ERL_API_HOST=0.0.0.0 ERL_API_PORT=8080 ERL_API_MAX_REQUEST_BYTES=131072 erl-memory serve

Run an external autonomous Codex loop:

python3 scripts/codex_loop.py \
  --goal "Continue improving this repository toward a production-ready ERL service" \
  --max-iterations 5

The loop stores run state and per-iteration transcripts under .codex-loop/.

Current API surface:

• GET /health
• GET /heuristics
• GET /heuristics?limit=...&offset=...
• GET /heuristics?source_task=...
• GET /heuristics/{heuristic_id}
• GET /heuristics?task=...&top_k=...
• POST /heuristics
• POST /reflect
• GET /trajectories
• GET /trajectories?limit=...&offset=...
• GET /trajectories?task=...&outcome=...
• GET /trajectories/{trajectory_id}
• POST /trajectories
• GET /evaluate

API notes:

• error responses use a stable envelope: {"error": {"code": "...", "message": "..."}}
• successfully completed POST /reflect, POST /heuristics, and POST /trajectories requests persist the full canonical JSON request/response pair whenever the request includes an Idempotency-Key header; repeated requests with the same payload replay the stored response, and conflicting payloads are rejected with 409 Conflict
• every API response includes an X-Request-ID header for request correlation
• collection endpoints return pagination: {"limit", "offset", "total"} and use deterministic ordering
• collection reads support exact-match filters on source_task, task, and outcome
• trajectories are persisted separately from heuristics
• GET /evaluate uses an isolated evaluation harness so it never mutates live heuristics or trajectory tables