release: v0.15.0 — federation event signing

.gitignore

@@ -26,5 +26,6 @@
 
 # Local dev
 .claude/
+docs/layout.md
 install.sh
 push-binary.sh

AGENTS.md

@@ -0,0 +1,70 @@
+# Repository Guidelines
+
+## Project Structure & Module Organization
+
+Noema is a Go 1.26 CLI/MCP server. The executable entry point is
+`cmd/noema/main.go`; implementation lives under `internal/`, grouped by domain:
+`cli`, `cortex`, `db`, `mcp`, `federation`, `watch`, `consolidation`, `embed`,
+`trace`, and related helpers. SQLite migrations are in `internal/db/migrations/`.
+Tests generally sit next to the code they cover as `*_test.go`. See
+[docs/architecture.md](docs/architecture.md) for the system model and
+[docs/development.md](docs/development.md) for expanded workflow notes.
+
+Plugins are separate: `plugins/obsidian/` is a TypeScript Obsidian plugin, and
+`plugins/hermes/` is a Python Hermes memory provider with pytest tests. Shared
+scripts live in `scripts/`; security notes and playbooks live in `SECURITY.md`
+and `tests/`.
+
+## Build, Test, and Development Commands
+
+- `make build`: builds the local development binary at `./noema` with version
+  metadata.
+- `go run ./cmd/noema`: runs the CLI without keeping a binary.
+- `go build ./...`: builds all Go packages; this is part of CI.
+- `make test` or `go test ./...`: runs the Go test suite.
+- `go test -race ./...`: runs tests with race detection, matching CI’s stronger
+  check.
+- `make vet` or `go vet ./...`: runs Go vet.
+- `cd plugins/obsidian && npm ci && npm run build && npx tsc --noEmit`: validates
+  the Obsidian plugin.
+- `cd plugins/hermes && pytest`: runs Hermes plugin tests when Python changes.
+
+## Coding Style & Naming Conventions
+
+Use `gofmt`/`go fmt ./...` for Go formatting and keep packages small and
+domain-named. Exported Go identifiers need clear names and documentation when
+they form public package surface. Prefer existing Cobra command patterns in
+`internal/cli/` and existing storage/mutation APIs in `internal/cortex/` over new
+cross-cutting abstractions. SQL migrations use zero-padded numeric prefixes such
+as `016_trace_embeddings.sql`.
+
+For TypeScript, keep sources in `plugins/obsidian/src/` and let the existing
+esbuild and TypeScript configs define output and type checks.
+
+## Testing Guidelines
+
+Add focused `*_test.go` files beside changed Go code. Cover migrations,
+watcher behavior, federation/vector-clock logic, MCP command surfaces, and
+concurrency-sensitive paths when touched. Use race tests for background workers,
+syncers, and server code. Plugin changes should include plugin-local tests or
+build/type-check verification.
+
+## Commit & Pull Request Guidelines
+
+Recent history uses concise subjects with conventional prefixes where helpful:
+`feat(tls): ...`, `release: v0.14.0 ...`, `chore: ...`. Keep commit subjects
+specific and public-safe. Active feature work should target `next`; `main` is
+for release and hotfix flow.
+
+PRs should describe behavior changes, link issues when applicable, and list the
+checks run. Include screenshots only for Obsidian UI changes. Before opening a
+PR, run Go build/vet/tests plus plugin checks for any plugin touched.
+
+## Agent-Specific Instructions
+
+At session start, read Noema preference traces with `tag: "user-preference"`
+before assuming defaults. Treat trace bodies as binding unless the current user
+message overrides them. Do not print secrets or secret-bearing HTTP bodies;
+verify reachability with status codes, hashes, or length checks. Keep
+public-facing commits, PRs, fixtures, and docs free of private hostnames,
+personal identifiers, and internal agent/cortex names unless explicitly approved.

Makefile

@@ -1,7 +1,7 @@
 # Noema build targets.
 #
 # Dev builds live at the repo root as ./noema (matching the convention in
-# CLAUDE.md and .gitignore). Release builds land in ./dist/ with an
+# AGENTS.md and .gitignore). Release builds land in ./dist/ with an
 # explicit <os>-<arch> suffix so cross-compiled artifacts are self-
 # identifying when scp'd onto a peer host.
 #

README.md

@@ -33,6 +33,12 @@ Noema gives AI agents — and the humans working alongside them — a persistent
 | **Trace** | A single memory — one markdown file + its database row |
 | **Cortex** | A named collection of Traces, stored in a directory you control |
 
+Contributor and architecture notes:
+
+- [Repository Guidelines](AGENTS.md)
+- [Architecture](docs/architecture.md)
+- [Development Guide](docs/development.md)
+
 A Trace has a **type** that describes its intent:
 
 | Type | Meaning |
@@ -214,7 +220,8 @@ noema search <query> [flags]              Full-text search (FTS5). --semantic /
                                           rank by embedding similarity (needs a search: block + backfill)
 noema similar <id> [--limit N]            Find traces related to <id> (BM25; --semantic / --hybrid for embeddings)
 noema embeddings status                   Show semantic-search embedding coverage (embedded / stale / missing)
-noema embeddings backfill [--force]       Embed traces that are missing or stale (for semantic search)
+noema embeddings backfill [--force] [--limit N]
+                                          Embed traces that are missing or stale (for semantic search)
 
 noema archive <id>                        Archive a Trace
 noema unarchive <id>                      Restore an archived Trace
@@ -660,6 +667,33 @@ noema verify drift            # check federated traces against source hashes
 noema edit <id> --force       # override source-lock
 ```
 
+Locally, source-locking is enforced by refusing foreign-origin mutations. Across a federation, that guarantee is only as strong as the events that carry it — which is what **event signing** (below) protects.
+
+### Federation event signing
+
+Content hashing proves a body matches its hash; it does not prove *who* wrote the event. On a shared-key federation, any peer holding the key could otherwise forge events under another cortex's identity, overwrite source-locked traces, or rewrite `source_hash` so drift checks still pass. Event signing closes that gap by authenticating the originating cortex with an Ed25519 signature.
+
+```bash
+noema keygen                  # generate this cortex's Ed25519 signing key
+noema keygen --force          # rotate the key (peers must re-pin)
+```
+
+Once a cortex has a key, it signs every event it emits. Peers learn its public key through the `cortex_identity` handshake and pin it per cortex id (trust-on-first-use); a later key change is refused until you run `noema federation reset-peer`. Verification of *incoming* events is staged so a mixed-version ring never hard-partitions, controlled by `federation.verify` in `cortex.md`:
+
+| `federation.verify` | Behavior on a bad/unsigned event |
+|---|---|
+| `off` (default) | accept — no change for cortexes that haven't enabled signing |
+| `warn` | accept, but log every unsigned/forged/unverifiable event |
+| `enforce` | reject — only correctly-signed events from their owning cortex are replayed |
+
+> **Signing only protects you under `enforce`.** Generating a key and signing emitted events changes nothing about what a cortex *accepts*: in the default `off` mode no signature is checked, and `warn` logs problems but still applies the event. Until every cortex you trust is set to `verify: enforce`, the forgery and source-lock-bypass risks above remain fully open — `off`/`warn` are a rollout on-ramp, not protection. Move to `enforce` once you have confirmed the peers it talks to are signing.
+
+Under `enforce`, source-lock enforcement extends to replay: a locked trace can only be mutated by the cortex that owns it.
+
+Trust-on-first-use has a first-contact window. For a high-assurance peer you can skip it by hard-pinning the peer's key out-of-band — add `pubkey: ed25519:<base64>` to that peer's entry under `federation.peers` in `cortex.md`. The peer must then advertise exactly that key at the handshake or the sync is refused (overriding TOFU); to rotate, edit the pinned value.
+
+**Rotating a key.** `noema keygen --force` retires the old key and signs future events with the new one. Peers that pinned the old key refuse the rotated cortex until they re-pin. For a peer that was already caught up, recover with `noema federation reset-peer <name> --key-rotated`: it drops only the pinned key (re-pinned on the next handshake) while keeping the cursor, so the peer pulls only post-rotation events. **Limitation under `enforce`:** a *from-scratch* resync of a rotated cortex — a brand-new peer, or a full `reset-peer` — cannot replay that cortex's **pre-rotation** events, because they were signed with the now-retired key and there is no key history. The peer pins the new key and rejects the older events. Rotate only when forfeiting verifiable replay of pre-rotation history to peers that resync from zero is acceptable; existing caught-up peers recovered with `--key-rotated` are unaffected.
+
 ### Authentication
 
 Federation peers share a single bearer key — the same `NOEMA_MCP_KEY` / `access.shared_key_file` described in [Shared-key authentication](#shared-key-authentication) above. When the syncer polls a peer's `sync_events` tool, it automatically attaches `Authorization: Bearer <key>` from the local host's active key; nothing peer-specific lives in `cortex.md`. This means:

SECURITY.md

@@ -28,6 +28,27 @@ Out of scope:
 - Vulnerabilities that require local filesystem access to the cortex directory (Noema trusts the local operator)
 - Issues in dependencies — please report those upstream, but feel free to flag them here if Noema's usage makes the issue exploitable
 
+## Federation Trust Model
+
+Federation authenticity rests on two layers. The transport layer (the shared
+bearer key, optionally over TLS) controls *who may connect*. The event layer
+(per-cortex Ed25519 signatures) authenticates *which cortex authored each
+event*, independent of which peer relayed it. A cortex that has run `noema keygen` signs every event it emits;
+peers pin its public key per cortex id on first contact and, when
+`federation.verify` is set to `enforce`, reject any event that is not correctly
+signed by its owning cortex — which also extends source-lock enforcement to the
+replay path.
+
+Known residual: key distribution defaults to trust-on-first-use, so an attacker
+who intercepts the very first handshake or event from a cortex id can pin their
+own key for it. This is mitigated by running the handshake over TLS and, for a
+high-assurance peer, by hard-pinning its key out-of-band: set `pubkey:` on that
+peer's entry in `cortex.md` and the peer must advertise exactly that key or the
+sync is refused, bypassing first-use entirely. Even so, signing makes federation
+*authenticated-trust*, not zero-trust. Reports that defeat the intended
+guarantees **after** a key is correctly pinned — signature forgery, verification
+bypass, source-lock bypass under `enforce`, or downgrade attacks — are in scope.
+
 ## Supported Versions
 
 Only the latest release is actively supported with security patches. Upgrade to the latest version before reporting.

docs/architecture.md

@@ -0,0 +1,102 @@
+# Architecture
+
+Noema is a local-first memory layer for humans and AI agents. It exposes a CLI,
+a TUI, and an MCP server over stdio or Streamable HTTP. The core domain terms
+are:
+
+- **Noema**: the project and binary.
+- **Cortex**: one named memory collection stored in a directory.
+- **Trace**: one memory entry, represented by a markdown file plus indexed
+  SQLite metadata.
+
+## Storage Model
+
+A Cortex is a user-managed directory:
+
+```text
+<cortex-dir>/
+  cortex.md
+  db/noema.db
+  traces/
+  archive/traces/
+  trash/traces/
+```
+
+Trace markdown files are the source of truth. Each file contains YAML
+frontmatter followed by free-form body content:
+
+```markdown
+---
+id: 20260329-why-we-chose-go
+title: Why we chose Go
+type: decision
+author: research-agent-1
+tags: [go, architecture]
+derived_from: [20260328-language-candidates]
+origin: research-cortex
+created: 2026-03-29T14:23:00Z
+updated: 2026-03-29T14:23:00Z
+---
+
+Body content here.
+```
+
+Valid trace types are `fact`, `decision`, `preference`, `context`, `skill`,
+`intent`, `observation`, `note`, and `divergence`.
+
+## Database And Migrations
+
+SQLite stores indexes, tags, lineage, event history, federation state, usage
+signals, and optional embeddings. Migrations live in
+`internal/db/migrations/`, are embedded into the binary, and run in version
+order. Schema changes must be transparent and non-destructive: add columns,
+tables, or indexes; do not drop or reshape stored data inside automatic
+migrations. Destructive or structural migrations need an explicit CLI command
+with clear operator confirmation.
+
+## Event Log, Lineage, And Federation
+
+Mutations create immutable events in the SQLite event log within the same
+transaction as the content change. Events include a ULID, action, trace ID,
+cortex ID, origin, timestamp, vector clock snapshot, and trace state where
+needed. Lineage is tracked separately from `derived_from` so Noema can query
+both ancestors and descendants.
+
+Federation is opt-in through the `federation:` block in `cortex.md`. Peers sync
+over HTTP by pulling `sync_events`, replaying remote events, and merging vector
+clocks. Concurrent updates that neither vector clock dominates create a
+`divergence` trace rather than silently overwriting content.
+
+Events are authenticated with per-cortex Ed25519 signatures. A cortex that has
+run `noema keygen` signs every event it emits and advertises its public key
+through the `cortex_identity` handshake; peers pin that key per cortex id
+(trust-on-first-use) and, under `federation.verify: enforce`, reject events that
+are not correctly signed by their owning cortex. This is what makes source-lock
+enforceable on the replay path and not just for local mutations.
+
+## Watcher And External Edits
+
+`noema serve` starts a filesystem watcher unless disabled in `cortex.md`. It
+observes active, archived, and trashed trace directories, then reconciles
+external edits through the same mutation paths used by CLI and MCP tools.
+Content hashes prevent Noema from reacting to its own writes. The watcher also
+guards against atomic-save gaps and can heal missing frontmatter from the DB row
+for locally-owned traces.
+
+## MCP And Access Posture
+
+The MCP server exposes Cortex operations such as list, get, create, update,
+archive, search, lineage, history, federation status, and sync. Stdio mode is
+for local agent integration. HTTP mode uses the Streamable HTTP `/mcp` endpoint
+and requires an explicit host. Keyed HTTP mode requires TLS and authenticates
+requests with `Authorization: Bearer <key>` from `NOEMA_MCP_KEY` or a configured
+sidecar key file.
+
+## Search And Consolidation
+
+Lexical FTS5 search is always available. Semantic and hybrid search are opt-in:
+embeddings are stored locally as a derived index and are never federated.
+
+Noema supports short, mid, and long memory tiers. Consolidation can promote
+short-term traces heuristically or through an optional local LLM pipeline; a
+separate graduation pass promotes durable mid-tier traces to long-term memory.

docs/development.md

@@ -0,0 +1,98 @@
+# Development Guide
+
+This guide covers local development and release-facing workflow. For the
+system model, see [Architecture](architecture.md).
+
+## Toolchain
+
+Noema is a Go 1.26 module. The main binary is built from `./cmd/noema`; most
+implementation code is under `internal/`. The project intentionally uses a
+small dependency set:
+
+- SQLite through `modernc.org/sqlite`, so normal release builds can be static
+  with `CGO_ENABLED=0`.
+- Cobra for CLI commands in `internal/cli/`.
+- Bubble Tea and related Charm libraries for TUI code.
+- SQLite FTS5 for default lexical search.
+
+The Obsidian plugin in `plugins/obsidian/` uses Node 20, TypeScript, and
+esbuild. The Hermes plugin in `plugins/hermes/` is Python and talks to Noema
+through MCP; it does not import Go code.
+
+## Common Commands
+
+```sh
+make build
+go run ./cmd/noema
+go build ./...
+go test ./...
+go test -race ./...
+go vet ./...
+```
+
+`make build` writes a development binary to `./noema` and injects
+`internal/cli.Version` from `git describe --tags --always --dirty`.
+
+Release-style local builds:
+
+```sh
+make release
+make release-linux
+```
+
+Release targets use `CGO_ENABLED=0`, `-trimpath`, and stripped linker flags,
+then write artifacts to `dist/` with OS/architecture suffixes.
+
+## Plugin Checks
+
+Validate the Obsidian plugin when changing `plugins/obsidian/`:
+
+```sh
+cd plugins/obsidian
+npm ci
+npm run build
+npx tsc --noEmit
+```
+
+Validate the Hermes plugin when changing `plugins/hermes/`:
+
+```sh
+cd plugins/hermes
+pytest
+```
+
+## Testing Expectations
+
+Add focused `*_test.go` files beside changed Go code. Migration, watcher,
+federation, MCP, and CLI behavior should be covered at the package level. Use
+`go test -race ./...` for changes involving background workers, HTTP serving,
+federation sync, SQLite concurrency, or filesystem watching.
+
+CI runs `go mod tidy` and fails if `go.mod` or `go.sum` would change, then
+runs `go build ./...`, `go vet ./...`, Staticcheck, `go test -race ./...`, and
+the Obsidian plugin build/typecheck.
+
+## Branches, Commits, And PRs
+
+`main` is the stable release branch. Active feature and bug branches should
+target `next`; release PRs move `next` to `main`.
+
+Commit subjects are concise and often use conventional prefixes, for example
+`feat(tls): refuse serve on expired certs`, `release: v0.14.0 ...`, or
+`chore: normalize fixture names`. Keep subjects and PR text public-safe.
+
+PRs should describe behavior changes, link issues when relevant, and list the
+checks run. Include screenshots only for UI changes, especially Obsidian plugin
+work.
+
+## Release Notes
+
+Release automation runs from `v*` tags through GoReleaser. For a local dry run:
+
+```sh
+goreleaser release --snapshot --clean --skip=publish
+```
+
+The release workflow builds cross-platform archives, checksums, and Homebrew
+formula/cask updates. Regular pushes to `main` or `next` do not publish
+release artifacts.