release: harden trust and consistency in 0.24.0

CHANGELOG.md

@@ -3,6 +3,37 @@
 All notable changes to **agentcairn** are documented here.
 Format: [Keep a Changelog](https://keepachangelog.com/en/1.1.0/); versioning: [SemVer](https://semver.org/).
 
+## [Unreleased]
+
+## [0.24.0] - 2026-07-12
+
+### Added
+- Per-vault, cross-process writer coordination for CLI, MCP, and Hermes mutations;
+  contention fails fast with retry guidance instead of racing DuckDB or dedup state.
+- MCP first-read reconciliation, explicit freshness/retrieval diagnostics, and
+  write-through `remember` so a successful save is immediately recallable.
+
+### Changed
+- Automatic recall is project-scoped by default and renders every recalled line as
+  untrusted quoted evidence in Claude Code, OpenCode, and Hermes (plugin 0.1.1).
+  Cross-project automatic recall now requires an explicit
+  `auto_recall_scope = "all"` opt-in.
+- New AgentCairn-owned files/directories default to `0600`/`0700`; replacements are
+  atomic and preserve modes chosen for existing user files.
+- Codex and Antigravity plugins (0.1.1) defer vault selection to the shared config file
+  rather than pinning `~/agentcairn`. Cloud embedding keys/endpoints are now
+  file-configurable.
+
+### Fixed
+- Reconciliation is transactional, so embedding/model/FTS failures preserve the last
+  good index. Model or vector-dimension mismatches and provider failures visibly
+  degrade retrieval to BM25 instead of producing incompatible vector queries.
+- Cloud embedding payloads are secret-redacted at the egress boundary; LLM-generated
+  titles/distillations are redacted before filenames or Markdown are written; vault
+  symlinks that escape the vault are rejected before indexing or network egress.
+- Temporal/project adjustments now work for negative cross-encoder logits, and
+  reranker failure falls back to the fused order.
+
 ## [0.23.0] - 2026-06-29
 
 ### Added
@@ -318,7 +349,23 @@ Format: [Keep a Changelog](https://keepachangelog.com/en/1.1.0/); versioning: [S
 - Out-of-band capture from coding-agent transcripts (redacted, non-lossy `remember`).
 - Published to PyPI via GitHub Trusted Publishing (OIDC, no stored secrets).
 
-[Unreleased]: https://github.com/ccf/agentcairn/compare/v0.9.6...HEAD
+[Unreleased]: https://github.com/ccf/agentcairn/compare/v0.24.0...HEAD
+[0.24.0]: https://github.com/ccf/agentcairn/compare/v0.23.0...v0.24.0
+[0.23.0]: https://github.com/ccf/agentcairn/compare/v0.22.1...v0.23.0
+[0.22.1]: https://github.com/ccf/agentcairn/compare/v0.22.0...v0.22.1
+[0.22.0]: https://github.com/ccf/agentcairn/compare/v0.21.0...v0.22.0
+[0.21.0]: https://github.com/ccf/agentcairn/compare/v0.20.1...v0.21.0
+[0.20.1]: https://github.com/ccf/agentcairn/compare/v0.20.0...v0.20.1
+[0.20.0]: https://github.com/ccf/agentcairn/compare/v0.19.0...v0.20.0
+[0.19.0]: https://github.com/ccf/agentcairn/compare/v0.18.0...v0.19.0
+[0.18.0]: https://github.com/ccf/agentcairn/compare/v0.17.0...v0.18.0
+[0.17.0]: https://github.com/ccf/agentcairn/compare/v0.16.0...v0.17.0
+[0.16.0]: https://github.com/ccf/agentcairn/compare/v0.15.0...v0.16.0
+[0.15.0]: https://github.com/ccf/agentcairn/compare/v0.14.0...v0.15.0
+[0.14.0]: https://github.com/ccf/agentcairn/compare/v0.13.0...v0.14.0
+[0.13.0]: https://github.com/ccf/agentcairn/compare/v0.12.0...v0.13.0
+[0.12.0]: https://github.com/ccf/agentcairn/compare/v0.11.0...v0.12.0
+[0.11.0]: https://github.com/ccf/agentcairn/compare/v0.10.1...v0.11.0
 [0.10.1]: https://github.com/ccf/agentcairn/compare/v0.10.0...v0.10.1
 [0.10.0]: https://github.com/ccf/agentcairn/compare/v0.9.8...v0.10.0
 [0.9.8]: https://github.com/ccf/agentcairn/compare/v0.9.7...v0.9.8

README.md

@@ -14,7 +14,7 @@ agentcairn gives your coding agent durable, high-quality memory — but instead
 
 ## Why agentcairn is different
 
-Most agent-memory systems make a database or cloud store the source of truth and treat files (if any) as a one-way export. agentcairn inverts that:
+Editable, file-backed memory is not unique to agentcairn. Its focus is the combination needed for coding-agent work: one inspectable vault shared across otherwise-independent agents, ambient transcript capture, provenance and temporal correctness, and a measured hybrid index that remains disposable:
 
 - **📂 Your vault is the source of truth — not an export.** Memory is human-readable Markdown with frontmatter and `[[wikilinks]]`. Edit it in Obsidian; the index honors your edits.
 - **♻️ The index is disposable.** DuckDB is a rebuildable cache (`cairn reindex`). Your memory survives a model upgrade, a corrupted index, a schema change, or uninstalling the tool — **zero data loss**, because the truth is just files on disk.
@@ -46,17 +46,23 @@ On install you pick a vault path (default `~/agentcairn`); it's **auto-created**
 a hybrid recall against what you just asked and injects the most relevant
 memories as context for that turn — not just the recency digest shown at session
 start. Trivially-short prompts (e.g. "yes", "go") are skipped. It is fail-open:
-if anything goes wrong it injects nothing and never blocks your prompt.
+if anything goes wrong it injects nothing and never blocks your prompt. Automatic
+recall is hard-limited to the current project by default. Each injected excerpt
+is quoted, tagged with its available permalink/project provenance, and explicitly
+framed as untrusted historical evidence — never as instructions for the agent.
 
 Configure it in `~/.agentcairn/config.toml` (flat top-level keys; env vars
 `CAIRN_AUTO_RECALL`, `CAIRN_AUTO_RECALL_K`, `CAIRN_AUTO_RECALL_SCOPE` override the file):
 
 ```toml
 auto_recall       = true    # master on/off (default: true)
 auto_recall_k     = 3       # memories injected per prompt
-auto_recall_scope = "all"   # "all" (boost, non-lossy) or "project" (hard filter)
+auto_recall_scope = "project" # project-only (default); "all" is an explicit cross-project opt-in
 ```
 
+Manual `cairn recall` and the `recall` MCP tool remain cross-project by default;
+use their existing `--scope project` / `scope="project"` option when desired.
+
 > Not on Claude Code or Codex? agentcairn is also a standalone MCP server + CLI for any host — see [Using it directly](#using-it-directly).
 
 ## How it works
@@ -71,9 +77,10 @@ flowchart LR
 
     T -- "redact → judge → distill → consolidate" --> V
     H -- "edit" --> V
-    V -- "parse / reconcile-on-spawn" --> I
-    I -- "READ_ONLY hybrid recall" --> M
-    M -. "remember (redacted write)" .-> V
+    V -- "transactional reconcile before first read" --> I
+    I -- "short-lived hybrid reads" --> M
+    M -. "remember: atomic Markdown write" .-> V
+    M -. "write-through under one writer lock" .-> I
 
     classDef truth fill:#eaf1ff,stroke:#317cff,color:#191919;
     classDef cache fill:#f5f5f3,stroke:#999999,color:#191919;
@@ -83,7 +90,7 @@ flowchart LR
 
 - **Capture** reads your agent harness's session transcripts (append-only, already on disk) *out-of-band* — robust by design, with no fragile live hooks — then redacts → dedups → judges (semantic durability; optional LLM distillation via `CAIRN_JUDGE=anthropic`) → gates → distills into the vault, non-lossily. `cairn sweep` auto-detects every present harness (Claude Code, Codex, Antigravity CLI, and Cursor are all supported, behind a `HarnessAdapter` seam) so you get unified memory across all four without any extra configuration. On the LLM tier it also **consolidates**: a new memory that duplicates an existing one is skipped, and a newer version of an evolving fact marks the older note `superseded_by` (kept + demoted in recall, never deleted) — fail-safe, so a wrong call never drops a distinct memory (`CAIRN_CONSOLIDATE=0` to disable). Plus an agent-driven `remember` tool for curated, high-value memories.
 - **Retrieval** fuses BM25 + semantic vectors with Reciprocal Rank Fusion, applies an optional graph-boost, and **degrades gracefully** down to keyword-only when no embedding model is available — so recall is *never* silently dead. An optional cross-encoder reranker adds precision.
-- **Hybrid intelligence:** offline local embeddings (FastEmbed / `nomic-embed-text-v1.5` by default) out of the box — strong on its own *and* in the hybrid fusion (with `nomic`, vector-only edges out BM25 even on short turns; see the benchmark). Set `CAIRN_EMBED_MODEL` to pick another FastEmbed model, or `CAIRN_EMBEDDER=ollama` for a local Ollama model. For higher recall quality at the cost of a network call, set `CAIRN_EMBEDDER=voyage` (default model `voyage-3`, requires `VOYAGE_API_KEY`) or `CAIRN_EMBEDDER=openai` (default `text-embedding-3-small`, requires `OPENAI_API_KEY`; set `OPENAI_BASE_URL` for any OpenAI-compatible endpoint). Both cloud embedders are opt-in — the default stays fully local. **Privacy:** with a cloud embedder enabled, your note text (already secret-redacted at write time) and your recall queries are sent to the provider. This is consistent with the optional `CAIRN_JUDGE=anthropic` LLM tier. **Cost:** switching embedder or model triggers a full-vault re-embed (dimension change) on the next `reindex`/`sweep` — real API cost and latency; plan accordingly.
+- **Hybrid intelligence:** offline local embeddings (FastEmbed / `nomic-embed-text-v1.5` by default) out of the box — strong on its own *and* in the hybrid fusion (with `nomic`, vector-only edges out BM25 even on short turns; see the benchmark). Set `CAIRN_EMBED_MODEL` to pick another FastEmbed model, or `CAIRN_EMBEDDER=ollama` for a local Ollama model. For higher recall quality at the cost of a network call, set `CAIRN_EMBEDDER=voyage` (default model `voyage-3`, requires `VOYAGE_API_KEY`) or `CAIRN_EMBEDDER=openai` (default `text-embedding-3-small`, requires `OPENAI_API_KEY`; set `OPENAI_BASE_URL` for any OpenAI-compatible endpoint). Both cloud embedders are opt-in — the default stays fully local. **Privacy:** with a cloud embedder enabled, note chunks and recall queries are secret-redacted again at the provider boundary before they leave the machine (including hand-edited notes; the Markdown itself is not modified). Vault symlinks that resolve outside the vault are rejected. The provider still receives the remaining redacted text, consistent with the optional `CAIRN_JUDGE=anthropic` tier. **Cost:** switching embedder or model triggers a full-vault re-embed on the next `reindex`/`sweep` — real API cost and latency; plan accordingly.
 - **Temporal memory:** notes may carry `valid_from`/`valid_until`/`superseded_by` frontmatter. Recall is validity-aware — it soft-demotes superseded and expired facts (the *current* fact wins) without ever hiding them (non-lossy), and annotates each result's status (`current`/`superseded`/`expired`/`not_yet_valid`) plus an `as_of` anchor so the agent can reason over time. Inert for notes with no validity fields.
 - **Provenance-aware recall:** notes carry `project`/`harness` provenance, and recall boosts your current project's memories (non-lossy — cross-project hits still surface, marked `[from: <project>]`). Pass `--project <repo>` to target another repo, or `--scope project` to hard-filter to just the current one.
 
@@ -136,9 +143,14 @@ judge = "anthropic"
 anthropic_api_key = "sk-ant-..."
 ```
 
+Cloud embedding settings use the same file (`voyage_api_key`,
+`openai_api_key`, and `openai_base_url`); keys are masked by `cairn config`.
+Codex and Antigravity plugin manifests deliberately do not pin a vault, so this
+shared config remains authoritative across hosts.
+
 ## Agents supported
 
-agentcairn works at two levels. **Plugin hosts** (Claude Code, Codex, and Antigravity) get a first-class plugin — a bundled MCP server (recall/search/`remember`), a memory skill, and (on Claude Code and Codex) ambient session hooks; `cairn install <host>` installs the plugin by calling the host's own CLI. **MCP hosts** (everything else) get the same recall/search/`remember` tools via the portable MCP server; `cairn install <host>` writes the MCP server config non-destructively (your other servers are preserved, the original is backed up to `<config>.bak`). The vault stays a single global `~/agentcairn`, so memory is shared across every host.
+agentcairn works at two levels. **Plugin hosts** (Claude Code, Codex, and Antigravity) get a first-class plugin — a bundled MCP server (recall/search/`remember`), a memory skill, and (on Claude Code and Codex) ambient session hooks; `cairn install <host>` installs the plugin by calling the host's own CLI. **MCP hosts** (everything else) get the same recall/search/`remember` tools via the portable MCP server; `cairn install <host>` writes the MCP server config non-destructively (your other servers are preserved, the original is backed up to `<config>.bak`). Every host resolves the same configured vault (`~/agentcairn` by default), so memory does not fragment by agent.
 
 | Host | Support | Set up with | Ambient capture |
 |---|---|---|---|