feat(memory): cross-project federation — FEDERATED scope (Phase 4 U3)

[fanhongy]

Summary

Every memory CAO stores today is trapped in one project — recall() only sees the current project's wiki plus shared global/. A lesson learned in project A is invisible in project B.

This adds federation: a machine-wide shared tier (FEDERATED scope) reachable through the existing memory_store/memory_recall/memory_forget verbs with scope="federated". No new MCP tools, no new verbs. Closes #313 (Phase 4 U3).

Design

• Lowest recall precedence (4) — a project-local fact with the same key always shadows a federated one. Federation is a fallback tier, never an override.
• Writable by any agent except session (SCOPE_RANK=0) — matches how agents already write global proactively. Mirrors the existing session asymmetry (lowest recall precedence, lowest write rank).
• Disjoint storage at MEMORY_BASE_DIR/federated/ — a top-level sibling of global/, inside the existing path-containment guard. Its own index.md.
• Secret-keyword gate, federated-only — a new services/secret_gate.py scans content on federated writes and rejects AWS keys, PEM blocks, bearer tokens, password=/secret= assignments, and GitHub/GitLab PATs. The blast radius is every project, so credentials are blocked mechanically. Other scopes are unaffected.
• Zero-cost when empty — the .exists() guard in _get_search_dirs keeps recall byte-identical to today when no federated memory exists (no presence cache needed).

Changes

• models/memory.py — FEDERATED enum value
• services/memory_scoring.py — SCOPE_RANK[federated]=0, SCOPE_PRECEDENCE[federated]=4 (+ asymmetry comment)
• services/memory_service.py — storage path, scope_id→None, both precedence dicts (incl. the duplicate literal), search-dir wiring, federated-only secret-gate hook in store()
• services/secret_gate.py (new) — scan_for_secrets(), 6 credential patterns
• services/cleanup_service.py, mcp_server/server.py, cli/commands/memory.py — scope plumbing + descriptions
• skills/cao-memory/SKILL.md — federated row + "share across projects" guidance

Testing

• test/services/test_secret_gate.py (new) — positive/negative pattern coverage incl. the canonical Authorization: Bearer <token> header form
• test/services/test_memory_service.py — federated store/recall roundtrip, ranks-last, file location, forget, secret-rejected-nothing-written, same-content-allowed-at-global (proves federated-only gating), empty-federated search-dirs byte-identical, scan_all no-double-add
• test/services/test_scoring.py — scope_write_allowed federated table; load-bearing recency invariant still green
• Full unit gate: 2734 passed; only known pre-existing flakes (bm25 perf, kiro integration) non-green. black/isort clean; mypy clean on touched files.

Built via a design → implement → 3-lens adversarial review (correctness/security/regression) → validate workflow. The review caught a real bug: the bearer-token regex required a :/= separator and missed the canonical Authorization: Bearer <token> header — fixed, with the documenting xfail flipped to a passing positive test.

Out of scope

Cross-machine sync; web UI surface; import/export conflict plumbing (Phase 4 U2, lands separately).

[Copilot]

Pull request overview

Adds a new machine-wide memory tier by introducing a federated scope that is readable across projects, stored in a disjoint on-disk root, and guarded by a federated-only secret/credential pattern gate to reduce blast radius.

Changes:

• Introduces MemoryScope.FEDERATED and wires it through store/recall/forget, search-dir enumeration, and scope precedence/write-rank behavior.
• Adds services/secret_gate.py and enforces it only on scope="federated" writes.
• Updates CLI/MCP descriptions + docs and adds/extends unit tests for federated behavior and the secret gate.

Reviewed changes

Copilot reviewed 11 out of 11 changed files in this pull request and generated 2 comments.

Show a summary per file

File	Description
test/services/test_secret_gate.py	New tests for scan_for_secrets() secret-pattern detection.
test/services/test_scoring.py	Adds federated-specific write-guard tests and asserts the recency invariant test still exists.
test/services/test_memory_service.py	Adds federated store/recall/forget/layout/precedence/search-dir/secret-gate coverage.
src/cli_agent_orchestrator/skills/cao-memory/SKILL.md	Documents federated scope usage and precedence.
src/cli_agent_orchestrator/services/secret_gate.py	Implements regex-based secret scanning for federated writes.
src/cli_agent_orchestrator/services/memory_service.py	Adds federated scope plumbing: scope_id resolution, pathing, search dirs, secret-gate hook.
src/cli_agent_orchestrator/services/memory_scoring.py	Extends precedence/rank tables and clarifies federated asymmetry in comments.
src/cli_agent_orchestrator/services/cleanup_service.py	Updates retention + parsing logic to recognize federated scope/dir.
src/cli_agent_orchestrator/models/memory.py	Adds FEDERATED enum value and updates scope field description.
src/cli_agent_orchestrator/mcp_server/server.py	Updates MCP tool parameter descriptions to include federated scope.
src/cli_agent_orchestrator/cli/commands/memory.py	Updates CLI help text to include federated scope.

---

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.