feat: federated knowledge layer (Plans 1–3 §7.2 + path coherence + wikilinks) - #77
Merged
Conversation
Supersedes the v1 federated-per-directory spec. One signature-homogeneous index per database ($MDM_HOME), partitioned per project by an inode-aware canonical-prefix filter; cross-db map/enumerate for the across-signature view. Consensus-clean: unanimous warroom sign-off (Opus/Codex/Grok) across two adversarial review rounds.
* feat: centralize mdm home and db paths * test: isolate CLI index fixture * test: adapt CLI fixture to explicit index roots * fix: align index consumers with mdm home
* feat: merge config and wire mdm home * test: normalize watcher paths across platforms
* feat: define canonical document identity * fix: enforce canonical path ordering and containment * fix: align prefix realpaths across platforms * fix: use one canonical realpath representation * test: use declared prefix through canonical boundary
* feat: persist canonical document keys * fix: normalize Windows canonical paths * fix: reconcile link index updates
* feat: index canonical document keys * fix: reconcile hardlink link aliases
* refactor: resolve source files by canonical key * fix: restore canonical path glob filters * fix: canonicalize path roots consistently on Windows * fix: relativize remaining document path globs
* feat: add manifest directory ownership * fix: normalize manifest path separators
* feat: honor nested ingest boundaries * fix: normalize discovery relative paths
* refactor: remove index compatibility paths * refactor: replace stale index cleanup with ignore
* fix: key embedding reuse by content hash * test: add document hashes to integration vectors
* feat: ingest the active manifest * fix: stabilize manifest index help and tests
…vectors, pure signature reads, extract persistence (#62) * refactor: decompose hybrid search * refactor: require namespaced vector stores * fix: make active provider reads pure * refactor: isolate embedding persistence
* feat: add generation durability primitives * fix: sync Windows files with write access
…reaper, validation (§7.2 T7-T10,T18,T20) (#66) * feat: identify process instances * fix: bind Windows process identity arguments * feat: serialize generation writers * refactor: share durable record links * feat: lease generation reads * feat: validate generation artifacts * feat: publish complete generations * feat: reap retired generations safely * fix: enforce generation validation before publish * refactor: split generation engine test suites * fix: close generation handoff races * fix: validate uncached generation bytes * refactor: share generation error codes * style: apply generation engine formatting * fix: normalize Windows generation paths * test: assert portable staging paths * fix: honor generation publication invariants
…r transaction + vector reconcile (§7.2 T11,T12,T19) (#67) * feat: route index writes through generation transactions * fix: complete generation writer reconciliation * test: retry generation fixture cleanup
…hes + §6.1 preflight seam (§7.2 T13-T17,T21) (#68) * feat: lease generation readers and caches * feat: expose generation preflight seam * fix: preserve portable generation read identities * fix: preserve reader degradation boundaries
* docs: align CLI help with generation model * docs: refresh provider pricing * docs: fix pricing note punctuation
…ath] scope (#71) * fix(cli): scope search paths and unify read guidance * docs: genericize knowledge corpus examples * fix(cli): address read surface review findings * test(cli): skip invalid Windows path fixture
…tract coherence, slice D) (#72) * fix(mcp): resolve path tools against served roots * fix(mcp): preserve round trips through root aliases
…lter/query miss) (#74) * feat(search): teach empty result causes * fix(search): pluralize guidance document counts * test(mcp): normalize path-dependent token count
…p generation publish (smoke fixes) (#70) * fix: honor query-time provider config across search paths + skip no-op generation publish (smoke fixes) * fix: always publish forced generation refreshes * fix: harden hybrid semantic degradation * chore: upgrade biome to 2.5.5 and bump deps (effect, openai, smol-toml, yaml, ignore, vitest) * test: seed MCP query provider acceptance fixture
* feat: add wikilink indexing * fix: address wikilink review findings * test: use robust generation cleanup
srobinson
force-pushed
the
design/federated-knowledge-layer
branch
from
July 21, 2026 22:32
db92e05 to
0c39369
Compare
srobinson
force-pushed
the
design/federated-knowledge-layer
branch
from
July 21, 2026 23:18
0c39369 to
6b02c6b
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Federated Knowledge Layer → main
Integrates the full
design/federated-knowledge-layerbranch (38 commits) intomain. Delivers a token-efficient, generation-safe markdown knowledge layer for LLM agents over CLI + MCP.What ships
Plan 1 — canonical document identity
DocumentKey/resolveSourceFile; hardlink dedup; ownership splits (indexer / searcher / vector store / embedding namespace); centralized MDM_HOME + DB paths. No backcompat (fresh DBs only).Plan 2 — consolidated manifest
[[sources]]); bounded discovery + nested ignore inheritance; consolidated manifest index; content-hash-aware embedding reuse; CLI + MCP index routed through the manifest.Plan 3 §7.2 — atomic generation swap
gen-<n>generations + a single atomiccurrentpointer; every write through onetransactGeneration(build → validate → fsync → atomic flip); every read through a per-request gated lease (generation-consistent,sourceRoot/indexRootsplit); PID + process-start-identity + boot-id defeat PID reuse; reaper closes the gate, drains + grace, never deletescurrent; freshgen-1, no migration; platform-aware fsync durability (Windows degrades, not fails). No-op re-index skips publish;--forcealways publishes.Smokefix (#70) — query-time provider config honored across all search paths (no leak to public OpenAI for custom providers); no-op generation publish skipped; semantic degradation hardening; biome 2.5.5 + dependency bump.
Path/root contract coherence (user-test driven: #71–#74) — one canonical path coordinate (absolute, realpath) that every MCP surface displays, accepts, and matches, with a round-trip invariant (any path
md_searchemits is reusable verbatim in every path tool +path_filter). CLI read-surface friendly first-run guidance + honoredsearch [path]scope (#71); MCP path tools resolve against the served source root (#72);path_filtercanonicalization (#73); teaching pointer on empty/error responses — 3-way corpus/filter/query miss distinction (#74).Wikilinks (#75) —
[[target]],[[target|alias]],[[target#heading]],[[folder/target]]; basename resolution reusing canonical identity; edges + backlinks through the same pipeline as standard links;INDEX_VERSION2→3 forces a clean rebuild (no migration).Index-update guidance (#76) — reframes a version mismatch as a non-destructive "just re-index" instead of "delete your .mdm".
Validation
path_filter, out-of-corpus typed pointer, teaching pointer).designbinary (mixed standard + wikilink corpus, no regression, smokefix intact, path coherence + Lever-1, embed/search, full~/.mdxre-index).Deferred (fast-follow, not blocking)
md_roots/md_corpusdiscoverability tool; generic all-zero framework; success-response corpus header; dependabot vulns on the default branch.