feat: add neural Mneme indexing and associative think#8
feat: add neural Mneme indexing and associative think#8georgeantonopoulos wants to merge 10 commits into
Conversation
📝 WalkthroughWalkthroughAdds a latent neuron index with hash or Ollama embeddings, introduces ChangesNeural Think Workflow
Prediction Compatibility
Estimated code review effort: 4 (Complex) | ~45 minutes Sequence Diagram(s)sequenceDiagram
participant Operator
participant MnemeCLI
participant build_latent_index
participant SQLite
participant neural_think
Operator->>MnemeCLI: run mneme index
MnemeCLI->>build_latent_index: build latent index
build_latent_index->>SQLite: read neurons and edges
build_latent_index->>SQLite: upsert latent vectors
Operator->>MnemeCLI: run mneme think with prompt
MnemeCLI->>neural_think: submit prompt
neural_think->>SQLite: load vectors and active edges
neural_think->>neural_think: rank seeds and spread activation
neural_think-->>MnemeCLI: return activations, provenance, and context
MnemeCLI-->>Operator: print JSON result
🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches📝 Generate docstrings
🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 5698647f95
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
There was a problem hiding this comment.
Actionable comments posted: 5
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@scripts/mneme_senses_context_hook.py`:
- Line 68: Update the production _strip_injected_context marker list to
recognize and remove COMPACT_MEMORY_REMINDER, preferably using a stable prefix
so quoted variants are stripped; add an end-to-end regression test for
scripts/mneme_senses_context_hook.py confirming the reminder is not reprocessed
as user content.
In `@src/mneme/cli.py`:
- Around line 324-336: Add argparse-level positive integer validation for
--batch-size, --dimensions, --seeds, --limit, and --max-neurons in the neural
command definitions, rejecting zero and negative values. Add non-negative
validation for --hops, allowing zero while rejecting negatives, and ensure
invalid values fail during argument parsing.
In `@src/mneme/neural.py`:
- Around line 143-145: The public neural APIs currently overwrite caller-owned
SQLite connection configuration. In build_latent_index at
src/mneme/neural.py:143-145 and think at src/mneme/neural.py:243-245, capture
the existing conn.row_factory before assigning sqlite3.Row, then restore the
original value before returning or closing the caller-provided connection;
preserve the current behavior for connections created from paths.
- Around line 103-110: The neural pipeline must ignore active edges with zero
effective strength or confidence in both stages. In _neuron_rows, filter the
active edge join so zero-weight edges are excluded from embedded neuron text; in
the later propagation logic at src/mneme/neural.py lines 292-293, default only
NULL weights and preserve explicit zero values.
- Around line 164-169: Update the incremental-index identity check around the
latent_neurons lookup to include the hash dimensions alongside content_hash,
provider, and model. Select the stored dimensions and require it to match the
current dimensions before continuing; a mismatch must proceed through rebuilding
so vectors are regenerated at the requested size.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: e7c09379-fd66-4ee4-a4c7-c1be40a482c1
📒 Files selected for processing (13)
NEURAL_EXPERIMENT.mdscripts/mneme_senses_context_hook.pyskills/mneme/SKILL.mdskills/mneme/references/hermes-live-vault-evaluation.mdskills/mneme/references/hook-directive-order.mdskills/mneme/references/neural-think-experiment.mdskills/mneme/references/person-entity-retrieval.mdsrc/mneme/cli.pysrc/mneme/neural.pysrc/mneme/world_model/predictions.pytests/test_hook_directive_order.pytests/test_neural.pytests/test_world_model_predictions.py
|
@codex review |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: 355c1884cc
ℹ️ About Codex in GitHub
Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".
Validate that stored latent vectors use one consistent width and that the live query embedding matches it before cosine ranking. If an Ollama model tag changes width, fail clearly with the mneme index --rebuild recovery path instead of silently truncating vectors. Add a regression test that simulates an Ollama model changing from three dimensions to two. Verified with 36 focused tests, 316 full-suite tests, the privacy scan, the example-vault smoke run, and the Mneme contract check.
Attach the existing Mneme agent-brain contract name and version to think results, and mark each activated neuron as provenance_not_fact so associative leads cannot be mistaken for verified facts. Add contract assertions to the neural propagation test. Verified with 7 focused tests, 316 full-suite tests, and the repository privacy scan.
830bdff to
9d7e1f1
Compare
There was a problem hiding this comment.
🧹 Nitpick comments (1)
src/mneme/neural.py (1)
596-604: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win
signals["latent"]["cosine"]reports the decayed score, not the raw cosine.
decayed_score_by_idholdscosine * temporal_decay(Line 544), so labeling itcosinemisrepresents the similarity signal in the activation reason payload consumers use for evaluation — especially since the lexical sibling already separatesraw_score,calibrated_score, and the decayedscore. Consider renaming this field (e.g.decayed_cosine/score) or emitting the pre-decay cosine alongsidetemporal_decayfor parity. Confirm no doc/contract inNEURAL_EXPERIMENT.mddepends on the current key.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@src/mneme/neural.py` around lines 596 - 604, The latent signal currently labels the decayed value from decayed_score_by_id as cosine. Update the signal payload construction near the in_latent branch to distinguish the decayed value, preferably by renaming the key to decayed_cosine or score, and preserve compatibility with any documented contract such as NEURAL_EXPERIMENT.md before changing the field name.
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Nitpick comments:
In `@src/mneme/neural.py`:
- Around line 596-604: The latent signal currently labels the decayed value from
decayed_score_by_id as cosine. Update the signal payload construction near the
in_latent branch to distinguish the decayed value, preferably by renaming the
key to decayed_cosine or score, and preserve compatibility with any documented
contract such as NEURAL_EXPERIMENT.md before changing the field name.
ℹ️ Review info
⚙️ Run configuration
Configuration used: defaults
Review profile: CHILL
Plan: Pro
Run ID: 180f1c3d-2bda-4ffe-9d05-5d9e34551e92
📒 Files selected for processing (8)
NEURAL_EXPERIMENT.mdscripts/mneme_senses_context_hook.pyskills/mneme/references/neural-think-experiment.mdsrc/mneme/cli.pysrc/mneme/neural.pytests/test_cli_contracts.pytests/test_hook_directive_order.pytests/test_neural.py
🚧 Files skipped from review as they are similar to previous changes (3)
- src/mneme/cli.py
- NEURAL_EXPERIMENT.md
- scripts/mneme_senses_context_hook.py
Summary
Adds an experimental neural interface for Mneme:
The existing relational model remains intact as auditable storage. This PR adds a smaller agent-facing path for local indexing and associative thought.
What changed
mneme indexandmneme thinkCLI commandsdescription/expected_byas compatible prediction fields while keeping canonical prediction outputTest plan
pytest -qwith the declared dev dependencies - 316 passedmneme run-once --vault ./examples/vault --db /tmp/mneme_smoke.sqlite --out /tmp/mneme_smoke_out- passed against the bundled example vaultmneme contract check --db /tmp/mneme_smoke.sqlite- passed with zero failures and zero warningspython scripts/privacy_scan.py- passedgit diff --check origin/main...HEAD- passedMneme contract checklist
truth_policyand contract metadata.Privacy checklist
Scope
This is an experimental vertical slice, not a replacement of the existing world model. Latent similarity remains an associative lead rather than factual evidence, and the source must be checked before making a factual claim.
Summary by CodeRabbit
New Features
mneme indexto build a local semantic memory index.mneme thinkfor semantic retrieval, controlled association, temporal decay, and source-backed evidence.Documentation
Bug Fixes
Tests