SIP-0001: Evolving Memory Graph via recall_count and Dynamic Edge Weights

sips/SIP.md

@@ -0,0 +1,54 @@
+# SoulScript Improvement Proposals (SIP)
+
+## What is a SIP?
+
+A SoulScript Improvement Proposal (SIP) is a design document providing information to the SoulScript community, describing a new feature or process for the language. SIPs are the primary mechanism for proposing major new features, collecting community input, and documenting design decisions.
+
+## SIP Types
+
+- **Standard**: Describes any change that affects most or all SoulScript implementations
+- **Informational**: Describes a design issue or provides general guidelines
+- **Process**: Describes a process surrounding SoulScript or proposes a change to a process
+
+## SIP Status
+
+- **Draft**: Initial proposal under discussion
+- **Review**: Formally submitted for peer review
+- **Accepted**: Approved for implementation
+- **Final**: Implemented in the language
+- **Deferred**: Postponed for future consideration
+- **Rejected**: Not accepted for implementation
+- **Withdrawn**: Removed by the author(s)
+
+## SIP Format
+
+# SIP-XXX: Title
+
+- **Status**: [Current status of the proposal]
+- **Authors**: [List of authors/contributors]
+- **Summary**: A brief (~200 word) description of the feature or change.
+- **Motivation**: Why is this change necessary? What problems does it solve?
+- **Specification**: Detailed technical description of the proposed change.
+- **Rationale**: Why was this design chosen over alternatives?
+- **Backwards Compatibility**: Impact on existing code and migration path if applicable.
+- **Reference Implementation**: Link to implementation or proof of concept if available.
+
+## Submitting a SIP
+
+1. Fork the SoulScript repository
+2. Create a new branch for your SIP
+3. Create your SIP using the format above
+4. Submit a pull request
+
+## SIP Workflow
+
+1. **Discussion**: Initial informal discussion in community discord
+2. **Draft**: Author submits initial proposal
+3. **Review**: Community and core team review
+4. **Update**: Author revises based on feedback
+5. **Decision**: Core team accepts or rejects
+6. **Implementation**: If accepted, feature is implemented
+
+## Current SIPs
+
+- [SIP-0001: Evolving Memory Graph via `recall_count` and Dynamic Edge Weights](./sip_0001.md)

sips/SIP_0001.md

@@ -0,0 +1,112 @@
+# SIP-0001: Evolving Memory Graph via `recall_count` and Dynamic Edge Weights
+
+**Status**: In Review  
+**Author**: @samsar  
+**Acknowledgements**: @cwm @felixnorden
+
+## 1. Overview
+
+**Problem**: Standard RAG setups rely purely on semantic similarity at query time, producing a static view of agent memory that doesn't adapt to real usage or user feedback.
+
+**Solution**: Introduce:
+
+- **`recall_count`** – A field that tracks how often a memory is retrieved.  
+- **Dynamic Edge Weights** – Each `connections` entry becomes an object with a `weight` that updates based on semantic overlap, co-occurrence, and user feedback (e.g., upvotes, confirmations, or direct edits).  
+
+This turns Soulscript's memory structure into a **living, usage-driven graph**, where core memories naturally strengthen if they see frequent use or positive user feedback, and weaken if they remain idle or receive negative signals.
+
+
+
+## 2. Proposed Changes
+
+### 2.1 `recall_count` Field
+
+- **Type**: Integer (≥ 0)  
+- **Definition**: Tracks how many times a memory is retrieved or referenced during generation.  
+- **Novelty**: Directly encodes usage frequency—helping the agent "reinforce" frequently recalled knowledge.
+
+    ```json
+    {
+        "id": "mem_01HKG9X5NJWT",
+        "connections": [...],
+        "importance_score": 0.85,
+        "recall_count": 5
+    }
+    ```
+
+### 2.2 Dynamic Edge Weights
+
+- **Current**: `connections` is a simple list of IDs.  
+- **Extension**: Each connection becomes a JSON object containing:  
+  - **`memory_id`**: The linked memory's ID  
+  - **`weight`**: Reflecting semantic similarity, co-occurrence, and user feedback signals  
+  - Optional fields like `last_updated` or `co_occurrences`
+
+    ```json
+    {
+        "connections": [
+            {
+            "memory_id": "mem_01HKG9X5MJKT",
+            "weight": 0.8,
+            "last_updated": 1703030400000
+            }
+        ]
+    }
+    ```
+
+### 2.3 Approaches for Integrating `recall_count` with `importance_score`
+
+The relationship between `recall_count` and `importance_score` can be clarified through two possible approaches:
+
+1. **`importance_score` as a Static-Plus-Dynamic Composite**  
+   - **Definition**: `importance_score` represents a **computed value** that combines static significance (e.g., event priority) and dynamic usage (`recall_count`).  
+   - **How It Works**:  
+     ```math
+     importance_score = (\alpha \times static_priority) + (\beta \times \log(1 + recall_count))
+     ```
+     In this approach, `recall_count` directly influences `importance_score`, ensuring it evolves over time with agent usage.
+
+2. **`recall_count` as a Separate Driver for Importance**  
+   - **Definition**: `importance_score` remains **static**, while `recall_count` provides **additional context** for dynamically ranking memories during retrieval.  
+   - **How It Works**: The system calculates a **dynamic priority** at query time by combining `importance_score` and `recall_count`:
+
+     ```math
+     dynamic_priority = importance_score + log(1 + recall_count)
+     ```
+
+     This keeps `importance_score` fixed while leveraging `recall_count` for retrieval adaptability.
+
+
+These approaches provide flexibility in integrating `recall_count`, allowing developers to choose between dynamic recalibration of `importance_score` or treating it as a static baseline augmented by usage signals.
+
+
+
+## 3. How User Feedback Fits In
+
+- **Positive Feedback**: If users confirm or "like" a memory's relevance (e.g., "Yes, that's correct!"), increment both `recall_count` and the relevant connection's `weight`.  
+- **Negative Feedback**: If users correct or "dislike" the memory link, reduce the edge `weight` or drop the connection entirely.  
+- **Adaptive Learning**: This ensures the graph isn't just passively updated by usage, but *actively refined* by human signals—leading to a more trustworthy knowledge base.
+
+
+## 4. What This Solves
+
+1. **Static Relevance Problem**: Memories are no longer permanently "stuck" at creation-time importance—usage and feedback continually update their rank.  
+2. **Human-Like Reinforcement**: Frequently recalled (and positively confirmed) memories become stronger and more likely to be retrieved again.  
+3. **Natural Forgetting**: If a memory sees little usage or negative feedback, it gradually loses weight in the graph and recedes from top-tier retrieval.
+
+
+## 5. Implementation & RAG Integration
+
+1. **Increment `recall_count`** each time a memory is selected by RAG or user queries.  
+2. **Update Edge Weights** using a combination of:  
+   - **Semantic Similarity** (embedding overlap)  
+   - **Co-Occurrence** (how often two memories appear together)  
+   - **User Feedback** (upvotes, confirmations, corrections)  
+3. **Run a PageRank-Like Algorithm**:  
+   - Node weight: importance_score + log(1 + recall_count)
+   - Edge weight: set by the fields above (semantic or feedback-based)  
+   - Distribute rank so frequently accessed, positively reinforced memories rise to the top.
+
+
+## 6. Conclusion
+By adding **`recall_count`** and **dynamic, feedback-driven edge weights**, Soulscript evolves from a static memory list to a robust, usage-sensitive knowledge graph. Agents not only pull semantically relevant chunks but also reinforce what's truly valuable, guided by real user feedback and a **human-like process of memory reinforcement**.