Description
The bot injects goals, tasks, and recent diary entries directly into the system prompt.
Impact
As goals become more complex and the diary grows, these metadata consume a significant portion of the LLM's context window, leaving less room for the actual task context and leading to model performance degradation.
Proposed Solution
- Implement a more aggressive summarization strategy for the diary.
- Use RAG (Retrieval-Augmented Generation) to fetch only relevant diary entries instead of a fixed number of recent ones.
- Only inject details for the currently active task rather than the entire goal tree.
Description
The bot injects goals, tasks, and recent diary entries directly into the system prompt.
Impact
As goals become more complex and the diary grows, these metadata consume a significant portion of the LLM's context window, leaving less room for the actual task context and leading to model performance degradation.
Proposed Solution