Integrate Cognee memory engine with SiliconFlow LLM provider#61
Integrate Cognee memory engine with SiliconFlow LLM provider#61HeDaas-Code with Copilot wants to merge 25 commits into
Conversation
Co-authored-by: HeDaas-Code <208586641+HeDaas-Code@users.noreply.github.com>
Co-authored-by: HeDaas-Code <208586641+HeDaas-Code@users.noreply.github.com>
Co-authored-by: HeDaas-Code <208586641+HeDaas-Code@users.noreply.github.com>
Co-authored-by: HeDaas-Code <208586641+HeDaas-Code@users.noreply.github.com>
There was a problem hiding this comment.
Pull request overview
This PR introduces a Cognee-backed memory layer (vector search + knowledge graph) and a Markdown-based “worldview” system, wiring both into the existing long-term memory pipeline and database GUI for management/editing.
Changes:
- Added
CogneeMemoryManager(async + sync wrapper) and integrated optional Cognee syncing intoLongTermMemoryManager. - Added
WorldviewBuilderfor Markdown worldview file management, parsing into modules, and syncing modules into Cognee / existing knowledge base. - Extended the database management GUI with new Cognee memory + worldview tabs, plus added docs and integration tests.
Reviewed changes
Copilot reviewed 9 out of 10 changed files in this pull request and generated 13 comments.
Show a summary per file
| File | Description |
|---|---|
src/core/cognee_memory.py |
New Cognee wrapper (add/search/cognify/prune) + sync wrapper helpers. |
src/core/long_term_memory.py |
Optional Cognee initialization and message/graph sync hooks; exposes Cognee search/clear. |
src/core/worldview_builder.py |
New Markdown worldview CRUD, section parsing to modules, LLM-assisted generation, sync to Cognee/KB. |
src/gui/cognee_gui.py |
New Tkinter UI for Cognee memory operations + worldview editor/sync actions. |
src/gui/database_gui.py |
Adds Cognee/worldview tabs and optional injection of managers/builders. |
tests/test_cognee_integration.py |
New integration tests covering imports/basic behaviors and GUI signature expectations. |
docs/COGNEE_INTEGRATION.md |
New integration guide and usage notes. |
requirements.txt |
Adds cognee>=0.5.0. |
example.env |
Documents COGNEE_ENABLED + LLM_API_KEY option. |
README.md |
Updates feature list and acknowledgements to include Cognee + worldview builder. |
| title_lower = title.lower() | ||
|
|
||
| category_keywords = { | ||
| "general": ["基本", "简介", "概述", "世界", "背景"], | ||
| "rules": ["规则", "法则", "限制", "物理", "魔法"], | ||
| "locations": ["地点", "场所", "位置", "环境", "地理"], | ||
| "characters": ["角色", "人物", "NPC", "种族"], | ||
| "events": ["事件", "历史", "时间", "故事"], |
There was a problem hiding this comment.
_infer_category() 里先对 title 做了 lower()，但关键字列表里包含大写的 "NPC"；在 title_lower 中永远匹配不到该关键字，导致包含 “NPC” 的标题无法归类到 characters。建议把关键字也统一小写（或去掉 lower()，改用大小写不敏感匹配）。
| loop = asyncio.new_event_loop() | ||
| asyncio.set_event_loop(loop) | ||
|
|
||
| # 格式化为对话记忆 | ||
| role_name = "用户" if role == "user" else "助手" | ||
| formatted_content = f"{role_name}: {content}" | ||
|
|
||
| loop.run_until_complete( | ||
| self.cognee_manager.add_memory(formatted_content, "conversation") | ||
| ) | ||
| except Exception as e: | ||
| print(f"⚠ 同步到 Cognee 失败: {e}") | ||
|
|
||
| def _run_cognee_cognify(self): | ||
| """运行 Cognee 知识图谱构建""" | ||
| try: | ||
| loop = asyncio.new_event_loop() | ||
| asyncio.set_event_loop(loop) | ||
| loop.run_until_complete(self.cognee_manager.cognify()) | ||
| print("✓ Cognee 知识图谱已更新") |
There was a problem hiding this comment.
这里每次同步消息都会 asyncio.new_event_loop() 并 set_event_loop()，但没有在结束时 loop.close()；在长时间运行/高频消息场景下容易造成资源泄漏，并且反复切换全局 event loop 也可能影响其他异步逻辑。建议改为：优先复用已有 loop（或用 asyncio.run 在无运行中 loop 时执行），并在 finally 中关闭临时 loop。
| loop = asyncio.new_event_loop() | |
| asyncio.set_event_loop(loop) | |
| # 格式化为对话记忆 | |
| role_name = "用户" if role == "user" else "助手" | |
| formatted_content = f"{role_name}: {content}" | |
| loop.run_until_complete( | |
| self.cognee_manager.add_memory(formatted_content, "conversation") | |
| ) | |
| except Exception as e: | |
| print(f"⚠ 同步到 Cognee 失败: {e}") | |
| def _run_cognee_cognify(self): | |
| """运行 Cognee 知识图谱构建""" | |
| try: | |
| loop = asyncio.new_event_loop() | |
| asyncio.set_event_loop(loop) | |
| loop.run_until_complete(self.cognee_manager.cognify()) | |
| print("✓ Cognee 知识图谱已更新") | |
| # 使用已有的 event loop（如果正在运行），否则使用 asyncio.run 执行 | |
| async def _do_sync() -> None: | |
| # 格式化为对话记忆 | |
| role_name = "用户" if role == "user" else "助手" | |
| formatted_content = f"{role_name}: {content}" | |
| await self.cognee_manager.add_memory(formatted_content, "conversation") | |
| try: | |
| loop = asyncio.get_running_loop() | |
| except RuntimeError: | |
| # 当前线程没有正在运行的 loop，安全地同步执行 | |
| asyncio.run(_do_sync()) | |
| else: | |
| # 已有运行中的 loop，复用它（异步调度，避免创建/切换新 loop） | |
| loop.create_task(_do_sync()) | |
| except Exception as e: | |
| print(f"⚠ 同步到 Cognee 失败: {e}") | |
| def _run_cognee_cognify(self): | |
| """运行 Cognee 知识图谱构建""" | |
| try: | |
| async def _do_cognify() -> None: | |
| await self.cognee_manager.cognify() | |
| try: | |
| loop = asyncio.get_running_loop() | |
| except RuntimeError: | |
| # 当前线程没有正在运行的 loop，安全地同步执行 | |
| asyncio.run(_do_cognify()) | |
| print("✓ Cognee 知识图谱已更新") | |
| else: | |
| # 已有运行中的 loop，复用它（异步调度，避免创建/切换新 loop） | |
| loop.create_task(_do_cognify()) | |
| print("✓ Cognee 知识图谱已更新") |
| try: | ||
| loop = asyncio.new_event_loop() | ||
| asyncio.set_event_loop(loop) | ||
| loop.run_until_complete(self.cognee_manager.clear_all_memory()) |
There was a problem hiding this comment.
clear_all_memory() 中为清空 Cognee 数据创建了新的 event loop，但没有关闭该 loop；这会在多次清空操作或测试场景下产生资源泄漏。建议在 try/finally 中关闭临时 loop，或改用统一的同步包装器来执行异步调用。
| loop = asyncio.new_event_loop() | ||
| asyncio.set_event_loop(loop) | ||
| return loop.run_until_complete( | ||
| self.cognee_manager.search(query, memory_type=memory_type) | ||
| ) |
There was a problem hiding this comment.
search_cognee_memory() 每次搜索都创建并设置新的 event loop，但未关闭，且会反复覆盖当前线程的默认 loop。建议复用/关闭 loop，或将该方法改为 async 并由调用方统一调度（GUI/同步场景可用 SyncWrapper）。
| # 解析一级和二级标题作为模块分隔 | ||
| # 匹配 ## 标题 或 ### 标题 | ||
| sections = re.split(r'\n(?=##\s+)', content) | ||
|
|
||
| current_category = "general" | ||
|
|
||
| for section in sections: | ||
| if not section.strip(): | ||
| continue | ||
|
|
||
| # 提取标题 | ||
| title_match = re.match(r'^##\s+(.+?)(?:\n|$)', section) | ||
| if title_match: |
There was a problem hiding this comment.
parse_worldview_to_modules() 的注释说明支持以 ## / ### 标题分段，但当前 re.split(r'\n(?=##\s+)', ...) 和 re.match(r'^##\s+...', ...) 只会识别二级标题，三级标题不会作为模块边界，导致模块解析不符合预期。建议把分割/匹配正则改为同时支持 ##+，并相应调整标题提取逻辑。
| try: | ||
| loop = asyncio.new_event_loop() | ||
| asyncio.set_event_loop(loop) | ||
|
|
||
| # 格式化为对话记忆 | ||
| role_name = "用户" if role == "user" else "助手" | ||
| formatted_content = f"{role_name}: {content}" | ||
|
|
||
| loop.run_until_complete( | ||
| self.cognee_manager.add_memory(formatted_content, "conversation") | ||
| ) | ||
| except Exception as e: | ||
| print(f"⚠ 同步到 Cognee 失败: {e}") | ||
|
|
||
| def _run_cognee_cognify(self): | ||
| """运行 Cognee 知识图谱构建""" | ||
| try: | ||
| loop = asyncio.new_event_loop() | ||
| asyncio.set_event_loop(loop) | ||
| loop.run_until_complete(self.cognee_manager.cognify()) | ||
| print("✓ Cognee 知识图谱已更新") | ||
| except Exception as e: | ||
| print(f"⚠ Cognee 知识图谱构建失败: {e}") |
There was a problem hiding this comment.
_run_cognee_cognify() 同样为每次调用创建新的 event loop 且未关闭，存在与 _sync_message_to_cognee() 相同的资源泄漏/全局 loop 污染问题。建议复用 loop 或在 finally 中 loop.close()，避免在对话过程中周期性触发时累计泄漏。
| try: | |
| loop = asyncio.new_event_loop() | |
| asyncio.set_event_loop(loop) | |
| # 格式化为对话记忆 | |
| role_name = "用户" if role == "user" else "助手" | |
| formatted_content = f"{role_name}: {content}" | |
| loop.run_until_complete( | |
| self.cognee_manager.add_memory(formatted_content, "conversation") | |
| ) | |
| except Exception as e: | |
| print(f"⚠ 同步到 Cognee 失败: {e}") | |
| def _run_cognee_cognify(self): | |
| """运行 Cognee 知识图谱构建""" | |
| try: | |
| loop = asyncio.new_event_loop() | |
| asyncio.set_event_loop(loop) | |
| loop.run_until_complete(self.cognee_manager.cognify()) | |
| print("✓ Cognee 知识图谱已更新") | |
| except Exception as e: | |
| print(f"⚠ Cognee 知识图谱构建失败: {e}") | |
| loop: Optional[asyncio.AbstractEventLoop] = None | |
| try: | |
| loop = asyncio.new_event_loop() | |
| # 格式化为对话记忆 | |
| role_name = "用户" if role == "user" else "助手" | |
| formatted_content = f"{role_name}: {content}" | |
| loop.run_until_complete( | |
| self.cognee_manager.add_memory(formatted_content, "conversation") | |
| ) | |
| except Exception as e: | |
| print(f"⚠ 同步到 Cognee 失败: {e}") | |
| finally: | |
| # 确保事件循环被正确关闭，避免资源泄漏 | |
| if loop is not None and not loop.is_closed(): | |
| loop.close() | |
| def _run_cognee_cognify(self): | |
| """运行 Cognee 知识图谱构建""" | |
| loop: Optional[asyncio.AbstractEventLoop] = None | |
| try: | |
| loop = asyncio.new_event_loop() | |
| loop.run_until_complete(self.cognee_manager.cognify()) | |
| print("✓ Cognee 知识图谱已更新") | |
| except Exception as e: | |
| print(f"⚠ Cognee 知识图谱构建失败: {e}") | |
| finally: | |
| # 确保事件循环被正确关闭，避免资源泄漏 | |
| if loop is not None and not loop.is_closed(): | |
| loop.close() |
| import tkinter as tk | ||
| from tkinter import ttk, scrolledtext, messagebox, simpledialog | ||
| from typing import Dict, Any, List | ||
| from typing import Dict, Any, List, Optional |
There was a problem hiding this comment.
typing.Optional 在该文件中未被使用。建议移除未使用的导入以保持代码整洁。
| from typing import Dict, Any, List, Optional | |
| from typing import Dict, Any, List |
| """测试前设置""" | ||
| # 禁用 Cognee 以避免依赖问题 | ||
| os.environ['COGNEE_ENABLED'] = 'false' | ||
|
|
There was a problem hiding this comment.
测试里在 setUp() 直接修改全局环境变量 COGNEE_ENABLED，但没有在 tearDown() 恢复原值；这会污染同一进程内后续测试（尤其是依赖默认 env 的测试）。建议在 setUp 保存旧值并在 tearDown 恢复/删除该变量，或使用 patch.dict(os.environ, ...) 做作用域隔离。
| """测试前设置""" | |
| # 禁用 Cognee 以避免依赖问题 | |
| os.environ['COGNEE_ENABLED'] = 'false' | |
| """测试前设置""" | |
| # 记录原始环境变量值，避免污染其他测试 | |
| self._old_cognee_enabled = os.environ.get('COGNEE_ENABLED') | |
| # 禁用 Cognee 以避免依赖问题 | |
| os.environ['COGNEE_ENABLED'] = 'false' | |
| def tearDown(self): | |
| """测试后恢复环境变量""" | |
| if self._old_cognee_enabled is None: | |
| # 原来未设置，则删除 | |
| os.environ.pop('COGNEE_ENABLED', None) | |
| else: | |
| # 恢复原始值 | |
| os.environ['COGNEE_ENABLED'] = self._old_cognee_enabled | |
|
@copilot [11:29:27] [ERROR] CogneeMemoryManager | 添加记忆失败: LLMAPIKeyNotSetError: LLM API key is not set. (Status code: 422) 我设置了主SILICONFLOW_API_KEY，没设置LLM_API_KEY，出现该错误，请修复 |
…ports, NPC keyword Co-authored-by: HeDaas-Code <208586641+HeDaas-Code@users.noreply.github.com>
Co-authored-by: HeDaas-Code <208586641+HeDaas-Code@users.noreply.github.com>
… OpenAI Co-authored-by: HeDaas-Code <208586641+HeDaas-Code@users.noreply.github.com>
|
@copilot [12:19:01] [ERROR] CogneeMemoryManager | 添加记忆失败: <failed_attempts> litellm.AuthenticationError: AuthenticationError: OpenAIException - Incorrect API key provided: sk-rarjf***************************************ebae. You can find your API key at https://platform.openai.com/account/api-keys. None litellm.AuthenticationError: AuthenticationError: OpenAIException - Incorrect API key provided: sk-rarjf***************************************ebae. You can find your API key at https://platform.openai.com/account/api-keys. None</failed_attempts> <last_exception> |
…on info Co-authored-by: HeDaas-Code <208586641+HeDaas-Code@users.noreply.github.com>
|
@copilot 对[15:15:20] [ERROR] CogneeMemoryManager | 添加记忆失败: [KeyError] 'Could not automatically map bge-large-zh-v1.5 to a tokeniser. Please use |
…_model_tokenizer_mapping Co-authored-by: HeDaas-Code <208586641+HeDaas-Code@users.noreply.github.com>
已修复 tokenizer 映射问题。现在直接在 |
|
@copilot [15:25:25] [ERROR] CogneeMemoryManager | Cognee 初始化失败: [AttributeError] 'DebugLogger' object has no attribute 'log_warning' |
Co-authored-by: HeDaas-Code <208586641+HeDaas-Code@users.noreply.github.com>
|
@copilot |
…el.MODEL_TO_ENCODING Co-authored-by: HeDaas-Code <208586641+HeDaas-Code@users.noreply.github.com>
已修复 tiktoken tokenizer 映射问题。错误来自 tiktoken 库而非 LiteLLM，需要在 |
|
@copilot 切换到 HuggingFace 分词器，让 Cognee 使用 HuggingFace 的 AutoTokenizer，同时处理好HuggingFace 在中国大陆容易出现的SSL问题 |
Co-authored-by: HeDaas-Code <208586641+HeDaas-Code@users.noreply.github.com>
已添加 HuggingFace tokenizer 支持并处理中国大陆 SSL 问题。修复在 commit 8ee9ed7。 新增配置选项： # HuggingFace 镜像站点（中国大陆加速）
HF_ENDPOINT=https://hf-mirror.com
# 禁用 SSL 验证（解决中国大陆 SSL 问题）
HF_HUB_DISABLE_SSL_VERIFY=1
# 指定 tokenizer 模型
HUGGINGFACE_TOKENIZER=BAAI/bge-large-zh-v1.5 |
|
将Cognee的可视化管理单拎出来，不要放在数据库管理下，同时在添加Cognee日志管理功能，读取.venv\Lib\site-packages\logs\下的日志 
|
Co-authored-by: HeDaas-Code <208586641+HeDaas-Code@users.noreply.github.com>
已完成以下修改（commit 184a165）： 1. 新增独立的 Cognee 管理器应用 ( python cognee_manager.py2. 添加 Cognee 日志查看器
3. 全面废止 tiktoken，只使用 HuggingFace 分词器 from transformers import AutoTokenizer
AutoTokenizer.from_pretrained(tokenizer_model, trust_remote_code=True)4. 在应用初始化时自动配置 HuggingFace
|
|
@copilot 不是让cognee_manager.py独立成新界面，而是独立出调试中的数据库界面，作为调试下的子界面 |
move-cognee-gui-to-debug-menu
💬 We'd love your input! Share your thoughts on Copilot coding agent in our 2 minute survey.