feat(memory): add Hindsight conversation memory

.dockerignore

@@ -0,0 +1,27 @@
+.git
+.gitignore
+.DS_Store
+.env
+.venv
+.pixi
+__pycache__
+*.py[cod]
+.pytest_cache
+.mypy_cache
+.ruff_cache
+
+node_modules
+frontend/node_modules
+frontend/dist
+dist
+build
+
+checkpoints
+.hf-cache
+data
+recordings
+tmp
+logs
+
+models/SoulX-LiveAct
+models/**/checkpoints

AGENTS.md

@@ -0,0 +1,70 @@
+# Project Guidelines
+
+1. All code comments must be written in English.
+2. Frontend changes must consider internationalization and include both Chinese and English user-facing text where applicable.
+3. When modifying code, follow the minimum-change principle and keep edits narrowly scoped.
+
+
+Behavioral guidelines to reduce common LLM coding mistakes. Merge with project-specific instructions as needed.
+
+**Tradeoff:** These guidelines bias toward caution over speed. For trivial tasks, use judgment.
+
+## 1. Think Before Coding
+
+**Don't assume. Don't hide confusion. Surface tradeoffs.**
+
+Before implementing:
+- State your assumptions explicitly. If uncertain, ask.
+- If multiple interpretations exist, present them - don't pick silently.
+- If a simpler approach exists, say so. Push back when warranted.
+- If something is unclear, stop. Name what's confusing. Ask.
+
+## 2. Simplicity First
+
+**Minimum code that solves the problem. Nothing speculative.**
+
+- No features beyond what was asked.
+- No abstractions for single-use code.
+- No "flexibility" or "configurability" that wasn't requested.
+- No error handling for impossible scenarios.
+- If you write 200 lines and it could be 50, rewrite it.
+
+Ask yourself: "Would a senior engineer say this is overcomplicated?" If yes, simplify.
+
+## 3. Surgical Changes
+
+**Touch only what you must. Clean up only your own mess.**
+
+When editing existing code:
+- Don't "improve" adjacent code, comments, or formatting.
+- Don't refactor things that aren't broken.
+- Match existing style, even if you'd do it differently.
+- If you notice unrelated dead code, mention it - don't delete it.
+
+When your changes create orphans:
+- Remove imports/variables/functions that YOUR changes made unused.
+- Don't remove pre-existing dead code unless asked.
+
+The test: Every changed line should trace directly to the user's request.
+
+## 4. Goal-Driven Execution
+
+**Define success criteria. Loop until verified.**
+
+Transform tasks into verifiable goals:
+- "Add validation" → "Write tests for invalid inputs, then make them pass"
+- "Fix the bug" → "Write a test that reproduces it, then make it pass"
+- "Refactor X" → "Ensure tests pass before and after"
+
+For multi-step tasks, state a brief plan:
+```
+1. [Step] → verify: [check]
+2. [Step] → verify: [check]
+3. [Step] → verify: [check]
+```
+
+Strong success criteria let you loop independently. Weak criteria ("make it work") require constant clarification.
+
+---
+
+**These guidelines are working if:** fewer unnecessary changes in diffs, fewer rewrites due to overcomplication, and clarifying questions come before implementation rather than after mistakes.

README.ja.md

@@ -111,6 +111,37 @@ git clone https://github.com/dsd2077/CyberVerse.git
 cd CyberVerse
 ```
 
+### 任意: Pixi で再現可能な開発環境を作成する
+
+CyberVerse には、Python / Go / フロントエンドの開発チェックを再現するための Pixi 環境が 2 つ用意されています。
+
+- `macos-arm`: Apple Silicon Mac（`osx-arm64`）
+- `linux-x86`: x86_64 Linux（`linux-64`）
+
+Pixi が未インストールの場合は、先に実行します。
+
+```bash
+curl -fsSL https://pixi.sh/install.sh | sh
+```
+
+Apple Silicon Mac:
+
+```bash
+pixi install -e macos-arm
+pixi run -e macos-arm python-test
+pixi run -e macos-arm go-test
+```
+
+x86_64 Linux:
+
+```bash
+pixi install -e linux-x86
+pixi run -e linux-x86 python-test
+pixi run -e linux-x86 go-test
+```
+
+Pixi 環境には、Python 3.10、Go 1.25、Node 22、FFmpeg、`soxr`、`libopus`、`opusfile` などの開発・テスト依存関係が含まれます。GPU アバター推論には、引き続き下記の CUDA / PyTorch とモデル重みの設定が必要です。
+
 ### ステップ 2: Python 環境を作成する
 
 ```bash

README.ko.md

@@ -111,6 +111,37 @@ git clone https://github.com/dsd2077/CyberVerse.git
 cd CyberVerse
 ```
 
+### 선택 사항: Pixi로 재현 가능한 개발 환경 만들기
+
+CyberVerse는 Python / Go / 프런트엔드 개발 검사를 재현하기 위한 두 가지 Pixi 환경을 제공합니다.
+
+- `macos-arm`: Apple Silicon Mac(`osx-arm64`)
+- `linux-x86`: x86_64 Linux(`linux-64`)
+
+Pixi가 없다면 먼저 설치합니다.
+
+```bash
+curl -fsSL https://pixi.sh/install.sh | sh
+```
+
+Apple Silicon Mac:
+
+```bash
+pixi install -e macos-arm
+pixi run -e macos-arm python-test
+pixi run -e macos-arm go-test
+```
+
+x86_64 Linux:
+
+```bash
+pixi install -e linux-x86
+pixi run -e linux-x86 python-test
+pixi run -e linux-x86 go-test
+```
+
+Pixi 환경에는 Python 3.10, Go 1.25, Node 22, FFmpeg, `soxr`, `libopus`, `opusfile` 등 개발 및 테스트 의존성이 포함됩니다. GPU 아바타 추론에는 아래 CUDA / PyTorch 및 모델 가중치 설정이 여전히 필요합니다.
+
 ### 2단계: Python 환경 만들기
 
 ```bash

README.md

@@ -112,6 +112,45 @@ git clone https://github.com/dsd2077/CyberVerse.git
 cd CyberVerse
 ```
 
+### Optional: Pixi reproducible development environments
+
+CyberVerse provides two Pixi environments for reproducible Python / Go / frontend development checks:
+
+- `macos-arm`: macOS on Apple Silicon (`osx-arm64`)
+- `linux-x86`: Linux on x86_64 (`linux-64`)
+
+Install Pixi if needed:
+
+```bash
+curl -fsSL https://pixi.sh/install.sh | sh
+```
+
+On macOS Apple Silicon:
+
+```bash
+pixi install -e macos-arm
+pixi run -e macos-arm python-test
+pixi run -e macos-arm go-test
+```
+
+After copying `infra/cyberverse_config.example.yaml` to `cyberverse_config.yaml` and configuring `.env`, you can start the local Mac-side services in one foreground process:
+
+```bash
+pixi run -e macos-arm dev
+```
+
+`Ctrl+C` stops the inference process, Go server, and frontend dev server. For weak Mac hardware, keep GPU/avatar inference on the remote Ubuntu deployment and point the local config or frontend environment at that remote API.
+
+On Linux x86_64:
+
+```bash
+pixi install -e linux-x86
+pixi run -e linux-x86 python-test
+pixi run -e linux-x86 go-test
+```
+
+The Pixi environments cover development and test dependencies such as Python 3.10, Go 1.25, Node 22, FFmpeg, `soxr`, `libopus`, and `opusfile`. GPU avatar inference still requires the CUDA / PyTorch and model-weight setup below.
+
 ### Step 2: Create Python environment
 
 ```bash
@@ -141,10 +180,19 @@ Edit `.env`, fill in your API keys:
 ```
 DOUBAO_ACCESS_TOKEN=your_doubao_access_token   # ByteDance Doubao omni model
 DOUBAO_APP_ID=your_doubao_app_id
+HINDSIGHT_API_KEY=your_hindsight_api_key       # Optional PersonaAgent long-term memory
+HINDSIGHT_BASE_URL=https://hindsight.jmsu.top
+HINDSIGHT_BANK_ID_TEMPLATE=cv:user:{user_id}:character:{character_id}
+HINDSIGHT_USER_ID=local-user                   # Stable cross-session memory namespace
+HINDSIGHT_USER_TAG=local-user                  # Legacy/common recall tag
 ```
 
 Doubao Voice: get **App ID** / **API Key** per [Volcengine quick start](https://www.volcengine.com/docs/6561/2119699?lang=zh) → `DOUBAO_APP_ID` / `DOUBAO_ACCESS_TOKEN`.
 
+PersonaAgent can use Hindsight for cross-session memory when `HINDSIGHT_BASE_URL` and optional `HINDSIGHT_API_KEY` are set in your local `.env`. By default it writes to banks shaped like `cv:user:{user_id}:character:{character_id}`; set `HINDSIGHT_USER_ID` to keep one user's memories stable across sessions. Keep real Hindsight keys only in `.env`; do not commit them.
+
+For the PersonaAgent digital-human path, the required external interfaces are the realtime omni model credentials, the text LLM credentials used by local subagents, optional `ZHIHU_ACCESS_SECRET` for research tools, optional Hindsight memory credentials, and the local or remote Go/API + avatar inference endpoints.
+
 After the stack is running, you can change these values (and other API keys / service endpoints) from the web UI at **`/settings`** instead of editing `.env` only.
 
 ### Step 4: Download model weights

README.zh-CN.md

@@ -113,6 +113,45 @@ git clone https://github.com/dsd2077/CyberVerse.git
 cd CyberVerse
 ```
 
+### 可选：使用 Pixi 创建可复现开发环境
+
+CyberVerse 提供两个 Pixi 环境，用于复现 Python / Go / 前端开发检查：
+
+- `macos-arm`：Apple Silicon Mac（`osx-arm64`）
+- `linux-x86`：x86_64 Linux（`linux-64`）
+
+如未安装 Pixi，先执行：
+
+```bash
+curl -fsSL https://pixi.sh/install.sh | sh
+```
+
+Apple Silicon Mac：
+
+```bash
+pixi install -e macos-arm
+pixi run -e macos-arm python-test
+pixi run -e macos-arm go-test
+```
+
+复制 `infra/cyberverse_config.example.yaml` 为 `cyberverse_config.yaml` 并配置 `.env` 后，可以用一个前台进程启动本地 Mac 侧服务：
+
+```bash
+pixi run -e macos-arm dev
+```
+
+按 `Ctrl+C` 会停止 inference、Go server 和前端 dev server。Mac 性能较弱时，把 GPU/avatar 推理留在远程 Ubuntu 部署上，本地配置或前端环境指向远程 API 即可。
+
+x86_64 Linux：
+
+```bash
+pixi install -e linux-x86
+pixi run -e linux-x86 python-test
+pixi run -e linux-x86 go-test
+```
+
+Pixi 环境会安装 Python 3.10、Go 1.25、Node 22、FFmpeg、`soxr`、`libopus`、`opusfile` 等开发和测试依赖。GPU 数字人推理仍需按下方步骤配置 CUDA / PyTorch 与模型权重。
+
 ### 第 2 步：创建 Python 环境
 
 ```bash
@@ -143,10 +182,19 @@ cp infra/.env.example .env
 ```
 DOUBAO_ACCESS_TOKEN=your_doubao_access_token   # ByteDance Doubao 语音 LLM
 DOUBAO_APP_ID=your_doubao_app_id
+HINDSIGHT_API_KEY=your_hindsight_api_key       # 可选：PersonaAgent 长期记忆
+HINDSIGHT_BASE_URL=https://hindsight.jmsu.top
+HINDSIGHT_BANK_ID_TEMPLATE=cv:user:{user_id}:character:{character_id}
+HINDSIGHT_USER_ID=local-user                   # 跨会话稳定记忆命名空间
+HINDSIGHT_USER_TAG=local-user                  # 兼容旧配置的通用 recall tag
 ```
 
 豆包语音：按 [火山引擎快速入门](https://www.volcengine.com/docs/6561/2119699?lang=zh) 获取 **App ID** / **API Key**，填入 `DOUBAO_APP_ID` / `DOUBAO_ACCESS_TOKEN`。
 
+PersonaAgent 在本地 `.env` 配置 `HINDSIGHT_BASE_URL` 和可选 `HINDSIGHT_API_KEY` 后，会使用 Hindsight 做跨会话长期记忆。默认 bank 形如 `cv:user:{user_id}:character:{character_id}`；设置 `HINDSIGHT_USER_ID` 可让同一用户跨会话稳定使用同一记忆库。真实 Hindsight key 只放本地 `.env`，不要提交到代码仓库。
+
+PersonaAgent 数字人链路需要的外部接口包括：实时 omni 模型凭证、本地 subagent 使用的文本 LLM 凭证、可选的 `ZHIHU_ACCESS_SECRET` 调研工具凭证、可选的 Hindsight 记忆凭证，以及本地或远程 Go/API + avatar inference endpoint。
+
 服务启动后，你也可以在 Web UI 的 **`/settings`** 页面修改这些值（以及其他 API Key / 服务端点），而不必只依赖编辑 `.env`。
 
 ### 第 4 步：下载模型权重

deploy/gpu/.env.example

@@ -0,0 +1,31 @@
+DASHSCOPE_API_KEY=your_dashscope_api_key
+DASHSCOPE_BASE_URL=https://dashscope-intl.aliyuncs.com/compatible-mode/v1
+
+# Direct WebRTC TURN relay secret. Change this before opening the service.
+TURN_PASSWORD=replace_with_a_long_random_password
+
+# Optional Go internal task API token. Leave empty for local development; set a
+# nonempty value before exposing internal task endpoints in production.
+AGENT_INTERNAL_TOKEN=
+
+# Optional PersonaAgent task/search tools.
+ZHIHU_ACCESS_SECRET=your_zhihu_access_secret
+
+# Optional PersonaAgent long-term memory. Keep real keys only in .env.
+HINDSIGHT_ENABLED=true
+HINDSIGHT_BASE_URL=https://hindsight.jmsu.top
+HINDSIGHT_API_KEY=your_hindsight_api_key
+HINDSIGHT_BANK_ID_TEMPLATE=cv:user:{user_id}:character:{character_id}
+HINDSIGHT_USER_ID=local-user
+HINDSIGHT_USER_TAG=local-user
+HINDSIGHT_TIMEOUT_SECONDS=30
+HINDSIGHT_RECALL_BUDGET=low
+HINDSIGHT_RECALL_MAX_RESULTS=5
+HINDSIGHT_RECALL_MAX_TOKENS=4096
+HINDSIGHT_RETAIN_CONTEXT=cyberverse realtime conversation
+HINDSIGHT_DOCUMENT_ID_TEMPLATE=session:{session_id}:turn:{turn_id}
+HINDSIGHT_RETAIN_MAX_CHARS=6000
+
+# Optional Hugging Face settings for weight download.
+# HF_ENDPOINT=https://hf-mirror.com
+# HF_TOKEN=

deploy/gpu/Dockerfile.inference

@@ -0,0 +1,52 @@
+FROM pytorch/pytorch:2.8.0-cuda12.8-cudnn9-devel
+
+ENV DEBIAN_FRONTEND=noninteractive \
+    PIP_NO_CACHE_DIR=1 \
+    PYTHONUNBUFFERED=1 \
+    PYTHONPATH=/workspace:/workspace/inference/generated
+
+RUN set -eux; \
+    echo 'Acquire::ForceIPv4 "true";' > /etc/apt/apt.conf.d/99force-ipv4; \
+    rm -f /etc/apt/sources.list.d/cuda*.list /etc/apt/sources.list.d/nvidia*.list; \
+    sed -i \
+        -e 's|http://archive.ubuntu.com/ubuntu|http://mirrors.aliyun.com/ubuntu|g' \
+        -e 's|http://security.ubuntu.com/ubuntu|http://mirrors.aliyun.com/ubuntu|g' \
+        /etc/apt/sources.list; \
+    apt-get update && apt-get install -y --no-install-recommends \
+    build-essential \
+    ca-certificates \
+    curl \
+    ffmpeg \
+    git \
+    libgl1 \
+    libglib2.0-0 \
+    libgomp1 \
+    libsm6 \
+    libxext6 \
+    libxrender1 \
+    pkg-config \
+    && rm -rf /var/lib/apt/lists/*
+
+WORKDIR /workspace
+
+COPY pyproject.toml README.md ./
+COPY proto/ ./proto/
+COPY inference/ ./inference/
+COPY agent_runtime/ ./agent_runtime/
+COPY models/flash_head/ ./models/flash_head/
+
+RUN python -m pip install --upgrade pip setuptools wheel \
+    && python -m pip install -e ".[inference,agent,qwen,omni,tts,flash_head]" \
+    && mkdir -p inference/generated \
+    && python -m grpc_tools.protoc \
+        -I proto/ \
+        --python_out=inference/generated \
+        --grpc_python_out=inference/generated \
+        proto/*.proto
+
+HEALTHCHECK --interval=30s --timeout=5s --retries=20 --start-period=600s \
+    CMD python -c "import grpc; ch = grpc.insecure_channel('127.0.0.1:50051'); grpc.channel_ready_future(ch).result(timeout=3)" || exit 1
+
+EXPOSE 50051
+
+CMD ["python", "-m", "inference.server", "--config", "/workspace/deploy/gpu/cyberverse_config.gpu.yaml"]