diff --git a/examples/openclaw-memory-plugin/.gitignore b/examples/openclaw-memory-plugin/.gitignore new file mode 100644 index 00000000..c2658d7d --- /dev/null +++ b/examples/openclaw-memory-plugin/.gitignore @@ -0,0 +1 @@ +node_modules/ diff --git a/examples/openclaw-memory-plugin/INSTALL-ZH.md b/examples/openclaw-memory-plugin/INSTALL-ZH.md new file mode 100644 index 00000000..241fde3c --- /dev/null +++ b/examples/openclaw-memory-plugin/INSTALL-ZH.md @@ -0,0 +1,679 @@ +# 为 OpenClaw 安装 OpenViking 记忆功能 + +通过 [OpenViking](https://github.com/volcengine/OpenViking) 为 [OpenClaw](https://github.com/openclaw/openclaw) 提供长效记忆能力。安装完成后,OpenClaw 将自动**记住**对话中的重要信息,并在回复前**回忆**相关内容。 + +--- + +## 一、快速开始(让 OpenClaw 自动安装) + +先将技能文件复制到 OpenClaw 技能目录,再让 OpenClaw 完成后续步骤: + +**Linux / macOS:** + +```bash +mkdir -p ~/.openclaw/skills/install-openviking-memory +cp examples/openclaw-memory-plugin/skills/install-openviking-memory/SKILL.md \ + ~/.openclaw/skills/install-openviking-memory/ +``` + +**Windows (cmd):** + +```cmd +mkdir "%USERPROFILE%\.openclaw\skills\install-openviking-memory" +copy examples\openclaw-memory-plugin\skills\install-openviking-memory\SKILL.md ^ + "%USERPROFILE%\.openclaw\skills\install-openviking-memory\" +``` + +然后对 OpenClaw 说:**「安装 OpenViking 记忆」** — 它会读取技能并自动完成安装。 + +如需手动安装,请继续阅读。 + +--- + +## 二、环境要求 + +### 总览 + +| 组件 | 版本要求 | 用途 | 必需? | +|------|----------|------|--------| +| **Python** | >= 3.10 | OpenViking 运行时 | 是 | +| **Node.js** | >= 22 | OpenClaw 运行时 + 安装助手 | 是 | +| **cmake** | — | 编译 C++ 扩展(OpenViking + OpenClaw 的 node-llama-cpp) | 是 | +| **g++ (gcc-c++)** | — | C++ 编译器 | 是 | +| **Go** | >= 1.25 | 编译 AGFS 服务端(仅 Linux 源码安装) | 源码安装必需 | +| **火山引擎 Ark API Key** | — | Embedding + VLM 模型调用 | 是 | + +> **PyPI 安装 vs 源码安装:** +> - `pip install openviking`(PyPI 预编译包):只需 Python、cmake、g++,**不需要 Go** +> - `pip install -e .`(源码安装):需要 Python、cmake、g++ **以及 Go >= 1.25**(Linux 上编译 AGFS) +> - **Windows** 用户可直接使用预编译 wheel 包,无需 Go + +### 快速检查 + +```bash +python3 --version # >= 3.10 +node -v # >= v22 +cmake --version # 已安装 +g++ --version # 已安装 +go version # >= go1.25(源码安装需要) +``` + +如果以上命令均正常,可跳过"环境准备"直接进入[第三节:安装步骤](#三安装步骤)。 + +--- + +## 三、环境准备(Linux) + +> 如果你的系统已满足上述环境要求,可跳过此节。 + +### 3.1 安装编译工具 + +> 已安装?运行 `cmake --version && g++ --version`,如果都有输出则跳过此步。 + +**RHEL / CentOS / openEuler / Fedora:** + +```bash +sudo dnf install -y gcc gcc-c++ cmake make +``` + +**Ubuntu / Debian:** + +```bash +sudo apt update +sudo apt install -y build-essential cmake +``` + +### 3.2 安装 Python 3.10+ + +> 已安装?运行 `python3 --version`,如果显示 >= 3.10 则跳过此步。 + +许多 Linux 发行版(如 openEuler 22.03、CentOS 7/8)自带 Python 3.9 或更低版本,且软件仓库中往往没有 Python 3.10+ 的包。推荐从源码编译。 + +#### 方式一:从源码编译(推荐) + +```bash +# 1. 安装编译依赖 +# RHEL / CentOS / openEuler / Fedora: +sudo dnf install -y gcc make openssl-devel bzip2-devel libffi-devel \ + zlib-devel readline-devel sqlite-devel xz-devel tk-devel + +# Ubuntu / Debian: +# sudo apt install -y build-essential libssl-dev libbz2-dev libffi-dev \ +# zlib1g-dev libreadline-dev libsqlite3-dev liblzma-dev tk-dev + +# 2. 下载并编译 +cd /tmp +curl -O https://www.python.org/ftp/python/3.11.12/Python-3.11.12.tgz +tar xzf Python-3.11.12.tgz +cd Python-3.11.12 +./configure --prefix=/usr/local --enable-optimizations --enable-shared \ + LDFLAGS="-Wl,-rpath /usr/local/lib" +make -j$(nproc) +sudo make altinstall + +# 3. 创建软链接,使 python3 / pip3 指向新版本 +sudo ln -sf /usr/local/bin/python3.11 /usr/local/bin/python3 +sudo ln -sf /usr/local/bin/pip3.11 /usr/local/bin/pip3 + +# 4. 验证 +python3 --version # 确认 >= 3.10 即可 +``` + +> **提示:** 使用 `altinstall` 而非 `install`,避免覆盖系统默认 Python。`/usr/local/bin` 在 `PATH` 中通常优先于 `/usr/bin`,创建软链接后 `python3` 即指向新版本。 + +#### 方式二:通过包管理器安装(部分发行版可用) + +```bash +# RHEL / CentOS / openEuler / Fedora(不一定可用) +sudo dnf install -y python3.11 python3.11-devel python3.11-pip + +# Ubuntu 22.04+ 自带 Python 3.10 +sudo apt install -y python3 python3-dev python3-pip python3-venv + +# Ubuntu 20.04 或更旧版本,需添加 PPA +sudo add-apt-repository ppa:deadsnakes/ppa +sudo apt install -y python3.11 python3.11-dev python3.11-venv +``` + +> 如果 `dnf install python3.11` 报 `No match for argument`,说明仓库中没有该包,请使用源码编译方式。 + +安装完成后升级 pip: + +```bash +python3 -m pip install --upgrade pip +``` + +> 下载 Python 包较慢?参见[附录:网络加速](#七网络加速镜像与代理配置)配置 pip 镜像。 + +### 3.3 安装 Node.js >= 22 + +> 已安装?运行 `node -v`,如果显示 >= v22 则跳过此步。 + +OpenClaw 要求 Node.js >= 22。安装助手脚本也依赖 Node.js 运行。 + +#### 方式一:通过 NodeSource 安装(推荐) + +```bash +# RHEL / CentOS / openEuler / Fedora +curl -fsSL https://rpm.nodesource.com/setup_22.x | sudo bash - +sudo dnf install -y nodejs + +# Ubuntu / Debian +# curl -fsSL https://deb.nodesource.com/setup_22.x | sudo bash - +# sudo apt install -y nodejs +``` + +#### 方式二:通过 nvm 安装(无需 root 权限) + +```bash +curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.40.1/install.sh | bash +source ~/.bashrc +nvm install 22 +nvm use 22 +``` + +#### 方式三:手动下载二进制包 + +```bash +wget https://nodejs.org/dist/v22.14.0/node-v22.14.0-linux-x64.tar.xz +sudo tar -C /usr/local -xJf node-v22.14.0-linux-x64.tar.xz +echo 'export PATH=$PATH:/usr/local/node-v22.14.0-linux-x64/bin' >> ~/.bashrc +source ~/.bashrc +``` + +> ARM 架构请将 `linux-x64` 替换为 `linux-arm64`。 + +验证: + +```bash +node -v # >= v22 +npm -v +``` + +### 3.4 安装 Go >= 1.25(仅源码安装需要) + +> 已安装?运行 `go version`,如果显示 >= go1.25 则跳过此步。 +> 使用 `pip install openviking`(PyPI 预编译包)的用户也可跳过。 + +Linux 源码安装 OpenViking 时需要 Go 编译 AGFS 服务端。 + +```bash +# 下载(ARM 请替换为 go1.25.6.linux-arm64.tar.gz) +wget https://go.dev/dl/go1.25.6.linux-amd64.tar.gz + +# 解压 +sudo rm -rf /usr/local/go +sudo tar -C /usr/local -xzf go1.25.6.linux-amd64.tar.gz + +# 配置环境变量 +cat >> ~/.bashrc << 'EOF' +export GOROOT=/usr/local/go +export GOPATH=$HOME/go +export PATH=$PATH:$GOROOT/bin:$GOPATH/bin +EOF +source ~/.bashrc + +# 验证 +go version # >= go1.25 +``` + +> Go 模块下载慢?参见[附录:网络加速](#七网络加速镜像与代理配置)配置 GOPROXY。 + +### 3.5 验证环境就绪 + +```bash +python3 --version # >= 3.10 +node -v # >= v22 +cmake --version # 已安装 +g++ --version # 已安装 +go version # >= go1.25(源码安装需要) +``` + +全部通过后即可开始安装。 + +--- + +## 四、安装步骤 + +### 4.1 安装 OpenClaw + +> **前置条件:** cmake 和 g++ 必须已安装(OpenClaw 依赖 `node-llama-cpp`,安装时需编译 C++ 代码)。 + +```bash +npm install -g openclaw +``` + +> 下载慢?参见[附录:网络加速](#七网络加速镜像与代理配置)配置 npm 镜像。 + +运行引导程序配置 LLM: + +```bash +openclaw onboard +``` + +验证: + +```bash +openclaw --version +``` + +### 4.2 安装 OpenViking + +```bash +git clone https://github.com/volcengine/OpenViking.git +cd OpenViking +``` + +#### 方式 A:从 PyPI 安装(推荐,无需 Go) + +```bash +python3 -m pip install openviking +``` + +#### 方式 B:从源码安装(开发者模式,需要 Go) + +**Linux / macOS:** + +```bash +go version && cmake --version && g++ --version # 确认工具已安装 +python3 -m pip install -e . +``` + +**Windows:** + +```powershell +python -m pip install -e . +``` + +> **提示:** Linux 源码安装**必须**安装 Go >= 1.25(编译 AGFS)。如确需跳过(高级用户): +> ```bash +> OPENVIKING_SKIP_AGFS_BUILD=1 python3 -m pip install -e . +> ``` + +验证: + +```bash +python3 -c "import openviking; print('ok')" +``` + +### 4.3 运行安装助手 + +在 OpenViking 仓库根目录下执行: + +```bash +npx ./examples/openclaw-memory-plugin/setup-helper +``` + +安装助手会依次完成: + +1. **环境检查** — 校验 cmake、g++、Python、Go、OpenClaw 是否就绪 +2. **安装 OpenViking**(如尚未安装) +3. **交互配置** — 提示输入以下信息: + - 数据存储路径(默认为绝对路径,如 `/home/yourname/.openviking/data`) + - 火山引擎 Ark API Key + - VLM 模型名称(默认 `doubao-seed-1-8-251228`) + - Embedding 模型名称(默认 `doubao-embedding-vision-250615`) + - 服务端口(默认 1933 / 1833) +4. **生成配置** — 创建 `~/.openviking/ov.conf` +5. **部署插件** — 将 `memory-openviking` 注册到 OpenClaw +6. **写入环境文件** — 生成 `~/.openclaw/openviking.env` + +> 非交互模式:`npx ./examples/openclaw-memory-plugin/setup-helper -y` + +### 4.4 启动并验证 + +**Linux / macOS:** + +```bash +source ~/.openclaw/openviking.env && openclaw gateway +``` + +**Windows (cmd):** + +```cmd +call "%USERPROFILE%\.openclaw\openviking.env.bat" && openclaw gateway +``` + +看到以下输出表示安装成功: + +``` +[gateway] listening on ws://127.0.0.1:18789 +[gateway] memory-openviking: local server started (http://127.0.0.1:1933, config: ...) +``` + +检查插件状态: + +```bash +openclaw status +# Memory 行应显示:enabled (plugin memory-openviking) +``` + +测试记忆功能: + +```bash +openclaw tui +``` + +输入:「请记住:我最喜欢的编程语言是 Python。」 + +在新对话中问:「我最喜欢的编程语言是什么?」 + +OpenClaw 应能从记忆中回忆并回答。 + +--- + +## 五、日常使用 + +每次使用带记忆功能的 OpenClaw: + +**Linux / macOS:** + +```bash +source ~/.openclaw/openviking.env && openclaw gateway +``` + +**Windows (cmd):** + +```cmd +call "%USERPROFILE%\.openclaw\openviking.env.bat" && openclaw gateway +``` + +> **便捷方式(Linux/macOS):** 加入 `~/.bashrc`: +> ```bash +> alias openclaw-start='source ~/.openclaw/openviking.env && openclaw gateway' +> ``` + +插件会自动启动和停止 OpenViking 服务。 + +--- + +## 六、配置参考 + +### `~/.openviking/ov.conf` + +```json +{ + "server": { + "host": "127.0.0.1", + "port": 1933 + }, + "storage": { + "workspace": "/home/yourname/.openviking/data", + "vectordb": { "backend": "local" }, + "agfs": { "backend": "local", "port": 1833 } + }, + "embedding": { + "dense": { + "backend": "volcengine", + "api_key": "", + "model": "doubao-embedding-vision-250615", + "api_base": "https://ark.cn-beijing.volces.com/api/v3", + "dimension": 1024, + "input": "multimodal" + } + }, + "vlm": { + "backend": "volcengine", + "api_key": "", + "model": "doubao-seed-1-8-251228", + "api_base": "https://ark.cn-beijing.volces.com/api/v3", + "temperature": 0.1, + "max_retries": 3 + } +} +``` + +> **注意:** `workspace` 必须使用**绝对路径**(如 `/home/yourname/.openviking/data`),不支持 `~` 或相对路径。安装助手会自动获取并填入。 + +### `~/.openclaw/openviking.env` + +由安装助手自动生成: + +```bash +export OPENVIKING_PYTHON='/usr/local/bin/python3' +export OPENVIKING_GO_PATH='/usr/local/go/bin' # 可选 +``` + +Windows 版(`openviking.env.bat`): + +```cmd +set OPENVIKING_PYTHON=C:\path\to\python.exe +set OPENVIKING_GO_PATH=C:\path\to\go\bin +``` + +### 安装助手选项 + +``` +npx ./examples/openclaw-memory-plugin/setup-helper [选项] + + -y, --yes 非交互模式,使用默认值 + -h, --help 显示帮助 + +环境变量: + OPENVIKING_PYTHON Python 解释器路径 + OPENVIKING_CONFIG_FILE 自定义 ov.conf 路径 + OPENVIKING_REPO 本地仓库路径(在仓库内运行时自动检测) + OPENVIKING_ARK_API_KEY 跳过 API Key 提示(用于 CI/脚本) +``` + +--- + +## 七、常见问题 + +### 安装阶段 + +#### Q: `cmake not found` / `g++ not found`(安装 OpenClaw 或 OpenViking 时) + +OpenClaw 依赖 `node-llama-cpp`(需编译 C++),OpenViking 的 C++ 扩展也需要 cmake/g++。 + +```bash +# RHEL / CentOS / openEuler +sudo dnf install -y gcc gcc-c++ cmake make + +# Ubuntu / Debian +sudo apt install -y build-essential cmake +``` + +#### Q: `No matching distribution found for python-multipart>=0.0.22` + +pip 使用了 Python 3.9 解释器。请确认使用 Python 3.10+: + +```bash +python3 --version # 确认 >= 3.10 +python3 -m pip install -e . +``` + +#### Q: `fatal error: Python.h: No such file or directory` + +缺少 Python 开发头文件: + +```bash +# RHEL / CentOS / openEuler +sudo dnf install -y python3-devel # 或 python3.11-devel + +# Ubuntu / Debian +sudo apt install -y python3-dev # 或 python3.11-dev +``` + +> 如果是源码编译的 Python,开发头文件已包含在内,无需额外安装。 + +#### Q: `Go compiler not found` / AGFS 构建失败 + +Linux 源码安装**必须**安装 Go >= 1.25,参见 [3.4 安装 Go](#34-安装-go--125仅源码安装需要)。 + +```bash +go version # 确认 >= 1.25 +python3 -m pip install -e . +``` + +#### Q: Go 模块下载超时(`dial tcp: i/o timeout`) + +配置 Go 代理,参见[附录:网络加速](#七网络加速镜像与代理配置)。 + +#### Q: npm 安装报 `ERR_INVALID_URL` + +通常是代理环境变量格式错误。代理地址**必须**包含 `http://` 前缀: + +```bash +# 错误 +export https_proxy=192.168.1.1:7897 + +# 正确 +export https_proxy=http://192.168.1.1:7897 +``` + +或清除代理: + +```bash +unset http_proxy https_proxy HTTP_PROXY HTTPS_PROXY +``` + +#### Q: npm 安装报 `ENOTEMPTY` + +上次安装中断留下残留文件,清理后重试: + +```bash +rm -rf $(npm root -g)/openclaw $(npm root -g)/.openclaw-* +npm install -g openclaw +``` + +### 运行阶段 + +#### Q: 网关输出中看不到 `memory-openviking` 插件 + +- 是否在 `openclaw gateway` 之前加载了环境变量? + - Linux/macOS:`source ~/.openclaw/openviking.env` + - Windows:`call "%USERPROFILE%\.openclaw\openviking.env.bat"` +- 运行 `openclaw status` 检查插件状态 +- 重新执行安装助手:`npx ./examples/openclaw-memory-plugin/setup-helper` + +#### Q: `health check timeout at http://127.0.0.1:1933` + +端口被旧进程占用,清理后重启: + +```bash +# Linux / macOS +lsof -ti tcp:1933 tcp:1833 | xargs kill -9 +source ~/.openclaw/openviking.env && openclaw gateway +``` + +```cmd +REM Windows +for /f "tokens=5" %a in ('netstat -ano ^| findstr "LISTENING" ^| findstr ":1933 :1833"') do taskkill /PID %a /F +call "%USERPROFILE%\.openclaw\openviking.env.bat" && openclaw gateway +``` + +#### Q: `extracted 0 memories` + +`ov.conf` 中的模型配置有误,请检查: + +- `embedding.dense.api_key` 为有效的火山引擎 Ark API Key +- `vlm.api_key` 已设置(通常与 embedding 相同) +- `vlm.model` 为模型名称(如 `doubao-seed-1-8-251228`),**不是** API Key + +### Python 版本问题 + +#### Q: `TypeError: unsupported operand type(s) for |: 'type' and 'NoneType'` + +Python 版本低于 3.10(`X | None` 语法是 3.10 引入的),需升级 Python,参见 [3.2 安装 Python](#32-安装-python-310)。 + +#### Q: `pip install -e .` 安装到了错误的 Python + +使用显式路径指定 Python: + +```bash +python3.11 -m pip install -e . +export OPENVIKING_PYTHON=python3.11 +npx ./examples/openclaw-memory-plugin/setup-helper +``` + +--- + +## 八、网络加速(镜像与代理配置) + +> 以下配置适用于下载速度较慢的网络环境。 + +### pip 镜像 + +```bash +# 永久配置(推荐) +python3 -m pip config set global.index-url https://pypi.tuna.tsinghua.edu.cn/simple +python3 -m pip config set global.trusted-host pypi.tuna.tsinghua.edu.cn + +# 其他可选镜像 +# 阿里云:https://mirrors.aliyun.com/pypi/simple/ +# 华为云:https://repo.huaweicloud.com/repository/pypi/simple/ +# 腾讯云:https://mirrors.cloud.tencent.com/pypi/simple/ +``` + +单次安装时临时指定: + +```bash +python3 -m pip install openviking -i https://pypi.tuna.tsinghua.edu.cn/simple +``` + +### npm 镜像 + +```bash +# 永久配置(推荐) +npm config set registry https://registry.npmmirror.com + +# 单次指定 +npm install -g openclaw --registry=https://registry.npmmirror.com +``` + +### Go 代理 + +```bash +# 七牛云(推荐) +go env -w GOPROXY=https://goproxy.cn,direct + +# 阿里云 +# go env -w GOPROXY=https://mirrors.aliyun.com/goproxy/,direct + +# 关闭校验(部分私有模块可能需要) +go env -w GONOSUMCHECK=* + +# 验证 +go env GOPROXY +``` + +> 配置后全局生效,后续 `pip install -e .` 编译 AGFS 时会自动使用。 + +--- + +## 九、卸载 + +**Linux / macOS:** + +```bash +# 停止服务 +lsof -ti tcp:1933 tcp:1833 tcp:18789 | xargs kill -9 + +# 卸载 OpenClaw +npm uninstall -g openclaw +rm -rf ~/.openclaw + +# 卸载 OpenViking +python3 -m pip uninstall openviking -y +rm -rf ~/.openviking +``` + +**Windows (cmd):** + +```cmd +REM 停止服务 +for /f "tokens=5" %a in ('netstat -ano ^| findstr "LISTENING" ^| findstr ":1933 :1833 :18789"') do taskkill /PID %a /F + +REM 卸载 OpenClaw +npm uninstall -g openclaw +rmdir /s /q "%USERPROFILE%\.openclaw" + +REM 卸载 OpenViking +python -m pip uninstall openviking -y +rmdir /s /q "%USERPROFILE%\.openviking" +``` + +--- + +**另见:** [INSTALL.md](./INSTALL.md)(English version) diff --git a/examples/openclaw-memory-plugin/INSTALL.md b/examples/openclaw-memory-plugin/INSTALL.md new file mode 100644 index 00000000..0c8b0e31 --- /dev/null +++ b/examples/openclaw-memory-plugin/INSTALL.md @@ -0,0 +1,679 @@ +# Install OpenViking Memory for OpenClaw + +Give [OpenClaw](https://github.com/openclaw/openclaw) long-term memory powered by [OpenViking](https://github.com/volcengine/OpenViking). After setup, OpenClaw will automatically **remember** facts from conversations and **recall** relevant context before responding. + +--- + +## 1. Quick Start (Let OpenClaw Install It) + +Copy the skill file into OpenClaw's skill directory, then let OpenClaw handle the rest: + +**Linux / macOS:** + +```bash +mkdir -p ~/.openclaw/skills/install-openviking-memory +cp examples/openclaw-memory-plugin/skills/install-openviking-memory/SKILL.md \ + ~/.openclaw/skills/install-openviking-memory/ +``` + +**Windows (cmd):** + +```cmd +mkdir "%USERPROFILE%\.openclaw\skills\install-openviking-memory" +copy examples\openclaw-memory-plugin\skills\install-openviking-memory\SKILL.md ^ + "%USERPROFILE%\.openclaw\skills\install-openviking-memory\" +``` + +Then tell OpenClaw: **"Install OpenViking memory"** — it will read the skill and complete the setup. + +For manual installation, continue reading. + +--- + +## 2. Prerequisites + +### Overview + +| Component | Version | Purpose | Required? | +|-----------|---------|---------|-----------| +| **Python** | >= 3.10 | OpenViking runtime | Yes | +| **Node.js** | >= 22 | OpenClaw runtime + setup helper | Yes | +| **cmake** | — | Compile C++ extensions (OpenViking + OpenClaw's node-llama-cpp) | Yes | +| **g++ (gcc-c++)** | — | C++ compiler | Yes | +| **Go** | >= 1.25 | Compile AGFS server (Linux source install only) | Source install only | +| **Volcengine Ark API Key** | — | Embedding + VLM model calls | Yes | + +> **PyPI vs Source install:** +> - `pip install openviking` (pre-built package): needs Python, cmake, g++ — **no Go required** +> - `pip install -e .` (source install): needs Python, cmake, g++ **and Go >= 1.25** (to compile AGFS on Linux) +> - **Windows** users can use pre-built wheel packages without Go + +### Quick Check + +```bash +python3 --version # >= 3.10 +node -v # >= v22 +cmake --version # installed +g++ --version # installed +go version # >= go1.25 (source install only) +``` + +If all commands pass, skip ahead to [Section 4: Installation Steps](#4-installation-steps). + +--- + +## 3. Environment Setup (Linux) + +> Skip this section if your system already meets the prerequisites above. + +### 3.1 Install Build Tools + +> Already installed? Run `cmake --version && g++ --version` — if both show output, skip this step. + +**RHEL / CentOS / openEuler / Fedora:** + +```bash +sudo dnf install -y gcc gcc-c++ cmake make +``` + +**Ubuntu / Debian:** + +```bash +sudo apt update +sudo apt install -y build-essential cmake +``` + +### 3.2 Install Python 3.10+ + +> Already installed? Run `python3 --version` — if it shows >= 3.10, skip this step. + +Many Linux distributions (e.g. openEuler 22.03, CentOS 7/8) ship with Python 3.9 or older, and their repositories often do not include Python 3.10+ packages. Building from source is recommended. + +#### Option A: Build from Source (recommended) + +```bash +# 1. Install build dependencies +# RHEL / CentOS / openEuler / Fedora: +sudo dnf install -y gcc make openssl-devel bzip2-devel libffi-devel \ + zlib-devel readline-devel sqlite-devel xz-devel tk-devel + +# Ubuntu / Debian: +# sudo apt install -y build-essential libssl-dev libbz2-dev libffi-dev \ +# zlib1g-dev libreadline-dev libsqlite3-dev liblzma-dev tk-dev + +# 2. Download and build +cd /tmp +curl -O https://www.python.org/ftp/python/3.11.12/Python-3.11.12.tgz +tar xzf Python-3.11.12.tgz +cd Python-3.11.12 +./configure --prefix=/usr/local --enable-optimizations --enable-shared \ + LDFLAGS="-Wl,-rpath /usr/local/lib" +make -j$(nproc) +sudo make altinstall + +# 3. Create symlinks so python3 / pip3 point to the new version +sudo ln -sf /usr/local/bin/python3.11 /usr/local/bin/python3 +sudo ln -sf /usr/local/bin/pip3.11 /usr/local/bin/pip3 + +# 4. Verify +python3 --version # Any version >= 3.10 is acceptable +``` + +> **Tip:** Use `altinstall` instead of `install` to avoid overwriting the system default Python. `/usr/local/bin` typically has higher priority in `PATH`, so the symlinks make `python3` point to the new version. + +#### Option B: Install via Package Manager (available on some distros) + +```bash +# RHEL / CentOS / openEuler / Fedora (may not be available) +sudo dnf install -y python3.11 python3.11-devel python3.11-pip + +# Ubuntu 22.04+ ships with Python 3.10 +sudo apt install -y python3 python3-dev python3-pip python3-venv + +# Ubuntu 20.04 or older, add the deadsnakes PPA first +sudo add-apt-repository ppa:deadsnakes/ppa +sudo apt install -y python3.11 python3.11-dev python3.11-venv +``` + +> If `dnf install python3.11` reports `No match for argument`, your repository does not have this package. Please use the source build method above. + +After installation, upgrade pip: + +```bash +python3 -m pip install --upgrade pip +``` + +> Downloads slow? See [Appendix: Network Acceleration](#8-network-acceleration-mirrors--proxies) to configure pip mirrors. + +### 3.3 Install Node.js >= 22 + +> Already installed? Run `node -v` — if it shows >= v22, skip this step. + +OpenClaw requires Node.js >= 22. The setup helper script also needs Node.js. + +#### Option A: Install via NodeSource (recommended) + +```bash +# RHEL / CentOS / openEuler / Fedora +curl -fsSL https://rpm.nodesource.com/setup_22.x | sudo bash - +sudo dnf install -y nodejs + +# Ubuntu / Debian +# curl -fsSL https://deb.nodesource.com/setup_22.x | sudo bash - +# sudo apt install -y nodejs +``` + +#### Option B: Install via nvm (no root required) + +```bash +curl -o- https://raw.githubusercontent.com/nvm-sh/nvm/v0.40.1/install.sh | bash +source ~/.bashrc +nvm install 22 +nvm use 22 +``` + +#### Option C: Download binary package manually + +```bash +wget https://nodejs.org/dist/v22.14.0/node-v22.14.0-linux-x64.tar.xz +sudo tar -C /usr/local -xJf node-v22.14.0-linux-x64.tar.xz +echo 'export PATH=$PATH:/usr/local/node-v22.14.0-linux-x64/bin' >> ~/.bashrc +source ~/.bashrc +``` + +> For ARM architecture, replace `linux-x64` with `linux-arm64`. + +Verify: + +```bash +node -v # >= v22 +npm -v +``` + +### 3.4 Install Go >= 1.25 (source install only) + +> Already installed? Run `go version` — if it shows >= go1.25, skip this step. +> Also skippable if using `pip install openviking` (pre-built package). + +Go is required on Linux to compile the AGFS server when installing from source. + +```bash +# Download (for ARM use go1.25.6.linux-arm64.tar.gz) +wget https://go.dev/dl/go1.25.6.linux-amd64.tar.gz + +# Extract +sudo rm -rf /usr/local/go +sudo tar -C /usr/local -xzf go1.25.6.linux-amd64.tar.gz + +# Configure environment variables +cat >> ~/.bashrc << 'EOF' +export GOROOT=/usr/local/go +export GOPATH=$HOME/go +export PATH=$PATH:$GOROOT/bin:$GOPATH/bin +EOF +source ~/.bashrc + +# Verify +go version # >= go1.25 +``` + +> Go module downloads slow? See [Appendix: Network Acceleration](#8-network-acceleration-mirrors--proxies) to configure GOPROXY. + +### 3.5 Verify Environment + +```bash +python3 --version # >= 3.10 +node -v # >= v22 +cmake --version # installed +g++ --version # installed +go version # >= go1.25 (source install only) +``` + +All checks pass? Proceed to installation. + +--- + +## 4. Installation Steps + +### 4.1 Install OpenClaw + +> **Prerequisite:** cmake and g++ must be installed (OpenClaw depends on `node-llama-cpp`, which compiles C++ code during installation). + +```bash +npm install -g openclaw +``` + +> Downloads slow? See [Appendix: Network Acceleration](#8-network-acceleration-mirrors--proxies) to configure npm mirrors. + +Run the onboarding wizard to configure your LLM: + +```bash +openclaw onboard +``` + +Verify: + +```bash +openclaw --version +``` + +### 4.2 Install OpenViking + +```bash +git clone https://github.com/volcengine/OpenViking.git +cd OpenViking +``` + +#### Option A: Install from PyPI (recommended, no Go needed) + +```bash +python3 -m pip install openviking +``` + +#### Option B: Install from Source (developer mode, requires Go) + +**Linux / macOS:** + +```bash +go version && cmake --version && g++ --version # Confirm tools are installed +python3 -m pip install -e . +``` + +**Windows:** + +```powershell +python -m pip install -e . +``` + +> **Note:** Go >= 1.25 is **required** on Linux for source install (to compile AGFS). To force-skip (advanced users only): +> ```bash +> OPENVIKING_SKIP_AGFS_BUILD=1 python3 -m pip install -e . +> ``` + +Verify: + +```bash +python3 -c "import openviking; print('ok')" +``` + +### 4.3 Run the Setup Helper + +From the OpenViking repo root: + +```bash +npx ./examples/openclaw-memory-plugin/setup-helper +``` + +The helper will walk you through: + +1. **Environment check** — verifies cmake, g++, Python, Go, OpenClaw +2. **Install OpenViking** (if not already installed) +3. **Interactive configuration** — prompts for: + - Data storage path (defaults to absolute path, e.g. `/home/yourname/.openviking/data`) + - Volcengine Ark API Key + - VLM model name (default: `doubao-seed-1-8-251228`) + - Embedding model name (default: `doubao-embedding-vision-250615`) + - Server ports (default: 1933 / 1833) +4. **Generate config** — creates `~/.openviking/ov.conf` +5. **Deploy plugin** — registers `memory-openviking` with OpenClaw +6. **Write env file** — generates `~/.openclaw/openviking.env` + +> Non-interactive mode: `npx ./examples/openclaw-memory-plugin/setup-helper -y` + +### 4.4 Start and Verify + +**Linux / macOS:** + +```bash +source ~/.openclaw/openviking.env && openclaw gateway +``` + +**Windows (cmd):** + +```cmd +call "%USERPROFILE%\.openclaw\openviking.env.bat" && openclaw gateway +``` + +You should see: + +``` +[gateway] listening on ws://127.0.0.1:18789 +[gateway] memory-openviking: local server started (http://127.0.0.1:1933, config: ...) +``` + +Check plugin status: + +```bash +openclaw status +# Memory line should show: enabled (plugin memory-openviking) +``` + +Test memory: + +```bash +openclaw tui +``` + +Say: "Please remember: my favorite programming language is Python." + +In a later conversation, ask: "What is my favorite programming language?" + +OpenClaw should recall the answer from OpenViking memory. + +--- + +## 5. Daily Usage + +Each time you want to use OpenClaw with memory: + +**Linux / macOS:** + +```bash +source ~/.openclaw/openviking.env && openclaw gateway +``` + +**Windows (cmd):** + +```cmd +call "%USERPROFILE%\.openclaw\openviking.env.bat" && openclaw gateway +``` + +> **Convenience (Linux/macOS):** Add to `~/.bashrc`: +> ```bash +> alias openclaw-start='source ~/.openclaw/openviking.env && openclaw gateway' +> ``` + +The plugin automatically starts and stops the OpenViking server. + +--- + +## 6. Configuration Reference + +### `~/.openviking/ov.conf` + +```json +{ + "server": { + "host": "127.0.0.1", + "port": 1933 + }, + "storage": { + "workspace": "/home/yourname/.openviking/data", + "vectordb": { "backend": "local" }, + "agfs": { "backend": "local", "port": 1833 } + }, + "embedding": { + "dense": { + "backend": "volcengine", + "api_key": "", + "model": "doubao-embedding-vision-250615", + "api_base": "https://ark.cn-beijing.volces.com/api/v3", + "dimension": 1024, + "input": "multimodal" + } + }, + "vlm": { + "backend": "volcengine", + "api_key": "", + "model": "doubao-seed-1-8-251228", + "api_base": "https://ark.cn-beijing.volces.com/api/v3", + "temperature": 0.1, + "max_retries": 3 + } +} +``` + +> **Note:** `workspace` must be an **absolute path** (e.g. `/home/yourname/.openviking/data`). Tilde (`~`) and relative paths are not supported. The setup helper fills this in automatically. + +### `~/.openclaw/openviking.env` + +Auto-generated by the setup helper: + +```bash +export OPENVIKING_PYTHON='/usr/local/bin/python3' +export OPENVIKING_GO_PATH='/usr/local/go/bin' # optional +``` + +Windows version (`openviking.env.bat`): + +```cmd +set OPENVIKING_PYTHON=C:\path\to\python.exe +set OPENVIKING_GO_PATH=C:\path\to\go\bin +``` + +### Setup Helper Options + +``` +npx ./examples/openclaw-memory-plugin/setup-helper [options] + + -y, --yes Non-interactive, use defaults + -h, --help Show help + +Environment variables: + OPENVIKING_PYTHON Python interpreter path + OPENVIKING_CONFIG_FILE Custom ov.conf path + OPENVIKING_REPO Local repo path (auto-detected when run from repo) + OPENVIKING_ARK_API_KEY Skip API key prompt (for CI/scripts) +``` + +--- + +## 7. Troubleshooting + +### Installation Issues + +#### `cmake not found` / `g++ not found` + +OpenClaw depends on `node-llama-cpp` (compiles C++), and OpenViking's C++ extensions also need cmake/g++. + +```bash +# RHEL / CentOS / openEuler +sudo dnf install -y gcc gcc-c++ cmake make + +# Ubuntu / Debian +sudo apt install -y build-essential cmake +``` + +#### `No matching distribution found for python-multipart>=0.0.22` + +pip is using Python 3.9. Make sure you're using Python 3.10+: + +```bash +python3 --version # Confirm >= 3.10 +python3 -m pip install -e . +``` + +#### `fatal error: Python.h: No such file or directory` + +Missing Python development headers: + +```bash +# RHEL / CentOS / openEuler +sudo dnf install -y python3-devel # or python3.11-devel + +# Ubuntu / Debian +sudo apt install -y python3-dev # or python3.11-dev +``` + +> If Python was built from source, development headers are already included. + +#### `Go compiler not found` / AGFS build failure + +Go >= 1.25 is **required** on Linux for source install. See [3.4 Install Go](#34-install-go--125-source-install-only). + +```bash +go version # Confirm >= 1.25 +python3 -m pip install -e . +``` + +#### Go module download timeout (`dial tcp: i/o timeout`) + +Configure Go proxy. See [Appendix: Network Acceleration](#8-network-acceleration-mirrors--proxies). + +#### npm `ERR_INVALID_URL` + +Usually caused by malformed proxy environment variables. Proxy URLs **must** include the `http://` prefix: + +```bash +# Wrong +export https_proxy=192.168.1.1:7897 + +# Correct +export https_proxy=http://192.168.1.1:7897 +``` + +Or clear proxies entirely: + +```bash +unset http_proxy https_proxy HTTP_PROXY HTTPS_PROXY +``` + +#### npm `ENOTEMPTY` + +Previous install was interrupted. Clean up and retry: + +```bash +rm -rf $(npm root -g)/openclaw $(npm root -g)/.openclaw-* +npm install -g openclaw +``` + +### Runtime Issues + +#### Plugin not showing in gateway output + +- Did you load the env file before `openclaw gateway`? + - Linux/macOS: `source ~/.openclaw/openviking.env` + - Windows: `call "%USERPROFILE%\.openclaw\openviking.env.bat"` +- Run `openclaw status` to check plugin state +- Re-run setup: `npx ./examples/openclaw-memory-plugin/setup-helper` + +#### `health check timeout at http://127.0.0.1:1933` + +A stale process is occupying the port. Kill it and restart: + +```bash +# Linux / macOS +lsof -ti tcp:1933 tcp:1833 | xargs kill -9 +source ~/.openclaw/openviking.env && openclaw gateway +``` + +```cmd +REM Windows +for /f "tokens=5" %a in ('netstat -ano ^| findstr "LISTENING" ^| findstr ":1933 :1833"') do taskkill /PID %a /F +call "%USERPROFILE%\.openclaw\openviking.env.bat" && openclaw gateway +``` + +#### `extracted 0 memories` + +Model configuration in `ov.conf` is incorrect. Check: + +- `embedding.dense.api_key` is a valid Volcengine Ark API key +- `vlm.api_key` is set (usually the same key) +- `vlm.model` is a model name (e.g. `doubao-seed-1-8-251228`), **not** the API key + +### Python Version Issues + +#### `TypeError: unsupported operand type(s) for |: 'type' and 'NoneType'` + +Python version is below 3.10 (`X | None` syntax requires 3.10+). Upgrade Python — see [3.2 Install Python](#32-install-python-310). + +#### `pip install -e .` installs to the wrong Python + +Use an explicit Python path: + +```bash +python3.11 -m pip install -e . +export OPENVIKING_PYTHON=python3.11 +npx ./examples/openclaw-memory-plugin/setup-helper +``` + +--- + +## 8. Network Acceleration (Mirrors & Proxies) + +> Use these if package downloads are slow in your network environment. + +### pip Mirror + +```bash +# Permanent (recommended) +python3 -m pip config set global.index-url https://pypi.tuna.tsinghua.edu.cn/simple +python3 -m pip config set global.trusted-host pypi.tuna.tsinghua.edu.cn + +# Alternatives +# Alibaba Cloud: https://mirrors.aliyun.com/pypi/simple/ +# Huawei Cloud: https://repo.huaweicloud.com/repository/pypi/simple/ +# Tencent Cloud: https://mirrors.cloud.tencent.com/pypi/simple/ +``` + +Single-use: + +```bash +python3 -m pip install openviking -i https://pypi.tuna.tsinghua.edu.cn/simple +``` + +### npm Mirror + +```bash +# Permanent (recommended) +npm config set registry https://registry.npmmirror.com + +# Single-use +npm install -g openclaw --registry=https://registry.npmmirror.com +``` + +### Go Proxy + +```bash +# goproxy.cn (recommended) +go env -w GOPROXY=https://goproxy.cn,direct + +# Alibaba Cloud +# go env -w GOPROXY=https://mirrors.aliyun.com/goproxy/,direct + +# Disable checksum verification (may be needed for some modules) +go env -w GONOSUMCHECK=* + +# Verify +go env GOPROXY +``` + +> Takes effect globally for the current user. Subsequent `pip install -e .` builds that compile AGFS will use this automatically. + +--- + +## 9. Uninstall + +**Linux / macOS:** + +```bash +# Stop services +lsof -ti tcp:1933 tcp:1833 tcp:18789 | xargs kill -9 + +# Remove OpenClaw +npm uninstall -g openclaw +rm -rf ~/.openclaw + +# Remove OpenViking +python3 -m pip uninstall openviking -y +rm -rf ~/.openviking +``` + +**Windows (cmd):** + +```cmd +REM Stop services +for /f "tokens=5" %a in ('netstat -ano ^| findstr "LISTENING" ^| findstr ":1933 :1833 :18789"') do taskkill /PID %a /F + +REM Remove OpenClaw +npm uninstall -g openclaw +rmdir /s /q "%USERPROFILE%\.openclaw" + +REM Remove OpenViking +python -m pip uninstall openviking -y +rmdir /s /q "%USERPROFILE%\.openviking" +``` + +--- + +**See also:** [INSTALL-ZH.md](./INSTALL-ZH.md) (Chinese version) diff --git a/examples/openclaw-memory-plugin/README.md b/examples/openclaw-memory-plugin/README.md new file mode 100644 index 00000000..dbe8ef69 --- /dev/null +++ b/examples/openclaw-memory-plugin/README.md @@ -0,0 +1,86 @@ +# OpenClaw + OpenViking Memory Plugin + +Use OpenViking as the long-term memory backend for [OpenClaw](https://github.com/openclaw/openclaw). + +## Quick Start + +```bash +cd /path/to/OpenViking +npx ./examples/openclaw-memory-plugin/setup-helper +openclaw gateway +``` + +The setup helper checks the environment, creates `~/.openviking/ov.conf`, deploys the plugin, and configures OpenClaw automatically. + +## Manual Setup + +Prerequisites: **OpenClaw** (`npm install -g openclaw`), **Python >= 3.10** with `openviking` (`pip install openviking`). + +```bash +# Install plugin +mkdir -p ~/.openclaw/extensions/memory-openviking +cp examples/openclaw-memory-plugin/{index.ts,config.ts,openclaw.plugin.json,package.json,.gitignore} \ + ~/.openclaw/extensions/memory-openviking/ +cd ~/.openclaw/extensions/memory-openviking && npm install + +# Configure (local mode — plugin auto-starts OpenViking) +openclaw config set plugins.enabled true +openclaw config set plugins.slots.memory memory-openviking +openclaw config set plugins.entries.memory-openviking.config.mode "local" +openclaw config set plugins.entries.memory-openviking.config.configPath "~/.openviking/ov.conf" +openclaw config set plugins.entries.memory-openviking.config.targetUri "viking://" +openclaw config set plugins.entries.memory-openviking.config.autoRecall true --json +openclaw config set plugins.entries.memory-openviking.config.autoCapture true --json + +# Start +openclaw gateway +``` + +## Setup Helper Options + +``` +npx openclaw-openviking-setup-helper [options] + + -y, --yes Non-interactive, use defaults + -h, --help Show help + +Env vars: + OPENVIKING_PYTHON Python path + OPENVIKING_CONFIG_FILE ov.conf path + OPENVIKING_REPO Local OpenViking repo path + OPENVIKING_ARK_API_KEY Volcengine API Key (skip prompt in -y mode) +``` + +## ov.conf Example + +```json +{ + "vlm": { + "backend": "volcengine", + "api_key": "", + "model": "doubao-seed-1-8-251228", + "api_base": "https://ark.cn-beijing.volces.com/api/v3", + "temperature": 0.1, + "max_retries": 3 + }, + "embedding": { + "dense": { + "backend": "volcengine", + "api_key": "", + "model": "doubao-embedding-vision-250615", + "api_base": "https://ark.cn-beijing.volces.com/api/v3", + "dimension": 1024, + "input": "multimodal" + } + } +} +``` + +## Troubleshooting + +| Symptom | Fix | +|---------|-----| +| Memory shows `disabled` / `memory-core` | `openclaw config set plugins.slots.memory memory-openviking` | +| `memory_store failed: fetch failed` | Check OpenViking is running; verify `ov.conf` and Python path | +| `health check timeout` | `lsof -ti tcp:1933 \| xargs kill -9` then restart | +| `extracted 0 memories` | Ensure `ov.conf` has valid `vlm` and `embedding.dense` with API key | diff --git a/examples/openclaw-memory-plugin/config.ts b/examples/openclaw-memory-plugin/config.ts new file mode 100644 index 00000000..246c4da2 --- /dev/null +++ b/examples/openclaw-memory-plugin/config.ts @@ -0,0 +1,268 @@ +import { homedir } from "node:os"; +import { join } from "node:path"; +import { resolve as resolvePath } from "node:path"; + +export type MemoryOpenVikingConfig = { + /** "local" = plugin starts OpenViking server as child process (like Claude Code); "remote" = use existing HTTP server */ + mode?: "local" | "remote"; + /** Path to ov.conf; used when mode is "local". Default ~/.openviking/ov.conf */ + configPath?: string; + /** Port for local server when mode is "local". Ignored when mode is "remote". */ + port?: number; + baseUrl?: string; + apiKey?: string; + targetUri?: string; + timeoutMs?: number; + autoCapture?: boolean; + captureMode?: "semantic" | "keyword"; + captureMaxLength?: number; + autoRecall?: boolean; + recallLimit?: number; + recallScoreThreshold?: number; + ingestReplyAssist?: boolean; + ingestReplyAssistMinSpeakerTurns?: number; + ingestReplyAssistMinChars?: number; +}; + +const DEFAULT_BASE_URL = "http://127.0.0.1:1933"; +const DEFAULT_PORT = 1933; +const DEFAULT_TARGET_URI = "viking://"; +const DEFAULT_TIMEOUT_MS = 15000; +const DEFAULT_CAPTURE_MODE = "semantic"; +const DEFAULT_CAPTURE_MAX_LENGTH = 24000; +const DEFAULT_RECALL_LIMIT = 6; +const DEFAULT_RECALL_SCORE_THRESHOLD = 0.01; +const DEFAULT_INGEST_REPLY_ASSIST = true; +const DEFAULT_INGEST_REPLY_ASSIST_MIN_SPEAKER_TURNS = 2; +const DEFAULT_INGEST_REPLY_ASSIST_MIN_CHARS = 120; +const DEFAULT_LOCAL_CONFIG_PATH = join(homedir(), ".openviking", "ov.conf"); + +function resolveEnvVars(value: string): string { + return value.replace(/\$\{([^}]+)\}/g, (_, envVar) => { + const envValue = process.env[envVar]; + if (!envValue) { + throw new Error(`Environment variable ${envVar} is not set`); + } + return envValue; + }); +} + +function toNumber(value: unknown, fallback: number): number { + if (typeof value === "number" && Number.isFinite(value)) { + return value; + } + if (typeof value === "string" && value.trim() !== "") { + const parsed = Number(value); + if (Number.isFinite(parsed)) { + return parsed; + } + } + return fallback; +} + +function assertAllowedKeys(value: Record, allowed: string[], label: string) { + const unknown = Object.keys(value).filter((key) => !allowed.includes(key)); + if (unknown.length === 0) { + return; + } + throw new Error(`${label} has unknown keys: ${unknown.join(", ")}`); +} + +function resolveDefaultBaseUrl(): string { + const fromEnv = process.env.OPENVIKING_BASE_URL || process.env.OPENVIKING_URL; + if (fromEnv) { + return fromEnv; + } + return DEFAULT_BASE_URL; +} + +export const memoryOpenVikingConfigSchema = { + parse(value: unknown): Required { + if (!value || typeof value !== "object" || Array.isArray(value)) { + value = {}; + } + const cfg = value as Record; + assertAllowedKeys( + cfg, + [ + "mode", + "configPath", + "port", + "baseUrl", + "apiKey", + "targetUri", + "timeoutMs", + "autoCapture", + "captureMode", + "captureMaxLength", + "autoRecall", + "recallLimit", + "recallScoreThreshold", + "ingestReplyAssist", + "ingestReplyAssistMinSpeakerTurns", + "ingestReplyAssistMinChars", + ], + "memory-openviking config", + ); + + const mode = (cfg.mode === "local" || cfg.mode === "remote" ? cfg.mode : "remote") as + | "local" + | "remote"; + const port = Math.max(1, Math.min(65535, Math.floor(toNumber(cfg.port, DEFAULT_PORT)))); + const rawConfigPath = + typeof cfg.configPath === "string" && cfg.configPath.trim() + ? cfg.configPath.trim() + : DEFAULT_LOCAL_CONFIG_PATH; + const configPath = resolvePath( + resolveEnvVars(rawConfigPath).replace(/^~/, homedir()), + ); + + const localBaseUrl = `http://127.0.0.1:${port}`; + const rawBaseUrl = + mode === "local" ? localBaseUrl : (typeof cfg.baseUrl === "string" ? cfg.baseUrl : resolveDefaultBaseUrl()); + const resolvedBaseUrl = resolveEnvVars(rawBaseUrl).replace(/\/+$/, ""); + const rawApiKey = typeof cfg.apiKey === "string" ? cfg.apiKey : process.env.OPENVIKING_API_KEY; + const captureMode = cfg.captureMode; + if ( + typeof captureMode !== "undefined" && + captureMode !== "semantic" && + captureMode !== "keyword" + ) { + throw new Error(`memory-openviking captureMode must be "semantic" or "keyword"`); + } + + return { + mode, + configPath, + port, + baseUrl: resolvedBaseUrl, + apiKey: rawApiKey ? resolveEnvVars(rawApiKey) : "", + targetUri: typeof cfg.targetUri === "string" ? cfg.targetUri : DEFAULT_TARGET_URI, + timeoutMs: Math.max(1000, Math.floor(toNumber(cfg.timeoutMs, DEFAULT_TIMEOUT_MS))), + autoCapture: cfg.autoCapture !== false, + captureMode: captureMode ?? DEFAULT_CAPTURE_MODE, + captureMaxLength: Math.max( + 200, + Math.min(200_000, Math.floor(toNumber(cfg.captureMaxLength, DEFAULT_CAPTURE_MAX_LENGTH))), + ), + autoRecall: cfg.autoRecall !== false, + recallLimit: Math.max(1, Math.floor(toNumber(cfg.recallLimit, DEFAULT_RECALL_LIMIT))), + recallScoreThreshold: Math.min( + 1, + Math.max(0, toNumber(cfg.recallScoreThreshold, DEFAULT_RECALL_SCORE_THRESHOLD)), + ), + ingestReplyAssist: cfg.ingestReplyAssist !== false, + ingestReplyAssistMinSpeakerTurns: Math.max( + 1, + Math.min( + 12, + Math.floor( + toNumber( + cfg.ingestReplyAssistMinSpeakerTurns, + DEFAULT_INGEST_REPLY_ASSIST_MIN_SPEAKER_TURNS, + ), + ), + ), + ), + ingestReplyAssistMinChars: Math.max( + 32, + Math.min( + 10000, + Math.floor(toNumber(cfg.ingestReplyAssistMinChars, DEFAULT_INGEST_REPLY_ASSIST_MIN_CHARS)), + ), + ), + }; + }, + uiHints: { + mode: { + label: "Mode", + help: "local = plugin starts OpenViking server (like Claude Code); remote = use existing HTTP server", + }, + configPath: { + label: "Config path (local)", + placeholder: DEFAULT_LOCAL_CONFIG_PATH, + help: "Path to ov.conf when mode is local", + }, + port: { + label: "Port (local)", + placeholder: String(DEFAULT_PORT), + help: "Port for local OpenViking server", + advanced: true, + }, + baseUrl: { + label: "OpenViking Base URL (remote)", + placeholder: DEFAULT_BASE_URL, + help: "HTTP URL when mode is remote (or use ${OPENVIKING_BASE_URL})", + }, + apiKey: { + label: "OpenViking API Key", + sensitive: true, + placeholder: "${OPENVIKING_API_KEY}", + help: "Optional API key for OpenViking server", + }, + targetUri: { + label: "Search Target URI", + placeholder: DEFAULT_TARGET_URI, + help: "Default OpenViking target URI for memory search", + }, + timeoutMs: { + label: "Request Timeout (ms)", + placeholder: String(DEFAULT_TIMEOUT_MS), + advanced: true, + }, + autoCapture: { + label: "Auto-Capture", + help: "Extract memories from recent conversation messages via OpenViking sessions", + }, + captureMode: { + label: "Capture Mode", + placeholder: DEFAULT_CAPTURE_MODE, + advanced: true, + help: '"semantic" captures all eligible user text and relies on OpenViking extraction; "keyword" uses trigger regex first.', + }, + captureMaxLength: { + label: "Capture Max Length", + placeholder: String(DEFAULT_CAPTURE_MAX_LENGTH), + advanced: true, + help: "Maximum sanitized user text length allowed for auto-capture.", + }, + autoRecall: { + label: "Auto-Recall", + help: "Inject relevant OpenViking memories into agent context", + }, + recallLimit: { + label: "Recall Limit", + placeholder: String(DEFAULT_RECALL_LIMIT), + advanced: true, + }, + recallScoreThreshold: { + label: "Recall Score Threshold", + placeholder: String(DEFAULT_RECALL_SCORE_THRESHOLD), + advanced: true, + }, + ingestReplyAssist: { + label: "Ingest Reply Assist", + help: "When transcript-like memory ingestion is detected, add a lightweight reply instruction to reduce NO_REPLY.", + advanced: true, + }, + ingestReplyAssistMinSpeakerTurns: { + label: "Ingest Min Speaker Turns", + placeholder: String(DEFAULT_INGEST_REPLY_ASSIST_MIN_SPEAKER_TURNS), + help: "Minimum speaker-tag turns (e.g. Name:) to detect transcript-like ingest text.", + advanced: true, + }, + ingestReplyAssistMinChars: { + label: "Ingest Min Chars", + placeholder: String(DEFAULT_INGEST_REPLY_ASSIST_MIN_CHARS), + help: "Minimum sanitized text length required before ingest reply assist can trigger.", + advanced: true, + }, + }, +}; + +export const DEFAULT_MEMORY_OPENVIKING_DATA_DIR = join( + homedir(), + ".openclaw", + "memory", + "openviking", +); diff --git a/examples/openclaw-memory-plugin/index.ts b/examples/openclaw-memory-plugin/index.ts new file mode 100644 index 00000000..a2cb16d9 --- /dev/null +++ b/examples/openclaw-memory-plugin/index.ts @@ -0,0 +1,1303 @@ +import { spawn, execSync } from "node:child_process"; +import { readFileSync, existsSync } from "node:fs"; +import { join } from "node:path"; +import { homedir, tmpdir, platform } from "node:os"; + +const IS_WIN = platform() === "win32"; +import type { OpenClawPluginApi } from "openclaw/plugin-sdk"; +import { Type } from "@sinclair/typebox"; +import { memoryOpenVikingConfigSchema } from "./config.js"; + +type FindResultItem = { + uri: string; + is_leaf?: boolean; + abstract?: string; + overview?: string; + category?: string; + score?: number; + match_reason?: string; +}; + +type FindResult = { + memories?: FindResultItem[]; + resources?: FindResultItem[]; + skills?: FindResultItem[]; + total?: number; +}; + +type CaptureMode = "semantic" | "keyword"; + +const MEMORY_URI_PREFIXES = ["viking://user/memories", "viking://agent/memories"]; + +const MEMORY_TRIGGERS = [ + /remember|preference|prefer|important|decision|decided|always|never/i, + /记住|偏好|喜欢|喜爱|崇拜|讨厌|害怕|重要|决定|总是|永远|优先|习惯|爱好|擅长|最爱|不喜欢/i, + /[\w.-]+@[\w.-]+\.\w+/, + /\+\d{10,}/, + /(?:我|my)\s*(?:是|叫|名字|name|住在|live|来自|from|生日|birthday|电话|phone|邮箱|email)/i, + /(?:我|i)\s*(?:喜欢|崇拜|讨厌|害怕|擅长|不会|爱|恨|想要|需要|希望|觉得|认为|相信)/i, + /(?:favorite|favourite|love|hate|enjoy|dislike|admire|idol|fan of)/i, +]; + +const CJK_CHAR_REGEX = /[\u3040-\u30ff\u3400-\u9fff\uf900-\ufaff\uac00-\ud7af]/; +const RELEVANT_MEMORIES_BLOCK_RE = /[\s\S]*?<\/relevant-memories>/gi; +const CONVERSATION_METADATA_BLOCK_RE = + /(?:^|\n)\s*(?:Conversation info|Conversation metadata|会话信息|对话信息)\s*(?:\([^)]+\))?\s*:\s*```[\s\S]*?```/gi; +const FENCED_JSON_BLOCK_RE = /```json\s*([\s\S]*?)```/gi; +const METADATA_JSON_KEY_RE = + /"(session|sessionid|sessionkey|conversationid|channel|sender|userid|agentid|timestamp|timezone)"\s*:/gi; +const LEADING_TIMESTAMP_PREFIX_RE = /^\s*\[[^\]\n]{1,120}\]\s*/; +const COMMAND_TEXT_RE = /^\/[a-z0-9_-]{1,64}\b/i; +const NON_CONTENT_TEXT_RE = /^[\p{P}\p{S}\s]+$/u; +const SUBAGENT_CONTEXT_RE = /^\s*\[Subagent Context\]/i; +const MEMORY_INTENT_RE = /记住|记下|remember|save|store|偏好|preference|规则|rule|事实|fact/i; +const QUESTION_CUE_RE = + /[??]|\b(?:what|when|where|who|why|how|which|can|could|would|did|does|is|are)\b|^(?:请问|能否|可否|怎么|如何|什么时候|谁|什么|哪|是否)/i; +const CAPTURE_LIMIT = 3; +const SPEAKER_TAG_RE = /(?:^|\s)([A-Za-z\u4e00-\u9fa5][A-Za-z0-9_\u4e00-\u9fa5-]{1,30}):\s/g; + +function resolveCaptureMinLength(text: string): number { + return CJK_CHAR_REGEX.test(text) ? 4 : 10; +} + +function looksLikeMetadataJsonBlock(content: string): boolean { + const matchedKeys = new Set(); + const matches = content.matchAll(METADATA_JSON_KEY_RE); + for (const match of matches) { + const key = (match[1] ?? "").toLowerCase(); + if (key) { + matchedKeys.add(key); + } + } + return matchedKeys.size >= 3; +} + +function sanitizeUserTextForCapture(text: string): string { + return text + .replace(RELEVANT_MEMORIES_BLOCK_RE, " ") + .replace(CONVERSATION_METADATA_BLOCK_RE, " ") + .replace(FENCED_JSON_BLOCK_RE, (full, inner) => + looksLikeMetadataJsonBlock(String(inner ?? "")) ? " " : full, + ) + .replace(LEADING_TIMESTAMP_PREFIX_RE, "") + .replace(/\u0000/g, "") + .replace(/\s+/g, " ") + .trim(); +} + +function looksLikeQuestionOnlyText(text: string): boolean { + if (!QUESTION_CUE_RE.test(text) || MEMORY_INTENT_RE.test(text)) { + return false; + } + // Multi-speaker transcripts often contain many "?" but should still be captured. + const speakerTags = text.match(/[A-Za-z\u4e00-\u9fa5]{2,20}:\s/g) ?? []; + if (speakerTags.length >= 2 || text.length > 280) { + return false; + } + return true; +} + +type TranscriptLikeIngestDecision = { + shouldAssist: boolean; + reason: string; + normalizedText: string; + speakerTurns: number; + chars: number; +}; + +function countSpeakerTurns(text: string): number { + let count = 0; + for (const _match of text.matchAll(SPEAKER_TAG_RE)) { + count += 1; + } + return count; +} + +function isTranscriptLikeIngest( + text: string, + options: { + minSpeakerTurns: number; + minChars: number; + }, +): TranscriptLikeIngestDecision { + const normalizedText = sanitizeUserTextForCapture(text.trim()); + if (!normalizedText) { + return { + shouldAssist: false, + reason: "empty_text", + normalizedText, + speakerTurns: 0, + chars: 0, + }; + } + + if (COMMAND_TEXT_RE.test(normalizedText)) { + return { + shouldAssist: false, + reason: "command_text", + normalizedText, + speakerTurns: 0, + chars: normalizedText.length, + }; + } + + if (SUBAGENT_CONTEXT_RE.test(normalizedText)) { + return { + shouldAssist: false, + reason: "subagent_context", + normalizedText, + speakerTurns: 0, + chars: normalizedText.length, + }; + } + + if (NON_CONTENT_TEXT_RE.test(normalizedText)) { + return { + shouldAssist: false, + reason: "non_content_text", + normalizedText, + speakerTurns: 0, + chars: normalizedText.length, + }; + } + + if (looksLikeQuestionOnlyText(normalizedText)) { + return { + shouldAssist: false, + reason: "question_text", + normalizedText, + speakerTurns: 0, + chars: normalizedText.length, + }; + } + + const chars = normalizedText.length; + if (chars < options.minChars) { + return { + shouldAssist: false, + reason: "chars_below_threshold", + normalizedText, + speakerTurns: 0, + chars, + }; + } + + const speakerTurns = countSpeakerTurns(normalizedText); + if (speakerTurns < options.minSpeakerTurns) { + return { + shouldAssist: false, + reason: "speaker_turns_below_threshold", + normalizedText, + speakerTurns, + chars, + }; + } + + return { + shouldAssist: true, + reason: "transcript_like_ingest", + normalizedText, + speakerTurns, + chars, + }; +} + +function normalizeCaptureDedupeText(text: string): string { + return normalizeDedupeText(text).replace(/[\p{P}\p{S}]+/gu, " ").replace(/\s+/g, " ").trim(); +} + +function pickRecentUniqueTexts(texts: string[], limit: number): string[] { + if (limit <= 0 || texts.length === 0) { + return []; + } + const seen = new Set(); + const picked: string[] = []; + for (let i = texts.length - 1; i >= 0; i -= 1) { + const text = texts[i]; + const key = normalizeCaptureDedupeText(text); + if (!key || seen.has(key)) { + continue; + } + seen.add(key); + picked.push(text); + if (picked.length >= limit) { + break; + } + } + return picked.reverse(); +} + +function getCaptureDecision(text: string, mode: CaptureMode, captureMaxLength: number): { + shouldCapture: boolean; + reason: string; + normalizedText: string; +} { + const trimmed = text.trim(); + const normalizedText = sanitizeUserTextForCapture(trimmed); + const hadSanitization = normalizedText !== trimmed; + if (!normalizedText) { + return { + shouldCapture: false, + reason: //i.test(trimmed) ? "injected_memory_context_only" : "empty_text", + normalizedText: "", + }; + } + + const compactText = normalizedText.replace(/\s+/g, ""); + const minLength = resolveCaptureMinLength(compactText); + if (compactText.length < minLength || normalizedText.length > captureMaxLength) { + return { + shouldCapture: false, + reason: "length_out_of_range", + normalizedText, + }; + } + + if (COMMAND_TEXT_RE.test(normalizedText)) { + return { + shouldCapture: false, + reason: "command_text", + normalizedText, + }; + } + + if (NON_CONTENT_TEXT_RE.test(normalizedText)) { + return { + shouldCapture: false, + reason: "non_content_text", + normalizedText, + }; + } + if (SUBAGENT_CONTEXT_RE.test(normalizedText)) { + return { + shouldCapture: false, + reason: "subagent_context", + normalizedText, + }; + } + if (looksLikeQuestionOnlyText(normalizedText)) { + return { + shouldCapture: false, + reason: "question_text", + normalizedText, + }; + } + + if (mode === "keyword") { + for (const trigger of MEMORY_TRIGGERS) { + if (trigger.test(normalizedText)) { + return { + shouldCapture: true, + reason: hadSanitization + ? `matched_trigger_after_sanitize:${trigger.toString()}` + : `matched_trigger:${trigger.toString()}`, + normalizedText, + }; + } + } + return { + shouldCapture: false, + reason: hadSanitization ? "no_trigger_matched_after_sanitize" : "no_trigger_matched", + normalizedText, + }; + } + + return { + shouldCapture: true, + reason: hadSanitization ? "semantic_candidate_after_sanitize" : "semantic_candidate", + normalizedText, + }; +} + +function clampScore(value: number | undefined): number { + if (typeof value !== "number" || Number.isNaN(value)) { + return 0; + } + return Math.max(0, Math.min(1, value)); +} + +function isMemoryUri(uri: string): boolean { + return MEMORY_URI_PREFIXES.some((prefix) => uri.startsWith(prefix)); +} + +function normalizeDedupeText(text: string): string { + return text.toLowerCase().replace(/\s+/g, " ").trim(); +} + +function isEventOrCaseMemory(item: FindResultItem): boolean { + const category = (item.category ?? "").toLowerCase(); + const uri = item.uri.toLowerCase(); + return ( + category === "events" || + category === "cases" || + uri.includes("/events/") || + uri.includes("/cases/") + ); +} + +function getMemoryDedupeKey(item: FindResultItem): string { + const abstract = normalizeDedupeText(item.abstract ?? item.overview ?? ""); + const category = (item.category ?? "").toLowerCase() || "unknown"; + if (abstract && !isEventOrCaseMemory(item)) { + return `abstract:${category}:${abstract}`; + } + return `uri:${item.uri}`; +} + +function postProcessMemories( + items: FindResultItem[], + options: { + limit: number; + scoreThreshold: number; + leafOnly?: boolean; + }, +): FindResultItem[] { + const deduped: FindResultItem[] = []; + const seen = new Set(); + const sorted = [...items].sort((a, b) => clampScore(b.score) - clampScore(a.score)); + for (const item of sorted) { + if (options.leafOnly && item.is_leaf !== true) { + continue; + } + if (clampScore(item.score) < options.scoreThreshold) { + continue; + } + const key = getMemoryDedupeKey(item); + if (seen.has(key)) { + continue; + } + seen.add(key); + deduped.push(item); + if (deduped.length >= options.limit) { + break; + } + } + return deduped; +} + +function formatMemoryLines(items: FindResultItem[]): string { + return items + .map((item, index) => { + const score = clampScore(item.score); + const abstract = item.abstract?.trim() || item.overview?.trim() || item.uri; + const category = item.category ?? "memory"; + return `${index + 1}. [${category}] ${abstract} (${(score * 100).toFixed(0)}%)`; + }) + .join("\n"); +} + +function trimForLog(value: string, limit = 260): string { + const normalized = value.trim(); + if (normalized.length <= limit) { + return normalized; + } + return `${normalized.slice(0, limit)}...`; +} + +function toJsonLog(value: unknown, maxLen = 6000): string { + try { + const json = JSON.stringify(value); + if (json.length <= maxLen) { + return json; + } + return JSON.stringify({ + truncated: true, + length: json.length, + preview: `${json.slice(0, maxLen)}...`, + }); + } catch { + return JSON.stringify({ error: "stringify_failed" }); + } +} + +function summarizeInjectionMemories(items: FindResultItem[]): Array> { + return items.map((item) => ({ + uri: item.uri, + category: item.category ?? null, + abstract: trimForLog(item.abstract?.trim() || item.overview?.trim() || item.uri, 180), + score: clampScore(item.score), + is_leaf: item.is_leaf === true, + })); +} + +function summarizeExtractedMemories( + items: Array>, +): Array> { + return items.slice(0, 10).map((item) => { + const abstractRaw = + typeof item.abstract === "string" + ? item.abstract + : typeof item.overview === "string" + ? item.overview + : typeof item.title === "string" + ? item.title + : ""; + return { + uri: typeof item.uri === "string" ? item.uri : null, + category: typeof item.category === "string" ? item.category : null, + abstract: trimForLog(abstractRaw, 180), + is_leaf: item.is_leaf === true, + }; + }); +} + +function isPreferencesMemory(item: FindResultItem): boolean { + return ( + item.category === "preferences" || + item.uri.includes("/preferences/") || + item.uri.endsWith("/preferences") + ); +} + +function isEventMemory(item: FindResultItem): boolean { + const category = (item.category ?? "").toLowerCase(); + return category === "events" || item.uri.includes("/events/"); +} + +function isLeafLikeMemory(item: FindResultItem): boolean { + return item.is_leaf === true || item.uri.endsWith(".md"); +} + +const PREFERENCE_QUERY_RE = /prefer|preference|favorite|favourite|like|偏好|喜欢|爱好|更倾向/i; +const TEMPORAL_QUERY_RE = + /when|what time|date|day|month|year|yesterday|today|tomorrow|last|next|什么时候|何时|哪天|几月|几年|昨天|今天|明天|上周|下周|上个月|下个月|去年|明年/i; +const QUERY_TOKEN_RE = /[a-z0-9]{2,}/gi; +const QUERY_TOKEN_STOPWORDS = new Set([ + "what", + "when", + "where", + "which", + "who", + "whom", + "whose", + "why", + "how", + "did", + "does", + "is", + "are", + "was", + "were", + "the", + "and", + "for", + "with", + "from", + "that", + "this", + "your", + "you", +]); + +type RecallQueryProfile = { + tokens: string[]; + wantsPreference: boolean; + wantsTemporal: boolean; +}; + +function buildRecallQueryProfile(query: string): RecallQueryProfile { + const text = query.trim(); + const allTokens = text.toLowerCase().match(QUERY_TOKEN_RE) ?? []; + const tokens = allTokens.filter((token) => !QUERY_TOKEN_STOPWORDS.has(token)); + return { + tokens, + wantsPreference: PREFERENCE_QUERY_RE.test(text), + wantsTemporal: TEMPORAL_QUERY_RE.test(text), + }; +} + +function lexicalOverlapBoost(tokens: string[], text: string): number { + if (tokens.length === 0 || !text) { + return 0; + } + const haystack = ` ${text.toLowerCase()} `; + let matched = 0; + for (const token of tokens.slice(0, 8)) { + if (haystack.includes(` ${token} `) || haystack.includes(token)) { + matched += 1; + } + } + return Math.min(0.2, (matched / Math.min(tokens.length, 4)) * 0.2); +} + +function rankForInjection(item: FindResultItem, query: RecallQueryProfile): number { + // Keep ranking simple and stable: semantic score + light query-aware boosts. + const baseScore = clampScore(item.score); + const abstract = (item.abstract ?? item.overview ?? "").trim(); + const leafBoost = isLeafLikeMemory(item) ? 0.12 : 0; + const eventBoost = query.wantsTemporal && isEventMemory(item) ? 0.1 : 0; + const preferenceBoost = query.wantsPreference && isPreferencesMemory(item) ? 0.08 : 0; + const overlapBoost = lexicalOverlapBoost(query.tokens, `${item.uri} ${abstract}`); + return baseScore + leafBoost + eventBoost + preferenceBoost + overlapBoost; +} + +function pickMemoriesForInjection( + items: FindResultItem[], + limit: number, + queryText: string, +): FindResultItem[] { + if (items.length === 0 || limit <= 0) { + return []; + } + + const query = buildRecallQueryProfile(queryText); + const sorted = [...items].sort((a, b) => rankForInjection(b, query) - rankForInjection(a, query)); + const deduped: FindResultItem[] = []; + const seen = new Set(); + for (const item of sorted) { + const abstractKey = (item.abstract ?? item.overview ?? "").trim().toLowerCase(); + const key = abstractKey || item.uri; + if (seen.has(key)) { + continue; + } + seen.add(key); + deduped.push(item); + } + const leaves = deduped.filter((item) => isLeafLikeMemory(item)); + if (leaves.length >= limit) { + return leaves.slice(0, limit); + } + + const picked = [...leaves]; + const used = new Set(leaves.map((item) => item.uri)); + for (const item of deduped) { + if (picked.length >= limit) { + break; + } + if (used.has(item.uri)) { + continue; + } + picked.push(item); + } + return picked; +} + +class OpenVikingClient { + constructor( + private readonly baseUrl: string, + private readonly apiKey: string, + private readonly timeoutMs: number, + ) {} + + private async request(path: string, init: RequestInit = {}): Promise { + const controller = new AbortController(); + const timer = setTimeout(() => controller.abort(), this.timeoutMs); + try { + const headers = new Headers(init.headers ?? {}); + if (this.apiKey) { + headers.set("X-API-Key", this.apiKey); + } + if (init.body && !headers.has("Content-Type")) { + headers.set("Content-Type", "application/json"); + } + + const response = await fetch(`${this.baseUrl}${path}`, { + ...init, + headers, + signal: controller.signal, + }); + + const payload = (await response.json().catch(() => ({}))) as { + status?: string; + result?: T; + error?: { code?: string; message?: string }; + }; + + if (!response.ok || payload.status === "error") { + const code = payload.error?.code ? ` [${payload.error.code}]` : ""; + const message = payload.error?.message ?? `HTTP ${response.status}`; + throw new Error(`OpenViking request failed${code}: ${message}`); + } + + return (payload.result ?? payload) as T; + } finally { + clearTimeout(timer); + } + } + + async healthCheck(): Promise { + await this.request<{ status: string }>("/health"); + } + + async find( + query: string, + options: { + targetUri: string; + limit: number; + scoreThreshold?: number; + sessionId?: string; + }, + ): Promise { + const body = { + query, + target_uri: options.targetUri, + limit: options.limit, + score_threshold: options.scoreThreshold, + session_id: options.sessionId, + }; + return this.request("/api/v1/search/search", { + method: "POST", + body: JSON.stringify(body), + }); + } + + async createSession(): Promise { + const result = await this.request<{ session_id: string }>("/api/v1/sessions", { + method: "POST", + body: JSON.stringify({}), + }); + return result.session_id; + } + + async addSessionMessage(sessionId: string, role: string, content: string): Promise { + await this.request<{ session_id: string }>( + `/api/v1/sessions/${encodeURIComponent(sessionId)}/messages`, + { + method: "POST", + body: JSON.stringify({ role, content }), + }, + ); + } + + async extractSessionMemories(sessionId: string): Promise>> { + return this.request>>( + `/api/v1/sessions/${encodeURIComponent(sessionId)}/extract`, + { method: "POST", body: JSON.stringify({}) }, + ); + } + + async deleteSession(sessionId: string): Promise { + await this.request(`/api/v1/sessions/${encodeURIComponent(sessionId)}`, { method: "DELETE" }); + } + + async deleteUri(uri: string): Promise { + await this.request(`/api/v1/fs?uri=${encodeURIComponent(uri)}&recursive=false`, { + method: "DELETE", + }); + } +} + +function extractTextsFromUserMessages(messages: unknown[]): string[] { + const texts: string[] = []; + for (const msg of messages) { + if (!msg || typeof msg !== "object") { + continue; + } + const msgObj = msg as Record; + if (msgObj.role !== "user") { + continue; + } + const content = msgObj.content; + if (typeof content === "string") { + texts.push(content); + continue; + } + if (Array.isArray(content)) { + for (const block of content) { + if (!block || typeof block !== "object") { + continue; + } + const blockObj = block as Record; + if (blockObj.type === "text" && typeof blockObj.text === "string") { + texts.push(blockObj.text); + } + } + } + } + return texts; +} + +function extractLatestUserText(messages: unknown[] | undefined): string { + if (!messages || messages.length === 0) { + return ""; + } + const texts = extractTextsFromUserMessages(messages); + for (let i = texts.length - 1; i >= 0; i -= 1) { + const normalized = sanitizeUserTextForCapture(texts[i] ?? ""); + if (normalized) { + return normalized; + } + } + return ""; +} + +function waitForHealth(baseUrl: string, timeoutMs: number, intervalMs: number): Promise { + const deadline = Date.now() + timeoutMs; + return new Promise((resolve, reject) => { + const tick = () => { + if (Date.now() > deadline) { + reject(new Error(`OpenViking health check timeout at ${baseUrl}`)); + return; + } + fetch(`${baseUrl}/health`) + .then((r) => r.json()) + .then((body: { status?: string }) => { + if (body?.status === "ok") { + resolve(); + return; + } + setTimeout(tick, intervalMs); + }) + .catch(() => setTimeout(tick, intervalMs)); + }; + tick(); + }); +} + +const memoryPlugin = { + id: "memory-openviking", + name: "Memory (OpenViking)", + description: "OpenViking-backed long-term memory with auto-recall/capture", + kind: "memory" as const, + configSchema: memoryOpenVikingConfigSchema, + + register(api: OpenClawPluginApi) { + const cfg = memoryOpenVikingConfigSchema.parse(api.pluginConfig); + + let clientPromise: Promise; + let localProcess: ReturnType | null = null; + let resolveLocalClient: ((c: OpenVikingClient) => void) | null = null; + + if (cfg.mode === "local") { + clientPromise = new Promise((resolve) => { + resolveLocalClient = resolve; + }); + } else { + clientPromise = Promise.resolve(new OpenVikingClient(cfg.baseUrl, cfg.apiKey, cfg.timeoutMs)); + } + + const getClient = (): Promise => clientPromise; + + api.registerTool( + { + name: "memory_recall", + label: "Memory Recall (OpenViking)", + description: + "Search long-term memories from OpenViking. Use when you need past user preferences, facts, or decisions.", + parameters: Type.Object({ + query: Type.String({ description: "Search query" }), + limit: Type.Optional( + Type.Number({ description: "Max results (default: plugin config)" }), + ), + scoreThreshold: Type.Optional( + Type.Number({ description: "Minimum score (0-1, default: plugin config)" }), + ), + targetUri: Type.Optional( + Type.String({ description: "Search scope URI (default: plugin config)" }), + ), + }), + async execute(_toolCallId, params) { + const { query } = params as { query: string }; + const limit = + typeof (params as { limit?: number }).limit === "number" + ? Math.max(1, Math.floor((params as { limit: number }).limit)) + : cfg.recallLimit; + const scoreThreshold = + typeof (params as { scoreThreshold?: number }).scoreThreshold === "number" + ? Math.max(0, Math.min(1, (params as { scoreThreshold: number }).scoreThreshold)) + : cfg.recallScoreThreshold; + const targetUri = + typeof (params as { targetUri?: string }).targetUri === "string" + ? (params as { targetUri: string }).targetUri + : cfg.targetUri; + const requestLimit = Math.max(limit * 4, limit); + const result = await (await getClient()).find(query, { + targetUri, + limit: requestLimit, + scoreThreshold: 0, + }); + const memories = postProcessMemories(result.memories ?? [], { + limit, + scoreThreshold, + }); + if (memories.length === 0) { + return { + content: [{ type: "text", text: "No relevant OpenViking memories found." }], + details: { count: 0, total: result.total ?? 0, scoreThreshold }, + }; + } + return { + content: [ + { + type: "text", + text: `Found ${memories.length} memories:\n\n${formatMemoryLines(memories)}`, + }, + ], + details: { + count: memories.length, + memories, + total: result.total ?? memories.length, + scoreThreshold, + requestLimit, + }, + }; + }, + }, + { name: "memory_recall" }, + ); + + api.registerTool( + { + name: "memory_store", + label: "Memory Store (OpenViking)", + description: + "Store text in OpenViking memory pipeline by writing to a session and running memory extraction.", + parameters: Type.Object({ + text: Type.String({ description: "Information to store as memory source text" }), + role: Type.Optional(Type.String({ description: "Session role, default user" })), + sessionId: Type.Optional(Type.String({ description: "Existing OpenViking session ID" })), + }), + async execute(_toolCallId, params) { + const { text } = params as { text: string }; + const role = + typeof (params as { role?: string }).role === "string" + ? (params as { role: string }).role + : "user"; + const sessionIdIn = (params as { sessionId?: string }).sessionId; + + api.logger.info?.( + `memory-openviking: memory_store invoked (textLength=${text?.length ?? 0}, sessionId=${sessionIdIn ?? "temp"})`, + ); + + let sessionId = sessionIdIn; + let createdTempSession = false; + try { + const c = await getClient(); + if (!sessionId) { + sessionId = await c.createSession(); + createdTempSession = true; + } + await c.addSessionMessage(sessionId, role, text); + const extracted = await c.extractSessionMemories(sessionId); + if (extracted.length === 0) { + api.logger.warn( + `memory-openviking: memory_store completed but extract returned 0 memories (sessionId=${sessionId}). ` + + "Check OpenViking server logs for embedding/extract errors (e.g. 401 API key, or extraction pipeline).", + ); + } else { + api.logger.info?.(`memory-openviking: memory_store extracted ${extracted.length} memories`); + } + return { + content: [ + { + type: "text", + text: `Stored in OpenViking session ${sessionId} and extracted ${extracted.length} memories.`, + }, + ], + details: { action: "stored", sessionId, extractedCount: extracted.length, extracted }, + }; + } catch (err) { + api.logger.warn(`memory-openviking: memory_store failed: ${String(err)}`); + throw err; + } finally { + if (createdTempSession && sessionId) { + const c = await getClient().catch(() => null); + if (c) await c.deleteSession(sessionId!).catch(() => {}); + } + } + }, + }, + { name: "memory_store" }, + ); + + api.registerTool( + { + name: "memory_forget", + label: "Memory Forget (OpenViking)", + description: + "Forget memory by URI, or search then delete when a strong single match is found.", + parameters: Type.Object({ + uri: Type.Optional(Type.String({ description: "Exact memory URI to delete" })), + query: Type.Optional(Type.String({ description: "Search query to find memory URI" })), + targetUri: Type.Optional( + Type.String({ description: "Search scope URI (default: plugin config)" }), + ), + limit: Type.Optional(Type.Number({ description: "Search limit (default: 5)" })), + scoreThreshold: Type.Optional( + Type.Number({ description: "Minimum score (0-1, default: plugin config)" }), + ), + }), + async execute(_toolCallId, params) { + const uri = (params as { uri?: string }).uri; + if (uri) { + if (!isMemoryUri(uri)) { + return { + content: [{ type: "text", text: `Refusing to delete non-memory URI: ${uri}` }], + details: { action: "rejected", uri }, + }; + } + await (await getClient()).deleteUri(uri); + return { + content: [{ type: "text", text: `Forgotten: ${uri}` }], + details: { action: "deleted", uri }, + }; + } + + const query = (params as { query?: string }).query; + if (!query) { + return { + content: [{ type: "text", text: "Provide uri or query." }], + details: { error: "missing_param" }, + }; + } + + const limit = + typeof (params as { limit?: number }).limit === "number" + ? Math.max(1, Math.floor((params as { limit: number }).limit)) + : 5; + const scoreThreshold = + typeof (params as { scoreThreshold?: number }).scoreThreshold === "number" + ? Math.max(0, Math.min(1, (params as { scoreThreshold: number }).scoreThreshold)) + : cfg.recallScoreThreshold; + const targetUri = + typeof (params as { targetUri?: string }).targetUri === "string" + ? (params as { targetUri: string }).targetUri + : cfg.targetUri; + const requestLimit = Math.max(limit * 4, 20); + + const result = await (await getClient()).find(query, { + targetUri, + limit: requestLimit, + scoreThreshold: 0, + }); + const candidates = postProcessMemories(result.memories ?? [], { + limit: requestLimit, + scoreThreshold, + leafOnly: true, + }).filter((item) => isMemoryUri(item.uri)); + if (candidates.length === 0) { + return { + content: [ + { + type: "text", + text: "No matching leaf memory candidates found. Try a more specific query.", + }, + ], + details: { action: "none", scoreThreshold }, + }; + } + const top = candidates[0]; + if (candidates.length === 1 && clampScore(top.score) >= 0.85) { + await (await getClient()).deleteUri(top.uri); + return { + content: [{ type: "text", text: `Forgotten: ${top.uri}` }], + details: { action: "deleted", uri: top.uri, score: top.score ?? 0 }, + }; + } + + const list = candidates + .map((item) => `- ${item.uri} (${(clampScore(item.score) * 100).toFixed(0)}%)`) + .join("\n"); + + return { + content: [ + { + type: "text", + text: `Found ${candidates.length} candidates. Specify uri:\n${list}`, + }, + ], + details: { action: "candidates", candidates, scoreThreshold, requestLimit }, + }; + }, + }, + { name: "memory_forget" }, + ); + + if (cfg.autoRecall || cfg.ingestReplyAssist) { + api.on("before_agent_start", async (event) => { + const queryText = extractLatestUserText(event.messages) || event.prompt.trim(); + if (!queryText) { + return; + } + const prependContextParts: string[] = []; + + if (cfg.autoRecall && queryText.length >= 5) { + try { + const candidateLimit = Math.max(cfg.recallLimit * 4, cfg.recallLimit); + const result = await (await getClient()).find(queryText, { + targetUri: cfg.targetUri, + limit: candidateLimit, + scoreThreshold: 0, + }); + const processed = postProcessMemories(result.memories ?? [], { + limit: candidateLimit, + scoreThreshold: cfg.recallScoreThreshold, + }); + const memories = pickMemoriesForInjection(processed, cfg.recallLimit, queryText); + if (memories.length > 0) { + const memoryContext = memories + .map((item) => `- [${item.category ?? "memory"}] ${item.abstract ?? item.uri}`) + .join("\n"); + api.logger.info?.( + `memory-openviking: injecting ${memories.length} memories into context`, + ); + api.logger.info?.( + `memory-openviking: inject-detail ${toJsonLog({ count: memories.length, memories: summarizeInjectionMemories(memories) })}`, + ); + prependContextParts.push( + "\nThe following OpenViking memories may be relevant:\n" + + `${memoryContext}\n` + + "", + ); + } + } catch (err) { + api.logger.warn(`memory-openviking: auto-recall failed: ${String(err)}`); + } + } + + if (cfg.ingestReplyAssist) { + const decision = isTranscriptLikeIngest(queryText, { + minSpeakerTurns: cfg.ingestReplyAssistMinSpeakerTurns, + minChars: cfg.ingestReplyAssistMinChars, + }); + if (decision.shouldAssist) { + api.logger.info?.( + `memory-openviking: ingest-reply-assist applied (reason=${decision.reason}, speakerTurns=${decision.speakerTurns}, chars=${decision.chars})`, + ); + prependContextParts.push( + "\n" + + "The latest user input looks like a multi-speaker transcript used for memory ingestion.\n" + + "Reply with 1-2 concise sentences to acknowledge or summarize key points.\n" + + "Do not output NO_REPLY or an empty reply.\n" + + "Do not fabricate facts beyond the provided transcript and recalled memories.\n" + + "", + ); + } + } + + if (prependContextParts.length > 0) { + return { + prependContext: prependContextParts.join("\n\n"), + }; + } + }); + } + + if (cfg.autoCapture) { + api.on("agent_end", async (event) => { + if (!event.success || !event.messages || event.messages.length === 0) { + api.logger.info( + `memory-openviking: auto-capture skipped (success=${String(event.success)}, messages=${event.messages?.length ?? 0})`, + ); + return; + } + try { + const texts = extractTextsFromUserMessages(event.messages); + api.logger.info( + `memory-openviking: auto-capture evaluating ${texts.length} text candidates`, + ); + const decisions = texts + .map((text) => { + const decision = getCaptureDecision(text, cfg.captureMode, cfg.captureMaxLength); + return { + captureText: decision.normalizedText, + decision, + }; + }) + .filter((item) => item.captureText); + for (const item of decisions.slice(0, 5)) { + const preview = + item.captureText.length > 80 + ? `${item.captureText.slice(0, 80)}...` + : item.captureText; + api.logger.info( + `memory-openviking: capture-check shouldCapture=${String(item.decision.shouldCapture)} reason=${item.decision.reason} text="${preview}"`, + ); + } + const toCapture = decisions + .filter((item) => item.decision.shouldCapture) + .map((item) => item.captureText); + const selected = pickRecentUniqueTexts(toCapture, CAPTURE_LIMIT); + if (selected.length === 0) { + api.logger.info("memory-openviking: auto-capture skipped (no matched texts)"); + return; + } + const c = await getClient(); + const sessionId = await c.createSession(); + try { + for (const text of selected) { + await c.addSessionMessage(sessionId, "user", text); + } + const extracted = await c.extractSessionMemories(sessionId); + api.logger.info( + `memory-openviking: auto-captured ${selected.length} messages, extracted ${extracted.length} memories`, + ); + api.logger.info( + `memory-openviking: capture-detail ${toJsonLog({ + capturedCount: selected.length, + captured: selected.map((text) => trimForLog(text, 260)), + extractedCount: extracted.length, + extracted: summarizeExtractedMemories(extracted), + })}`, + ); + if (extracted.length === 0) { + api.logger.warn( + "memory-openviking: auto-capture completed but extract returned 0 memories. Check OpenViking server logs for embedding/extract errors.", + ); + } + } finally { + await c.deleteSession(sessionId).catch(() => {}); + } + } catch (err) { + api.logger.warn(`memory-openviking: auto-capture failed: ${String(err)}`); + } + }); + } + + api.registerService({ + id: "memory-openviking", + start: async () => { + if (cfg.mode === "local" && resolveLocalClient) { + const baseUrl = cfg.baseUrl; + // Local mode: startup (embedder load, AGFS) can take 1–2 min; use longer health timeout + const timeoutMs = Math.max(cfg.timeoutMs, 120_000); + const intervalMs = 500; + const defaultPy = IS_WIN ? "python" : "python3"; + let pythonCmd = process.env.OPENVIKING_PYTHON; + if (!pythonCmd) { + if (IS_WIN) { + const envBat = join(homedir(), ".openclaw", "openviking.env.bat"); + if (existsSync(envBat)) { + try { + const content = readFileSync(envBat, "utf-8"); + const m = content.match(/set\s+OPENVIKING_PYTHON=(.+)/i); + if (m?.[1]) pythonCmd = m[1].trim(); + } catch { /* ignore */ } + } + } else { + const envFile = join(homedir(), ".openclaw", "openviking.env"); + if (existsSync(envFile)) { + try { + const content = readFileSync(envFile, "utf-8"); + const m = content.match(/OPENVIKING_PYTHON=['"]([^'"]+)['"]/); + if (m?.[1]) pythonCmd = m[1]; + } catch { + /* ignore */ + } + } + } + } + if (!pythonCmd) { + if (IS_WIN) { + try { + pythonCmd = execSync("where python", { encoding: "utf-8", shell: true }).split(/\r?\n/)[0].trim(); + } catch { + pythonCmd = "python"; + } + } else { + try { + pythonCmd = execSync("command -v python3 || which python3", { + encoding: "utf-8", + env: process.env, + shell: "/bin/sh", + }).trim(); + } catch { + pythonCmd = "python3"; + } + } + } + if (pythonCmd === defaultPy) { + api.logger.warn?.( + `memory-openviking: 未解析到 ${defaultPy} 路径,将用 "${defaultPy}"。若 openviking 在自定义 Python 下,请设置 OPENVIKING_PYTHON` + + (IS_WIN ? ' 或 call "%USERPROFILE%\\.openclaw\\openviking.env.bat"' : " 或 source ~/.openclaw/openviking.env"), + ); + } + // Kill stale OpenViking processes occupying the target port + if (IS_WIN) { + try { + const netstatOut = execSync(`netstat -ano | findstr "LISTENING" | findstr ":${cfg.port}"`, { + encoding: "utf-8", shell: true, + }).trim(); + if (netstatOut) { + const pids = new Set(); + for (const line of netstatOut.split(/\r?\n/)) { + const m = line.trim().match(/\s(\d+)\s*$/); + if (m) pids.add(Number(m[1])); + } + for (const pid of pids) { + if (pid > 0) { + api.logger.info?.(`memory-openviking: killing stale process on port ${cfg.port} (pid ${pid})`); + try { execSync(`taskkill /PID ${pid} /F`, { shell: true }); } catch { /* already gone */ } + } + } + await new Promise((r) => setTimeout(r, 500)); + } + } catch { /* netstat not available or no stale process */ } + } else { + try { + const lsofOut = execSync(`lsof -ti tcp:${cfg.port} -s tcp:listen 2>/dev/null || true`, { + encoding: "utf-8", + shell: "/bin/sh", + }).trim(); + if (lsofOut) { + for (const pidStr of lsofOut.split(/\s+/)) { + const pid = Number(pidStr); + if (pid > 0) { + api.logger.info?.(`memory-openviking: killing stale process on port ${cfg.port} (pid ${pid})`); + try { process.kill(pid, "SIGKILL"); } catch { /* already gone */ } + } + } + await new Promise((r) => setTimeout(r, 500)); + } + } catch { /* lsof not available or no stale process */ } + } + + // Inherit system environment; optionally override Go/Python paths via env vars + const pathSep = IS_WIN ? ";" : ":"; + const env = { + ...process.env, + OPENVIKING_CONFIG_FILE: cfg.configPath, + ...(process.env.OPENVIKING_GO_PATH && { PATH: `${process.env.OPENVIKING_GO_PATH}${pathSep}${process.env.PATH || ""}` }), + ...(process.env.OPENVIKING_GOPATH && { GOPATH: process.env.OPENVIKING_GOPATH }), + ...(process.env.OPENVIKING_GOPROXY && { GOPROXY: process.env.OPENVIKING_GOPROXY }), + }; + const child = spawn( + pythonCmd, + [ + "-m", + "openviking.server.bootstrap", + "--config", + cfg.configPath, + "--host", + "127.0.0.1", + "--port", + String(cfg.port), + ], + { env, cwd: IS_WIN ? tmpdir() : "/tmp", stdio: ["ignore", "pipe", "pipe"] }, + ); + localProcess = child; + child.on("error", (err) => api.logger.warn(`memory-openviking: local server error: ${String(err)}`)); + child.stderr?.on("data", (chunk) => api.logger.debug?.(`[openviking] ${String(chunk).trim()}`)); + try { + await waitForHealth(baseUrl, timeoutMs, intervalMs); + const client = new OpenVikingClient(baseUrl, cfg.apiKey, cfg.timeoutMs); + resolveLocalClient(client); + api.logger.info( + `memory-openviking: local server started (${baseUrl}, config: ${cfg.configPath})`, + ); + } catch (err) { + localProcess = null; + child.kill("SIGTERM"); + throw err; + } + } else { + await (await getClient()).healthCheck().catch(() => {}); + api.logger.info( + `memory-openviking: initialized (url: ${cfg.baseUrl}, targetUri: ${cfg.targetUri}, search: hybrid endpoint)`, + ); + } + }, + stop: () => { + if (localProcess) { + localProcess.kill("SIGTERM"); + localProcess = null; + api.logger.info("memory-openviking: local server stopped"); + } else { + api.logger.info("memory-openviking: stopped"); + } + }, + }); + }, +}; + +export default memoryPlugin; diff --git a/examples/openclaw-memory-plugin/openclaw.plugin.json b/examples/openclaw-memory-plugin/openclaw.plugin.json new file mode 100644 index 00000000..bbb8b0f3 --- /dev/null +++ b/examples/openclaw-memory-plugin/openclaw.plugin.json @@ -0,0 +1,143 @@ +{ + "id": "memory-openviking", + "kind": "memory", + "uiHints": { + "mode": { + "label": "Mode", + "help": "local = plugin starts OpenViking (like Claude Code); remote = use existing HTTP server" + }, + "configPath": { + "label": "Config path (local)", + "placeholder": "~/.openviking/ov.conf", + "help": "Path to ov.conf when mode is local" + }, + "port": { + "label": "Port (local)", + "placeholder": "1933", + "help": "Port for local OpenViking server", + "advanced": true + }, + "baseUrl": { + "label": "OpenViking Base URL (remote)", + "placeholder": "http://127.0.0.1:1933", + "help": "HTTP URL when mode is remote (or ${OPENVIKING_BASE_URL})" + }, + "apiKey": { + "label": "OpenViking API Key", + "sensitive": true, + "placeholder": "${OPENVIKING_API_KEY}", + "help": "Optional API key for OpenViking server" + }, + "targetUri": { + "label": "Search Target URI", + "placeholder": "viking://", + "help": "Default OpenViking target URI for memory search" + }, + "timeoutMs": { + "label": "Request Timeout (ms)", + "placeholder": "15000", + "advanced": true + }, + "autoCapture": { + "label": "Auto-Capture", + "help": "Extract memories from recent conversation messages via OpenViking sessions" + }, + "captureMode": { + "label": "Capture Mode", + "placeholder": "semantic", + "advanced": true, + "help": "semantic captures all eligible user text; keyword uses trigger regex first" + }, + "captureMaxLength": { + "label": "Capture Max Length", + "placeholder": "24000", + "advanced": true, + "help": "Maximum sanitized user text length allowed for auto-capture" + }, + "autoRecall": { + "label": "Auto-Recall", + "help": "Inject relevant OpenViking memories into agent context" + }, + "recallLimit": { + "label": "Recall Limit", + "placeholder": "6", + "advanced": true + }, + "recallScoreThreshold": { + "label": "Recall Score Threshold", + "placeholder": "0.01", + "advanced": true + }, + "ingestReplyAssist": { + "label": "Ingest Reply Assist", + "help": "When transcript-like memory ingestion is detected, add a lightweight reply instruction to reduce NO_REPLY.", + "advanced": true + }, + "ingestReplyAssistMinSpeakerTurns": { + "label": "Ingest Min Speaker Turns", + "placeholder": "2", + "help": "Minimum speaker-tag turns (e.g. Name:) to detect transcript-like ingest text.", + "advanced": true + }, + "ingestReplyAssistMinChars": { + "label": "Ingest Min Chars", + "placeholder": "120", + "help": "Minimum sanitized text length required before ingest reply assist can trigger.", + "advanced": true + } + }, + "configSchema": { + "type": "object", + "additionalProperties": false, + "properties": { + "mode": { + "type": "string" + }, + "configPath": { + "type": "string" + }, + "port": { + "type": "number" + }, + "baseUrl": { + "type": "string" + }, + "apiKey": { + "type": "string" + }, + "targetUri": { + "type": "string" + }, + "timeoutMs": { + "type": "number" + }, + "autoCapture": { + "type": "boolean" + }, + "captureMode": { + "type": "string" + }, + "captureMaxLength": { + "type": "number" + }, + "autoRecall": { + "type": "boolean" + }, + "recallLimit": { + "type": "number" + }, + "recallScoreThreshold": { + "type": "number" + }, + "ingestReplyAssist": { + "type": "boolean" + }, + "ingestReplyAssistMinSpeakerTurns": { + "type": "number" + }, + "ingestReplyAssistMinChars": { + "type": "number" + } + } + } +} diff --git a/examples/openclaw-memory-plugin/package-lock.json b/examples/openclaw-memory-plugin/package-lock.json new file mode 100644 index 00000000..71a8e81b --- /dev/null +++ b/examples/openclaw-memory-plugin/package-lock.json @@ -0,0 +1,21 @@ +{ + "name": "@openclaw/memory-openviking", + "version": "2026.2.6-3", + "lockfileVersion": 3, + "requires": true, + "packages": { + "": { + "name": "@openclaw/memory-openviking", + "version": "2026.2.6-3", + "dependencies": { + "@sinclair/typebox": "0.34.48" + } + }, + "node_modules/@sinclair/typebox": { + "version": "0.34.48", + "resolved": "https://registry.npmjs.org/@sinclair/typebox/-/typebox-0.34.48.tgz", + "integrity": "sha512-kKJTNuK3AQOrgjjotVxMrCn1sUJwM76wMszfq1kdU4uYVJjvEWuFQ6HgvLt4Xz3fSmZlTOxJ/Ie13KnIcWQXFA==", + "license": "MIT" + } + } +} diff --git a/examples/openclaw-memory-plugin/package.json b/examples/openclaw-memory-plugin/package.json new file mode 100644 index 00000000..5900ecaa --- /dev/null +++ b/examples/openclaw-memory-plugin/package.json @@ -0,0 +1,12 @@ +{ + "name": "@openclaw/memory-openviking", + "version": "2026.2.6-3", + "description": "OpenClaw OpenViking-backed long-term memory plugin (install to ~/.openclaw/extensions)", + "type": "module", + "dependencies": { + "@sinclair/typebox": "0.34.48" + }, + "openclaw": { + "extensions": ["./index.ts"] + } +} diff --git a/examples/openclaw-memory-plugin/setup-helper/cli.js b/examples/openclaw-memory-plugin/setup-helper/cli.js new file mode 100755 index 00000000..10e166e0 --- /dev/null +++ b/examples/openclaw-memory-plugin/setup-helper/cli.js @@ -0,0 +1,833 @@ +#!/usr/bin/env node +/** + * OpenClaw + OpenViking setup helper + * Usage: npx openclaw-openviking-setup-helper + * Or: npx openclaw-openviking-setup-helper --help + */ + +import { spawn } from "node:child_process"; +import { mkdir, writeFile, access, readFile, rm } from "node:fs/promises"; +import { createInterface } from "node:readline"; +import { join, dirname } from "node:path"; +import { fileURLToPath } from "node:url"; +import { existsSync } from "node:fs"; + +const __dirname = dirname(fileURLToPath(import.meta.url)); +const GITHUB_RAW = + process.env.OPENVIKING_GITHUB_RAW || + "https://raw.githubusercontent.com/OpenViking/OpenViking/main"; + +const IS_WIN = process.platform === "win32"; +const IS_LINUX = process.platform === "linux"; +const HOME = process.env.HOME || process.env.USERPROFILE || ""; +const OPENCLAW_DIR = join(HOME, ".openclaw"); +const OPENVIKING_DIR = join(HOME, ".openviking"); +const EXT_DIR = join(OPENCLAW_DIR, "extensions"); +const PLUGIN_DEST = join(EXT_DIR, "memory-openviking"); + +// ─── Utility helpers ─── + +function log(msg, level = "info") { + const icons = { info: "\u2139", ok: "\u2713", err: "\u2717", warn: "\u26A0" }; + console.log(`${icons[level] || ""} ${msg}`); +} + +function run(cmd, args, opts = {}) { + return new Promise((resolve, reject) => { + const p = spawn(cmd, args, { + stdio: opts.silent ? "pipe" : "inherit", + shell: opts.shell ?? true, + ...opts, + }); + p.on("close", (code) => (code === 0 ? resolve() : reject(new Error(`exit ${code}`)))); + }); +} + +function runCapture(cmd, args, opts = {}) { + return new Promise((resolve) => { + const p = spawn(cmd, args, { + stdio: ["ignore", "pipe", "pipe"], + shell: opts.shell ?? false, + ...opts, + }); + let out = ""; + let err = ""; + p.stdout?.on("data", (d) => (out += d)); + p.stderr?.on("data", (d) => (err += d)); + p.on("error", (e) => { + if (e.code === "ENOENT") resolve({ code: -1, out: "", err: `command not found: ${cmd}` }); + else resolve({ code: -1, out: "", err: String(e) }); + }); + p.on("close", (code) => resolve({ code, out: out.trim(), err: err.trim() })); + }); +} + +function runCaptureWithTimeout(cmd, args, timeoutMs, opts = {}) { + return new Promise((resolve) => { + const p = spawn(cmd, args, { + stdio: ["ignore", "pipe", "pipe"], + shell: opts.shell ?? false, + ...opts, + }); + let out = ""; + let err = ""; + let settled = false; + const done = (result) => { if (!settled) { settled = true; resolve(result); } }; + const timer = setTimeout(() => { p.kill(); done({ code: out ? 0 : -1, out: out.trim(), err: err.trim() }); }, timeoutMs); + p.stdout?.on("data", (d) => (out += d)); + p.stderr?.on("data", (d) => (err += d)); + p.on("error", (e) => { clearTimeout(timer); done({ code: -1, out: "", err: String(e) }); }); + p.on("close", (code) => { clearTimeout(timer); done({ code, out: out.trim(), err: err.trim() }); }); + }); +} + +async function question(prompt, defaultValue = "") { + const rl = createInterface({ input: process.stdin, output: process.stdout }); + const def = defaultValue ? ` [${defaultValue}]` : ""; + return new Promise((resolve) => { + rl.question(`${prompt}${def}: `, (answer) => { + rl.close(); + resolve((answer ?? defaultValue).trim() || defaultValue); + }); + }); +} + +async function questionApiKey(prompt) { + const rl = createInterface({ input: process.stdin, output: process.stdout }); + return new Promise((resolve) => { + rl.question(prompt, (answer) => { + rl.close(); + resolve((answer ?? "").trim()); + }); + }); +} + +// ─── Distro detection ─── + +async function detectDistro() { + if (IS_WIN) return "windows"; + try { + const { out } = await runCapture("sh", ["-c", "cat /etc/os-release 2>/dev/null"]); + const lower = out.toLowerCase(); + if (lower.includes("ubuntu") || lower.includes("debian")) return "debian"; + if (lower.includes("centos") || lower.includes("rhel") || lower.includes("openeuler") || lower.includes("fedora") || lower.includes("rocky") || lower.includes("alma")) return "rhel"; + } catch {} + const { code: aptCode } = await runCapture("sh", ["-c", "command -v apt"]); + if (aptCode === 0) return "debian"; + const { code: dnfCode } = await runCapture("sh", ["-c", "command -v dnf || command -v yum"]); + if (dnfCode === 0) return "rhel"; + return "unknown"; +} + +// ─── Environment checks ─── + +const DEFAULT_PYTHON = IS_WIN ? "python" : "python3"; + +async function checkOpenclaw() { + if (IS_WIN) { + const { code } = await runCaptureWithTimeout("openclaw", ["--version"], 10000, { shell: true }); + return code === 0 ? { ok: true } : { ok: false }; + } + const { code } = await runCapture("openclaw", ["--version"]); + return code === 0 ? { ok: true } : { ok: false }; +} + +async function checkPython() { + const py = process.env.OPENVIKING_PYTHON || DEFAULT_PYTHON; + const { code, out } = await runCapture(py, ["-c", "import sys; print(f'{sys.version_info.major}.{sys.version_info.minor}')"]); + if (code !== 0) return { ok: false, version: null, cmd: py, msg: `Python not found (tried: ${py})` }; + const [major, minor] = out.split(".").map(Number); + if (major < 3 || (major === 3 && minor < 10)) + return { ok: false, version: out, cmd: py, msg: `Python ${out} too old, need >= 3.10` }; + return { ok: true, version: out, cmd: py, msg: `${out} (${py})` }; +} + +async function checkGo() { + const goDir = process.env.OPENVIKING_GO_PATH?.replace(/^~/, HOME); + const goCmd = goDir ? join(goDir, "go") : "go"; + const { code, out } = await runCapture(goCmd, ["version"]); + if (code !== 0) return { ok: false, version: null, msg: "Go not found" }; + const m = out.match(/go([0-9]+)\.([0-9]+)/); + if (!m) return { ok: false, version: null, msg: "Cannot parse Go version" }; + const [, major, minor] = m.map(Number); + if (major < 1 || (major === 1 && minor < 25)) + return { ok: false, version: `${major}.${minor}`, msg: `Go ${major}.${minor} too old, need >= 1.25` }; + return { ok: true, version: `${major}.${minor}`, msg: `${major}.${minor}` }; +} + +async function checkCmake() { + const { code } = await runCapture("cmake", ["--version"]); + return { ok: code === 0 }; +} + +async function checkGpp() { + const { code } = await runCapture("g++", ["--version"]); + return { ok: code === 0 }; +} + +async function checkOpenvikingModule() { + const py = process.env.OPENVIKING_PYTHON || DEFAULT_PYTHON; + const { code } = await runCapture(py, ["-c", "import openviking"]); + return code === 0 ? { ok: true } : { ok: false }; +} + +async function checkOvvConf() { + const cfg = process.env.OPENVIKING_CONFIG_FILE || join(OPENVIKING_DIR, "ov.conf"); + try { + await access(cfg); + return { ok: true, path: cfg }; + } catch { + return { ok: false, path: cfg }; + } +} + +// ─── Config helpers ─── + +const DEFAULT_SERVER_PORT = 1933; +const DEFAULT_AGFS_PORT = 1833; +const DEFAULT_VLM_MODEL = "doubao-seed-1-8-251228"; +const DEFAULT_EMBEDDING_MODEL = "doubao-embedding-vision-250615"; + +const DEFAULT_WORKSPACE = join(HOME, ".openviking", "data"); + +function buildOvvConfJson(opts = {}) { + const { + apiKey = "", + serverPort = DEFAULT_SERVER_PORT, + agfsPort = DEFAULT_AGFS_PORT, + vlmModel = DEFAULT_VLM_MODEL, + embeddingModel = DEFAULT_EMBEDDING_MODEL, + workspace = DEFAULT_WORKSPACE, + } = opts; + return JSON.stringify({ + server: { + host: "127.0.0.1", + port: serverPort, + root_api_key: null, + cors_origins: ["*"], + }, + storage: { + workspace, + vectordb: { name: "context", backend: "local", project: "default" }, + agfs: { port: agfsPort, log_level: "warn", backend: "local", timeout: 10, retry_times: 3 }, + }, + embedding: { + dense: { + backend: "volcengine", + api_key: apiKey || null, + model: embeddingModel, + api_base: "https://ark.cn-beijing.volces.com/api/v3", + dimension: 1024, + input: "multimodal", + }, + }, + vlm: { + backend: "volcengine", + api_key: apiKey || null, + model: vlmModel, + api_base: "https://ark.cn-beijing.volces.com/api/v3", + temperature: 0.1, + max_retries: 3, + }, + }, null, 2); +} + +function parsePort(val, defaultVal) { + const n = parseInt(val, 10); + return Number.isFinite(n) && n >= 1 && n <= 65535 ? n : defaultVal; +} + +async function ensureOvvConf(cfgPath, opts = {}) { + await mkdir(dirname(cfgPath), { recursive: true }); + await writeFile(cfgPath, buildOvvConfJson(opts)); + log(`Created config: ${cfgPath}`, "ok"); + if (!opts.apiKey) { + log("API Key not set; memory features may be unavailable. Edit ov.conf to add later.", "warn"); + } +} + +async function getApiKeyFromOvvConf(cfgPath) { + let raw; + try { + raw = await readFile(cfgPath, "utf-8"); + const cfg = JSON.parse(raw); + return cfg?.embedding?.dense?.api_key || ""; + } catch { + const m = raw?.match(/api_key\s*:\s*["']?([^"'\s#]+)["']?/); + return m ? m[1].trim() : ""; + } +} + +async function getOvvConfPorts(cfgPath) { + try { + const raw = await readFile(cfgPath, "utf-8"); + const cfg = JSON.parse(raw); + return { + serverPort: cfg?.server?.port ?? DEFAULT_SERVER_PORT, + agfsPort: cfg?.storage?.agfs?.port ?? DEFAULT_AGFS_PORT, + }; + } catch { + return { serverPort: DEFAULT_SERVER_PORT, agfsPort: DEFAULT_AGFS_PORT }; + } +} + +async function isOvvConfInvalid(cfgPath) { + try { + JSON.parse(await readFile(cfgPath, "utf-8")); + return false; + } catch { + return true; + } +} + +async function updateOvvConf(cfgPath, opts = {}) { + let cfg; + try { + cfg = JSON.parse(await readFile(cfgPath, "utf-8")); + } catch { + log("ov.conf is not valid JSON, will create new config", "warn"); + await ensureOvvConf(cfgPath, opts); + return; + } + if (opts.apiKey !== undefined) { + if (!cfg.embedding) cfg.embedding = {}; + if (!cfg.embedding.dense) cfg.embedding.dense = {}; + cfg.embedding.dense.api_key = opts.apiKey || null; + if (!cfg.vlm) cfg.vlm = {}; + cfg.vlm.api_key = opts.apiKey || null; + } + if (opts.vlmModel !== undefined) { + if (!cfg.vlm) cfg.vlm = {}; + cfg.vlm.model = opts.vlmModel; + if (!cfg.vlm.api_base) cfg.vlm.api_base = "https://ark.cn-beijing.volces.com/api/v3"; + if (!cfg.vlm.backend) cfg.vlm.backend = "volcengine"; + } + if (opts.embeddingModel !== undefined) { + if (!cfg.embedding) cfg.embedding = {}; + if (!cfg.embedding.dense) cfg.embedding.dense = {}; + cfg.embedding.dense.model = opts.embeddingModel; + } + if (opts.serverPort !== undefined && cfg.server) cfg.server.port = opts.serverPort; + if (opts.agfsPort !== undefined && cfg.storage?.agfs) cfg.storage.agfs.port = opts.agfsPort; + await writeFile(cfgPath, JSON.stringify(cfg, null, 2)); +} + +// ─── Interactive config collection ─── + +async function collectOvvConfInteractive(nonInteractive) { + const opts = { + apiKey: process.env.OPENVIKING_ARK_API_KEY || "", + serverPort: DEFAULT_SERVER_PORT, + agfsPort: DEFAULT_AGFS_PORT, + vlmModel: DEFAULT_VLM_MODEL, + embeddingModel: DEFAULT_EMBEDDING_MODEL, + workspace: DEFAULT_WORKSPACE, + }; + if (nonInteractive) return opts; + + console.log("\n╔══════════════════════════════════════════════════════════╗"); + console.log("║ OpenViking Configuration (ov.conf) ║"); + console.log("╚══════════════════════════════════════════════════════════╝"); + + console.log("\n--- Data Storage ---"); + console.log("Workspace is where OpenViking stores all data (vector database, files, etc.)."); + opts.workspace = await question(`Workspace path`, DEFAULT_WORKSPACE); + + console.log("\nOpenViking requires a Volcengine Ark API Key for:"); + console.log(" - Embedding model: vectorizes text for semantic search"); + console.log(" - VLM model: analyzes conversations to extract memories"); + console.log("\nGet your API Key at: https://console.volcengine.com/ark\n"); + + opts.apiKey = (await questionApiKey("Volcengine Ark API Key (leave blank to skip, configure later): ")) || opts.apiKey; + + console.log("\n--- Model Configuration ---"); + console.log("VLM model is used to extract and analyze memories from conversations."); + opts.vlmModel = await question(`VLM model name`, DEFAULT_VLM_MODEL); + + console.log("\nEmbedding model is used to vectorize text for semantic search."); + opts.embeddingModel = await question(`Embedding model name`, DEFAULT_EMBEDDING_MODEL); + + console.log("\n--- Server Ports ---"); + const serverPortStr = await question(`OpenViking HTTP port`, String(DEFAULT_SERVER_PORT)); + opts.serverPort = parsePort(serverPortStr, DEFAULT_SERVER_PORT); + const agfsPortStr = await question(`AGFS port`, String(DEFAULT_AGFS_PORT)); + opts.agfsPort = parsePort(agfsPortStr, DEFAULT_AGFS_PORT); + + return opts; +} + +// ─── Installation helpers ─── + +async function installOpenviking(repoRoot) { + const py = process.env.OPENVIKING_PYTHON || DEFAULT_PYTHON; + log(`Installing openviking (using ${py})...`); + if (repoRoot && existsSync(join(repoRoot, "pyproject.toml"))) { + await run(py, ["-m", "pip", "install", "-e", repoRoot]); + return; + } + await run(py, ["-m", "pip", "install", "openviking"]); +} + +async function fetchPluginFromGitHub(dest) { + log("Downloading memory-openviking plugin from GitHub..."); + const files = [ + "examples/openclaw-memory-plugin/index.ts", + "examples/openclaw-memory-plugin/config.ts", + "examples/openclaw-memory-plugin/openclaw.plugin.json", + "examples/openclaw-memory-plugin/package.json", + "examples/openclaw-memory-plugin/package-lock.json", + "examples/openclaw-memory-plugin/.gitignore", + ]; + await mkdir(dest, { recursive: true }); + for (let i = 0; i < files.length; i++) { + const rel = files[i]; + const name = rel.split("/").pop(); + process.stdout.write(` Downloading ${i + 1}/${files.length}: ${name} ... `); + const url = `${GITHUB_RAW}/${rel}`; + const res = await fetch(url); + if (!res.ok) throw new Error(`Download failed: ${url}`); + const buf = await res.arrayBuffer(); + await writeFile(join(dest, name), Buffer.from(buf)); + process.stdout.write("\u2713\n"); + } + log(`Plugin downloaded to ${dest}`, "ok"); + process.stdout.write(" Installing plugin deps (npm install)... "); + await run("npm", ["install", "--no-audit", "--no-fund"], { cwd: dest, silent: true }); + process.stdout.write("\u2713\n"); + log("Plugin deps installed", "ok"); +} + +async function fixStalePluginPaths(pluginPath) { + const cfgPath = join(OPENCLAW_DIR, "openclaw.json"); + if (!existsSync(cfgPath)) return; + try { + const cfg = JSON.parse(await readFile(cfgPath, "utf8")); + let changed = false; + const paths = cfg?.plugins?.load?.paths; + if (Array.isArray(paths)) { + const cleaned = paths.filter((p) => existsSync(p)); + if (!cleaned.includes(pluginPath)) cleaned.push(pluginPath); + if (JSON.stringify(cleaned) !== JSON.stringify(paths)) { + cfg.plugins.load.paths = cleaned; + changed = true; + } + } + const installs = cfg?.plugins?.installs; + if (installs) { + for (const [k, v] of Object.entries(installs)) { + if (v?.installPath && !existsSync(v.installPath)) { + delete installs[k]; + changed = true; + } + } + } + if (changed) { + await writeFile(cfgPath, JSON.stringify(cfg, null, 2) + "\n"); + log("Cleaned stale plugin paths from openclaw.json", "ok"); + } + } catch {} +} + +async function configureOpenclawViaJson(pluginPath, serverPort) { + const cfgPath = join(OPENCLAW_DIR, "openclaw.json"); + let cfg = {}; + try { cfg = JSON.parse(await readFile(cfgPath, "utf8")); } catch { /* start fresh */ } + if (!cfg.plugins) cfg.plugins = {}; + cfg.plugins.enabled = true; + cfg.plugins.allow = ["memory-openviking"]; + if (!cfg.plugins.slots) cfg.plugins.slots = {}; + cfg.plugins.slots.memory = "memory-openviking"; + if (!cfg.plugins.load) cfg.plugins.load = {}; + const paths = Array.isArray(cfg.plugins.load.paths) ? cfg.plugins.load.paths : []; + if (!paths.includes(pluginPath)) paths.push(pluginPath); + cfg.plugins.load.paths = paths; + if (!cfg.plugins.entries) cfg.plugins.entries = {}; + cfg.plugins.entries["memory-openviking"] = { + config: { + mode: "local", + configPath: "~/.openviking/ov.conf", + port: serverPort, + targetUri: "viking://", + autoRecall: true, + autoCapture: true, + }, + }; + if (!cfg.gateway) cfg.gateway = {}; + cfg.gateway.mode = "local"; + await mkdir(OPENCLAW_DIR, { recursive: true }); + await writeFile(cfgPath, JSON.stringify(cfg, null, 2) + "\n"); +} + +async function configureOpenclawViaCli(pluginPath, serverPort, mode) { + const runNoShell = (cmd, args, opts = {}) => run(cmd, args, { ...opts, shell: false }); + if (mode === "link") { + if (existsSync(PLUGIN_DEST)) { + log(`Removing old plugin dir ${PLUGIN_DEST}...`, "info"); + await rm(PLUGIN_DEST, { recursive: true, force: true }); + } + await run("openclaw", ["plugins", "install", "-l", pluginPath]); + } else { + await runNoShell("openclaw", ["config", "set", "plugins.load.paths", JSON.stringify([pluginPath])], { silent: true }).catch(() => {}); + } + await runNoShell("openclaw", ["config", "set", "plugins.enabled", "true"]); + await runNoShell("openclaw", ["config", "set", "plugins.allow", JSON.stringify(["memory-openviking"]), "--json"]); + await runNoShell("openclaw", ["config", "set", "gateway.mode", "local"]); + await runNoShell("openclaw", ["config", "set", "plugins.slots.memory", "memory-openviking"]); + await runNoShell("openclaw", ["config", "set", "plugins.entries.memory-openviking.config.mode", "local"]); + await runNoShell("openclaw", ["config", "set", "plugins.entries.memory-openviking.config.configPath", "~/.openviking/ov.conf"]); + await runNoShell("openclaw", ["config", "set", "plugins.entries.memory-openviking.config.port", String(serverPort)]); + await runNoShell("openclaw", ["config", "set", "plugins.entries.memory-openviking.config.targetUri", "viking://"]); + await runNoShell("openclaw", ["config", "set", "plugins.entries.memory-openviking.config.autoRecall", "true", "--json"]); + await runNoShell("openclaw", ["config", "set", "plugins.entries.memory-openviking.config.autoCapture", "true", "--json"]); +} + +async function configureOpenclaw(pluginPath, serverPort = DEFAULT_SERVER_PORT, mode = "link") { + await fixStalePluginPaths(pluginPath); + if (IS_WIN) { + await configureOpenclawViaJson(pluginPath, serverPort); + } else { + await configureOpenclawViaCli(pluginPath, serverPort, mode); + } + log("OpenClaw plugin config done", "ok"); +} + +async function resolveCommand(cmd) { + if (IS_WIN) { + const { code, out } = await runCapture("where", [cmd], { shell: true }); + return code === 0 ? out.split(/\r?\n/)[0].trim() : ""; + } + const { out } = await runCapture("sh", ["-c", `command -v ${cmd} 2>/dev/null || which ${cmd}`]); + return out || ""; +} + +async function writeOpenvikingEnv() { + const pyCmd = process.env.OPENVIKING_PYTHON || DEFAULT_PYTHON; + const pyPath = await resolveCommand(pyCmd); + const goOut = await resolveCommand("go"); + const goPath = goOut ? dirname(goOut) : ""; + await mkdir(OPENCLAW_DIR, { recursive: true }); + + if (IS_WIN) { + const lines = []; + if (pyPath) lines.push(`set OPENVIKING_PYTHON=${pyPath}`); + if (goPath) lines.push(`set OPENVIKING_GO_PATH=${goPath}`); + if (process.env.GOPATH) lines.push(`set OPENVIKING_GOPATH=${process.env.GOPATH}`); + if (process.env.GOPROXY) lines.push(`set OPENVIKING_GOPROXY=${process.env.GOPROXY}`); + await writeFile(join(OPENCLAW_DIR, "openviking.env.bat"), lines.join("\r\n") + "\r\n"); + log(`Written ~/.openclaw/openviking.env.bat`, "ok"); + } else { + const lines = []; + if (pyPath) lines.push(`export OPENVIKING_PYTHON='${pyPath}'`); + if (goPath) lines.push(`export OPENVIKING_GO_PATH='${goPath}'`); + if (process.env.GOPATH) lines.push(`export OPENVIKING_GOPATH='${process.env.GOPATH}'`); + if (process.env.GOPROXY) lines.push(`export OPENVIKING_GOPROXY='${process.env.GOPROXY}'`); + await writeFile(join(OPENCLAW_DIR, "openviking.env"), lines.join("\n") + "\n"); + log(`Written ~/.openclaw/openviking.env`, "ok"); + } +} + +// ─── Main flow ─── + +async function main() { + const args = process.argv.slice(2); + const help = args.includes("--help") || args.includes("-h"); + const nonInteractive = args.includes("--yes") || args.includes("-y"); + + if (help) { + console.log(` +OpenClaw + OpenViking setup helper + +Usage: npx openclaw-openviking-setup-helper [options] + +Options: + -y, --yes Non-interactive, use defaults + -h, --help Show help + +Steps: + 1. Check OpenClaw + 2. Check build environment (Python, Go, cmake, g++) + 3. Install openviking module if needed + 4. Configure ov.conf (API Key, VLM, Embedding, ports) + 5. Deploy memory-openviking plugin + 6. Write ~/.openclaw/openviking.env + +Env vars: + OPENVIKING_PYTHON Python path + OPENVIKING_CONFIG_FILE ov.conf path + OPENVIKING_REPO Local OpenViking repo path (use local plugin if set) + OPENVIKING_ARK_API_KEY Volcengine Ark API Key (used in -y mode, skip prompt) + OPENVIKING_GO_PATH Go bin dir (when Go not in PATH, e.g. ~/local/go/bin) +`); + process.exit(0); + } + + console.log("\n\ud83e\udd9e OpenClaw + OpenViking setup helper\n"); + + const distro = await detectDistro(); + + // ════════════════════════════════════════════ + // Phase 1: Check build tools & runtime environment + // cmake/g++ must be present before OpenClaw install (node-llama-cpp needs them) + // ════════════════════════════════════════════ + console.log("── Step 1/5: Checking build environment ──\n"); + + const missing = []; + + // cmake check (needed by OpenClaw's node-llama-cpp AND OpenViking C++ extension) + const cmakeResult = await checkCmake(); + if (cmakeResult.ok) { + log("cmake: installed", "ok"); + } else { + log("cmake: not found", "err"); + missing.push({ name: "cmake", detail: "Required by OpenClaw (llama.cpp) and OpenViking (C++ extension)" }); + } + + // g++ check (needed by OpenClaw's node-llama-cpp AND OpenViking C++ extension) + const gppResult = await checkGpp(); + if (gppResult.ok) { + log("g++: installed", "ok"); + } else { + log("g++: not found", "err"); + missing.push({ name: "g++ (gcc-c++)", detail: "Required by OpenClaw (llama.cpp) and OpenViking (C++ extension)" }); + } + + // Python check + const pyResult = await checkPython(); + if (pyResult.ok) { + log(`Python: ${pyResult.msg}`, "ok"); + } else { + log(`Python: ${pyResult.msg}`, "err"); + missing.push({ name: "Python >= 3.10", detail: pyResult.version ? `Current: ${pyResult.version}` : "Not found" }); + } + + // Go check (required on Linux for source install) + const goResult = await checkGo(); + if (goResult.ok) { + log(`Go: ${goResult.msg}`, "ok"); + } else if (IS_LINUX) { + log(`Go: ${goResult.msg}`, "err"); + missing.push({ name: "Go >= 1.25", detail: goResult.msg }); + } else { + log(`Go: not found (not required on ${process.platform})`, "warn"); + } + + if (missing.length > 0) { + console.log("\n\u2717 Missing dependencies:\n"); + for (const m of missing) { + console.log(` - ${m.name}: ${m.detail}`); + } + + console.log("\n Please install the missing dependencies:\n"); + + if (distro === "rhel") { + const needBuild = missing.some((m) => m.name === "cmake" || m.name === "g++ (gcc-c++)"); + const needPython = missing.some((m) => m.name.startsWith("Python")); + const needGo = missing.some((m) => m.name.startsWith("Go")); + + if (needBuild) console.log(" sudo dnf install -y gcc gcc-c++ cmake make"); + if (needPython) { + console.log(" # Install Python 3.11 (try package manager first):"); + console.log(" sudo dnf install -y python3.11 python3.11-devel python3.11-pip"); + console.log(" # If unavailable, build from source:"); + console.log(" # See INSTALL-ZH.md 'Linux Environment Setup' section"); + } + if (needGo) { + console.log(" # Install Go >= 1.25:"); + console.log(" wget https://go.dev/dl/go1.25.6.linux-amd64.tar.gz"); + console.log(" sudo rm -rf /usr/local/go"); + console.log(" sudo tar -C /usr/local -xzf go1.25.6.linux-amd64.tar.gz"); + console.log(" echo 'export PATH=$PATH:/usr/local/go/bin' >> ~/.bashrc"); + console.log(" source ~/.bashrc"); + console.log(" # Configure Go module proxy (recommended if downloads are slow):"); + console.log(" go env -w GOPROXY=https://goproxy.cn,direct"); + } + } else if (distro === "debian") { + const needBuild = missing.some((m) => m.name === "cmake" || m.name === "g++ (gcc-c++)"); + const needPython = missing.some((m) => m.name.startsWith("Python")); + const needGo = missing.some((m) => m.name.startsWith("Go")); + + if (needBuild) console.log(" sudo apt update && sudo apt install -y build-essential cmake"); + if (needPython) { + console.log(" # Install Python 3.11:"); + console.log(" sudo add-apt-repository ppa:deadsnakes/ppa"); + console.log(" sudo apt install -y python3.11 python3.11-dev python3.11-venv"); + } + if (needGo) { + console.log(" # Install Go >= 1.25:"); + console.log(" wget https://go.dev/dl/go1.25.6.linux-amd64.tar.gz"); + console.log(" sudo rm -rf /usr/local/go"); + console.log(" sudo tar -C /usr/local -xzf go1.25.6.linux-amd64.tar.gz"); + console.log(" echo 'export PATH=$PATH:/usr/local/go/bin' >> ~/.bashrc"); + console.log(" source ~/.bashrc"); + console.log(" # Configure Go module proxy (recommended if downloads are slow):"); + console.log(" go env -w GOPROXY=https://goproxy.cn,direct"); + } + } else { + console.log(" Please install: cmake, g++, Python >= 3.10, Go >= 1.25"); + console.log(" See INSTALL-ZH.md for detailed instructions."); + } + + console.log("\n After installing, re-run this script:"); + console.log(" npx ./examples/openclaw-memory-plugin/setup-helper\n"); + process.exit(1); + } + + // ════════════════════════════════════════════ + // Phase 2: Check OpenClaw + // ════════════════════════════════════════════ + console.log("\n── Step 2/5: Checking OpenClaw ──\n"); + + const hasOpenclaw = await checkOpenclaw(); + if (!hasOpenclaw.ok) { + log("OpenClaw is not installed.", "err"); + console.log("\n Please install OpenClaw:\n"); + console.log(" npm install -g openclaw\n"); + console.log(" If downloads are slow, use npmmirror registry:"); + console.log(" npm install -g openclaw --registry=https://registry.npmmirror.com\n"); + console.log(" After installation, run onboarding to configure your LLM:"); + console.log(" openclaw onboard\n"); + console.log(" Then re-run this script:"); + console.log(" npx ./examples/openclaw-memory-plugin/setup-helper\n"); + process.exit(1); + } + log("OpenClaw: installed", "ok"); + + // ════════════════════════════════════════════ + // Phase 3: Check & install openviking module + // ════════════════════════════════════════════ + console.log("\n── Step 3/5: Checking openviking module ──\n"); + + const ovMod = await checkOpenvikingModule(); + if (ovMod.ok) { + log("openviking module: installed", "ok"); + } else { + log("openviking module: not found", "warn"); + const inferredRepoRoot = join(__dirname, "..", "..", ".."); + const hasLocalRepo = existsSync(join(inferredRepoRoot, "pyproject.toml")); + const repo = process.env.OPENVIKING_REPO || (hasLocalRepo ? inferredRepoRoot : ""); + + if (nonInteractive) { + await installOpenviking(repo); + } else { + const choice = await question( + repo + ? "Install openviking from local repo? (y=local repo / n=skip)" + : "Install openviking from PyPI? (y/n)", + "y" + ); + if (choice.toLowerCase() === "y") { + await installOpenviking(repo); + } else { + log("Please install openviking manually and re-run this script.", "err"); + if (repo) console.log(` cd ${repo} && python3.11 -m pip install -e .`); + else console.log(" python3.11 -m pip install openviking"); + process.exit(1); + } + } + + const recheck = await checkOpenvikingModule(); + if (!recheck.ok) { + log("openviking module installation failed. Check errors above.", "err"); + process.exit(1); + } + log("openviking module: installed", "ok"); + } + + // ════════════════════════════════════════════ + // Phase 4: Configure ov.conf (interactive) + // ════════════════════════════════════════════ + console.log("\n── Step 4/5: Configuring OpenViking ──\n"); + + const ovConf = await checkOvvConf(); + const ovConfPath = ovConf.path; + let ovOpts = { + apiKey: process.env.OPENVIKING_ARK_API_KEY || "", + serverPort: DEFAULT_SERVER_PORT, + agfsPort: DEFAULT_AGFS_PORT, + vlmModel: DEFAULT_VLM_MODEL, + embeddingModel: DEFAULT_EMBEDDING_MODEL, + }; + + if (!ovConf.ok) { + log(`ov.conf not found: ${ovConfPath}`, "info"); + const create = nonInteractive || (await question("Create ov.conf now? (y/n)", "y")).toLowerCase() === "y"; + if (create) { + ovOpts = await collectOvvConfInteractive(nonInteractive); + await ensureOvvConf(ovConfPath, ovOpts); + } else { + log("Please create ~/.openviking/ov.conf manually", "err"); + process.exit(1); + } + } else { + log(`ov.conf found: ${ovConfPath}`, "ok"); + const invalid = await isOvvConfInvalid(ovConfPath); + const existingKey = await getApiKeyFromOvvConf(ovConfPath); + const existingPorts = await getOvvConfPorts(ovConfPath); + + if (invalid) { + log("ov.conf format is invalid, will recreate", "warn"); + ovOpts = await collectOvvConfInteractive(nonInteractive); + await ensureOvvConf(ovConfPath, ovOpts); + } else if (!existingKey && !nonInteractive) { + log("API Key is not configured in ov.conf", "warn"); + console.log("\nOpenViking needs a Volcengine Ark API Key for memory features."); + console.log("Get your API Key at: https://console.volcengine.com/ark\n"); + const apiKey = (await questionApiKey("Volcengine Ark API Key (leave blank to skip): ")) || process.env.OPENVIKING_ARK_API_KEY || ""; + if (apiKey) { + await updateOvvConf(ovConfPath, { apiKey }); + log("Written API Key to ov.conf", "ok"); + } else { + log("API Key not set; memory features may be unavailable. Edit ov.conf to add later.", "warn"); + } + ovOpts = { ...existingPorts, apiKey }; + } else if (!existingKey && process.env.OPENVIKING_ARK_API_KEY) { + await updateOvvConf(ovConfPath, { apiKey: process.env.OPENVIKING_ARK_API_KEY }); + log("Written API Key from env to ov.conf", "ok"); + ovOpts = { ...existingPorts, apiKey: process.env.OPENVIKING_ARK_API_KEY }; + } else { + ovOpts = { ...existingPorts, apiKey: existingKey }; + } + } + + // ════════════════════════════════════════════ + // Phase 5: Deploy plugin & finalize + // ════════════════════════════════════════════ + console.log("\n── Step 5/5: Deploying plugin ──\n"); + + const inferredRepoRoot = join(__dirname, "..", "..", ".."); + const repoRoot = process.env.OPENVIKING_REPO || + (existsSync(join(inferredRepoRoot, "examples", "openclaw-memory-plugin", "index.ts")) ? inferredRepoRoot : ""); + let pluginPath; + if (repoRoot && existsSync(join(repoRoot, "examples", "openclaw-memory-plugin", "index.ts"))) { + pluginPath = join(repoRoot, "examples", "openclaw-memory-plugin"); + log(`Using local plugin: ${pluginPath}`, "ok"); + if (!existsSync(join(pluginPath, "node_modules"))) { + await run("npm", ["install", "--no-audit", "--no-fund"], { cwd: pluginPath, silent: true }); + } + } else { + await fetchPluginFromGitHub(PLUGIN_DEST); + pluginPath = PLUGIN_DEST; + } + + await configureOpenclaw(pluginPath, ovOpts?.serverPort); + await writeOpenvikingEnv(); + + // Done + console.log("\n╔══════════════════════════════════════════════════════════╗"); + console.log("║ \u2705 Setup complete! ║"); + console.log("╚══════════════════════════════════════════════════════════╝"); + console.log("\nTo start OpenClaw with memory:"); + if (IS_WIN) { + console.log(' call "%USERPROFILE%\\.openclaw\\openviking.env.bat" && openclaw gateway'); + } else { + console.log(" source ~/.openclaw/openviking.env && openclaw gateway"); + } + console.log("\nTo verify:"); + console.log(" openclaw status"); + console.log(""); +} + +main().catch((e) => { + console.error(e); + process.exit(1); +}); diff --git a/examples/openclaw-memory-plugin/setup-helper/package.json b/examples/openclaw-memory-plugin/setup-helper/package.json new file mode 100644 index 00000000..d02501ce --- /dev/null +++ b/examples/openclaw-memory-plugin/setup-helper/package.json @@ -0,0 +1,21 @@ +{ + "name": "openclaw-openviking-setup-helper", + "version": "0.1.0", + "description": "Setup helper for installing OpenViking memory plugin into OpenClaw", + "type": "module", + "bin": { + "openclaw-openviking-setup-helper": "./cli.js" + }, + "keywords": ["openviking", "openclaw", "setup", "memory", "agent"], + "author": "OpenViking", + "license": "Apache-2.0", + "repository": { + "type": "git", + "url": "https://github.com/OpenViking/OpenViking.git", + "directory": "examples/openclaw-memory-plugin/setup-helper" + }, + "files": ["cli.js", "README.md"], + "engines": { + "node": ">=22.0.0" + } +} diff --git a/examples/openclaw-memory-plugin/skills/install-openviking-memory/SKILL.md b/examples/openclaw-memory-plugin/skills/install-openviking-memory/SKILL.md new file mode 100644 index 00000000..2a8b3c31 --- /dev/null +++ b/examples/openclaw-memory-plugin/skills/install-openviking-memory/SKILL.md @@ -0,0 +1,266 @@ +--- +name: install-openviking-memory +description: Install and configure OpenViking as the long-term memory backend for OpenClaw. Run this skill once to set up persistent memory that automatically remembers and recalls context across conversations. Trigger when the user asks to "install openviking", "set up memory", or "enable long-term memory". +compatibility: OpenClaw installed (npm install -g openclaw), Python >= 3.10, Node.js >= 22 +--- + +# Install OpenViking Memory for OpenClaw + +This skill installs [OpenViking](https://github.com/volcengine/OpenViking) as the memory backend for OpenClaw. After installation, OpenClaw will automatically **remember** facts from conversations and **recall** relevant context. + +> **For AI Agents:** Follow the steps below **in order**. Run each command, check the output, and only proceed when the step passes. Ask the user for input only where indicated with **[ASK USER]**. + +--- + +## Step 1: Check Prerequisites + +Run **all** checks below. Every check must pass before continuing. + +### 1.1 Check Node.js (>= 22) + +```bash +node -v +``` + +- **Pass:** Output shows `v22.x.x` or higher. +- **Fail:** Tell the user: "Node.js >= 22 is required. Install it from https://nodejs.org/ or use nvm: `nvm install 22`" + +### 1.2 Check Python (>= 3.10) + +**Linux / macOS:** + +```bash +python3 --version +``` + +**Windows:** + +```powershell +python --version +``` + +- **Pass:** Output shows `Python 3.10.x` or higher. +- **Fail:** Tell the user: "Python >= 3.10 is required. See INSTALL.md for installation instructions, or build from source: https://www.python.org/downloads/" + +### 1.3 Check Build Tools (cmake, g++) + +```bash +cmake --version +g++ --version +``` + +- **Pass:** Both commands return version info. +- **Fail:** Tell the user which tool is missing and how to install: + - RHEL/CentOS/openEuler: `sudo dnf install -y gcc gcc-c++ cmake make` + - Ubuntu/Debian: `sudo apt install -y build-essential cmake` + - **These tools are required** — OpenClaw's `node-llama-cpp` dependency compiles C++ during installation, and OpenViking also has C++ extensions. + +### 1.4 Check OpenClaw + +```bash +openclaw --version +``` + +- **Pass:** Shows version output. +- **Fail:** Tell the user: "OpenClaw is required. Install with: `npm install -g openclaw` (if downloads are slow: `npm install -g openclaw --registry=https://registry.npmmirror.com`). After installing, run `openclaw onboard` to configure your LLM." + +### 1.5 Check Go (>= 1.25) — only for source install on Linux + +> Skip this check if the user will install via `pip install openviking` (PyPI pre-built package). + +```bash +go version +``` + +- **Pass:** Output shows `go1.25.x` or higher. +- **Fail:** Tell the user: "Go >= 1.25 is required for source installation on Linux (to compile AGFS). Download from https://go.dev/dl/ — see INSTALL.md for detailed steps." + +--- + +## Step 2: Clone and Install OpenViking + +### 2.1 Clone Repository + +If the OpenViking repo is not already present: + +```bash +git clone https://github.com/volcengine/OpenViking.git +cd OpenViking +``` + +### 2.2 Install Python Package + +Two options — choose based on the user's needs: + +#### Option A: Install from PyPI (recommended, no Go needed) + +```bash +python3 -m pip install openviking +``` + +#### Option B: Install from Source (developer mode) + +Requires Go >= 1.25 on Linux (check passed in Step 1.5). + +**Linux / macOS:** + +```bash +python3 -m pip install -e . +``` + +**Windows:** + +```powershell +python -m pip install -e . +``` + +> If pip downloads are slow, suggest using a mirror: +> `python3 -m pip install openviking -i https://pypi.tuna.tsinghua.edu.cn/simple` + +### 2.3 Verify Installation + +**Linux / macOS:** + +```bash +python3 -c "import openviking; print('openviking module: ok')" +``` + +**Windows:** + +```powershell +python -c "import openviking; print('openviking module: ok')" +``` + +- **Pass:** Prints `openviking module: ok`. +- **Fail — multiple Python versions:** Ask the user which Python to use, then install with that path: `/path/to/python3.11 -m pip install openviking` +- **Fail — `TypeError: unsupported operand type(s) for |`:** Python version is below 3.10. The user needs to upgrade. +- **Fail — `Go compiler not found`:** Go is not installed (source install only). See Step 1.5. + +--- + +## Step 3: Run the Setup Helper + +From the OpenViking repo root: + +```bash +npx ./examples/openclaw-memory-plugin/setup-helper +``` + +The helper will interactively prompt for: + +1. **Workspace path** — data storage location (default: absolute path of `~/.openviking/data`, auto-detected) +2. **Volcengine Ark API Key** — **[ASK USER]** Direct them to https://console.volcengine.com/ark if they don't have one +3. **VLM model** — default `doubao-seed-1-8-251228`, press Enter to accept +4. **Embedding model** — default `doubao-embedding-vision-250615`, press Enter to accept +5. **Server ports** — default 1933 (HTTP) and 1833 (AGFS), press Enter to accept + +The helper will automatically: +- Create `~/.openviking/ov.conf` +- Deploy the `memory-openviking` plugin into OpenClaw +- Configure OpenClaw to use local mode +- Write `~/.openclaw/openviking.env` (Linux/macOS) or `openviking.env.bat` (Windows) + +Wait for `Setup complete!` before proceeding. + +--- + +## Step 4: Start OpenClaw with Memory + +**Always load the env file first**, then start the gateway: + +**Linux / macOS:** + +```bash +source ~/.openclaw/openviking.env && openclaw gateway +``` + +**Windows (cmd):** + +```cmd +call "%USERPROFILE%\.openclaw\openviking.env.bat" && openclaw gateway +``` + +Wait a few seconds. Verify this line appears in the output: + +``` +[gateway] memory-openviking: local server started (http://127.0.0.1:1933, ...) +``` + +- **Pass:** Tell the user: "OpenViking memory is now active. I will automatically remember important facts from our conversations and recall them when relevant." +- **Fail — `health check timeout`:** A stale process is blocking the port. Fix with: + ```bash + lsof -ti tcp:1933 tcp:1833 | xargs kill -9 + source ~/.openclaw/openviking.env && openclaw gateway + ``` + +--- + +## Step 5: Verify (Optional) + +```bash +openclaw status +``` + +The **Memory** line should show: `enabled (plugin memory-openviking)` + +--- + +## Troubleshooting Quick Reference + +| Symptom | Cause | Fix | +|---------|-------|-----| +| `cmake not found` during npm install | Missing build tools | `sudo dnf install -y gcc gcc-c++ cmake make` | +| `Python.h: No such file or directory` | Missing Python dev headers | `sudo dnf install -y python3-devel` (or `python3.11-devel`) | +| `Go compiler not found` | Go not installed | Install Go >= 1.25 from https://go.dev/dl/ | +| `dial tcp: i/o timeout` (Go modules) | Network issue | `go env -w GOPROXY=https://goproxy.cn,direct` | +| `ERR_INVALID_URL` (npm) | Proxy missing `http://` prefix | `export https_proxy=http://host:port` | +| `extracted 0 memories` | Wrong API key or model name | Check `api_key` and `model` in `~/.openviking/ov.conf` | +| `health check timeout` | Stale process on port | Kill with `lsof -ti tcp:1933 tcp:1833 \| xargs kill -9` | +| Plugin not loaded | Env file not sourced | Run `source ~/.openclaw/openviking.env` before gateway | + +--- + +## Daily Usage + +Each time the user wants to start OpenClaw with memory: + +**Linux / macOS:** + +```bash +source ~/.openclaw/openviking.env && openclaw gateway +``` + +**Windows (cmd):** + +```cmd +call "%USERPROFILE%\.openclaw\openviking.env.bat" && openclaw gateway +``` + +> Suggest adding an alias for convenience: +> ```bash +> echo 'alias openclaw-start="source ~/.openclaw/openviking.env && openclaw gateway"' >> ~/.bashrc +> ``` + +--- + +## Uninstall + +**Linux / macOS:** + +```bash +lsof -ti tcp:1933 tcp:1833 tcp:18789 | xargs kill -9 +npm uninstall -g openclaw +rm -rf ~/.openclaw +python3 -m pip uninstall openviking -y +rm -rf ~/.openviking +``` + +**Windows (cmd):** + +```cmd +for /f "tokens=5" %a in ('netstat -ano ^| findstr "LISTENING" ^| findstr ":1933 :1833 :18789"') do taskkill /PID %a /F +npm uninstall -g openclaw +rmdir /s /q "%USERPROFILE%\.openclaw" +python -m pip uninstall openviking -y +rmdir /s /q "%USERPROFILE%\.openviking" +```