From f7c2c00ad56bc69a42c102347c8872f1f36f4c01 Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Wed, 5 Aug 2026 04:55:19 +0800 Subject: [PATCH 1/4] feat(subos): a subos describes itself, and packages can declare its environment (2026.8.5.1) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit A program needs three things to run: a loader and a libc (bootstrap), a way to find its binaries and libraries (discovery), and the environment its subsystems look at (configuration). xlings had the first two -- glibc plus elfpatch, xvm plus shims -- and nothing for the third. That gap is mcpp-community/mcpp#352: a GLFW binary that links correctly and exits 255, because a GL driver is found through LIBGL_DRIVERS_PATH, an EGL vendor through __EGL_VENDOR_LIBRARY_DIRS, and a font config through XDG_DATA_DIRS. None of those can be linked in, and the process that has to see them is the user's own binary -- which xlings never wraps, so the per-shim `envs` on xvm.add cannot reach it. A subos now carries a `subos_info` block in its own .xlings.json saying what it IS, next to the `workspace` that says what it HAS: which runtime its binaries were built against, and which variables its processes need. subos_info: { schema_version, runtime, envs, created_at, created_by } `runtime` is self-describing ("glibc@2.39" is Linux/glibc) and settable with `xlings subos new --runtime`; the family is derived, never stored, so it cannot contradict the runtime it came from. `envs` is keyed by the declaring package's binding -- the same provider-scoped ownership xvm.add uses -- so uninstall drops exactly what a package added and a recipe writes no cleanup. Values must use ${pkgdir} / ${subosdir} / ${home} / ${xlings_home}. A value holding this machine's absolute paths describes this machine, and a subos description that only works where it was written is not a description. An unresolvable placeholder is left verbatim rather than blanked: "${pkgdir}/lib" collapsing to "/lib" is a real host path a driver search would follow. Packages declare through `subos.env{}` (libxpkg 0.0.48). Entering the subos -- `subos use`, `--shell`, `--cmd` -- expands and applies them, and reports what it injected on stderr. A variable the user already exported wins over `set`; `prepend` still composes with it. Conflicts resolve by binding order, not install order. Install order is not in the manifest, and recording it would add a field whose only effect is to make the result depend on history: two machines holding identical manifests would export different values. doctor reports every conflict rather than resolving it quietly. Three things this needed that the design did not anticipate: * `self init` writes the block too. The `default` subos is not created through subos::create, so without this the whole layer would be inert on the one subos everybody actually uses -- and inert is indistinguishable from "no package needed anything". It doubles as the migration for homes that predate the block. * env ops are consumed BEFORE process_xvm_operations_'s early return. A package that declares only env registers nothing with xvm, and would otherwise install cleanly with its declarations dropped. * doctor's renderer had a `default: break;`. A new FindingKind without a case is detected and never printed. doctor gains five checks: structure, sections owned by packages that are not installed, values that would not expand, contested variables, and a declared runtime that is missing. Reporter and repairer call one predicate, so `--fix` touches exactly what was reported. Verified: 29 unit tests over schema, invariants, expansion, conflicts and order-independence; E2E-60 walks install → declaration → --shell → --cmd → user override → doctor → uninstall; E2E-61 runs one recipe through a real released binary and this build. E2E-61 also pins the probe rule, which the design got wrong. `subos.env` arrives as a NEW MODULE, and import() answers an unknown module with a permissive proxy whose every key is truthy -- so `if subos.env then` is true on clients that will accept the call and discard it. The rule is `type(subos.env) == "function"`; `if xvm.files then` was only ever safe because `xvm` is a module those clients already ship. Design: .agents/docs/2026-08-05-subos-minimum-design.md Plan: .agents/docs/2026-08-05-subos-slice1-landing-plan.md --- ...system-three-tier-and-composable-distro.md | 488 +++++++++++++++ .../docs/2026-08-05-subos-minimum-design.md | 577 ++++++++++++++++++ .../2026-08-05-subos-slice1-landing-plan.md | 176 ++++++ ...8-05-userspace-distro-hermetic-strategy.md | 304 +++++++++ docs/spec/xlings-json-schema.md | 46 ++ mcpp.lock | 6 +- mcpp.toml | 4 +- src/cli/spec.cppm | 2 +- src/core/config.cppm | 2 +- src/core/subos.cppm | 296 ++++++++- src/core/subos/manifest.cppm | 488 +++++++++++++++ src/core/xim/installer.cppm | 139 +++++ src/core/xself/doctor.cppm | 285 +++++++++ src/core/xself/init.cppm | 44 +- tests/e2e/run_all.sh | 2 + tests/e2e/subos_env_declaration_test.sh | 254 ++++++++ tests/e2e/subos_env_probe_compat_test.sh | 200 ++++++ tests/unit/test_subos_manifest.cpp | 345 +++++++++++ 18 files changed, 3643 insertions(+), 15 deletions(-) create mode 100644 .agents/docs/2026-08-05-ecosystem-three-tier-and-composable-distro.md create mode 100644 .agents/docs/2026-08-05-subos-minimum-design.md create mode 100644 .agents/docs/2026-08-05-subos-slice1-landing-plan.md create mode 100644 .agents/docs/2026-08-05-userspace-distro-hermetic-strategy.md create mode 100644 src/core/subos/manifest.cppm create mode 100755 tests/e2e/subos_env_declaration_test.sh create mode 100755 tests/e2e/subos_env_probe_compat_test.sh create mode 100644 tests/unit/test_subos_manifest.cpp diff --git a/.agents/docs/2026-08-05-ecosystem-three-tier-and-composable-distro.md b/.agents/docs/2026-08-05-ecosystem-three-tier-and-composable-distro.md new file mode 100644 index 00000000..0023e7d1 --- /dev/null +++ b/.agents/docs/2026-08-05-ecosystem-three-tier-and-composable-distro.md @@ -0,0 +1,488 @@ +# xlings 生态三分层定位与可组合发行版:讨论纪要 + +**日期**: 2026-08-05 +**类型**: 讨论纪要(discussion memo)—— 综合 2026-08-05 会话讨论,作为后续逐条深入的锚点,不是策略也不是实施计划 +**触发**: 关于"xlings 生态究竟是什么、多 glibc 能不能做、可组合发行版如何表达、预构建 vs 源码构建如何取舍、消费者依赖硬钉能否用版本语义救"的连续追问 +**关联**: +- `2026-08-05-userspace-distro-hermetic-strategy.md`(运行时边界策略,本文的**下位配套**) +- `2026-05-22-subos-sandbox-gpu-passthrough.md`(sandbox 层 GPU 设备节点透传) +- `2026-06-26-multiarch-package-description-design.md`(payload 多架构表达) +- `2026-06-21-linux-root-usability-survey.md`(权限边界) +- xim-pkgindex `docs/V2/xpackage-spec.md`(新字段/能力的 probe-first 迁移范式) + +--- + +## 0. TL;DR + +**xlings 生态定位为三分层**: +- **kernel**(宿主提供)—— 只提供 syscall 边界 +- **xlings**(用户态发行版底座)—— subos 隔离 + xim-pkgindex 包管理 + 版本/binding 解析 +- **mcpp**(生态主构建工具)—— C/C++ 是核心域,**不限于** C/C++;其他语言按需包裹/集成 + +**分发采用预构建 + 源码构建混合架构**:命中预构建即取,不命中走 mcpp 源构;主流软件通过 xlings-res 预热覆盖长尾。 + +**多 glibc / 可组合发行版通过 platform manifest 承载**,不做"任意组合"(Nix 模式对 xlings 团队规模不现实),而是"用户从官方 platform 里选一份"(Flatpak runtime / Homebrew bottle 模式)。 + +**消费者依赖从字面量硬钉演进为三轴模型**: +- `deps` 范围(装机期选择器) +- `runtime_floor`(构件期物理烙定,不可变) +- `platform_membership`(coupled-ABI 群的集合成员,短路 range 解析) + +**与 hermetic 策略正交且互补**:hermetic 谈"运行时边界",本文谈"生态角色 + 分发架构"。合并后形成 xlings 生态的完整技术定位。 + +--- + +## 1. 起点与范围 + +会话由前一份 hermetic 策略延伸而来。hermetic 策略解决"什么 `.so` 能穿越到宿主",但一个更基础的问题从未明文写过:**xlings 究竟是什么?** 生态里的很多具体决策(compat.* 归属、xim-x-* 命名、pkgindex 与 mcpp-index 的分工、xlings-res 的作用、subos 与 mcpp sandbox 的关系)因为缺一份"我们是什么"的定位文档而反复重新推导。 + +本文的目标: +- **梳理**上述讨论,把散在多个 issue/PR/记忆里的决策浮到明面 +- **命名**已经在做的模式(用户态发行版、三分层、混合分发、platform 化),让后续 review 有共同锚 +- **枚举**已识别的阻塞项与开放问题,不给结论,留给后续逐条深入 + +本文**不是**策略、不是实施计划、不承诺时间表。任何进入代码的动作都需要独立的设计文档评审。 + +--- + +## 2. 三分层定位:kernel + xlings + mcpp + +### 2.1 分工与职责 + +``` +┌─────────────────────────────────────────────────────────┐ +│ kernel (host) │ +│ syscall, /dev/*, /proc/*, vendor kmod(如 nvidia.ko)│ +│ ↑ 通过 capabilities_host 白名单穿越 │ +├─────────────────────────────────────────────────────────┤ +│ xlings │ +│ subos 隔离层 (bwrap sandbox, 独立 sysroot) │ +│ xim-pkgindex 包管理 (recipe, xvm, elfpatch) │ +│ 版本/binding 解析 (xvm.add, xvm 节点绑定) │ +│ platform manifest (新: 承载"这个 subos 是哪个发行版") │ +├─────────────────────────────────────────────────────────┤ +│ mcpp │ +│ 生态主构建工具 │ +│ C/C++ 核心域: compile, link, test, canonical build │ +│ 构建 fingerprint、缓存、增量 │ +│ 其他语言按需包裹/集成 (见 §2.3 scope 修正) │ +├─────────────────────────────────────────────────────────┤ +│ 用户项目 / 应用 │ +└─────────────────────────────────────────────────────────┘ +``` + +**语义收敛**:每一层只对上一层负责,每一层解决一类问题。当前生态里的角色混淆(比如 recipe 里塞 build 逻辑、mcpp 反过来管 subos 状态)是**层次未分开**的症状。 + +### 2.2 参照系(定位判据的来源) + +| 生态 | Kernel | 用户态底座 | 包管理器 | 构建工具 | 分发形态 | +|---|---|---|---|---|---| +| Debian/Fedora | Linux | glibc + rootfs | apt/dnf | (每包各带) | 预构建为主 | +| NixOS | Linux(自建) | glibc(自建) | nix | nix build | binary cache 命中即取,不中现构 | +| Homebrew | XNU(宿主) | libSystem(宿主) | brew | (每 formula) | bottle(预构)+ 源构混合 | +| Gentoo | Linux(宿主) | glibc(自建) | portage | ebuild | 源构为主,binhost 兜底 | +| **xlings 提议** | Linux(宿主) | xlings subos + xim-pkgindex | xlings CLI | **mcpp** | **预构命中即取,不中走 mcpp 源构** | + +**最贴近的类比**:Homebrew 精神 + Nix 分层。 +- 从 Homebrew 继承:不接管 kernel、不自建 rootfs、host 只提供 syscall 边界(呼应 hermetic 策略 non-goals) +- 从 Nix 继承:用一个构建工具(nix build ↔ mcpp)承担源→二进制的正典转换,预构建是同一 recipe 的产物缓存,不是另一套并行的东西 +- **业界没有完美先例**——Homebrew 的 bottle/source 有微差(build flag 漂移),Nix 严格但要求自建整个 rootfs。用户态发行版 + host kernel + 单一构建正典的组合是空生态位 + +### 2.3 mcpp scope 定位(用户 2026-08-05 更正) + +**mcpp 是生态的主要构建工具,C/C++ 是核心域,不限于 C/C++。** + +原会话草案曾表述为"mcpp 是 C/C++ 构建规范,非 C/C++ 走各自 upstream build system"。这个表述过窄,已按用户更正修改。修正后的定位: + +- mcpp 首先服务 C/C++——现有能力、canonical build、fingerprint 机制、hermetic build 环境都以 C/C++ 为一等公民 +- mcpp **可以且应该**承担多语言场景下的**协调/包裹**角色——具体形态(是原生集成、是插件、还是 wrapper)是**开放问题 A** +- **不承诺 mcpp 亲自实现 cargo/npm/pip 的求解器/构建管线**;这类生态的求解和构建有其成熟工具,mcpp 侧更可能是"调用它们并统一 fingerprint/缓存/hermetic 边界" + +**为什么这个边界重要**:如果不明确 scope,mcpp 会被要求覆盖所有语言生态,scope 爆炸后没人能维护;如果把 mcpp 限死为 C/C++,又切断了"xlings 生态用一套工具"的定位价值。**折中是"C/C++ 一等公民 + 多语言协调层"**,协调层的具体形态另议。 + +### 2.4 subos 与 mcpp sandbox 的关系 + +- **subos**:运行 OS 实例(bwrap-based,独立 sysroot / xvm 状态 / 包安装目录) +- **mcpp sandbox**:构建时环境(bwrap-based,提供 hermetic 构建) + +现状:两者的 sandbox 实现应已复用。**长期方向**:mcpp build 应能直接消费"当前 subos"作为构建环境——subos 是 mcpp 的输入(工具链/依赖来源),构建产物落回 subos。这样 mcpp canonical build 天然遵循 subos 的 platform 约束。 + +**开放问题**:mcpp build 究竟应"消费 subos"(在 subos 里跑)还是"消费 platform 描述"(在临时环境里按 platform 组装工具链)?两者对 fingerprint 稳定性影响不同。留作开放问题 J 的子问题。 + +--- + +## 3. 可组合发行版:subos + platform manifest + +### 3.1 subos 是容器,platform manifest 是"是哪一款" + +subos 已经是"一个用户态 OS 实例"的物理位置。但**subos 现在没有整体身份**——每个包各自声明 `xim:glibc@2.39`,合起来隐式产生了一个"发行版",但没人拥有它、没人能整体替换它。 + +**可组合发行版 = subos + platform manifest**: + +``` +distro := subos + platform_manifest +platform_manifest := { + id: "xlings-2026.08-el9", + kernel_abi_floor: "3.10.0", + glibc: "2.39", + gcc: "16.1.0", + libstdcxx: "gcc@16.1.0", + binutils: "2.42", + linux_headers: "6.6", + ... +} +``` + +一个 subos 生成时选一份 platform,之后**subos 内所有包的依赖解析都以 platform 为锚**。用户"切换发行版" = "换 subos",不是"改配置"。 + +### 3.2 platform manifest 的形态(开放问题 B) + +至少三种承载形式,尚未定: + +**形式 1 —— 显式声明包**(platform 本身是 xim-pkgindex 的一个 xpkg): +```lua +-- pkgs/x/xlings-platform-2026.08.lua +package = { + name = "xlings-platform-2026.08", + type = "platform", + members = { + { "xim:glibc", "2.39" }, + { "xim:gcc", "16.1.0" }, + { "xim:binutils", "2.42" }, + { "xim:linux-headers", "6.6" }, + ... + }, + kernel_abi_floor = "3.10.0", +} +``` + +**形式 2 —— subos 元数据**(subos 创建时选一份,记在 subos 目录里): +```json +// $XLINGS_HOME/subos//platform.json +{ "id": "xlings-2026.08-el9", ... } +``` + +**形式 3 —— `.xlings.json` 用户侧**(用户项目声明依赖的 platform): +```json +{ "platform": "xlings-2026.08-el9" } +``` + +三者可以共存(索引侧发布定义 + subos 侧固化实例 + 项目侧声明需求),但**谁是权威源**、**谁与谁一致性校验**需要单独设计。留作开放问题 B。 + +### 3.3 为什么不是"任意组合"(Nix 模式) + +Nix 允许用户任意组合"glibc 2.31 + gcc 13.3 + llvm 20 + boost 1.85",代价是 nixpkgs 社区的巨大规模(数千 committer,构建集群,binary cache)。**xlings 团队规模是两个数量级之外**。 + +务实定位:**"用户从官方 platform 里选一份"**,不是"用户自由组合"。 +- 官方发布节奏:比如季度一份 platform(2026.05 / 2026.08 / 2026.11 / ...) +- 每份 platform 是**原子集合**:内部互测通过,一起 release +- 用户"自定义发行版" = "从官方 platform 库里选" + "在 platform 之上叠加应用层包" +- 极端情况下用户可以 fork platform recipe 自建,但那是社区扩展,不是核心承诺 + +**对齐 Flatpak runtime `//24.08` 模型**,也匹配 hermetic 策略 §5.3 P2 #8 的"Release 套装"表述。 + +--- + +## 4. 多 glibc:能力 vs 现实 + +### 4.1 xvm 数据模型:已经够 + +事实核对(xim-pkgindex): +- `glibc.lua` 有 `xvm_enable = true` 且每个 .so 通过 `xvm.add(lib, {version=glibc_version, ...})` 版本绑定注册 +- gcc 已有 5 个共存版本(9.4.0 / 11.5.0 / 13.3.0 / 15.1.0 / 16.1.0),llvm 有 2 个(20.1.7 / 22.1.8) +- **glibc 现在只有 2.39 是"没做",不是"做不到"** +- 每个 subos 有独立 `subos_sysrootdir()`,glibc payload 走 xvm 独立目录——存储上完全隔离 + +### 4.2 阻塞项(优先级排序) + +即使 xvm 模型够,**加第二份 glibc 到索引里,今天只会产生孤儿包**,因为: + +1. **消费者硬钉字面量**(最硬的阻塞) + - `pkgs/g/gcc.lua:35`: `"xim:glibc@2.39", "xim:binutils@2.42"` + - `pkgs/l/llvm.lua:27`: `"xim:glibc@2.39"` + - **所有 5 个 gcc 版本、2 个 llvm 版本全部锁 2.39**——加 glibc 2.28 索引里没消费者用 + +2. **elfpatch loader 单值** + - `glibc.exports.runtime.loader = "lib64/ld-linux-x86-64.so.2"` 无版本轴 + - 预设"当前 subos 只有一个 glibc"——PT_INTERP 该写哪份必须由 subos platform 决定,不是 recipe 决定 + +3. **sysroot include 平面** + - `glibc.lua:__config_header` 塞 130 项进 `subos_sysrootdir()/usr/include`,`declare_headers` first-claimant-wins + - 两份 glibc 会在同一平面 include 目录打架 + - 需要 `usr/include/glibc-2.28/` 版本分片 + gcc `--sysroot`/`-isysroot` 联动 + +4. **binding 是 pin 不是 range** + - 无法表达"我兼容 glibc >= 2.28" + - manylinux 那种"编译对老、运行对新"的复用能力现在无法表达 + +5. **libstdc++/libc++ 与 glibc 隐式耦合** + - gcc-15 的 libstdc++ 依赖新 glibc 符号 + - 换 glibc 就得换 gcc → 佐证"platform 是原子单位"的判断 + +6. **kernel syscall 版本地板** + - 最新 glibc 会隐式要求某个 kernel(2.39 要 3.2+,2.42 要 3.10+) + - platform 应显式声明 `kernel_abi_floor`,首装时探 `uname -r` 拒绝不合规宿主 + +**前三条不修,加第二份 glibc 到索引里就是纯技术债。** + +### 4.3 platform 化 = 多 glibc 的最小可行形式 + +多 glibc 应该表现为**多 platform**,而不是"任意 glibc 组合"。用户不定义"glibc 2.31 + gcc 13.3.0 + llvm 20"这种随意组合(nixpkgs 规模),而是从 xlings 官方发布的一组 platform 里选。 + +**渐进路径**: +1. 定义 `xlings-platform-2026.08 = {glibc=2.39, gcc=16.1.0, ...}`,让当前 recipe 全都隐式属这一个 platform。**零功能变化,只是给现状命名** +2. 补 recipe DSL 里的 `${platform.*}` 变量,仅新写 recipe 用 +3. 做第二份 platform 当作压力测试(如 `xlings-platform-legacy-el7 = {glibc=2.28, gcc=11.5.0, ...}`)——此时同时暴露阻塞项 1/2/3/6 +4. compat.mesa 等新 payload 一律标 `platform: 2026.08`,不试图跨 platform 复用 + +--- + +## 5. 分发架构:预构建 + 源码构建混合 + +### 5.1 三 tier 分层 + +用户提议的 "预构建不中就源构" 有一个例外:**底座层没有源构选项**(bootstrap 循环:构 glibc 需要 gcc,构 gcc 需要 glibc)。 + +``` +Tier 0 — Bootstrap 底座 (永远预构建,不允许源构) + glibc, binutils, gcc(至少一份能自举的), linux-headers, libstdc++ + → 就是 §3 的 xim-platform-YYYY.MM + → 份数由 xlings 官方发布节奏决定 (季度一份 × 3 arch × 3 年 ≈ 36 份底座) + +Tier 1 — 工具链 / 主流库 (优先预构建,可源构兜底) + llvm, cmake, ninja, boost, freetype, libpng, mesa, ... + → 预构命中率应 > 95% (受众都是主流版本) + → 不中时 mcpp 本地构 + +Tier 2 — 长尾 / 用户项目 (源构常态,预构增值) + 用户 mcpp 项目、compat.* 库、私域包 + → 预构建是可选优化 + → 组织可架私有 substituter 加速团队 +``` + +**关键性质**:Tier 0 的存在恰好是"xlings 是发行版"最硬的一条证据——发行版的定义就是"我发布一组同步过的底座包"。 + +### 5.2 Canonical build 与 fingerprint + +用户提议的"预构不中就源构"要 sound,必须解决"等价性"问题。 + +**Canonical build**:一个包在 mcpp 侧只有一套 build 描述(compile 命令 + flags + 依赖版本),预构建 tarball 与本机源构走**同一份 recipe**。这样: +- 预构 tarball = canonical build 的 CI 产物 +- 源构产物 = 用户机上跑同一 canonical build +- 两者行为等价(不承诺字节等价——那是未来目标) + +**xim-pkgindex 现状** vs **迁移目标**: +- 现状:100+ 个 recipe 的 `install()` 里塞满 build 逻辑(makefile 参数、cmake flags、post-build patch) +- 目标:recipe 只做 metadata,build 逻辑交给 mcpp canonical build 描述 +- **这是多年重构**。务实做法:**新包 mcpp-first,老包按需迁移**(参照 xpackage-spec V2 是 V1 严格超集的模式) + +**Fingerprint**:mcpp 已有 fingerprint 概念(memory 里 `mcpp fingerprint stale binary` 提到 `target//bin/xlings` 路径结构)。扩展方向: + +``` +fingerprint = hash( + source_tarball_sha256, + toolchain_version + toolchain_fp, # 当前 subos 的 gcc/glibc 版本 + build_flags, + target_platform, # §3 的 platform id + canonical_build_recipe_version +) +``` + +查询 xlings-res:`//.tar.gz` 存在 → 下载;不存在 → mcpp 本地构 → 落 `target//`。 + +**Fingerprint 定义的字段列表与稳定性**是开放问题 J。 + +### 5.3 "命中 = 相等"的定义 + +两级选项: +- **字节级重现**(reproducible builds):同源 + 同工具链 + 同 flags → bit-identical。Nix/Bazel 承诺。工程代价高,签名可从 hash 派生 +- **行为等价**(functional equivalence):跑起来一样,字节不必一样。apt/brew 现状。代价低,但预构与源构结果不同时 bug 只在其中一条路径复现,定位困难 + +**对 xlings 现状**:行为等价起步,字节重现作为未来目标。 + +### 5.4 迁移代价与现实约束 + +1. **本地源构对宿主要求上升**:用户机装 xlings 几百 MB,但源构 mcpp 项目要 gcc + headers + 磁盘 + CPU。教育场景下源构兜底可能是"永远打不开的开关"——**xlings-res 预构建覆盖率必须 >> 95%**。这不推翻架构,但决定了 **xlings-res 预构建 CI 是生态的关键基础设施**。开放问题 F +2. **Fingerprint 传染性**:一个包的 fingerprint 依赖工具链 fingerprint,工具链依赖 libc fingerprint。**libc 换一次,fingerprint 全部失效,tier 1 所有包需重构**。对策:libc/toolchain 变更走 platform 原子发布,平时不动 +3. **信任模型统一**:预构建来自 xlings-res(CA/signing 假设),源构建来自本机(信任 upstream sha256)。两者语义等价意味着"不介意从哪拿"——已经是 Nix binary cache 信任模型 + +--- + +## 6. 版本语义:三轴模型 + +### 6.1 现在的硬钉字符串同时承担四种语义 + +`"xim:glibc@2.39"` 现在暗含四件事,没显式拆开: + +| 语义 | 含义 | 生命周期 | +|---|---|---| +| **构建期符号地板** | "gcc 这份预构件是拿 glibc 2.39 的 headers/loader 编的" | 不可变,tarball 出厂就烙死 | +| **装机期选择器** | "装 gcc 时,给我把 glibc 2.39 也装上/找到" | 可由 policy 改 | +| **兼容承诺** | "我保证这份 gcc 在 glibc 2.39 上能跑" | 人工断言,断言错了用户炸 | +| **平台成员** | "我是 xlings-platform-2026.08 的一员" | 集合成员,不是比较 | + +**Semver 范围解决的是第 2 和第 3 类**。但 glibc 的向前兼容是**单向的**: +- 对 2.28 编译的二进制 → 在 2.39 上跑得动 +- 对 2.39 编译的二进制 → 在 2.28 上跑不动(缺 GLIBC_2.30+) + +写 `"xim:glibc@>=2.28"` 只有在**这份 tarball 真的是拿 2.28 编的**时才是真话。tarball 拿 2.39 编,再声明 `>=2.28`——resolver 满心欢喜给 2.28 环境,运行时炸——**范围声明与物理事实脱节比硬钉更危险**,因为默认"绿"实际"红"。 + +**对 glibc 这类的正确模型是最小版本地板,不是范围**。 + +### 6.2 拆开:三轴模型 + +```lua +package = { + name = "gcc", + version = "16.1.0", + + -- 装机期(resolver 看这个,可以是范围/平台变量) + deps = { + { "xim:glibc", version = ">=2.39", reason = "gcc 16 build-time floor" }, + { "xim:binutils", version = "^2.42", reason = "ld 兼容" }, + }, + + -- 构件期(不可变,elfpatch/loader-选择/CI hermetic 都读这里) + runtime_floor = { + loader = "glibc@2.39", -- PT_INTERP 烙定 + libc = "glibc@2.39", -- 编译时 headers + libstdcxx = "gcc@16.1.0", -- 自带 libstdc++.so.6 + }, + + -- 平台成员(若属某个原子发布集合) + platform_membership = "xlings-2026.08", +} +``` + +**关键性质**: +- `deps` 里的范围**只在 mcpp 源构或纯 header-only 包时是自由的**。对预构 tarball,`deps` 的范围下界必须 == `runtime_floor` 对应字段——**CI 强校验** +- `runtime_floor` 是 tarball 元数据,发布时确定,**从此再不能改**——就像 PT_INTERP 一样物理烙定 +- `platform_membership` 存在时,resolver **短路**——直接锁 platform 定义的版本,不走 `deps` 范围解析。这条路径吃掉 80% 的现实场景 + +### 6.3 按包类型分级采用 + +| 包类型 | 举例 | 版本语义 | 理由 | +|---|---|---|---| +| **coupled-ABI 底座** | glibc, gcc, libstdc++, libc++ | 只用 platform 成员 | 三者符号 ABI 相互烙定,任意范围都是幻觉 | +| **系统运行时** | libpng, freetype, openssl | 最小版本地板 (`>=X.Y`) | 通常向前兼容,反向不成立;上界几乎不需要 | +| **纯 header-only C/C++ 库** | fmt, spdlog, catch2 | 完整语义范围 (`^X.Y.Z`) | 无 ABI,只在 build-time 解析 | +| **应用层预构件** | gh, ollama, godot | 单版本 pin (现状) | 上游各自发一个,没有多版本选择必要 | +| **mcpp 源构包** (未来) | 用户项目、compat.mesa 源版 | 完整语义范围 | 源构环境已知,范围能真被 solver 利用 | + +**推论**:"引入 semver 范围"不是全局决策,是按类型渐进的政策。 + +### 6.4 Silent-success 陷阱 + +xlings 生态踩过多次"从不发生和已经成功产生一样的日志"(memory: silent-success pattern)。版本范围的天然坑就是这一族: +- 声明 `>=2.28`,预构建实际拿 2.39 编——CI 里全都是 2.39,永远绿 +- 用户在 2.28 环境装,报 `GLIBC_2.30 not found`——错误信息不指向 recipe 的 range 声明,而是指向 loader + +**防坑必须的两件事**: +1. `runtime_floor` 与 `deps` 下界一致性,**CI 强校验**——范围声明与物理事实必须联动 +2. 范围声明的**每个"点"都必须 CI 覆盖**——`>=2.28, <3.0` 意味着 CI 至少跑过 2.28/2.35/2.39 三个采样点的 install + smoke test + +只做第一件,范围诚实但受限;只做第二件,CI 矩阵爆炸。**两件必须同时做**,而**平台成员机制**恰好把 CI 矩阵砍下来:成员天然把"跑几个采样点"变成"跑几个 platform",数量可控。CI 采样点矩阵设计是开放问题 H。 + +### 6.5 迁移(V2 spec probe pattern 复用) + +xpackage-spec V2 §"Adopting a capability" 已写清:新字段/新语法必须通过 probe 让老 client 走 legacy 分支,不能靠 `min_xlings` 挡门。三轴模型同样适用: + +```lua +if pkgindex.version_constraints then + deps = { { "xim:glibc", version = ">=2.39" } } +else + deps = { "xim:glibc@2.39" } -- 老 client 退化到硬钉最低点 +end +``` + +代价:每次 range 变更都要维护 legacy 侧硬钉,直到 dropping 老 client 那天。 + +--- + +## 7. 与 hermetic 策略的关系 + +正交、互补: + +| | hermetic 策略 | 三分层 + 分发架构 | +|---|---|---| +| **谈的是** | 运行时**边界**(什么 `.so` 能穿越到宿主) | 生态**角色** + 分发**架构** | +| **主要产物** | `capabilities_host` 元数据 + bwrap 空 host CI | mcpp canonical build + fingerprint 分发协议 + platform manifest | +| **单独价值** | 收敛"跨越宿主"的口径 | 收敛"我们是什么 + 如何来到用户机器上"的口径 | +| **联动价值** | mcpp 本地构出的二进制**天然遵循 hermetic**(build 环境本身 hermetic);预构建 tarball CI 用 bwrap 空 host 验证 hermetic 合规 | + +**建议**:把三分层定位写成上位文档,hermetic 策略作为其下位的**运行时边界章节**;分发架构作为下位的**分发章节**。合并后生态里所有决策(新增包、新增依赖、新增 CI check)都能找到锚。 + +--- + +## 8. 阻塞项汇总(优先级排序) + +从"要做多 glibc / 可组合发行版"角度倒推,阻塞项排序: + +| # | 阻塞项 | 影响 | 相关章节 | +|---|---|---|---| +| 1 | 消费者硬钉字面量 | 加第二份 glibc 无人使用 | §4.2 | +| 2 | 无 platform manifest 概念 | subos 无整体身份,无从"选一份" | §3 | +| 3 | elfpatch loader 单值 | PT_INTERP 无法按 subos platform 选择 | §4.2 | +| 4 | sysroot include 平面 | 多 glibc headers 打架 | §4.2 | +| 5 | recipe 里塞 build 逻辑 | 无法做 canonical build → 预构/源构等价性无从谈起 | §5.2 | +| 6 | 无 fingerprint 分发协议 | "命中即取,不中现构"无判定 | §5.2 | +| 7 | binding 是 pin 不是 range | 无法表达跨版本兼容 | §6 | +| 8 | 无 CI 一致性校验(range vs runtime_floor) | Silent-success 陷阱 | §6.4 | +| 9 | libstdc++ 与 glibc 隐式耦合无声明 | 换任一即换另一,但没人显式表达这个约束 | §4.2 §6.3 | +| 10 | kernel syscall 地板无声明 | 最新 glibc 隐式要求某 kernel,无人校验 | §4.2 | + +--- + +## 9. 最小可行下一步(仅建议顺序,不承诺时间) + +1. **写定位文档**(本文的继任者):把三分层 + hermetic 合成一份 `docs/design/ecosystem-positioning.md`。**零代码,当天可做** +2. **命名现状为 platform-2026.08**:定义 `xlings-platform-2026.08 = {现有 gcc/glibc/binutils/...}`,让当前 recipe 隐式属这份 platform。**零功能变化,零迁移** +3. **加 `runtime_floor` 字段**(spec V3):走 V2 spec 的 probe pattern 引入,recipe 侧填充,尚不强制。**逐包填,老 client 兼容** +4. **加 `deps` range 语法**(spec V3):同上,同样 probe 引入。CI 强校验 `runtime_floor` 与 `deps` 下界一致性 +5. **mcpp canonical build 探索**:选 1~2 个包做 pilot(不要选 glibc/gcc 这种硬骨头,选 libpng/fmt 这类简单库),验证 fingerprint 稳定性 + 预构/源构等价性 +6. **第二份 platform 作为压力测试**:当上面 5 步走通,做一份 `xlings-platform-legacy-el7`,一次性暴露多 glibc 剩余阻塞项(elfpatch / sysroot / libstdc++ 耦合) +7. **compat.mesa 等 hermetic 策略新 payload**:严格标 `platform_membership`,不跨 platform 复用 + +--- + +## 10. 开放问题(留待后续逐条讨论) + +- **A. mcpp scope 到底延伸到多远?** C/C++ 一等公民 + 多语言协调层——协调层是原生集成、插件、还是 wrapper?对 cargo/npm/pip 的态度分别是什么? +- **B. platform manifest 的 CLI/配置形态**:形式 1(xpkg)/形式 2(subos 元数据)/形式 3(`.xlings.json`)如何选?权威源是谁?一致性校验谁做? +- **C. compat.* vs xim-x-* 命名归属**:上一份分析建议归 xim-pkgindex,但保留 `compat.` 前缀会打破现有命名分层 +- **D. macOS / Windows 上 hermetic 与 platform 的对应**:macOS 有 system frameworks 与 libSystem,Windows 有 UCRT。platform 概念如何跨平台一致? +- **E. xlings-res 存储 / 带宽预算**:多 platform × 多 arch × tier 1 全预构 → 存储和 CDN 成本量级? +- **F. 源构建 UX 在教育场景的可行性**:mcpp 本地构对宿主 CPU/磁盘/网络的最低要求,教育场景能否兜底? +- **G. 老 recipe 迁移政策**:canonical build 迁移是强制(deadline)还是渐进(新包新政策)? +- **H. Range 的 CI 采样点矩阵设计**:每个 `>=X.Y` 声明至少覆盖哪几个点?矩阵如何砍? +- **I. Bootstrap 层如何独立更新**:比如 glibc 出安全补丁 2.39.1,能不能不动 platform 集合的其它成员单独发? +- **J. Fingerprint 定义的字段列表与稳定性**:哪些字段进 fingerprint?工具链版本进入的话,platform 微调 fingerprint 全炸,如何权衡? +- **K. mcpp 消费 subos vs 消费 platform 描述**:mcpp build 究竟在 subos 内跑,还是在临时环境按 platform 组装工具链? +- **L. 私有 substituter 支持**:组织内的私域预构建缓存,如何在信任模型 + fingerprint 协议里表达? +- **M. platform 弃用政策**:一份 platform release 后多久 EOL?EOL 后旧 subos 是不可迁移的? + +--- + +## 11. 决策未定与假设标注 + +本文所有具体形态(字段名、目录布局、CLI 语法)都是**示例**,不是决策。真正的决策在后续 §10 各开放问题单独讨论时才作出。 + +**已经作出的假设(可讨论)**: +- xlings 定位为"用户态发行版"(呼应 hermetic 策略 TL;DR)——如果这个定位本身要改,本文整体重来 +- mcpp 是生态主要构建工具(用户 2026-08-05 更正确认) +- kernel 由宿主提供,不接管(呼应 hermetic 策略 non-goals) +- Bootstrap 底座必须预构建(bootstrap 循环的技术必然) +- coupled-ABI 群走 platform 而非 range(basd on glibc symver 单向兼容的物理事实) + +**故意不作的假设**(留待后续): +- mcpp 对非 C/C++ 语言的具体覆盖形态 +- platform manifest 承载的确切数据结构 +- fingerprint 具体字段 +- range 语法(采纳 semver 还是自定义) +- 迁移 deadline + +--- + +## 12. 一句话总括 + +> **xlings 是"用户态 Linux 发行版",三分层 = kernel(host) + xlings(底座) + mcpp(主构建工具)。分发采用预构 + 源构混合(canonical build 保等价性,fingerprint 判命中)。多 glibc / 可组合发行版通过 platform manifest 承载,不做自由组合。消费者依赖从硬钉演进为 deps range + runtime_floor + platform_membership 三轴模型。与 hermetic 策略正交互补,合并后形成完整生态定位。** diff --git a/.agents/docs/2026-08-05-subos-minimum-design.md b/.agents/docs/2026-08-05-subos-minimum-design.md new file mode 100644 index 00000000..2d0e08d1 --- /dev/null +++ b/.agents/docs/2026-08-05-subos-minimum-design.md @@ -0,0 +1,577 @@ +# xlings subos slice 1 详细设计:Configuration 基质补完 + +**日期**: 2026-08-05 +**类型**: 详细设计(detailed design)—— slice 1 可实施规格 +**范围**: xlings 侧新增最小机制,让 `compat.mesa` 端到端可装可跑;不引入 platform 抽象、不引入新 CLI 动词 +**关联**: +- `2026-08-05-userspace-distro-hermetic-strategy.md`(运行时边界策略) +- `2026-08-05-ecosystem-three-tier-and-composable-distro.md`(生态定位讨论) + +**状态**:所有 schema 与 CLI 决策已定案(2026-08-05 会话收敛);实施细节标 [OPEN] 待具体施工时定 + +--- + +## 0. TL;DR + +**问题**:issue #352 类场景——xlings 的 subos 缺 Configuration 基质,GL/Vulkan/字体等子系统的发现协议(env vars + config 目录)没有 per-subos 承载。 + +**Slice 1 目标**:在 xlings 侧引入最小机制补完 Configuration 基质,让 `compat.mesa` payload 装进 subos 后 GLFW 用户程序能跑起来。 + +**核心决策**: +- Manifest 新增单块 `subos_info`(对齐 `xlings subos info` CLI) +- 字段:`schema_version` / `runtime` / `envs` / `created_at` / `created_by` +- **不引入 platform 抽象**——`runtime` 就是一个 xvm binding 字符串,self-describing +- **不新增 CLI 动词**——复用 `xlings subos use --shell` 与 `--cmd`,slice 1 扩展它们的行为 +- 新 xpkg API:`subos.env{}` 让 pkg config() 声明 env + +**Slice 1 明确不做**: +- Platform manifest 概念 +- 多 runtime(多 glibc 等)—— 生态阻塞项另议,slice 1 只需字段就位 +- Capabilities_host / activation_hooks +- `xlings subos export/import/diff/snapshot/migrate` +- 跨 shell/OS 完备性(bash 优先,fish/zsh 之后补) +- 自动 schema upgrade + +--- + +## 1. 分析框架:三层基质(分析工具,不是 schema 分块) + +用户程序在 subos 里跑起来需要三层基质齐备: + +| 层 | 内容 | xlings 现状 | Slice 1 | +|---|---|---|---| +| **1. Bootstrap** | PT_INTERP + CRT + libc/runtime | ✅ glibc pkg + elfpatch | 不动 | +| **2. Discovery** | PATH + RPATH + xvm binding | ✅ xvm.add/xvm.files + elfpatch | 不动 | +| **3. Configuration** | env vars + config 发现约定 | 🟡 无 per-subos 承载 | **补完** | + +三层是分析工具,不是 schema 组织形式——schema 只有一个 `subos_info` 块。 + +--- + +## 2. 最小 subos 不变量 + +`xlings subos new ` 完成后,以下条件必须同时成立(doctor 用同一套判据检查): + +**文件系统**: +- I1. `$XLINGS_HOME/subos//` 目录存在 +- I2. `$XLINGS_HOME/subos//.xlings.json` 文件存在且合法 JSON +- I3. xvm 状态目录已初始化(即使为空) + +**Manifest 结构**: +- I4. 文件顶级存在 `subos_info` 对象 +- I5. `subos_info.schema_version` == `1` +- I6. `subos_info.runtime` 是非空字符串,格式 `@` +- I7. `subos_info.envs` 是对象(可空 `{}`) +- I8. `subos_info.created_at` / `created_by` 是非空字符串 + +**注册状态**: +- I9. `$XLINGS_HOME/.xlings.json` 的 `subos` 注册表包含 `` 条目 + +**基质就绪**(逻辑判据,非文件检查): +- I10. `subos_info.runtime` 引用的 pkg 在 xvm 里已注册(Bootstrap 可用) +- I11. `envs` 段每个 key 对应的 pkg 在 xvm 里已装(env 与 xvm 一致) + +**违反任一 → doctor 报错并给出修复动作**。Reporter 与 repairer 用同一函数,遵循 xlings 生态"reporter/repairer 谓词一致"原则。 + +**"空集合 ≠ 缺失"原则**:所有可增长段(`envs`)在 `new` 时初始化为空对象/数组,不省略 key。避免 reader 处理 optional。 + +--- + +## 3. subos_info schema(完整) + +### 3.1 位置 + +**Per-subos** `.xlings.json`: +- Home 全局 subos:`$XLINGS_HOME/subos//.xlings.json` +- 项目 subos:`/.xlings/subos//.xlings.json` + +**不出现在**: +- 项目根 `.xlings.json`(那里 `subos` 是字符串选择器) +- Home 根 `.xlings.json`(那里 `subos` 是注册表 map) + +### 3.2 完整结构 + +```json +{ + "workspace": { ... }, // 现有段,不动 + + "subos_info": { // slice 1 新增 + "schema_version": 1, + "runtime": "glibc@2.39", + "envs": { }, + "created_at": "2026-08-05T14:23:11Z", + "created_by": "xlings 0.4.71" + } +} +``` + +### 3.3 字段规约 + +| 字段 | 类型 | 必填 | 说明 | +|---|---|---|---| +| `schema_version` | integer | ✅ | 当前 = 1,未来通过 probe pattern 演化 | +| `runtime` | string | ✅ | xvm binding 格式 `@`,self-describing | +| `envs` | object | ✅ | Per-pkg env 声明,可空 `{}` | +| `created_at` | string (ISO-8601) | ✅ | wall clock,信息字段 | +| `created_by` | string | ✅ | 创建时 xlings 版本,信息字段 | + +### 3.4 runtime 字段:self-describing 约定 + +``` +runtime: "glibc@2.39" → Linux + glibc 家族 +runtime: "musl@1.2.5" → Linux + musl 家族 +runtime: "wasi-libc@0.1" → WASI 家族(未来) +runtime: "macos_sdk@14.5" → macOS 家族(未来) +runtime: "ucrt@10.0.22000" → Windows 家族(未来) +``` + +**家族信息通过 pkg 名字前缀由 xlings 侧辅助函数派生**,不作为独立字段冗余存储: + +``` +family_of(runtime_binding) → + glibc@* → linux--glibc + musl@* → linux--musl + wasi-libc@* → wasm32-wasi + macos_sdk@* → darwin- + ucrt@* → windows--ucrt + 默认 → "unknown" +``` + +映射表在 xlings 侧代码中维护,新 OS 加入时更新。**Slice 1 只需 `glibc@*` 一项**。 + +### 3.5 envs 段结构 + +```json +"envs": { + "compat.mesa@25.0.0": [ + { "var": "LIBGL_DRIVERS_PATH", "op": "set", "value": "${pkgdir}/lib/dri" }, + { "var": "__EGL_VENDOR_LIBRARY_DIRS", "op": "set", "value": "${pkgdir}/share/glvnd/egl_vendor.d" }, + { "var": "XDG_DATA_DIRS", "op": "prepend", "value": "${pkgdir}/share" } + ] +} +``` + +**Schema**: +- Key:pkg binding,格式 `@` +- Value:数组,每项 `{ var: string, op: enum, value: string }` + +**op 允许取值**: +| op | 语义 | Slice 1 | +|---|---|---| +| `set` | 无条件设置(覆盖既有值) | ✅ 实现 | +| `prepend` | 前置到既有值(冒号分隔) | ✅ 实现 | +| `set-if-unset` | 仅在未设时设置 | ⬜ 未来 | +| `append` | 追加到既有值 | ⬜ 未来 | + +**Slice 1 只实现 `set` 与 `prepend`**——足够覆盖 GL/Vulkan/fontconfig 类场景。 + +### 3.6 value 占位符 + +在 emit 时(activate 或 run)展开: + +| 占位符 | 展开为 | +|---|---| +| `${pkgdir}` | 该 pkg 的 install 目录绝对路径 | +| `${subosdir}` | 该 subos 的根目录绝对路径 | +| `${home}` | 用户 home 目录 | +| `${xlings_home}` | `$XLINGS_HOME` | + +**占位符是可迁移性的关键**——manifest 内 value 不含机器特定绝对路径。 + +### 3.7 Provider-scoped 所有权 + +`envs` 以 pkg binding 为 key,与 xvm.add / xvm.files 同一所有权模型: +- pkg install 时其 config() 通过 `subos.env{}` 写入 +- pkg uninstall 时 xlings 自动删除该 key 整段 +- 不允许 cross-pkg 声明(A pkg config() 声明 B pkg 的 env) + +### 3.8 冲突语义(activate 时) + +同一变量被多个 pkg 声明时: + +| 场景 | 规则 | +|---|---| +| 多个 `set` 同变量 | doctor 报 warning;运行期以 **binding 序后到者**为准 | +| 多个 `prepend` | **binding 序后到者**在前(靠近变量头) | +| `set` 与 `prepend` 混合 | `set` 生效,`prepend` 忽略,doctor 报 warning | +| 用户已在 shell 里 export | **用户值优先**,pkg 声明不覆盖(见 UC-1) | + +> **施工修正(2026-08-05)**:本节原写"以**装机顺序**后到者为准"。**装机顺序不在 +> manifest 里**,要实现它就得加一个字段,而那个字段唯一的作用是让结果取决于历史 —— +> 两台机器持有逐字节相同的 manifest 会导出不同的值。这与 SH-1(可分享性)直接矛盾: +> 一份只在写它的机器上成立的描述不算描述。 +> +> 改为按 **binding 字典序**。它由数据本身决定,可复现,且让 manifest 成为完整答案。 +> 已由单测 `DoesNotDependOnDeclarationOrder` 钉住。 + +**冲突检测在 activate 时进行**,不在 write 时(简化 write 路径,写多次不 fail)。 + +--- + +## 4. xlings subos new(最小改动) + +### 4.1 CLI 签名 + +现有: +``` +xlings subos new [--storage ] [--image-size ] [--from ] +``` + +Slice 1 新增: +``` +xlings subos new [--runtime ] [其他现有 flag] + +--runtime subos 的 runtime pkg binding,如 "glibc@2.39" + 缺省:xlings 内嵌 fallback(当前为 "glibc@2.39") +``` + +**不新增其他 flag**——`--platform` / `default_platform` 不做。 + +### 4.2 执行流程 + +``` +xlings subos new [--runtime ] + +Step 1: 前置校验 + - 未占用、命名合法 + - Home 目录可写 + +Step 2: 解析 runtime + --runtime CLI 参数 → 若无 → xlings 编译时嵌入默认(slice 1: "glibc@2.39") + → 记为 R + +Step 3: 校验 R 可用 + - R 对应的 pkg 在索引里存在 + - 若目标机未装 → prompt "install now? [Y/n]"(默认 Yes) + +Step 4: 物理创建 + - mkdir -p $XLINGS_HOME/subos// + - 初始化 xvm 状态子目录 + - 若需要 → 装 R + +Step 5: 写 subos_info 到 subos//.xlings.json + { + "workspace": {}, + "subos_info": { + "schema_version": 1, + "runtime": "", + "envs": {}, + "created_at": "", + "created_by": "xlings " + } + } + +Step 6: 更新 home .xlings.json subos 注册表 + +Step 7: 验证不变量 I1~I11,任一失败回滚 + +Step 8: 输出提示 + "Created subos '' with runtime . + Enter with: xlings subos use + Run one command: xlings subos use --cmd '' + Emit env script: xlings subos use --shell" +``` + +### 4.3 错误场景 + +| 场景 | 退出码 | 提示 | +|---|---|---| +| `` 已存在 | 1 | "use another name or `subos remove`" | +| `--runtime` 引用的 pkg 索引里无 | 2 | "no such runtime pkg; check `xlings install --search`" | +| Runtime 与 host arch 不容 | 3 | 提示 arch 详情 | +| Runtime 未装且用户拒绝安装 | 4 | 提示手动安装 | +| Step 4~6 部分成功后失败 | 5 | 反向回滚;若回滚失败,提示手动清理路径 | + +失败必须显式,不 silent-success(memory: silent-success pattern)。 + +--- + +## 5. subos.env{} xpkg API + +### 5.1 Lua API 签名 + +```lua +subos.env{ + var = "LIBGL_DRIVERS_PATH", + op = "set", -- 或 "prepend" + value = "${pkgdir}/lib/dri", + binding = "compat.mesa@" .. pkginfo.version() +} +``` + +### 5.2 参数 + +| 参数 | 类型 | 必填 | 说明 | +|---|---|---|---| +| `var` | string | ✅ | 环境变量名 | +| `op` | string | ✅ | `set` 或 `prepend`(slice 1 只这两种) | +| `value` | string | ✅ | 值,支持 §3.6 占位符 | +| `binding` | string | ✅ | Provider 标识,必须与本 pkg 一致(cross-pkg 拒绝) | + +### 5.3 语义 + +- 调用时 append 到 `subos_info.envs[binding]` 数组 +- 幂等:同 binding 内 (var, op, value) 三元组已存在则不重复写 +- 写入原子:走 atomic write 路径(避免 flock 场景;memory: atomic write vs flock) +- **write 时不做冲突检测**——只在 activate 时统一按 §3.8 规则解冲突 + +### 5.4 老 client 兼容(V2 spec probe 模式) + +> **施工修正(2026-08-05)**:本节原写的是 `if subos.env then`。**那个探针在所有 +> client 上恒为真**,必须用 `type(...)`: +> +> ```lua +> if type(subos.env) == "function" then +> subos.env{ var = "...", op = "set", value = "...", binding = binding } +> else +> -- 老 client 无此能力,走原有路径 +> end +> ``` +> +> 原因在 libxpkg `prelude.lua::import()`:未知模块返回 permissive proxy stub, +> 它的 `__index` 对任意 key 都返回一个 truthy 的可调用 table。老 client 没有 +> `xim.libxpkg.subos`,`subos.env` 于是是个 truthy proxy —— recipe 走"新分支", +> 调用静默无效,装完什么也没发生。 +> +> V2 spec 里 `if xvm.files then` 成立,只因为 `xvm` 是**已存在的模块**,老 client +> 上那个 *field* 才真的是 `nil`。缺失的 *module* 永远不是。stub 是带 `__call` 的 +> **table**,真函数才是 `function`,`type()` 是唯一能区分两者的判据。 +> +> 已由 e2e 用真实旧二进制钉住(见 §11)。规则已写进 +> `xim-pkgindex/docs/V2/xpackage-spec.md`。 + +### 5.5 卸载路径 + +pkg `uninstall()` 无需显式清理 env——xlings 侧 uninstall 流程会按 binding 删除对应 envs key。 + +--- + +## 6. 复用 `xlings subos use`(不新增命令) + +### 6.1 现有 CLI(不动) + +``` +xlings subos use + --global Persist the active SubOS + --shell [KIND] Emit shell activation code + --sandbox [BACKEND] Enable sandbox + --cmd Run one command + --keep / --no-keep Namespace keeper + --ttl Keeper idle timeout + --gpu Expose GPU devices +``` + +### 6.2 Slice 1 扩展:`--shell` 输出内容 + +现有 `--shell` 已经 emit shell 激活源码。**Slice 1 在其输出里追加 subos_info.envs 展开的 export 语句**: + +Bash 输出示例: +```bash +# 现有 xvm binding 相关 export(不动) +export PATH="..." + +# Slice 1 新增:subos_info.envs 展开 +export LIBGL_DRIVERS_PATH="/home/user/.xlings/subos/default/pkg/compat.mesa-25.0.0/lib/dri" +export __EGL_VENDOR_LIBRARY_DIRS="/home/user/.xlings/subos/default/pkg/compat.mesa-25.0.0/share/glvnd/egl_vendor.d" +export XDG_DATA_DIRS="/home/user/.xlings/subos/default/pkg/compat.mesa-25.0.0/share${XDG_DATA_DIRS:+:$XDG_DATA_DIRS}" +``` + +**注入规则**: +1. 读 subos_info.envs,按 binding 迭代 +2. 按 §3.8 冲突规则汇总 +3. 展开 §3.6 占位符 +4. 探测目标 shell(bash / fish / zsh),emit 对应语法 +5. **不覆盖用户已 export 的变量**——通过 `${VAR:+...}`(bash)、`$VAR`(fish)保持既有值 + +### 6.3 Slice 1 扩展:`--cmd` 环境注入 + +现有 `--cmd ` 已经在 subos 上下文里执行命令。**Slice 1 在 exec 前把 subos_info.envs 计算后的 env dict 注入进程**: + +- 保留当前进程 env 作为基线 +- 叠加 envs 段计算结果(用户已设的 env 优先,pkg 声明不覆盖) +- `execvpe(cmd, argv, effective_env)`——不启动 shell + +### 6.4 Shell 覆盖(Slice 1 范围) + +| Shell | Slice 1 支持 | +|---|---| +| bash | ✅ 主 emitter | +| zsh | ✅(与 bash 兼容语法) | +| fish | ✅(独立 emitter) | +| PowerShell/cmd | ⬜ 未来 | + +用户可通过 `--shell [KIND]` 显式指定(已在现有 CLI)。 + +### 6.5 UC-2 输出实际生效清单 + +`--shell` stderr 输出实际注入的变量清单(不进 eval,只做提示): + +``` +[xlings] subos: default (runtime glibc@2.39) +[xlings] env: 3 vars from 1 packages +[xlings] compat.mesa: LIBGL_DRIVERS_PATH, __EGL_VENDOR_LIBRARY_DIRS, XDG_DATA_DIRS +``` + +避免用户 `eval` 后不清楚加了什么。 + +--- + +## 7. 卸载 + doctor 集成 + +### 7.1 pkg uninstall 路径 + +Xlings 侧 uninstall 流程新增一步: + +``` +uninstall(pkg): + 1. 调用 pkg uninstall() hook(既有) + 2. 从 xvm 撤销 binding(既有) + 3. [新增] 从 subos_info.envs 移除 @ 为 key 的整段 + 4. Atomic write .xlings.json + 5. 若移除后 envs 为空 → 保留 {}(遵循 I7) +``` + +### 7.2 doctor 检查项 + +| # | 检查 | 判据 | 修复动作 | +|---|---|---|---| +| D1 | Manifest 结构完整 | I1~I8 全成立 | 引导用户重建 subos | +| D2 | envs binding 与 xvm 一致 | 每个 envs key 对应 pkg 在 xvm 已装 | 移除孤儿 envs 段 or 提示重装 | +| D3 | envs value 占位符可解 | `${pkgdir}` 等指向目录存在 | 提示重装对应 pkg | +| D4 | envs 变量无冲突 | 同 var 无多个 `set` | warning(非 error) | +| D5 | runtime 引用一致 | subos_info.runtime 引用的 pkg 在 xvm 已装 | 提示装 runtime | + +**Reporter 与 repairer 用同一函数**(memory: reporter/repairer predicate drift),不允许两处描述判据。 + +--- + +## 8. 契约条款 + +### 8.1 SH(可分享性) + +- **SH-1** 值可迁移:`subos_info` 内所有 value 必须用占位符,禁止绝对路径、机器特定 ID +- **SH-2** envs 是 derived cache:pkg config() 是权威源;recreate 时重跑 config() 而非直接 apply + +### 8.2 UC(用户覆盖权) + +- **UC-1** 用户 env 优先:用户在 shell 里 export 的变量,activate/run 不覆盖 +- **UC-2** 无静默选择:activate 输出实际生效清单到 stderr +- **UC-3** 遵循 v2 spec probe pattern:新字段/API 通过 probe 兼容老 client,不用 `min_xlings` + +--- + +## 9. Slice 1 边界 + +### 9.1 已完成(不动) + +- glibc pkg + elfpatch PT_INTERP/RPATH(Bootstrap) +- xvm.add / xvm.files provider-scoped 注册(Discovery) +- xvm shim + PATH(Discovery) +- `xlings subos new/list/remove/info/use/stop` 基础 CLI +- `~/.xlings.json` subos 注册表 + activeSubos 机制 +- doctor 框架 + +### 9.2 Slice 1 新增(xlings 侧) + +代码估算 ~800~1200 行: + +| 组件 | 估算 | +|---|---| +| `subos_info` 读写 + schema 校验 | ~200 | +| `subos.env{}` xpkg API + provider-scoped 记账 | ~150 | +| `xlings subos use --shell` env emitter (bash/zsh/fish) | ~200 | +| `xlings subos use --cmd` env 注入 | ~100 | +| `xlings subos new --runtime` + 不变量校验 | ~150 | +| Uninstall envs 清理 | ~60 | +| Doctor D1~D5 检查项 | ~120 | +| 单元 + 集成测试 | ~250 | + +### 9.3 Slice 1 新增(xim-pkgindex 侧) + +- `compat.mesa` recipe(独立设计,见 task #8~9) +- 一份预构建 tarball 上传 xlings-res +- dlopen 闭包一次性人工审计 +- 一份 bwrap 空 host smoke test + +### 9.4 明确不做(non-goals) + +- Platform manifest / 可组合发行版 +- 多 runtime(多 glibc / musl 并存)—— 生态阻塞项另议 +- `capabilities_host` / `activation_hooks` 段 +- `xlings subos export/import/diff/snapshot/migrate` CLI +- 用户 `--force` flag(UC-1 是原则,CLI 后续 slice 补) +- 自动 schema upgrade +- PowerShell / cmd shell 支持 +- macOS / Windows 上的 runtime(仅数据模型预留,不实现) +- NVIDIA 闭源栈 payload(slice 2) + +--- + +## 10. 开放问题 + +| # | 问题 | 阻塞谁 | +|---|---|---| +| O1 | `--shell fish` 与 bash 的语义等价 fixture 是否需要? | 测试 | +| O2 | Runtime 未装且用户拒绝 `--yes` 时的具体退出码语义 | subos new UX | +| O3 | Doctor D4(env 冲突)是 warning 还是 error?判据严格性统一 | doctor | +| O4 | `compat.mesa` payload 实际 size(可能超 800MB,是否分包)| pkgindex 侧 slice 1 | +| O5 | subos 迁移/克隆场景下 `${pkgdir}` 是否需要在 subos_info 里持久化解析结果? | 未来 subos 迁移能力 | + +不阻塞 slice 1 起步,施工时定。 + +--- + +## 11. 施工顺序建议 + +1. **Task #2 定案**:本 spec §3 schema 已定,可直接进入实施 +2. **Task #4**:实现 `subos.env{}` API + 写入路径(2~3 天) +3. **Task #3**:`xlings subos new --runtime` + 不变量校验(2~3 天) +4. **Task #5**:`xlings subos use --shell` env emitter (bash 优先,~5 天) +5. **Task #6**:`xlings subos use --cmd` env 注入(2 天) +6. **Task #7**:uninstall 清理 + doctor D1~D5(3~5 天) +7. 并行 **Task #8~10**:pkgindex 侧 compat.mesa(2~3 周) +8. 集成 + e2e smoke test(1 周) + +**总估算 4~6 周**。 + +--- + +## 13. 施工实况(2026-08-05) + +Slice 1 的 xlings + libxpkg 两侧已实现并验证。落地拆分见 +`2026-08-05-subos-slice1-landing-plan.md`。 + +### 13.1 与本设计的三处偏离 + +| # | 设计原文 | 实际 | 原因 | +|---|---|---|---| +| 1 | 探针 `if subos.env then` | `type(subos.env) == "function"` | 前者恒真,见 §5.4 修正 | +| 2 | 冲突按装机顺序 | 按 binding 序 | 装机顺序不在 manifest 里,见 §3.8 修正 | +| 3 | (未提及)`default` subos | `self init` 也写 manifest | 它不走 `subos::create`,否则新能力在**所有人实际在用的那个 subos 上**全程静默无效,同时成为老 home 的迁移路径 | + +### 13.2 设计未覆盖、施工中必须补的 + +- **早退分支**:`process_xvm_operations_` 在没有 xvm 注册项时会提前 return。 + 只声明 env、不注册任何 xvm 节点的包会被静默丢弃 —— env 消费必须放在早退**之前**。 +- **doctor 渲染器的 `default: break;`**:新 FindingKind 不加 case 就不打印, + 查得出、报不出。 +- **卸载按包名而非精确 binding 匹配**:removal 会经命名空间和 group 成员解析版本, + 安装时记录的 binding 在卸载点无法可靠重建;同一包装过两个版本时会留下另一段。 + +### 13.3 覆盖 + +- 单测 `tests/unit/test_subos_manifest.cpp` — 29 例(schema/不变量/占位符/冲突/确定性) +- libxpkg `tests/test_executor.cpp` — 6 例(op 收集/校验/探针语义) +- E2E-60 `subos_env_declaration_test.sh` — 装→声明落盘→`--shell`→`--cmd`→用户覆盖→doctor→卸载清段 +- E2E-61 `subos_env_probe_compat_test.sh` — 真实旧二进制上的双读数差分 + +### 13.4 仍未做 + +§9.4 全部照旧。§10 的 O1(fish/bash 等价 fixture)已由 E2E-60 覆盖 bash 侧, +fish emitter 只有构造正确性、无行为断言;O4、O5 仍开放,随 compat.mesa 一起定。 + +--- + +## 12. 一句话总括 + +> **Slice 1 = 一个 `subos_info` 块 + 一个 `subos.env{}` API + `xlings subos use --shell/--cmd` 两处扩展。总代码 ~800~1200 行。补完 Configuration 基质,让 compat.mesa 端到端可跑;所有更大的野心(platform、多 runtime、canonical build 等)明确挂到后续 slice,不阻塞当前工作。** diff --git a/.agents/docs/2026-08-05-subos-slice1-landing-plan.md b/.agents/docs/2026-08-05-subos-slice1-landing-plan.md new file mode 100644 index 00000000..d9b90a20 --- /dev/null +++ b/.agents/docs/2026-08-05-subos-slice1-landing-plan.md @@ -0,0 +1,176 @@ +# subos slice 1 落地计划:任务拆分与跨仓依赖 + +**日期**: 2026-08-05 +**类型**: 落地计划(landing plan) +**上游**: `2026-08-05-subos-minimum-design.md`(详细设计,schema/CLI 已定案) +**目标**: 把 slice 1 从设计变成已发布、可验证的能力 + +--- + +## 0. 设计文档的一处修正(施工前必须先改) + +设计文档 §5.4 给出的老 client 探针是: + +```lua +if subos.env then ... else ... end +``` + +**这个探针在所有 client 上恒为真,是错的。** + +原因在 libxpkg 的 `prelude.lua::import()`:未知模块返回一个 permissive +proxy stub —— 它的 `__index` 对任意 key 都返回一个可调用的 proxy。老 client +没有 `xim.libxpkg.subos`,`import()` 于是返回 stub,`subos.env` 是一个 truthy +的 proxy table。recipe 会走"新路径",调用静默无效,装完什么也没发生。这正是 +xlings 生态反复出现的 silent-success 形态。 + +V2 spec 里 `if xvm.files then` 之所以成立,是因为 `xvm` 是**已存在的模块**—— +老 client 上 `xvm` 是真模块,`xvm.files` 才真的是 `nil`。**新模块不适用这个 +写法。** + +**新模块的探针必须判类型**: + +```lua +if type(subos.env) == "function" then ... end +``` + +proxy 是带 `__call` 元方法的 **table**,真函数才是 `function`,两者可区分。 + +这条规则要写进 `xim-pkgindex/docs/V2/xpackage-spec.md`,并由一个跑真实旧 +xlings 二进制的 e2e 断言(模板:`tests/e2e/xvm_files_probe_compat_test.sh`)。 + +--- + +## 1. 为什么必须动 libxpkg(而不是只动 xlings) + +考虑过三条路,只有一条站得住: + +| 方案 | 结论 | +|---|---| +| `subos.env` 放 xim-pkgindex 的 `libs/`(像 `xim.pkgindex.sysroot`) | ❌ index 是滚动更新的,模块在所有 client 上都存在,而消费 op 的 C++ 在 xlings 里。探针恒真 → silent-success | +| xlings 在执行 recipe 前往 `_LIBXPKG_MODULES` 注入模块 | ❌ 模块表在 libxpkg 的 `load_stdlib` 里写死,没有宿主注入口 | +| **放 libxpkg** | ✅ Lua 函数与消费它的 C++ 静态链接进同一个 xlings 二进制,"函数在不在"就等于"这个 client 支不支持"。单一真相源 | + +所以 **libxpkg 0.0.48 是硬前置**,不是可选项。 + +--- + +## 2. 跨仓依赖图 + +``` +┌─ libxpkg ─────────────────────────────────────────────┐ +│ #12 subos.lua + XvmOp{var,value,mode} + 模块注册 + 测试 │ +└───────────────┬───────────────────────┬───────────────┘ + │ │ + ┌────────────▼──────────┐ ┌────────▼────────────────┐ + │ #13 发布链(串行) │ │ #14 本地 registry 播种 │ + │ bump 0.0.48 → merge │ │ ~/.mcpp/registry/data/ │ + │ → tag → gtc 镜像 │ │ …/0.0.48 + .mcpp_ok │ + │ → mcpp-index 条目 │ │ (让 xlings 侧立刻能编) │ + │ → publish-artifact │ └────────┬────────────────┘ + └────────────┬──────────┘ │ + │ ┌────────▼──────────────────┐ + │ │ #15 manifest.cppm 核心 │ + │ │ schema/不变量/占位符/合并 │ + │ └────────┬──────────────────┘ + │ │ + │ ┌──────┬───────┴──────┬──────────┐ + │ ▼ ▼ ▼ ▼ + │ #16 #17 #18 #19 + │ new install/ use doctor + │ --runtime uninstall --shell D1–D5 + │ env ops --cmd + │ └──────┴───────┬──────┴──────────┘ + │ ▼ + │ #20 测试(unit + e2e) + │ │ + #21 规范/文档(可全程并行)──────────┤ + │ │ + └───────────┬───────────┘ + ▼ + #22 版本号 + 单 PR + CI 全绿 + ▼ + #23 release + gtc 补 gitcode 资源 + ▼ + #24 xlings subos 真实验证 +``` + +**并行窗口**:#14 一旦完成,#15→#16/#17/#18/#19 与 #13 的发布链完全并行。 +#21 全程可并行。真正的串行瓶颈只有 #12 → #14 → #15 → 四路扇出 → #20 → #22。 + +--- + +## 3. 各仓改动面 + +### 3.1 openxlings/libxpkg(#12, #13) + +| 文件 | 改动 | +|---|---| +| `src/lua-stdlib/xim/libxpkg/subos.lua` | 新增。`M.env{var, op, value, binding}` | +| `src/xpkg-executor.cppm` | `XvmOp` 加 `var` / `value` / `mode`;`xvm_operations()` 读取;`load_stdlib` 注册 `subos` 模块 | +| `xmake.lua` | embed 列表加 `subos_lua` | +| `src/xpkg-lua-stdlib.cppm` | 由 xmake 从 `.lua` **自动生成**,不手改 | +| `tests/test_executor.cpp` | subos_env op 收集用例 | +| `mcpp.toml` | `0.0.47` → `0.0.48` | + +**命名冲突处理**:用户面参数是 `op = "set"|"prepend"`,而 `XvmOp.op` 已经 +表示 op 类别。内部 entry 用 `op = "subos_env"` + `mode = "set"|"prepend"`, +用户面不变。 + +### 3.2 openxlings/xlings(#14–#20, #22, #23) + +| 文件 | 改动 | +|---|---| +| `src/core/subos/manifest.cppm` | **新增**,slice 1 的地基 | +| `src/core/subos.cppm` | `create()` 写 subos_info;`run()` 解析 `--runtime`;`use_emit_shell` / `use_spawn_shell` 注入 env | +| `src/core/xim/installer.cppm` | 消费 `subos_env` op;uninstall 按 binding 清段 | +| `src/core/xself/doctor.cppm` | D1–D5 | +| `src/cli/spec.cppm` | `subos new --runtime` 帮助文本 | +| `mcpp.toml` | 版本号 + `mcpplibs.xpkg = "0.0.48"` | +| `tests/unit/`、`tests/e2e/` | 见 #20 | + +### 3.3 openxlings/xim-pkgindex(#21, #24) + +| 文件 | 改动 | +|---|---| +| `docs/V2/xpackage-spec.md` | `subos.env` API + **新模块探针规则** | +| 验证载体 recipe | #24 需要一个真实声明 env 的包 | + +### 3.4 mcpplibs/mcpp-index(#13 的一环) + +`pkgs/x/xpkg.lua` 加 0.0.48 条目,GLOBAL + CN 双 URL + sha256,**三个平台块都要写**。 + +--- + +## 4. 已知的坑(来自既往教训,不是推测) + +1. **`bump-index` 和 `mirror-binaries` 都吞掉自己的失败**(结尾 `|| echo "…(non-blocking)"`)。两个都可能报绿而什么都没干。#23 必须查产物:xlings-res 的版本条目 + `latest.ref`,以及用 **GET(不是 HEAD)** 验证 gitcode 资源。 +2. **mcpp index 是 artifact 不是 git clone**。合并到 mcpp-index 后要等 `publish-artifact.yml`,客户端还有 TTL —— `rm -rf ~/.mcpp/registry/data/` 强制刷新。手改缓存里的 `pkgs/**` 无效。 +3. **`mcpp build` 不重建测试二进制,只有 `mcpp test` 会**。改完测试跑 `mcpp build` 会留下过期 `test_main`,红相位可能假绿。 +4. **libxpkg CI 会静默腐烂**,失败读起来像网络错误(`fetch 'mcpplibs.capi.lua@0.0.3' failed`)。`test_executor` 另有 4 个既存 elfpatch 失败,CI 用 `--gtest_filter=-ExecutorTest.ApplyElfpatchAuto_*` 过滤。 +5. **隔离 home 测试不要预置 `data/`** —— 会让 index 变成 symlink,差分断言变得不可证伪。 +6. **绝不写入被 flock 的文件**:rename 换掉 inode 会静默破坏锁。subos_info 的原子写要与 home config 的锁路径区分清楚。 +7. **`quick_install` 忽略 `XLINGS_HOME`**,验证 release 要用 tarball 自带的 `self install`。 + +--- + +## 5. 本次落地的范围边界 + +**做**:libxpkg 0.0.48 + xlings slice 1 全部(设计文档 §9.2)+ 规范文档 + 发布 + 真实验证。 + +**不做**(设计文档 §9.4 已明确,外加): +- `compat.mesa` payload 本身(#8–#10,设计文档 §11 估 2–3 周,独立并行轨) +- 多 runtime 并存 +- platform 抽象 + +#24 的验证载体因此不是 compat.mesa,而是一个小体量的真实 recipe —— 目的是证明 +**机制**端到端通,payload 规模是另一件事。 + +--- + +## 6. 完成判据 + +- [ ] libxpkg 0.0.48 已发布,mcpp-index 可解析,gitcode 资源 GET 可下载 +- [ ] xlings 单 PR 含全部 slice 1 实现 + 测试 + 版本号,CI 全绿 +- [ ] 新老 client 探针差分测试通过(老二进制走 legacy 分支且不 silent-success) +- [ ] release 产物已验证:版本条目、`latest.ref`、gitcode 资源三者一致 +- [ ] `xlings subos` 真实跑通:new → install → use → env 生效 → uninstall 清段 → doctor 干净 diff --git a/.agents/docs/2026-08-05-userspace-distro-hermetic-strategy.md b/.agents/docs/2026-08-05-userspace-distro-hermetic-strategy.md new file mode 100644 index 00000000..e0fdd158 --- /dev/null +++ b/.agents/docs/2026-08-05-userspace-distro-hermetic-strategy.md @@ -0,0 +1,304 @@ +# xlings 作为用户态发行版:hermetic 策略与 GPU/GLIBC 边界设计 + +**日期**: 2026-08-05 +**类型**: 策略 (strategy) — 尚未细化到实施计划,先立框架 +**触发**: mcpp-community/mcpp#352(Fedora 44 上 GLFW/OpenGL 程序静默 exit 255,三层根因:观测性 + `compat.glx-runtime` symlink 到宿主 `/usr/lib` 32-bit 库 + 沙盒 glibc 2.39 与宿主 Mesa 要求 GLIBC_2.43 冲突) +**关联**: +- `2026-05-22-subos-sandbox-gpu-passthrough.md`(sandbox 内 GPU 设备节点透传,本文的**下层配套**) +- `2026-06-21-linux-root-usability-survey.md`(用户身份/权限边界) +- `2026-06-26-multiarch-package-description-design.md`(payload 的多架构表达,本文所有新包都要沿用) +- mcpp-index `pkgs/c/compat.glx-runtime.lua`(现存"半 hermetic"的典型样本) +- xim-pkgindex `pkgs/g/glibc.lua`(只有 2.39,`XLINGS_RES` payload) + +--- + +## 0. TL;DR + +**将 xlings 明确定位为"用户态 Linux 发行版",执行"能不依赖宿主就不依赖宿主"策略。** +物理上不可自带的资源(kernel syscall、wire protocol、GPU vendor 私有栈)显式列入 `capabilities_host` 白名单,除此之外一律走 xlings 内自建 payload。glibc 版本冲突是"任何宿主 `.so` 被拽进 xlings loader 进程"的**症状**,而非独立问题;策略解决"依赖宿主的范围",症状自动消失。 + +**近期最优选择(不锁死路径)**: +- Intel/AMD GPU 场景 → 走 hermetic-full(自带 Mesa/libdrm/GLVND) +- NVIDIA 场景 → 走版本化 payload(参考 Flatpak `org.freedesktop.Platform.GL.nvidia-XXX-YY`) +- Kernel module 编译**不接管**(留给宿主/DKMS),xlings 只负责 userspace 侧对齐 + +**保留的未来迁移路径**(不现在做,但**不封堵**): +- linker namespace 分层(参考 Steam pressure-vessel `libcapsule`)—— 长期解 NVIDIA 无版本 payload 兜底 +- 接管 kernel module 编译(参考 NixOS `hardware.nvidia.package`)—— 若 xlings 自建 kernel/subos 深化时再评 +- Release 套装模型(参考 NixOS channel / Flatpak runtime `//24.08`)—— 版本轴纪律 + +**明确不做**: +- 不做完整 Linux distro(不发行 init、systemd、包管理器替代)—— xlings 是**"用户态 app 侧的发行版"**,不是 rootfs 侧的发行版 +- 不接管 kernel(不发行 kernel、不管 firmware 装载、不管 udev) + +--- + +## 1. 深度分析:问题与现状 + +### 1.1 issue #352 的三层根因(具体表症) + +| 层 | 归属 | 缺陷性质 | 修复形态 | +|---|---|---|---| +| L1 观测性静默 | `sudoevolve/EUI-NEO` `glfw_app_main.cpp` | 无 `glfwSetErrorCallback` + 各失败路径无 stderr | 上游 30 行 PR | +| L2 glx-runtime 布局假设错 | `mcpplibs/mcpp-index` `compat.glx-runtime.lua:63-127` | 候选目录顺序把 `/usr/lib` 放最后,`ln -sf` 覆盖 64-bit symlink 为 32-bit | 单 patch | +| L3 glibc 版本鸿沟 | `xim-pkgindex` `glibc.lua` + mcpp 引擎 | 沙盒 glibc 2.39 vs 宿主 Mesa 要求 GLIBC_2.43;`LD_LIBRARY_PATH=/usr/lib64` 变通亦不可用(ld.so 与 libc.so.6 版本必须成对) | **架构级** | + +**L1、L2 是叶子,L3 是骨** —— 只补 L1/L2 会让用户从"静默 exit 255"变成"至少能看到 GLIBC_2.43 not found",但**不解决根本问题**。 + +### 1.2 L3 的真实性质:glibc 是症状,不是成因 + +- 崩溃的必要条件是:xlings loader(`xim-x-glibc/2.39/ld-linux-x86-64.so.2`)加载了**一份别人编译时锁死 glibc 版本的 `.so`** +- issue #352 的具体路径:用户 exe `dlopen("libGLX.so.0")` → symlink → 宿主 `libGLX_mesa.so.0` → 后者 `DT_NEEDED libc.so.6` 且 symver 表含 `GLIBC_2.43` → 我们的 2.39 libc 提不出该符号 → SIGABRT +- **如果 loader 全程只加载"xlings 自建、都链到 2.39"的 `.so`,`GLIBC_*` 这个词根本不会出现在任何 error message 里** + +这个观察的**推论**: +- 问题不是"glibc 太老要追新",而是"何时应该跨越到宿主库"这个边界不清晰 +- 追 glibc 版本是**追不完的**(发行版每半年一版,rawhide 永远领先),且**老 mcpp 二进制不受益**(PT_INTERP 编译时烙定) +- **正解是收敛"跨越边界"的地方**,不是提高沙盒 glibc 的版本 + +### 1.3 生态里已经在"实质做 hermetic"但没写成明文 + +现有历史决策(散落在若干 issue/PR)已经**在无意贯彻这个策略**,只是从未被明确提炼: + +| 记录 | 决策 | 隐含语义 | +|---|---|---| +| xim-pkgindex termux musl DNS | musl-static `getaddrinfo` 读不到 `$PREFIX/etc/resolv.conf` → 手写 UDP DNS | 不依赖宿主 `/etc/resolv.conf` | +| mcpp e2e elfpatch 前缀替换 | 测试 elfpatch 曾"写穿 `ln -sf`"烙死 `mktemp -d` 路径 → 改为前缀替换 | payload 不能借宿主路径当锚 | +| mcpp macOS 静态 libc++ SIOF | 归档成员 `.init_array.` 排最后 → 静态 vs 动态是分界线 | 自带 C++ 运行时的 ABI 独立性 | +| xim-pkgindex llvm22 slim asset | slim 删了 `libatomic.so.1`、libc++ 硬依赖它、沙盒 glibc loader**不回退系统** → 崩 | loader 不回退系统 = 隐式 hermetic | +| mcpp Windows DLL deploy | PE 无 RPATH → 拷 DLL 到 exe 旁,`mcpp run` 会塞 PATH 掩盖漏洞 → 直接执行 `.exe` 才是发布态 | Windows 上早就是 hermetic | + +**结论**:"用户态发行版 + hermetic 优先"不是新方向,是**给已经在做的事一个明确的名字和判据**,以后新决策不用重新推导。 + +--- + +## 2. 策略陈述:边界与优先级 + +### 2.1 一句话原则 + +> **只有 kernel syscall 和显式声明的 wire protocol 允许穿越到宿主;所有 `.so`(除 GPU vendor 私有栈)必须来自 xlings 自建 payload。** + +### 2.2 允许穿越的边界(**枚举而非例外**) + +| 边界类型 | 具体 | 为什么无法自带 | +|---|---|---| +| kernel syscall | 全部 syscall,含 `/dev/dri/*` ioctl、`/dev/snd/*`、futex、io_uring、`/dev/nvidia*` ioctl | 内核 ABI,唯一入口 | +| wire protocol(socket 字节流)| X11(`/tmp/.X11-unix/`)、Wayland(`$XDG_RUNTIME_DIR/wayland-*`)、D-Bus(`/run/user/*/bus`)、PulseAudio/PipeWire(socket)、CUPS | 协议稳定,进程外通信,**不引入宿主 libc** | +| 硬件/固件 | `/lib/firmware`(kernel 自动加载) | 硬件绑定 | +| GPU vendor 强制耦合 | NVIDIA 专有栈(`libGLX_nvidia.so`、`libEGL_nvidia.so`、`libcuda.so` 等) | vendor 用私有 IOCTL 与 `nvidia.ko` 校验版本;`.so` 与内核模块必须严格对齐 | + +### 2.3 不允许穿越的(即使"就一次也不行") + +- 任何 `/usr/lib*` `/lib*` 下的 `.so`,含 `libc`、`libssl`、`libGL`(Mesa)、`libX11`、`libpulse`、`libdbus`、`libcups`、`libfontconfig`、`libfreetype`、`libgio` … +- `/usr/share/*` 下的**运行期数据**:zoneinfo、locale、CA bundle、fontconfig cache、图标主题 +- xlings 有观点的 `/etc/*` 项:`/etc/ssl/certs`(信任链应归 xlings 管)、`/etc/resolv.conf`(DNS 已在 musl 场景踩过) + +### 2.4 每一条穿越必须**在包元数据里写理由** + +新增字段 `capabilities_host`,每一项带 `reason`: + +```lua +capabilities_host = { + { "kernel.drm", reason = "GPU ioctl,唯一入口" }, + { "wire.wayland", reason = "compositor 是宿主实例" }, + { "hw.nvidia-gpu", reason = "vendor 私有 IOCTL 版本绑定" }, -- 仅在探到 nvidia.ko 时激活 +} +``` + +引擎侧在 target 分析阶段:**任何 dlopen 目标不在包自身链接闭包 ∪ `capabilities_host` 隐含允许集**,报错拒绝。当前 `compat.glx-runtime.lua:76-80` 那种"直接 `ln -sf /usr/lib64/*`"就会在这一步被拦下。 + +--- + +## 3. 场景枚举(不只是 GPU) + +hermetic 策略的判据要在多个场景上自洽,不是只解 GPU: + +| 场景 | 现状 | hermetic 策略下的形态 | +|---|---|---| +| **GPU/GL** (issue #352) | `compat.glx-runtime` symlink 宿主 | Intel/AMD 自带 Mesa;NVIDIA 版本化 payload | +| **音频** (PulseAudio/PipeWire) | 未系统性处理;示例包偶发使用 | `compat.libpulse`(client-side)+ `capabilities_host = wire.pulse`(server 是宿主) | +| **字体/i18n** | 依赖宿主 `/usr/share/fonts`、`fontconfig` | `compat.fontconfig` + `xim-x-fonts-default` payload(挑一套 Noto/DejaVu) | +| **HTTPS/CA** | 静态 OpenSSL 编译时 OPENSSLDIR=/etc/ssl → Debian 挂 | `xim-x-ca-certificates` payload(Mozilla bundle),客户端库改从这里读 | +| **DNS** | musl-static 已改手写 UDP | 已 hermetic,补写策略文档确认此为**原型** | +| **DBus** | 应用 dlopen 宿主 libdbus | `compat.libdbus`(client)+ `wire.dbus` | +| **打印** (CUPS) | 未涉及 | 若涉及则 `compat.libcups` + `wire.cups` | +| **timezone/locale** | 用宿主 `/usr/share/zoneinfo` `/usr/share/locale` | `xim-x-tzdata` + `xim-x-locale` 小 payload | +| **CLI 工具运行** | 已 hermetic(自带 glibc/gcc/llvm) | 现状即目标态,无变化 | + +**重要**:并非所有场景要一次性做完。策略给的是**判据**;每次新包 review 时按判据决定 —— 这样以后不会有人再走"这次借一下 host"的短路。 + +--- + +## 4. 方案对比:GPU 边界四条路 + +聚焦最难的 GPU 场景,横向对比业界四种解法: + +### 4.1 Flatpak — 版本化 runtime,一驱动一份 + +- **机制**:extension `org.freedesktop.Platform.GL.nvidia-575-42-01` 等,一版一份;首次运行探 `/proc/driver/nvidia/version` 拉对应 +- **关键细节**:extension 里的 `.so` **重新链接到 Flatpak runtime 的 glibc**,不是 NVIDIA `.run` 原样;所以沙盒里 loader 加载它 glibc 天然对齐 +- **代价**:CDN 存几百份(每份 ~200MB)、老驱动 extension 永远不能删 +- **优点**:工程简单、5+ 年验证、有现成 CI 模板可参考 + +### 4.2 NixOS — 驱动是 nix derivation + +- **机制**:`hardware.nvidia.package` 从 NVIDIA `.run` 抽源码,**用 nixpkgs 的 gcc + kernel headers 一起 build**,产出 `nvidia.ko` + 所有 `.so`,挂 `/run/opengl-driver/lib/` +- **glibc 对齐方式**:整个 NixOS 是**同一版 nixpkgs channel**,glibc 全局一致 +- **代价**:xlings 必须接管 kernel module 编译(每 kernel 版本 × 每 driver 版本的构建矩阵)+ nixpkgs 级别的社区规模 +- **不适合近期**:接管 kernel module 是 xlings 团队规模两个数量级之外的承诺 + +### 4.3 传统发行版(Debian/Fedora)— 没沙盒,自然没冲突 + +- **机制**:`nvidia-driver-575` / `akmod-nvidia`,DKMS 在每次 kernel 升级时重 build,userspace `.so` 按发行版 glibc 编,落 `/usr/lib*` +- **glibc 对齐方式**:整个系统只有一个 glibc,不存在两份 +- **代价**:xlings 需要成为**真正的发行版**(rootfs 级、init 级、包管理器级),超出定位 + +### 4.4 Steam pressure-vessel + libcapsule — linker namespace 分层 + +- **机制**:利用 glibc `dlmopen` + `LM_ID_NEWLM`,让**同一进程内同时容纳两份 glibc** —— 沙盒 glibc 归应用,宿主 glibc 归 GL 驱动,由 `libcapsule` proxy 做跨 namespace marshaling +- **glibc 对齐方式**:根本不需要对齐,通过 linker namespace 隔离 +- **代价**:GL 近千函数需生成 proxy stub(可自动化,但每次 GL 版本升要跟);目前只 Valve/Collabora 在维护 +- **技术上最优雅**,但**工程量对小团队不现实** —— 作为未来兜底方案保留 + +### 4.5 决策矩阵 + +| 方案 | 存储代价 | 首次运行延迟 | 工程复杂度 | Kernel 侧责任 | xlings 定位契合度 | +|---|---|---|---|---|---| +| Flatpak | 高(几百份) | 高(下 ~200MB) | **低** | 无 | ★★★★ | +| NixOS | 中 | 无 | 中 | **接管 `.ko` 编译** | ★★(超定位) | +| 传统发行版 | 低 | 无 | 低(政策成本高) | 全接管 | ★(超定位) | +| pressure-vessel | 低(宿主单份) | 无 | **极高** | 无 | ★★★★★(但远期) | + +**近期最优 = Flatpak-style,兼顾工程可行与定位契合**。 + +**长期不封堵 pressure-vessel-style**,一旦 GLVND 抽象已就位、包元数据已齐备,`libcapsule` 只是"实现细节替换",不需要改架构。这就是"最优选择 + 未来可迁移"的具体含义。 + +--- + +## 5. 推荐近期方案(P0 → P1) + +### 5.1 P0 — 三个最小落地(2-4 周) + +1. **明文写策略**(本文即是初稿),在 xlings 主 repo `docs/design/` 下发一份 policy doc +2. **补 L1/L2 止血**: + - L1:向 sudoevolve/EUI-NEO 上游提 PR,`glfwSetErrorCallback` + 各失败点 stderr(**约 30 行**) + - L2:改 `compat.glx-runtime.lua` 候选目录顺序 + ELF class 探测(约 10 行)—— **不解决 L3,但让用户至少能看到真实错误** +3. **发 `compat.mesa` payload**(Intel/AMD 路径的第一个 non-trivial 落地): + - 内容:libgallium + libGLX_mesa + libEGL_mesa + libdrm(必要项)+ shader compiler 依赖的 llvm-runtime + - 大小预算:< 500MB 压缩后 + - 依赖闭包必须审:任何 `.so` `DT_NEEDED` 指向宿主 = 缺包 + - **capabilities**:`opengl.glvnd`、`opengl.mesa.driver` + - **capabilities_host**:`kernel.drm`(附 reason) + +### 5.2 P1 — 4-6 周 + +4. **加 `capabilities_host` 元数据字段** + 索引侧校验: + - `xim-pkgindex` 与 `mcpp-index` 的 lua 包描述加字段 + - 校验脚本(参考 `check_version_pins.sh`、`check_cross_package_refs.lua` 的做法):静态扫描每个包的 dlopen list,与 `capabilities_host` 允许集比对,不匹配报错 +5. **NVIDIA 分支**(Flatpak-style): + - `xim-x-nvidia-driver-575` `xim-x-nvidia-driver-570` 等一组 payload + - xlings-res CI 自动从 NVIDIA `.run` 抽包 → 重链到 xim-x-glibc → 发 release + - 首装时探测:读 `/proc/driver/nvidia/version` → 拉对应 + - **kernel module 不接管**:如果用户 kernel 上没有 `nvidia.ko`,报明确错误("请通过发行版安装 nvidia driver kernel module"),不试图代劳 +6. **CI 空-host 校验**: + - bwrap 构一个 host 只有 `/dev /proc /sys /tmp` 的 rootfs + - 在其中跑 `mcpp build && mcpp test && mcpp run`(选一组 examples) + - 任何 `LD_DEBUG=libs` 显示的 search path 命中 `/usr/lib*` 都失败 + - 参考 mcpp `link-argv-max-arg-strlen` 教训:**没有真实覆盖 = 没有验证** + +### 5.3 P2 — 3-6 个月 + +7. **其他 hermetic 缺口**(章节 3 的表)按优先级补:CA bundle、fontconfig、pulseaudio client → dbus → cups +8. **Release 套装模型**(参考 Flatpak runtime `//24.08`): + - 引入 `xlings-platform-2026.08` 这样的**同步版本集合** + - Mesa + libdrm + libX11 + xim-x-glibc + libc++ 等作为**一组**发布,内部互测通过 + - `.xlings.json` 可 pin 平台版本而非每个包单独 pin +9. **考虑 linker namespace 兜底**(P3,不承诺): + - 如果 NVIDIA payload 矩阵开始成本失控(比如 Chinese fork 分支太多),或用户"就是插了没见过的 GPU"的场景增多,启动 `libcapsule` 抽象研究 + +--- + +## 6. 未来可迁移路径(**不封堵、不预实现**) + +以下都是**在近期方案架构上可增量演化**的迁移方向,现在**不做**但**不封堵**: + +### 6.1 → NixOS-style 接管 kernel module + +- **触发条件**:xlings subos 深化到自建 kernel(现在只是 sandbox 层),或**社区规模足以维护 driver × kernel 编译矩阵** +- **迁移复杂度**:`xim-x-nvidia-driver-XXX` payload 现有元数据结构可原地扩展,增加 `kmod` 段,由 xlings 侧编译 +- **保留的选择**:即使做了,也可以保留 Flatpak-style 作为 fallback(用户没有 build toolchain 时) + +### 6.2 → pressure-vessel-style linker namespace + +- **触发条件**:NVIDIA payload 矩阵爆炸,或 pre-built 二进制场景增多 +- **迁移复杂度**:GLVND 层已经就位(方案 4.1 已在做 GLVND 分发),`libcapsule` 只替换分发层实现 +- **保留的选择**:可以只对 GL 一族做 capsule 化,其他包(音频、DBus)仍用普通 hermetic + +### 6.3 → Release 套装 → Rolling / LTS 双通道 + +- **触发条件**:企业/教育场景要求"两年不动" +- **迁移复杂度**:`xim-platform-YYYY.MM` 的语义已经支持"套装",加个 `.lts` 标签即可 +- **保留的选择**:rolling 用户零感知 + +### 6.4 → 完整 rootfs distro + +- **触发条件**:xlings 定位主动扩大(未来讨论,不现在承诺) +- **迁移复杂度**:显著,涉及 init/包管理器/kernel +- **本文明确**:P0-P2 不做这一步 + +--- + +## 7. 明确不做(non-goals) + +以下事项即使技术可行也**不做**,避免任务蔓延: + +- **不发行 kernel**:kernel 由宿主提供,`capabilities_host = kernel.*` 是永久边界 +- **不接管 NVIDIA kernel module 编译**(近期):DKMS 交给宿主发行版 +- **不发行 systemd/init**:xlings 是用户态,不管进程 1 +- **不做 Flatpak-style 应用沙盒**:xlings 是**开发工具生态**,不是应用分发;subos 有 sandbox 但那是隔离,不是应用运行时 +- **不追 glibc 版本**:2.39 就是 2.39,不发 2.40/2.41/...(除非有独立强 need);策略从"追新"转向"收敛边界" + +--- + +## 8. 开放问题(等实施前决策) + +1. **NVIDIA payload 命名 schema**: + - Flatpak: `nvidia-575-42-01`(3 段) + - 是否需要区分 open-kernel-module vs proprietary?(NVIDIA 从 R515 起有 open kernel module,但 userspace 仍闭源) + - 建议参考 Flatpak 命名,后续按需扩展 +2. **capabilities_host 声明的 review 边界**: + - 谁批准新增一条? + - 是否需要每季度 audit 一次 `capabilities_host` 集合避免膨胀? +3. **兼容"混合环境"用户**: + - 一台机器同时有 Intel iGPU 和 NVIDIA dGPU(Optimus) + - GLVND 分发能处理(靠 `__NV_PRIME_RENDER_OFFLOAD` 或 `DRI_PRIME`),但需要 payload 都装齐 + - 探测策略:探到 `/proc/driver/nvidia/version` 就装 NVIDIA payload,同时也装 Mesa payload +4. **payload 首次下载体验**: + - Mesa payload ~500MB、NVIDIA payload ~200MB,首次运行 GUI 项目会明显停顿 + - 是否 seed 到 xlings quick_install?或让用户显式 `xlings install compat.mesa`? + - 建议参考 Flatpak:第一次 `mcpp run` 触发下载,显示进度条,明确告知 +5. **`compat.mesa` 与 `xim-x-mesa` 的归属**: + - mcpp-index 归属 → 用户项目侧 + - xim-pkgindex 归属 → 工具链侧 + - GL 库属于 runtime,理论上两侧都可以;建议放 **xim-pkgindex**(更接近"用户态发行版底座") +6. **老 mcpp 二进制的兼容**: + - 已发布 mcpp 二进制 PT_INTERP 是烙定的 + - 新策略下产出的二进制才 hermetic + - 需要**发版说明明确**:某个 mcpp 版本起 GUI 项目走新路径 + +--- + +## 9. 与已有 xlings 文档的关系 + +- **不冲突**:本文是**上位策略**,不推翻任何已有实施设计 +- **补齐**: + - `2026-05-22-subos-sandbox-gpu-passthrough.md` 解决"sandbox 里能看到 GPU 设备节点",本文解决"看到之后跑什么 GL 库"—— 一上一下,配套 + - `2026-06-26-multiarch-package-description-design.md` 提供 payload 的多架构表达,本文新增的所有 payload 沿用 + - `2026-06-21-linux-root-usability-survey.md` 讨论权限边界,本文的 `capabilities_host` 是"资源边界"版本,理念一致 +- **更新**:本文落地后,`compat.glx-runtime.lua` 那种"直接 symlink 宿主"的模式应逐步淘汰,mcpp-index 的 `docs/CONTRIBUTING` 应加"新包不得直接 symlink 宿主 `.so`"约束 + +--- + +## 10. 一句话总括 + +> **xlings 是"用户态 Linux 发行版",执行"能不依赖宿主就不依赖宿主"策略。glibc 冲突是策略未明确时的症状;把边界写清楚,症状自然消失。近期抄 Flatpak-style 的做法把 GPU 边界收敛;长期不封堵 NixOS-style / pressure-vessel-style / rolling+LTS 等演化路径。** diff --git a/docs/spec/xlings-json-schema.md b/docs/spec/xlings-json-schema.md index 2fe34a6e..fb7e91a7 100644 --- a/docs/spec/xlings-json-schema.md +++ b/docs/spec/xlings-json-schema.md @@ -159,9 +159,48 @@ | 字段 | 类型 | 说明 | |------|------|------| | `workspace` | `object` | 已安装工具的版本状态（见下方格式） | +| `subos_info` | `object` | SubOS 自身的描述：运行时与环境声明（见下方） | | `storage` | `string` | 存储模式：`"shared"`、`"tmpfs"`、`"image"` | | `imageSize` | `string` | 当 storage 为 `image` 时的磁盘映像大小（如 `"4G"`） | +### subos_info（2026.8.5+） + +`workspace` 记录的是这个 SubOS **装了什么**；`subos_info` 记录的是它**是什么** —— +二进制针对哪个运行时构建，以及进入它的进程需要哪些环境变量。 + +```json +"subos_info": { + "schema_version": 1, + "runtime": "glibc@2.39", + "envs": { + "compat.mesa@25.0.0": [ + { "var": "LIBGL_DRIVERS_PATH", "op": "set", "value": "${pkgdir}/lib/dri" }, + { "var": "XDG_DATA_DIRS", "op": "prepend", "value": "${pkgdir}/share" } + ] + }, + "created_at": "2026-08-05T14:23:11Z", + "created_by": "xlings 2026.8.5.1" +} +``` + +| 字段 | 类型 | 说明 | +|------|------|------| +| `schema_version` | `integer` | 当前为 `1` | +| `runtime` | `string` | 运行时 binding `@`，自描述（`glibc@2.39` 即 Linux/glibc）。由 `xlings subos new --runtime` 指定 | +| `envs` | `object` | 以**声明包的 binding** 为键。包卸载时 xlings 删除整段；recipe 不写清理代码 | +| `created_at` / `created_by` | `string` | 创建时间与创建者版本 | + +`envs` 的值由包在 `config()` 里通过 `subos.env{}` 写入（见 xim-pkgindex 的 +xpackage-spec V2）。**值必须使用占位符** —— `${pkgdir}` / `${subosdir}` / +`${home}` / `${xlings_home}` —— 写死绝对路径会让这份描述只在写它的机器上成立。 +占位符在进入 SubOS 时展开。 + +`envs` 为空时保留 `{}`,不省略键：缺失和空是两种写法、同一个意思,会让每个读者都要处理两遍。 + +进入 SubOS(`xlings subos use`)时这些变量被注入进程；**用户自己已 export 的值优先**, +`set` 不覆盖它,`prepend` 仍然与它拼接。`xlings self doctor` 检查这一段的完整性、 +与已装包的一致性、占位符可解性,以及是否有多个包争抢同一变量。 + ### workspace 条目格式（SubOS） SubOS 工作区中每个工具支持三种值形式： @@ -228,6 +267,13 @@ SubOS 工作区中每个工具支持三种值形式： { "storage": "image", "imageSize": "4G", + "subos_info": { + "schema_version": 1, + "runtime": "glibc@2.39", + "envs": {}, + "created_at": "2026-08-05T14:23:11Z", + "created_by": "xlings 2026.8.5.1" + }, "workspace": { "gcc": { "active": "16.1.0", diff --git a/mcpp.lock b/mcpp.lock index dd0ef538..6518a6b1 100644 --- a/mcpp.lock +++ b/mcpp.lock @@ -33,7 +33,7 @@ hash = "fnv1a:3465dd0bd5d7aa20" [package."mcpplibs.xpkg"] namespace = "mcpplibs" -version = "0.0.47" -source = "index+mcpplibs@0.0.47" -hash = "fnv1a:0cd8e39df748a206" +version = "0.0.48" +source = "index+mcpplibs@0.0.48" +hash = "fnv1a:2e16f253753e1665" diff --git a/mcpp.toml b/mcpp.toml index 78fd25d3..49d31f93 100644 --- a/mcpp.toml +++ b/mcpp.toml @@ -1,6 +1,6 @@ [package] name = "xlings" -version = "2026.8.4.2" +version = "2026.8.5.1" description = "Universal package management infrastructure tool with SubOS isolation" license = "Apache-2.0" repo = "https://github.com/openxlings/xlings" @@ -39,7 +39,7 @@ libarchive = "3.8.7" [dependencies.mcpplibs] cmdline = "0.0.2" -xpkg = "0.0.47" +xpkg = "0.0.48" tinyhttps = "0.2.9" capi.lua = "0.0.3" diff --git a/src/cli/spec.cppm b/src/cli/spec.cppm index a805335d..95b513c6 100644 --- a/src/cli/spec.cppm +++ b/src/cli/spec.cppm @@ -63,7 +63,7 @@ const CommandSpec& root() { {"config", "Show or modify configuration", {}, {}, {{"--lang ", "Set language"}, {"--mirror ", "Set mirror"}, {"--add-xpkg ", "Add package recipe"}, {"--index-repo ", "Add index repository"}}, {}}, {"subos", "Manage SubOS environments", {}, {}, {}, { - {"new", "Create a SubOS", {}, {{"name", "SubOS name", true}}, {{"--storage ", "shared, tmpfs or image"}, {"--image-size ", "Image size"}, {"--from ", "Fork source"}}, {}}, + {"new", "Create a SubOS", {}, {{"name", "SubOS name", true}}, {{"--storage ", "shared, tmpfs or image"}, {"--image-size ", "Image size"}, {"--from ", "Fork source"}, {"--runtime ", "Runtime binding, e.g. glibc@2.39"}}, {}}, {"use", "Enter a SubOS", {}, {{"name", "SubOS name", true}}, {{"--global", "Persist the active SubOS"}, {"--shell [KIND]", "Emit shell activation code"}, {"--sandbox [BACKEND]", "Enable sandbox (bwrap or proot on Linux)"}, {"--cmd ", "Run one command"}, {"--keep", "Keep the namespace keeper"}, {"--no-keep", "Disable the namespace keeper"}, {"--ttl ", "Keeper idle timeout"}, {"--gpu", "Expose GPU devices (bwrap only)"}}, {}}, {"list", "List SubOS environments", {"ls"}, {}, {}, {}}, {"remove", "Remove a SubOS", {"rm"}, {{"name", "SubOS name", true}}, {}, {}}, diff --git a/src/core/config.cppm b/src/core/config.cppm index 8cfd17fa..a1716798 100644 --- a/src/core/config.cppm +++ b/src/core/config.cppm @@ -13,7 +13,7 @@ import xlings.core.xvm.db; namespace xlings { export struct Info { - static constexpr std::string_view VERSION = "2026.8.4.2"; + static constexpr std::string_view VERSION = "2026.8.5.1"; static constexpr std::string_view REPO = "https://github.com/openxlings/xlings"; }; diff --git a/src/core/subos.cppm b/src/core/subos.cppm index e39d2d23..2a2993a5 100644 --- a/src/core/subos.cppm +++ b/src/core/subos.cppm @@ -31,6 +31,7 @@ import xlings.core.xim.commands; // auto_install_backend_ needs cmd_install import xlings.core.subos.keeper; import xlings.core.subos.gpu; import xlings.core.subos.sandbox; +import xlings.core.subos.manifest; // Leaf module (std + json only). Same source for "is this a global option" // that the CLI validator uses, so the two cannot disagree about `--yes`. import xlings.cli.spec; @@ -140,11 +141,48 @@ void update_current_symlink_(EventStream& stream, // ───────────────────────────────────────────────────────────────────── +// Give a subos directory a `subos_info` block, or leave the one it has. +// +// Idempotent, and called from every path that produces a subos directory +// (create, new_from, and the migration of a subos made before this block +// existed). A subos without it violates invariant I4 and cannot be described, +// checked or entered with its environment. +// +// The block is added even when `.xlings.json` already exists, which is the +// difference from the surrounding code: the file predates the block, so +// "the file is there" does not mean "the subos describes itself". +bool ensure_subos_info_(const fs::path& dir, std::string_view runtime) { + auto json = read_config_json_(dir / ".xlings.json"); + if (!json.is_object()) json = nlohmann::json::object(); + if (!json.contains("workspace")) json["workspace"] = nlohmann::json::object(); + + // Only replace a block that is absent or unusable. Rewriting a valid one + // would discard the envs a package declared into it. + if (manifest::validate_block(json).empty()) return true; + + json[std::string(manifest::BLOCK)] = manifest::make_block( + runtime, std::format("xlings {}", Info::VERSION)); + try { + write_config_json_(dir / ".xlings.json", json); + } catch (const std::exception& e) { + log::error("failed to write subos manifest {}: {}", + (dir / ".xlings.json").string(), e.what()); + return false; + } + return true; +} + // Create a subos. V6: storage mode is a creation-time property // (`--storage image|tmpfs|shared`). Non-shared modes force sandbox // entry at use-time. The sandbox-private dirs are laid down lazily. +// +// `runtime` is the subos's declared runtime binding ("glibc@2.39"): what its +// binaries are built against. Empty means the built-in default. It is a +// creation-time property because changing it after the fact would invalidate +// every payload already installed. export int create(const std::string& name, const fs::path& customDir, sandbox::StorageMode storage, const std::string& imageSize, + const std::string& runtime, EventStream& stream) { auto& p = Config::paths(); @@ -157,6 +195,21 @@ export int create(const std::string& name, const fs::path& customDir, return 1; } + // Checked before anything is laid down. A malformed runtime that only + // surfaced at write time would leave a registered subos that cannot + // satisfy its own invariants. + const std::string effectiveRuntime = + runtime.empty() ? std::string(manifest::DEFAULT_RUNTIME) : runtime; + if (!manifest::is_binding(effectiveRuntime)) { + stream.emit(ErrorEvent{ + .code = ErrorCode::InvalidInput, + .message = "invalid --runtime '" + effectiveRuntime + + "' (expected @, e.g. glibc@2.39)", + .recoverable = false, + }); + return 1; + } + for (char c : name) { if (!std::isalnum(static_cast(c)) && c != '_' && c != '-') { stream.emit(ErrorEvent{ @@ -197,7 +250,17 @@ export int create(const std::string& name, const fs::path& customDir, j["storage"] = sandbox::storage_to_string_(storage); if (storage == sandbox::StorageMode::Image) j["imageSize"] = imageSize; + j[std::string(manifest::BLOCK)] = manifest::make_block( + effectiveRuntime, std::format("xlings {}", Info::VERSION)); write_config_json_(subosConfig, j); + } else if (!ensure_subos_info_(dir, effectiveRuntime)) { + stream.emit(ErrorEvent{ + .code = ErrorCode::Internal, + .message = "failed to write the subos manifest for '" + name + "'", + .recoverable = false, + .hint = "check write permission on " + subosConfig.string(), + }); + return 1; } // Image mode: create sparse ext4 image @@ -266,18 +329,58 @@ export int create(const std::string& name, const fs::path& customDir, return 1; } + // The subos is registered; check it can actually satisfy the invariants + // before saying so. A creation that reports success and leaves a subos + // that doctor immediately condemns is the failure mode this whole slice + // exists to remove -- "it happened" and "it worked" must not look alike. + if (auto findings = manifest::validate(dir); !findings.empty()) { + std::string detail; + for (const auto& f : findings) { + if (!detail.empty()) detail += "; "; + detail += std::string(manifest::describe(f.kind)); + if (!f.detail.empty()) detail += " (" + f.detail + ")"; + } + // Roll back to the state before the command: the registry entry first, + // since that is what makes the name unusable a second time. + (void)update_home_config(p.homeDir, [&](nlohmann::json& json) { + if (json.contains("subos") && json["subos"].is_object()) + json["subos"].erase(name); + return true; + }); + std::error_code rmec; + fs::remove_all(dir, rmec); + stream.emit(ErrorEvent{ + .code = ErrorCode::Internal, + .message = "subos '" + name + "' did not come out valid: " + detail, + .recoverable = false, + .hint = rmec + ? "rolled back the registry entry, but " + dir.string() + + " could not be removed -- delete it before retrying" + : "nothing was left behind; retry, or report this", + }); + return 1; + } + nlohmann::json payload; payload["name"] = name; payload["dir"] = dir.string(); payload["storage"] = sandbox::storage_to_string_(storage); + payload["runtime"] = effectiveRuntime; stream.emit(DataEvent{"subos_created", payload.dump()}); return 0; } -// Back-compat overload (no storage argument → shared). +// Back-compat overloads. Callers that predate the runtime argument get the +// built-in default, and callers that predate storage get shared as before. +export int create(const std::string& name, const fs::path& customDir, + sandbox::StorageMode storage, const std::string& imageSize, + EventStream& stream) { + return create(name, customDir, storage, imageSize, "", stream); +} + export int create(const std::string& name, const fs::path& customDir, EventStream& stream) { - return create(name, customDir, sandbox::StorageMode::Shared, "50G", stream); + return create(name, customDir, sandbox::StorageMode::Shared, "50G", "", stream); } // ───────────────────────────────────────────────────────────────────── @@ -424,7 +527,8 @@ fs::path locate_base_pkg_(const PkgRef& ref) { export int new_from(const std::string& name, const fs::path& customDir, sandbox::StorageMode storage, const std::string& imageSize, - const std::string& fromSpec, EventStream& stream) { + const std::string& fromSpec, const std::string& runtime, + EventStream& stream) { auto& p = Config::paths(); fs::path baseDir; @@ -498,7 +602,8 @@ export int new_from(const std::string& name, const fs::path& customDir, // Create target subos via standard `create`. This sets up // bin/lib/usr/generations, writes initial .xlings.json, optionally // creates home.img, and registers the subos. - if (auto rc = create(name, customDir, storage, imageSize, stream); rc != 0) { + if (auto rc = create(name, customDir, storage, imageSize, runtime, stream); + rc != 0) { return rc; } @@ -526,6 +631,18 @@ export int new_from(const std::string& name, const fs::path& customDir, subosCfg["imageSize"] = imageSize; else subosCfg.erase("imageSize"); + // Same restoration, same reason. copy_tree_ replaced the manifest create() + // wrote with the base's, and a base built before subos_info existed has + // none -- which would leave the fork registered and failing its own + // invariants. A base that does carry one keeps it: it describes the very + // content that was just copied in, envs included. + if (!subosCfg.contains("workspace")) + subosCfg["workspace"] = nlohmann::json::object(); + if (!manifest::validate_block(subosCfg).empty()) { + subosCfg[std::string(manifest::BLOCK)] = manifest::make_block( + runtime.empty() ? manifest::DEFAULT_RUNTIME : runtime, + std::format("xlings {}", Info::VERSION)); + } write_config_json_(subosCfgPath, subosCfg); // Re-mint subos shims (they may have been clobbered by copy_tree_ @@ -623,6 +740,87 @@ inline std::string rebuild_path_for_subos_(const std::string& orig_path, return out; } +// Where a provider's payload lives, for `${pkgdir}`. +// +// A binding is `@` and carries no namespace, while the store +// directory is `-x-` (or a bare `` for the primary index). So +// the bare spelling is tried first and the namespaced ones are found by scan. +// The scan is over the handful of bindings that actually appear in `envs`, +// not over the whole store. +// +// Returning empty is meaningful: expansion then leaves `${pkgdir}` in place +// rather than collapsing the value to a host path, and doctor D3 reports it. +inline fs::path pkgdir_for_binding_(std::string_view binding) { + const auto at = binding.find('@'); + if (at == std::string_view::npos) return {}; + const std::string name(binding.substr(0, at)); + const std::string version(binding.substr(at + 1)); + if (name.empty() || version.empty()) return {}; + + const auto store = Config::paths().dataDir / "xpkgs"; + std::error_code ec; + + if (auto direct = store / name / version; fs::is_directory(direct, ec)) + return direct; + + if (!fs::is_directory(store, ec)) return {}; + const auto suffix = "-x-" + name; + for (const auto& entry : platform::dir_entries(store)) { + if (!entry.is_directory(ec)) continue; + if (!entry.path().filename().string().ends_with(suffix)) continue; + if (auto candidate = entry.path() / version; + fs::is_directory(candidate, ec)) { + return candidate; + } + } + return {}; +} + +inline manifest::Placeholders placeholders_for_(const fs::path& subosDir) { + return manifest::Placeholders{ + .subosdir = subosDir, + .home = platform::get_home_dir(), + .xlings_home = Config::paths().homeDir, + .pkgdir_of = pkgdir_for_binding_, + }; +} + +// The variables a subos exports, resolved and ready to apply. +// +// Empty for a subos with no declarations, which is every subos until a package +// makes one -- so both call sites below stay silent in the common case. +inline std::vector subos_env_for_(const std::string& name) { + const auto dir = Config::subos_dir(name); + auto doc = manifest::read_document(dir); + if (!doc) return {}; + return manifest::resolve(manifest::parse(*doc), placeholders_for_(dir)); +} + +// UC-2: say what was injected. +// +// To stderr, always -- the `--shell` path's stdout is eval'd by the caller's +// shell, and this is a report, not code. Without it the user pipes an opaque +// blob into `eval` and cannot tell which package changed what. +inline void report_injected_env_(const std::string& subosName, + const std::vector& vars) { + if (vars.empty()) return; + std::set providers; + for (const auto& v : vars) + providers.insert(v.providers.begin(), v.providers.end()); + + std::println(stderr, "[xlings] subos {}: {} env var(s) from {} package(s)", + subosName, vars.size(), providers.size()); + for (const auto& v : vars) { + if (v.unresolved) { + std::println(stderr, "[xlings] {} — unresolved path, skipped " + "(run `xlings self doctor`)", v.var); + } else if (v.conflicted) { + std::println(stderr, "[xlings] {} — declared by {} packages, " + "conflicting", v.var, v.providers.size()); + } + } +} + } // namespace use_detail_ // Internal — not exported. `xlings subos use --global ` and @@ -695,6 +893,17 @@ int use_emit_shell(const std::string& name, bool is_pwsh = (shell_kind == "pwsh" || shell_kind == "powershell" || shell_kind == "ps1" || shell_kind == "ps"); + // The subos's own declared environment (GL driver paths, EGL vendor dirs, + // and whatever else a package needs a *user's* binary to see). Emitted + // after the xvm/PATH lines so a declaration cannot displace them. + // + // UC-1 -- a variable the user already exported wins. The emitted code + // tests the live variable rather than what this process happens to see: + // `--shell` output is frequently captured once and eval'd later, in a + // shell whose environment has moved on. + const auto envVars = use_detail_::subos_env_for_(name); + use_detail_::report_injected_env_(name, envVars); + if (is_fish) { std::println(R"(set -gx XLINGS_ACTIVE_SUBOS "{}";)", name); std::println(R"(set -gx XLINGS_BIN "{}";)", bin_dir.string()); @@ -702,6 +911,20 @@ int use_emit_shell(const std::string& name, // bin. fish's $PATH is a list, so we use string match -v. std::println(R"(set -gx PATH "{}" (string match -v -r "^{}/subos/[^/]+/bin$" -- $PATH);)", bin_dir.string(), p.homeDir.string()); + for (const auto& v : envVars) { + if (v.unresolved) continue; + // R"SH(...)SH": the fish source below contains `)"`, which ends a + // plain R"(...)" literal early -- and the truncation compiles, + // because what is left is still a valid string. + if (v.op == manifest::OP_PREPEND) { + std::println( + R"SH(if set -q {0}; set -gx {0} "{1}:${0}"; else; set -gx {0} "{1}"; end;)SH", + v.var, v.value); + } else { + std::println(R"SH(if not set -q {0}; set -gx {0} "{1}"; end;)SH", + v.var, v.value); + } + } return 0; } if (is_pwsh) { @@ -709,6 +932,19 @@ int use_emit_shell(const std::string& name, std::println(R"($env:XLINGS_BIN = '{}')", bin_dir.string()); std::println(R"($env:Path = '{}' + ';' + (($env:Path -split ';') -notmatch '^{}\\subos\\[^\\]+\\bin$' -join ';'))", bin_dir.string(), p.homeDir.string()); + for (const auto& v : envVars) { + if (v.unresolved) continue; + // ';' rather than ':' -- these are path lists, and on Windows the + // separator is the one the platform's own tools split on. + if (v.op == manifest::OP_PREPEND) { + std::println( + R"($env:{0} = if ($env:{0}) {{ '{1}' + ';' + $env:{0} }} else {{ '{1}' }})", + v.var, v.value); + } else { + std::println(R"(if (-not $env:{0}) {{ $env:{0} = '{1}' }})", + v.var, v.value); + } + } return 0; } // POSIX (sh/bash/zsh) default @@ -718,6 +954,17 @@ int use_emit_shell(const std::string& name, std::println(R"(export XLINGS_ACTIVE_SUBOS="{}";)", name); std::println(R"(export XLINGS_BIN="{}";)", bin_dir.string()); std::println(R"(export PATH="{}";)", new_path); + for (const auto& v : envVars) { + if (v.unresolved) continue; + if (v.op == manifest::OP_PREPEND) { + // ${VAR:+:$VAR} appends the separator only when VAR is non-empty, + // so an unset variable does not become a trailing ':' -- which an + // empty PATH-list element reads as "the current directory". + std::println(R"(export {0}="{1}${{{0}:+:${0}}}";)", v.var, v.value); + } else { + std::println(R"(: "${{{0}:={1}}}"; export {0};)", v.var, v.value); + } + } return 0; } @@ -797,6 +1044,30 @@ int use_spawn_shell(const std::string& name, EventStream& stream, platform::set_env_variable("XLINGS_BIN", bin_dir.string()); platform::set_env_variable("PATH", new_path); + // The subos's declared environment, applied to this process before it is + // replaced -- so the shell (or the single `--cmd`) inherits it, and so + // does every user binary run inside. This is the path that matters for + // issue #352: nothing xlings wraps needs LIBGL_DRIVERS_PATH, the user's + // own GL program does. + // + // UC-1 -- a variable already set in this environment is the user's, and + // `set` leaves it alone. `prepend` still contributes, since composing is + // what prepend means. + { + const auto envVars = use_detail_::subos_env_for_(name); + use_detail_::report_injected_env_(name, envVars); + for (const auto& v : envVars) { + if (v.unresolved) continue; + const auto existing = utils::get_env_or_default(v.var); + if (v.op == manifest::OP_PREPEND) { + platform::set_env_variable( + v.var, existing.empty() ? v.value : v.value + ":" + existing); + } else if (existing.empty()) { + platform::set_env_variable(v.var, v.value); + } + } + } + nlohmann::json payload; payload["name"] = name; payload["mode"] = "spawn"; @@ -1013,9 +1284,20 @@ export int run(int argc, char* argv[], EventStream& stream) { // (auto-installs the base xpkg if missing); bare name is treated // as a local subos to fork from. std::string fromSpec; + // --runtime : what this subos's binaries are built against + // ("glibc@2.39"). Creation-time, because changing it later would + // invalidate every payload already installed. Absent → the built-in + // default, so existing invocations keep working unchanged. + std::string runtime; for (int i = 3; i < argc; ++i) { std::string a = argv[i]; - if (a == "--storage" && i + 1 < argc) { + if (a == "--runtime" && i + 1 < argc) { + runtime = argv[++i]; + } + else if (a.rfind("--runtime=", 0) == 0) { + runtime = a.substr(10); + } + else if (a == "--storage" && i + 1 < argc) { auto s = std::string(argv[++i]); if (s == "image") storage = sandbox::StorageMode::Image; else if (s == "tmpfs") storage = sandbox::StorageMode::Tmpfs; @@ -1048,9 +1330,9 @@ export int run(int argc, char* argv[], EventStream& stream) { return 1; } if (!fromSpec.empty()) { - return new_from(name, {}, storage, imageSize, fromSpec, stream); + return new_from(name, {}, storage, imageSize, fromSpec, runtime, stream); } - return create(name, {}, storage, imageSize, stream); + return create(name, {}, storage, imageSize, runtime, stream); } if (sub == "use") { // Flags supported: diff --git a/src/core/subos/manifest.cppm b/src/core/subos/manifest.cppm new file mode 100644 index 00000000..65d3808b --- /dev/null +++ b/src/core/subos/manifest.cppm @@ -0,0 +1,488 @@ +export module xlings.core.subos.manifest; + +import std; + +import xlings.libs.json; +import xlings.platform; + +// `subos_info` — what a subos is, recorded in the subos's own `.xlings.json`. +// +// A subos already had a directory, a bin/, an xvm scope and a registry entry. +// What it did not have was a statement of *what it is*: which runtime its +// binaries were built against, and which environment its processes need. Both +// were implicit in whatever happened to be installed, which is why a subos +// could not be described, checked, or reproduced. +// +// Layer this closes: a program needs bootstrap (PT_INTERP + CRT + libc), +// discovery (PATH + RPATH), and configuration (env vars). xlings had the first +// two — glibc + elfpatch, xvm + shims — and nothing for the third. That is the +// gap behind mcpp-community/mcpp#352: a GLFW binary that links fine and exits +// 255 because no one told it where the GL drivers are. +// +// Placement is the per-subos `.xlings.json`, not a new file and not the home +// or project one. Those two already use the key `subos` for other things (a +// selector string at project level, a registry map at home level), and a subos +// describing itself belongs in the subos. +// +// This module is deliberately free of Config/xvm imports: it takes paths and a +// binding→dir resolver from the caller. That keeps invariant checking and +// placeholder expansion testable without a home on disk, and it is what lets +// the doctor reporter and the repairer share one predicate instead of +// describing the rules twice and drifting apart. + +export namespace xlings::subos::manifest { + +namespace fs = std::filesystem; + +inline constexpr int SCHEMA_VERSION = 1; +inline constexpr std::string_view BLOCK = "subos_info"; +// The runtime a subos gets when the caller names none. A constant rather than +// a lookup: slice 1 ships one runtime, and inventing a "pick the newest libc +// present" rule would make two homes with the same command produce different +// subos. +inline constexpr std::string_view DEFAULT_RUNTIME = "glibc@2.39"; + +inline constexpr std::string_view OP_SET = "set"; +inline constexpr std::string_view OP_PREPEND = "prepend"; + +// ── data ──────────────────────────────────────────────────────────────── + +// One variable a package asks its subos to export. +struct EnvDecl { + std::string var; + std::string op; // OP_SET | OP_PREPEND + std::string value; // may contain ${...} placeholders +}; + +// A provider's whole section. Keyed by binding so uninstalling the package +// removes exactly what it added — the same provider-scoped ownership xvm.add +// and xvm.files already use. +struct Provider { + std::string binding; // "@" + std::vector decls; +}; + +struct Info { + int schema_version = 0; + std::string runtime; + std::vector envs; // sorted by binding; see resolve() + std::string created_at; + std::string created_by; +}; + +// ── runtime family ────────────────────────────────────────────────────── + +// The runtime string is self-describing: "glibc@2.39" says Linux/glibc without +// a second field to disagree with it. Families are derived here rather than +// stored, so a manifest cannot claim a family its runtime contradicts. +// +// Slice 1 needs only the glibc row. The rest are listed to make the shape of +// the mapping explicit — a new OS adds a row, not a schema field. +std::string family_of(std::string_view runtime, std::string_view arch = "x86_64") { + const auto at = runtime.find('@'); + const auto name = runtime.substr(0, at == std::string_view::npos + ? runtime.size() : at); + if (name == "glibc") return std::format("linux-{}-glibc", arch); + if (name == "musl") return std::format("linux-{}-musl", arch); + if (name == "wasi-libc") return "wasm32-wasi"; + if (name == "macos_sdk") return std::format("darwin-{}", arch); + if (name == "ucrt") return std::format("windows-{}-ucrt", arch); + return "unknown"; +} + +// "@", both halves non-empty. Used for `runtime` and for every +// envs key. +bool is_binding(std::string_view s) { + const auto at = s.find('@'); + return at != std::string_view::npos && at > 0 && at + 1 < s.size(); +} + +std::string_view binding_name(std::string_view binding) { + const auto at = binding.find('@'); + return at == std::string_view::npos ? binding : binding.substr(0, at); +} + +// ── invariants ────────────────────────────────────────────────────────── + +enum class Defect { + DirMissing, // I1 + ConfigMissing, // I2 + ConfigUnreadable, // I2 + BlockMissing, // I4 + SchemaUnsupported, // I5 + RuntimeMalformed, // I6 + EnvsMalformed, // I7 + EnvDeclMalformed, // I7 + ProvenanceMissing, // I8 +}; + +struct Finding { + Defect kind; + std::string detail; +}; + +std::string_view describe(Defect d) { + switch (d) { + case Defect::DirMissing: return "subos directory is missing"; + case Defect::ConfigMissing: return "subos has no .xlings.json"; + case Defect::ConfigUnreadable: return ".xlings.json is not readable JSON"; + case Defect::BlockMissing: return "no subos_info block"; + case Defect::SchemaUnsupported: return "unsupported subos_info schema_version"; + case Defect::RuntimeMalformed: return "runtime is not @"; + case Defect::EnvsMalformed: return "envs is not an object of provider sections"; + case Defect::EnvDeclMalformed: return "env declaration is malformed"; + case Defect::ProvenanceMissing: return "created_at / created_by missing"; + } + return "unknown defect"; +} + +fs::path config_path(const fs::path& subosDir) { + return subosDir / ".xlings.json"; +} + +// Read the whole document. An unreadable or malformed file yields nullopt, and +// the caller must not paper over it with an empty object: for the home config +// "corrupt means absent" is the historical behavior, but here it would let +// `--fix` rewrite a file it never managed to read. +std::optional read_document(const fs::path& subosDir) { + const auto path = config_path(subosDir); + std::error_code ec; + if (!fs::exists(path, ec) || ec) return std::nullopt; + try { + auto content = platform::read_file_to_string(path.string()); + auto parsed = nlohmann::json::parse(content, nullptr, false); + if (parsed.is_discarded() || !parsed.is_object()) return std::nullopt; + return parsed; + } catch (...) { return std::nullopt; } +} + +// I4–I8, over a document already in hand. Split out from validate() so a +// caller that has just built a document in memory can check it before writing. +std::vector validate_block(const nlohmann::json& doc) { + std::vector out; + + if (!doc.contains(std::string(BLOCK)) || !doc[std::string(BLOCK)].is_object()) { + out.push_back({Defect::BlockMissing, std::string(BLOCK)}); + return out; // nothing below can be checked + } + const auto& b = doc[std::string(BLOCK)]; + + if (!b.contains("schema_version") || !b["schema_version"].is_number_integer() + || b["schema_version"].get() != SCHEMA_VERSION) { + out.push_back({Defect::SchemaUnsupported, + b.contains("schema_version") ? b["schema_version"].dump() + : "absent"}); + } + + const auto runtime = b.value("runtime", std::string{}); + if (!is_binding(runtime)) { + out.push_back({Defect::RuntimeMalformed, + runtime.empty() ? "absent" : runtime}); + } + + // "An empty collection is not a missing one": `envs` is written as {} at + // creation and stays {} when the last provider is removed, so no reader + // has to treat absent and empty as the same thing. + if (!b.contains("envs") || !b["envs"].is_object()) { + out.push_back({Defect::EnvsMalformed, + b.contains("envs") ? b["envs"].type_name() : "absent"}); + } else { + for (auto it = b["envs"].begin(); it != b["envs"].end(); ++it) { + if (!is_binding(it.key())) { + out.push_back({Defect::EnvsMalformed, + std::format("provider key '{}' is not @", + it.key())}); + continue; + } + if (!it.value().is_array()) { + out.push_back({Defect::EnvsMalformed, + std::format("provider '{}' is not an array", it.key())}); + continue; + } + for (const auto& d : it.value()) { + const auto var = d.is_object() ? d.value("var", std::string{}) + : std::string{}; + const auto op = d.is_object() ? d.value("op", std::string{}) + : std::string{}; + if (var.empty() || (op != OP_SET && op != OP_PREPEND)) { + out.push_back({Defect::EnvDeclMalformed, + std::format("{}: {}", it.key(), d.dump())}); + } + } + } + } + + if (b.value("created_at", std::string{}).empty() + || b.value("created_by", std::string{}).empty()) { + out.push_back({Defect::ProvenanceMissing, ""}); + } + return out; +} + +// I1–I8 for a subos on disk. The single predicate the doctor reporter and the +// repairer both call — two descriptions of the same rule is how a reporter +// ends up flagging what `--fix` will not touch. +std::vector validate(const fs::path& subosDir) { + std::error_code ec; + if (!fs::is_directory(subosDir, ec) || ec) + return {{Defect::DirMissing, subosDir.string()}}; + if (!fs::exists(config_path(subosDir), ec) || ec) + return {{Defect::ConfigMissing, config_path(subosDir).string()}}; + + auto doc = read_document(subosDir); + if (!doc) return {{Defect::ConfigUnreadable, config_path(subosDir).string()}}; + return validate_block(*doc); +} + +// ── parse / build ─────────────────────────────────────────────────────── + +// Providers come back sorted by binding. That ordering is the resolution order +// (see resolve) and it is derived from the data rather than from install +// history, so two homes holding the same manifest resolve it identically. +Info parse(const nlohmann::json& doc) { + Info info; + if (!doc.contains(std::string(BLOCK)) || !doc[std::string(BLOCK)].is_object()) + return info; + const auto& b = doc[std::string(BLOCK)]; + + info.schema_version = b.value("schema_version", 0); + info.runtime = b.value("runtime", std::string{}); + info.created_at = b.value("created_at", std::string{}); + info.created_by = b.value("created_by", std::string{}); + + if (b.contains("envs") && b["envs"].is_object()) { + for (auto it = b["envs"].begin(); it != b["envs"].end(); ++it) { + if (!it.value().is_array()) continue; + Provider p{.binding = it.key()}; + for (const auto& d : it.value()) { + if (!d.is_object()) continue; + EnvDecl e{ + .var = d.value("var", std::string{}), + .op = d.value("op", std::string{}), + .value = d.value("value", std::string{}), + }; + if (e.var.empty()) continue; + if (e.op != OP_SET && e.op != OP_PREPEND) continue; + p.decls.push_back(std::move(e)); + } + info.envs.push_back(std::move(p)); + } + } + std::ranges::sort(info.envs, {}, &Provider::binding); + return info; +} + +std::string utc_now_iso() { + const auto now = std::chrono::system_clock::to_time_t( + std::chrono::system_clock::now()); + char buf[32]; + std::strftime(buf, sizeof(buf), "%Y-%m-%dT%H:%M:%SZ", std::gmtime(&now)); + return buf; +} + +// The block a freshly created subos gets. `envs` is an explicit empty object. +nlohmann::json make_block(std::string_view runtime, std::string_view createdBy) { + nlohmann::json b; + b["schema_version"] = SCHEMA_VERSION; + b["runtime"] = std::string(runtime); + b["envs"] = nlohmann::json::object(); + b["created_at"] = utc_now_iso(); + b["created_by"] = std::string(createdBy); + return b; +} + +// ── env declarations ──────────────────────────────────────────────────── + +// Record one declaration under its provider. Idempotent on the whole +// (var, op, value) triple: config() runs again on every dependent install, and +// a re-run must not grow the section. +// +// Returns whether the document changed, so a caller can skip a write. +bool add_env(nlohmann::json& doc, std::string_view binding, const EnvDecl& decl) { + auto& b = doc[std::string(BLOCK)]; + if (!b.is_object()) b = nlohmann::json::object(); + if (!b.contains("envs") || !b["envs"].is_object()) + b["envs"] = nlohmann::json::object(); + + auto& section = b["envs"][std::string(binding)]; + if (!section.is_array()) section = nlohmann::json::array(); + + for (const auto& existing : section) { + if (existing.value("var", std::string{}) == decl.var + && existing.value("op", std::string{}) == decl.op + && existing.value("value", std::string{}) == decl.value) { + return false; + } + } + section.push_back({{"var", decl.var}, {"op", decl.op}, {"value", decl.value}}); + return true; +} + +// Drop a provider's whole section — the uninstall counterpart of add_env. The +// package never writes cleanup code for this; ownership is by binding, so +// removing the key removes exactly what that package added and nothing else. +// +// The `envs` object itself stays, empty, per the invariant above. +bool remove_provider(nlohmann::json& doc, std::string_view binding) { + if (!doc.contains(std::string(BLOCK))) return false; + auto& b = doc[std::string(BLOCK)]; + if (!b.is_object() || !b.contains("envs") || !b["envs"].is_object()) + return false; + return b["envs"].erase(std::string(binding)) > 0; +} + +// Every provider whose package name matches, regardless of version. Uninstall +// knows the name and version it removed, but a home that installed the same +// package twice under different versions can hold a stale section from the +// earlier one; matching on name is what lets doctor see it. +std::vector providers_named(const nlohmann::json& doc, + std::string_view name) { + std::vector out; + if (!doc.contains(std::string(BLOCK))) return out; + const auto& b = doc[std::string(BLOCK)]; + if (!b.is_object() || !b.contains("envs") || !b["envs"].is_object()) return out; + for (auto it = b["envs"].begin(); it != b["envs"].end(); ++it) + if (binding_name(it.key()) == name) out.push_back(it.key()); + return out; +} + +// ── placeholders ──────────────────────────────────────────────────────── + +// What a value may refer to. `${pkgdir}` differs per provider, so it arrives +// as a resolver rather than a path — and the resolver is the caller's, which +// is what keeps this module independent of the version database. +struct Placeholders { + fs::path subosdir; + fs::path home; + fs::path xlings_home; + std::function pkgdir_of; +}; + +// Expand ${...} in a declared value. +// +// Placeholders are why a manifest is portable at all: a value carrying +// /home/alice/... describes one machine, and a subos description that only +// works on the machine that wrote it is not a description. +// +// An unknown or unresolvable placeholder is left verbatim rather than replaced +// with an empty string. Empty would turn "${pkgdir}/lib/dri" into "/lib/dri" — +// a real path, on the host, outside the subos. Leaving the text intact makes +// the failure visible to doctor D3 instead of pointing a driver search at /. +std::string expand(std::string_view value, std::string_view binding, + const Placeholders& ph) { + std::string out; + out.reserve(value.size()); + + for (std::size_t i = 0; i < value.size();) { + if (value[i] != '$' || i + 1 >= value.size() || value[i + 1] != '{') { + out += value[i++]; + continue; + } + const auto close = value.find('}', i + 2); + if (close == std::string_view::npos) { // unterminated: verbatim + out += value.substr(i); + break; + } + const auto name = value.substr(i + 2, close - i - 2); + fs::path resolved; + bool known = true; + if (name == "subosdir") resolved = ph.subosdir; + else if (name == "home") resolved = ph.home; + else if (name == "xlings_home") resolved = ph.xlings_home; + else if (name == "pkgdir") resolved = ph.pkgdir_of ? ph.pkgdir_of(binding) + : fs::path{}; + else known = false; + + if (!known || resolved.empty()) out += value.substr(i, close - i + 1); + else out += resolved.string(); + i = close + 1; + } + return out; +} + +// True once expansion left any `${...}` behind — i.e. something could not be +// resolved. Doctor D3 reports on this rather than on the directory existing, +// because an unexpanded value is a defect regardless of what is on disk. +bool has_unresolved(std::string_view expanded) { + return expanded.find("${") != std::string_view::npos; +} + +// ── resolution ────────────────────────────────────────────────────────── + +// One variable as it will actually be exported. +struct Resolved { + std::string var; + std::string op; // the winning op + std::string value; // expanded, prepends already joined + std::vector providers; // every binding that declared this var + bool conflicted = false; + bool unresolved = false; +}; + +// Fold the manifest into the variables to export. +// +// Conflict rules, when more than one provider names the same variable: +// * several `set` — the last provider wins, and it is a conflict +// * several `prepend` — all contribute, later providers land nearer the front +// * `set` and `prepend` mixed — `set` wins, the prepends are dropped, conflict +// +// "Later" means later in binding order, not install order. Install order is +// not recorded in the manifest and recording it would add a field whose only +// job is to make the outcome depend on history — two homes with byte-identical +// manifests would then export different values. Sorting by binding keeps the +// manifest the whole answer. Every conflict is reported (doctor D4) rather than +// resolved quietly. +std::vector resolve(const Info& info, const Placeholders& ph) { + struct Acc { + std::string setValue; + bool hasSet = false; + std::vector prepends; // front-most last + std::vector providers; + bool unresolved = false; + std::size_t order = 0; // first appearance, for stable output + }; + std::map acc; + std::size_t seen = 0; + + for (const auto& p : info.envs) { // already sorted by binding + for (const auto& d : p.decls) { + auto expanded = expand(d.value, p.binding, ph); + auto& a = acc[d.var]; + if (a.providers.empty()) a.order = seen++; + a.providers.push_back(p.binding); + if (has_unresolved(expanded)) a.unresolved = true; + if (d.op == OP_SET) { a.setValue = std::move(expanded); a.hasSet = true; } + else { a.prepends.push_back(std::move(expanded)); } + } + } + + std::vector out; + out.reserve(acc.size()); + for (auto& [var, a] : acc) { + Resolved r{.var = var, .providers = a.providers, .unresolved = a.unresolved}; + // More than one declaration for a variable is a conflict unless they + // are all prepends, which compose by construction. + r.conflicted = a.providers.size() > 1 + && !(!a.hasSet && !a.prepends.empty()); + if (a.hasSet) { + r.op = std::string(OP_SET); + r.value = std::move(a.setValue); + if (!a.prepends.empty()) r.conflicted = true; + } else { + r.op = std::string(OP_PREPEND); + // Later provider nearer the front, matching "the newest thing + // installed is found first". + for (auto it = a.prepends.rbegin(); it != a.prepends.rend(); ++it) { + if (!r.value.empty()) r.value += ':'; + r.value += *it; + } + } + out.push_back(std::move(r)); + } + // Stable, and stable for a reason: this list is echoed to the user and + // diffed in tests, so it must not reorder because a map rehashed. + std::ranges::sort(out, {}, &Resolved::var); + return out; +} + +} // namespace xlings::subos::manifest diff --git a/src/core/xim/installer.cppm b/src/core/xim/installer.cppm index fabcab3b..6a14513b 100644 --- a/src/core/xim/installer.cppm +++ b/src/core/xim/installer.cppm @@ -25,6 +25,7 @@ import xlings.core.xvm.bindings; import xlings.core.xvm.removal; import xlings.core.xvm.registration; import xlings.core.xvm.errors; +import xlings.core.subos.manifest; import xlings.core.xvm.commands; import xlings.core.xvm.shim; import xlings.core.xim.libxpkg.types.script; @@ -1436,11 +1437,105 @@ void detach_current_subos_(const std::string& target, } } +// Record a package's `subos.env` declarations into the subos it installs into. +// +// Provider-scoped, exactly like xvm registrations: everything lands under the +// declaring package's binding, and uninstall drops that key. A recipe never +// writes cleanup for it. +// +// Cross-package declarations are refused. A package may describe its own +// runtime needs; letting it write a section owned by another name would make +// uninstall unable to clean up (the owner's key is what removal keys on) and +// would hand any recipe the ability to edit any other's environment. +bool apply_subos_env_ops_(const std::vector& operations, + const PlanNode& node) { + namespace mf = xlings::subos::manifest; + + std::vector declarations; + for (const auto& op : operations) + if (op.op == "subos_env") declarations.push_back(&op); + if (declarations.empty()) return true; + + const auto canonical = node.canonicalName.empty() + ? canonical_package_name(node.namespaceName, node.name) + : node.canonicalName; + + const auto subosDir = Config::xvm_artifact_subos_dir(); + auto docPath = mf::config_path(subosDir); + auto doc = mf::read_document(subosDir); + if (!doc) { + // Absent is recoverable (an old subos predates the block); unreadable + // is not, and must not be papered over by starting from {} -- that + // would discard a workspace we never managed to parse. + std::error_code ec; + if (std::filesystem::exists(docPath, ec)) { + log::error("[xim] {} is not readable JSON; refusing to record " + "subos env declarations for {}@{}", + docPath.string(), canonical, node.version); + return false; + } + nlohmann::json fresh; + fresh["workspace"] = nlohmann::json::object(); + fresh[std::string(mf::BLOCK)] = mf::make_block( + mf::DEFAULT_RUNTIME, std::format("xlings {}", Info::VERSION)); + doc = std::move(fresh); + } + if (!doc->contains(std::string(mf::BLOCK)) + || !doc->at(std::string(mf::BLOCK)).is_object()) { + (*doc)[std::string(mf::BLOCK)] = mf::make_block( + mf::DEFAULT_RUNTIME, std::format("xlings {}", Info::VERSION)); + } + + bool changed = false; + for (const auto* op : declarations) { + if (!mf::is_binding(op->binding)) { + log::error("[xim] {}@{} declared env '{}' with binding '{}' " + "(expected @); nothing recorded", + canonical, node.version, op->var, op->binding); + return false; + } + const auto owner = mf::binding_name(op->binding); + if (owner != node.name && owner != canonical) { + log::error("[xim] {}@{} declared env '{}' for '{}', which it does " + "not own; nothing recorded", + canonical, node.version, op->var, op->binding); + return false; + } + changed |= mf::add_env(*doc, op->binding, + {.var = op->var, .op = op->mode, .value = op->value}); + } + if (!changed) return true; + + if (auto findings = mf::validate_block(*doc); !findings.empty()) { + log::error("[xim] recording env declarations for {}@{} would leave an " + "invalid subos manifest: {}", canonical, node.version, + mf::describe(findings.front().kind)); + return false; + } + + try { + platform::write_string_to_file(docPath.string(), doc->dump(2)); + } catch (const std::exception& e) { + log::error("[xim] failed to write {}: {}", docPath.string(), e.what()); + return false; + } + log::debug("[xim] recorded {} subos env declaration(s) for {}@{}", + declarations.size(), canonical, node.version); + return true; +} + bool process_xvm_operations_(const PlanNode& node, const std::filesystem::path& dataDir, mcpplibs::xpkg::PackageExecutor& executor, bool useAfterInstall) { auto xvm_ops = executor.xvm_operations(); + + // Before the early return below. A package that declares only env and + // registers nothing with xvm has an empty registration batch, and would + // otherwise install cleanly with its declarations dropped on the floor -- + // the exact shape where "nothing happened" and "it worked" look alike. + if (!apply_subos_env_ops_(xvm_ops, node)) return false; + auto& paths = Config::paths(); auto& scopedDb = Config::versions_mut(); auto& scopedWorkspace = Config::workspace_mut(); @@ -2791,6 +2886,50 @@ public: removalResult.error().version)); } + // Drop this package's subos env section. Keyed by binding, so it takes + // exactly what the package added and leaves every other provider's + // alone -- the recipe's uninstall() writes nothing for this. + // + // Matched by package *name* rather than the exact binding: the removal + // above resolves versions through namespaces and group members, so the + // binding recorded at install time is not reliably reconstructible + // here. A home that installed two versions of the same package can + // hold a section for each, and leaving the other one behind would + // point a driver search at a payload that is being deleted. + { + namespace mf = xlings::subos::manifest; + if (auto doc = mf::read_document(artifactSubosDir)) { + bool changed = false; + for (const auto& binding : + mf::providers_named(*doc, executingProvider)) { + changed |= mf::remove_provider(*doc, binding); + } + // providers_named matches the bare name; a namespaced install + // records the canonical one, so ask again under it. + if (executingProvider != detachTarget) { + for (const auto& binding : + mf::providers_named(*doc, detachTarget)) { + changed |= mf::remove_provider(*doc, binding); + } + } + if (changed) { + try { + platform::write_string_to_file( + mf::config_path(artifactSubosDir).string(), + doc->dump(2)); + } catch (const std::exception& e) { + // Not fatal to the uninstall -- the payload is already + // going -- but it must be said. A stale section points + // at a directory that no longer exists, which doctor + // D2/D3 will report. + log::warn("[xim] could not clear subos env " + "declarations for {}: {}", + executingProvider, e.what()); + } + } + } + } + for (const auto& op : xvm_ops) { if (op.op == "remove_headers") { xvm::remove_headers(op.includedir, sysroot_include); diff --git a/src/core/xself/doctor.cppm b/src/core/xself/doctor.cppm index 1e64be0c..e31b68d2 100644 --- a/src/core/xself/doctor.cppm +++ b/src/core/xself/doctor.cppm @@ -23,6 +23,8 @@ import xlings.core.xvm.owner; import xlings.core.xself.repair; import xlings.core.xim.payload; // classify_payload_platform import xlings.core.profile; +import xlings.core.subos.manifest; +import xlings.platform.target; // platform::host().arch for the runtime family namespace xlings::xself { @@ -128,6 +130,27 @@ enum class FindingKind { SysrootDangling, BindingState, OtherSubos, + // The subos does not describe itself: no `subos_info` block, or one that + // cannot be read. Every configuration-layer feature is inert without it, + // and inert looks exactly like "no package needed anything". + SubosManifest, + // An `envs` section owned by a package that is not installed here. Left + // behind by an uninstall that could not write the manifest, or copied in + // by a fork from a home where that package existed. The values point into + // a payload directory that is not there. + SubosEnvOrphan, + // A declared value that still contains `${...}` after expansion, so the + // variable would be exported pointing at a literal placeholder. Reported + // rather than exported: see manifest::expand on why it is not blanked. + SubosEnvUnresolved, + // One variable claimed by several packages. Resolution is deterministic, + // so this is not a breakage -- but one of the two packages is not getting + // what it asked for, and only a human can say which should. + SubosEnvConflict, + // The subos names a runtime that is not installed in it. The payloads + // built against it may still run off the host's libc, which is precisely + // the hermetic boundary the runtime field exists to make checkable. + SubosRuntimeMissing, }; enum class FindingLevel { @@ -385,11 +408,187 @@ std::string activation_conflict_(const DoctorState& st, return reason; } +// D1–D5 over the active subos's `subos_info`. +// +// One function, called by detection and again by the repair pass. The repair +// acts on what this returns rather than on its own reading of the rules -- +// a reporter and a repairer that each describe the criteria end up describing +// them differently, and then fight. +// +// Nothing here needs the version DB except D2 and D5, which take it as an +// argument, so this stays checkable against a directory. +std::vector detect_subos_manifest_(const xvm::VersionDB& db, + const fs::path& subosDir, + const std::string& subosName) { + namespace mf = xlings::subos::manifest; + std::vector out; + + // D1 — structure. Anything wrong here makes the rest unreadable, so it is + // the only finding produced when it fires. + if (auto structural = mf::validate(subosDir); !structural.empty()) { + std::string detail; + for (const auto& f : structural) { + if (!detail.empty()) detail += "; "; + detail += std::string(mf::describe(f.kind)); + if (!f.detail.empty()) detail += " (" + f.detail + ")"; + } + const bool unreadable = std::ranges::any_of( + structural, [](const auto& f) { + return f.kind == mf::Defect::ConfigUnreadable; + }); + out.push_back({ + .kind = FindingKind::SubosManifest, + .level = FindingLevel::Error, + .target = subosName, + .detail = detail, + // An unreadable file is not repaired: rewriting it would discard a + // workspace no one has managed to parse. Everything else is a + // missing or unusable block, which `--fix` can add. + .remedy = unreadable + ? std::format("inspect {}", + Config::display_path(mf::config_path(subosDir))) + : "xlings self doctor --fix", + }); + return out; + } + + auto doc = mf::read_document(subosDir); + if (!doc) return out; // D1 already covered this + const auto info = mf::parse(*doc); + + const auto installed = [&](std::string_view binding) { + const auto at = binding.find('@'); + if (at == std::string_view::npos) return false; + const std::string name(binding.substr(0, at)); + const std::string version(binding.substr(at + 1)); + const auto* vi = xvm::get_vinfo(db, name); + if (!vi) return false; + // Namespaced installs record `:`, so a bare match is not + // enough -- compare the version tail. + return std::ranges::any_of(vi->versions, [&](const auto& entry) { + const auto& key = entry.first; + if (key == version) return true; + const auto colon = key.find(':'); + return colon != std::string::npos && key.substr(colon + 1) == version; + }); + }; + + // D2 — an envs section whose owner is not installed here. + for (const auto& provider : info.envs) { + if (installed(provider.binding)) continue; + out.push_back({ + .kind = FindingKind::SubosEnvOrphan, + .level = FindingLevel::Error, + .target = subosName, + .version = provider.binding, + .detail = std::format( + "subos '{}' exports {} variable(s) for '{}', which is not " + "installed here", subosName, provider.decls.size(), + provider.binding), + .remedy = "xlings self doctor --fix", + }); + } + + // D3/D4 — over the resolved set, so both see exactly what activation will. + const auto resolved = mf::resolve(info, mf::Placeholders{ + .subosdir = subosDir, + .home = platform::get_home_dir(), + .xlings_home = Config::paths().homeDir, + .pkgdir_of = [](std::string_view binding) -> fs::path { + const auto at = binding.find('@'); + if (at == std::string_view::npos) return {}; + const std::string name(binding.substr(0, at)); + const std::string version(binding.substr(at + 1)); + const auto store = Config::paths().dataDir / "xpkgs"; + std::error_code ec; + if (auto direct = store / name / version; + fs::is_directory(direct, ec)) { + return direct; + } + if (!fs::is_directory(store, ec)) return {}; + const auto suffix = "-x-" + name; + for (const auto& entry : platform::dir_entries(store)) { + if (!entry.is_directory(ec)) continue; + if (!entry.path().filename().string().ends_with(suffix)) continue; + if (auto candidate = entry.path() / version; + fs::is_directory(candidate, ec)) { + return candidate; + } + } + return {}; + }, + }); + + for (const auto& v : resolved) { + if (v.unresolved) { + out.push_back({ + .kind = FindingKind::SubosEnvUnresolved, + .level = FindingLevel::Error, + .target = subosName, + .version = v.var, + .detail = std::format( + "{} would export an unexpanded path; its provider's " + "payload is missing", v.var), + // Reinstalling the provider is what puts the payload back. + // Only one is named even when several declared the variable: + // a remedy the user can paste beats an exhaustive one. + .remedy = v.providers.empty() ? std::string{} + : std::format("xlings install {}", v.providers.front()), + }); + } + if (v.conflicted) { + std::string names; + for (const auto& b : v.providers) { + if (!names.empty()) names += ", "; + names += b; + } + out.push_back({ + .kind = FindingKind::SubosEnvConflict, + // A warning, not an error: resolution is deterministic and the + // subos works. What is wrong is that one of these packages is + // silently not getting what it asked for. + .level = FindingLevel::Warning, + .target = subosName, + .version = v.var, + .detail = std::format("{} is claimed by {}", v.var, names), + }); + } + } + + // D5 — the declared runtime is not installed here. + if (mf::is_binding(info.runtime) && !installed(info.runtime)) { + out.push_back({ + .kind = FindingKind::SubosRuntimeMissing, + // Warning: binaries generally still run, off the host's libc. + // That is the hermetic boundary being crossed silently, which is + // the thing worth saying rather than failing over. + .level = FindingLevel::Warning, + .target = subosName, + .version = info.runtime, + .detail = std::format( + "subos '{}' declares runtime {} ({}), which is not installed " + "here", subosName, info.runtime, + mf::family_of(info.runtime, platform::host().arch)), + .remedy = std::format("xlings install {}", info.runtime), + }); + } + return out; +} + Scan detect_(const DoctorState& st, const CoordinateProbe& probe) { auto& p = Config::paths(); Scan scan; const auto add = [&](Finding f) { scan.findings.push_back(std::move(f)); }; + // The subos this run is actually in. Other subos are not inspected from + // here for the same reason their payloads are not repaired: a second + // shell may be inside one right now. + for (auto&& f : detect_subos_manifest_(st.db, p.subosDir, + p.activeSubos.empty() ? "default" + : p.activeSubos)) { + add(std::move(f)); + } + // The PATH an aliased command would inherit. Read once, and read from // THIS process: doctor is normally started from the user's shell, so this // is the same PATH the shim would get. It is not guaranteed to be -- a @@ -1267,6 +1466,66 @@ void repair_local_(const DoctorState& st, const Scan& scan, out.notes.emplace_back(std::move(label), std::move(text)); }; + // Subos manifest repairs. Driven by the findings detection produced, not + // by a second reading of the rules -- so what `--fix` touches is exactly + // what was reported. + { + namespace mf = xlings::subos::manifest; + const bool wantsBlock = std::ranges::any_of( + scan.findings, [](const Finding& f) { + return f.kind == FindingKind::SubosManifest + && f.remedy == "xlings self doctor --fix"; + }); + std::vector orphans; + for (const auto& f : scan.findings) + if (f.kind == FindingKind::SubosEnvOrphan) orphans.push_back(f.version); + + if (wantsBlock || !orphans.empty()) { + auto doc = mf::read_document(p.subosDir); + nlohmann::json document = doc ? *doc : nlohmann::json::object(); + if (!doc && fs::exists(mf::config_path(p.subosDir))) { + // Unreadable rather than absent. Detection already said so and + // offered no `--fix` remedy; do not overwrite it here either. + note(glyph::mark(glyph::failed, "subos manifest"), + std::format("{} is not readable JSON; left untouched", + Config::display_path( + mf::config_path(p.subosDir)))); + } else { + bool changed = false; + if (!document.contains("workspace")) + document["workspace"] = nlohmann::json::object(); + if (wantsBlock) { + document[std::string(mf::BLOCK)] = mf::make_block( + mf::DEFAULT_RUNTIME, + std::format("xlings {}", Info::VERSION)); + changed = true; + note(glyph::mark(glyph::bullet, "subos manifest"), + std::format("described subos '{}' (runtime {})", + p.activeSubos, mf::DEFAULT_RUNTIME)); + } + for (const auto& binding : orphans) { + if (!mf::remove_provider(document, binding)) continue; + changed = true; + note(glyph::mark(glyph::bullet, "subos env dropped"), + std::format("{} is not installed here", binding)); + } + if (changed) { + try { + platform::write_string_to_file( + mf::config_path(p.subosDir).string(), + document.dump(2)); + } catch (const std::exception& e) { + note(glyph::mark(glyph::failed, "subos manifest"), + std::format("could not write {}: {}", + Config::display_path( + mf::config_path(p.subosDir)), + e.what())); + } + } + } + } + } + for (const auto& f : scan.findings) { if (f.kind == FindingKind::MissingShim) { if (!fs::exists(st.xlingsBin)) continue; @@ -2026,6 +2285,32 @@ void render_(const Scan& scan, const RepairReport& repair, bool fix, case FindingKind::ReleaseAnchor: if (verbose) add(glyph::mark(glyph::note, "release anchor"), f.detail); break; + case FindingKind::SubosManifest: + add(glyph::mark(glyph::failed, "subos manifest"), f.detail); + // The remedy for an unreadable file is not `--fix`, so it has + // to be printed rather than left to the generic footer. + if (!f.remedy.empty() && f.remedy != "xlings self doctor --fix") + add(" " + glyph::mark(glyph::remedy, "run"), f.remedy); + break; + case FindingKind::SubosEnvOrphan: + add(glyph::mark(glyph::failed, "subos env orphan"), f.detail); + break; + case FindingKind::SubosEnvUnresolved: + add(glyph::mark(glyph::failed, "subos env unresolved"), f.detail); + if (!f.remedy.empty()) + add(" " + glyph::mark(glyph::remedy, "run"), f.remedy); + break; + case FindingKind::SubosEnvConflict: + // Never collapsed into a count, unlike the notice categories: + // there is one per contested variable, the number is small, + // and which packages disagree is the whole content. + add(glyph::mark(glyph::warn, "subos env conflict"), f.detail); + break; + case FindingKind::SubosRuntimeMissing: + add(glyph::mark(glyph::warn, "subos runtime"), f.detail); + if (!f.remedy.empty()) + add(" " + glyph::mark(glyph::remedy, "run"), f.remedy); + break; default: break; } } diff --git a/src/core/xself/init.cppm b/src/core/xself/init.cppm index 51971696..60daf0b0 100644 --- a/src/core/xself/init.cppm +++ b/src/core/xself/init.cppm @@ -11,6 +11,7 @@ import xlings.platform; import xlings.core.xself.compat; // Generated at build time from config/shell/*.{sh,fish,ps1}; see mcpp.toml. import xlings.core.xself.profile_resources; +import xlings.core.subos.manifest; namespace xlings::xself { @@ -166,6 +167,38 @@ static void write_if_missing_(const fs::path& path, std::string_view content) { platform::write_string_to_file(path.string(), std::string(content)); } +// Give a subos directory a valid `subos_info`, preserving everything else in +// the file. Idempotent: a block that already validates is left alone, so the +// envs packages declared into it survive. +static void ensure_subos_manifest_(const fs::path& subos_dir) { + namespace mf = xlings::subos::manifest; + auto path = subos_dir / ".xlings.json"; + + nlohmann::json json = nlohmann::json::object(); + if (fs::exists(path)) { + try { + auto parsed = nlohmann::json::parse( + platform::read_file_to_string(path.string()), nullptr, false); + if (parsed.is_discarded() || !parsed.is_object()) { + // Unreadable, not absent. Overwriting would throw away a + // workspace we cannot see; leave it for doctor to report. + log::warn("[xlings:self]: {} is not readable JSON; leaving it " + "alone (run `xlings self doctor`)", + Config::display_path(path)); + return; + } + json = std::move(parsed); + } catch (...) { return; } + } + if (!json.contains("workspace")) json["workspace"] = nlohmann::json::object(); + if (mf::validate_block(json).empty()) return; + + json[std::string(mf::BLOCK)] = + mf::make_block(mf::DEFAULT_RUNTIME, std::format("xlings {}", Info::VERSION)); + ensure_parent_dirs_(path); + platform::write_string_to_file(path.string(), json.dump(2)); +} + // Extract the value following `# xlings-profile-version: ` on any line of // `text`. Returns an empty string when the marker is absent, which we // interpret as "legacy v1" — anything older than the time we started @@ -286,7 +319,16 @@ bool ensure_home_layout(const fs::path& home_dir) { auto current_link = home_dir / "subos" / "current"; platform::create_directory_link(current_link, default_subos); - write_if_missing_(default_subos / ".xlings.json", "{\"workspace\":{}}"); + // Not write_if_missing_: every home that predates `subos_info` already has + // this file, so "missing" is exactly the case that never fires on the + // subos that matters most. `default` is where an ordinary user installs + // everything, and a default without the block would make the whole + // configuration layer inert on upgrade while looking installed. + // + // This is also the migration: `self init` runs on install and update, so an + // old home repairs its default subos on the next either. Other subos in + // that home are `subos doctor --fix`'s job -- init does not enumerate them. + ensure_subos_manifest_(default_subos); write_if_missing_(home_dir / "data" / "xim-index-repos" / "xim-indexrepos.json", "{}"); // Profile content lives in xlings.core.xself.profile_resources. We use // the version-aware writer so users who installed an older xlings get diff --git a/tests/e2e/run_all.sh b/tests/e2e/run_all.sh index 16774774..27d16517 100755 --- a/tests/e2e/run_all.sh +++ b/tests/e2e/run_all.sh @@ -109,6 +109,8 @@ TESTS=( "E2E-57 |subos_use_process_model_test.sh||" "E2E-58 |arch_evidence_contract_test.sh||" "E2E-59 |index_version_contract_test.sh||" + "E2E-60 |subos_env_declaration_test.sh||" + "E2E-61 |subos_env_probe_compat_test.sh||" ) PASS=0; FAIL=0; SOFTFAIL=0 diff --git a/tests/e2e/subos_env_declaration_test.sh b/tests/e2e/subos_env_declaration_test.sh new file mode 100755 index 00000000..f93ac6f6 --- /dev/null +++ b/tests/e2e/subos_env_declaration_test.sh @@ -0,0 +1,254 @@ +#!/usr/bin/env bash +# E2E: subos.env — a package declares an environment variable, and a program +# the package does not own can see it. +# +# This is the whole of subos slice 1 end to end. The layer being tested is +# configuration: PATH and RPATH get a binary loaded, and neither can tell it +# where to find a GL driver. That is mcpp-community/mcpp#352 — a GLFW binary +# that links fine and exits 255 because LIBGL_DRIVERS_PATH points nowhere. +# +# What has to hold: +# 1. install — the declaration lands in the subos manifest, keyed by +# the declaring package +# 2. --shell — eval'ing the emitted code sets the variable +# 3. --cmd — a command run in the subos inherits it +# 4. user override — a value the user exported already wins (UC-1) +# 5. uninstall — the section goes, and `envs` stays as {} +# 6. doctor — clean afterwards, and it catches an orphaned section +# +# The probe rule has its own test (E2E: subos_env_probe_compat_test.sh); this +# one assumes the capability is present. +# +# Design: .agents/docs/2026-08-05-subos-minimum-design.md + +set -uo pipefail + +# shellcheck source=./project_test_lib.sh +source "$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/project_test_lib.sh" + +require_fixture_index + +RUNTIME_DIR="$ROOT_DIR/tests/e2e/runtime/subos_env" +LOCAL_INDEX_DIR="$RUNTIME_DIR/xim-pkgindex" +HOME_DIR="$RUNTIME_DIR/home" + +cleanup() { rm -rf "$RUNTIME_DIR"; } +trap cleanup EXIT +cleanup +mkdir -p "$RUNTIME_DIR" + +BIN="$(find_xlings_bin)" +log "client: $("$BIN" --version 2>&1 | head -1)" + +# ── a package that declares two variables ─────────────────────────────── +cp -r "$FIXTURE_INDEX_DIR" "$LOCAL_INDEX_DIR" +printf 'xim_indexrepos = {}\n' > "$LOCAL_INDEX_DIR/xim-indexrepos.lua" +rm -f "$LOCAL_INDEX_DIR/.xlings-index-cache.json" +mkdir -p "$LOCAL_INDEX_DIR/pkgs/e" + +cat > "$LOCAL_INDEX_DIR/pkgs/e/envfixture.lua" <<'LUA' +package = { + spec = "1", + name = "envfixture", + description = "Local fixture for tests/e2e/subos_env_declaration_test.sh", + authors = {"xlings-ci"}, + licenses = {"MIT"}, + type = "package", + archs = {"x86_64"}, + status = "stable", + categories = {"test-fixture"}, + + xpm = { + linux = { ["1.0.0"] = {} }, + macosx = { ["1.0.0"] = {} }, + windows = { ["1.0.0"] = {} }, + }, +} + +import("xim.libxpkg.pkginfo") +import("xim.libxpkg.xvm") +import("xim.libxpkg.subos") + +function install() + local dir = pkginfo.install_dir() + os.tryrm(dir) + os.mkdir(path.join(dir, "bin")) + os.mkdir(path.join(dir, "drivers")) + os.mkdir(path.join(dir, "share")) + io.writefile(path.join(dir, "bin", "envfixture"), "#!/bin/sh\necho envfixture\n") + os.exec("chmod +x " .. path.join(dir, "bin", "envfixture")) + return true +end + +function config() + xvm.add(package.name, { bindir = path.join(pkginfo.install_dir(), "bin") }) + + -- type(), not truthiness: import() hands back a permissive proxy for a + -- module the client does not ship, and every key on it is truthy. + if type(subos.env) == "function" then + local binding = package.name .. "@" .. pkginfo.version() + subos.env{ var = "E2E_DRIVERS_PATH", op = "set", + value = "${pkgdir}/drivers", binding = binding } + subos.env{ var = "E2E_DATA_DIRS", op = "prepend", + value = "${pkgdir}/share", binding = binding } + end + return true +end + +function uninstall() + -- Nothing here on purpose. The env section is provider-scoped, and xlings + -- drops it with the package; a recipe writing its own cleanup would be + -- the second owner of that state. + return true +end +LUA + +# ── an isolated home ──────────────────────────────────────────────────── +mkdir -p "$HOME_DIR/subos/default/bin" "$HOME_DIR/data/xim-index-repos" +cat > "$HOME_DIR/.xlings.json" < "$HOME_DIR/data/xim-index-repos/xim-indexrepos.json" +cp "$BIN" "$HOME_DIR/xlings" + +# env -i: the variables under test must come from the subos, not be inherited +# from whatever shell is running the suite. +x() { ( cd /tmp && env -i HOME="$HOME" PATH=/usr/bin:/bin \ + XLINGS_HOME="$HOME_DIR" "$BIN" "$@" ) } + +x self init >/dev/null 2>&1 || true + +MANIFEST="$HOME_DIR/subos/default/.xlings.json" +[[ -f "$MANIFEST" ]] || fail "self init produced no subos manifest" + +# ── 1. install records the declaration ────────────────────────────────── +OUT="$(x install envfixture@1.0.0 -y 2>&1)" || { echo "$OUT" >&2; fail "install failed"; } + +python3 - "$MANIFEST" <<'PY' || exit 1 +import json, sys +info = json.load(open(sys.argv[1])).get("subos_info") +if info is None: + raise SystemExit("no subos_info block after install") +envs = info.get("envs", {}) +key = "envfixture@1.0.0" +if key not in envs: + raise SystemExit(f"no section for {key}; envs={envs}") +got = {d["var"]: (d["op"], d["value"]) for d in envs[key]} +want = { + "E2E_DRIVERS_PATH": ("set", "${pkgdir}/drivers"), + "E2E_DATA_DIRS": ("prepend", "${pkgdir}/share"), +} +if got != want: + raise SystemExit(f"recorded {got}, expected {want}") +# The stored value must stay a placeholder. A manifest holding this machine's +# absolute paths describes this machine, and a subos description that only +# works where it was written is not a description. +print("[project-e2e] ✓ recorded, and still portable") +PY + +# Installing again must not grow the section — config() re-runs on every +# dependent install. +x install envfixture@1.0.0 -y >/dev/null 2>&1 +COUNT="$(python3 -c ' +import json,sys +print(len(json.load(open(sys.argv[1]))["subos_info"]["envs"]["envfixture@1.0.0"]))' "$MANIFEST")" +[[ "$COUNT" == "2" ]] || fail "reinstall grew the section to $COUNT declarations" +log " ✓ re-install is idempotent" + +# ── 2. --shell emits code that sets them ──────────────────────────────── +SHELL_OUT="$(x subos use default --shell sh 2>/dev/null)" +EVALED="$( env -i PATH=/usr/bin:/bin bash -c "eval '$SHELL_OUT' + echo \"DRIVERS=\$E2E_DRIVERS_PATH\" + echo \"DATA=\$E2E_DATA_DIRS\"" )" +grep -q "DRIVERS=.*/drivers$" <<<"$EVALED" \ + || { echo "$EVALED" >&2; fail "--shell did not set E2E_DRIVERS_PATH"; } +grep -q "DATA=.*/share$" <<<"$EVALED" \ + || { echo "$EVALED" >&2; fail "--shell did not set E2E_DATA_DIRS"; } +# The expanded path must exist — the placeholder resolved to a real payload. +DRIVERS_PATH="$(sed -n 's/^DRIVERS=//p' <<<"$EVALED")" +[[ -d "$DRIVERS_PATH" ]] || fail "E2E_DRIVERS_PATH=$DRIVERS_PATH is not a directory" +log " ✓ --shell exports both, expanded to a real payload" + +# UC-2: the user is told what was injected, on stderr so the stdout stays +# eval-safe. +REPORT="$(x subos use default --shell sh 2>&1 >/dev/null)" +grep -q "2 env var(s) from 1 package(s)" <<<"$REPORT" \ + || { echo "$REPORT" >&2; fail "--shell did not report what it injected"; } +log " ✓ --shell reports the injected set on stderr" + +# ── 3. --cmd inherits them ────────────────────────────────────────────── +CMD_OUT="$(x subos use default --cmd 'echo CMD=$E2E_DRIVERS_PATH' 2>/dev/null)" +grep -q "CMD=.*/drivers$" <<<"$CMD_OUT" \ + || { echo "$CMD_OUT" >&2; fail "--cmd did not inject E2E_DRIVERS_PATH"; } +log " ✓ --cmd injects into the process environment" + +# ── 4. the user's own value wins (UC-1) ───────────────────────────────── +OVERRIDE="$( cd /tmp && env -i HOME="$HOME" PATH=/usr/bin:/bin \ + XLINGS_HOME="$HOME_DIR" \ + E2E_DRIVERS_PATH=/user/choice E2E_DATA_DIRS=/user/share \ + "$BIN" subos use default \ + --cmd 'echo "D=$E2E_DRIVERS_PATH"; echo "X=$E2E_DATA_DIRS"' 2>/dev/null )" +grep -q "^D=/user/choice$" <<<"$OVERRIDE" \ + || { echo "$OVERRIDE" >&2; fail "a 'set' declaration overwrote the user's value"; } +# prepend still composes — that is what prepend means — but it must not +# discard what was there. +grep -q "^X=.*/share:/user/share$" <<<"$OVERRIDE" \ + || { echo "$OVERRIDE" >&2; fail "'prepend' did not compose with the user's value"; } +log " ✓ the user's value survives 'set' and is kept by 'prepend'" + +# ── 5. doctor is clean ────────────────────────────────────────────────── +DOC="$(x self doctor 2>&1)" +grep -qiE "subos env (orphan|unresolved)" <<<"$DOC" \ + && { echo "$DOC" >&2; fail "doctor reports a defect on a healthy subos"; } +log " ✓ doctor is clean while the package is installed" + +# ...and catches a section whose owner is gone. Removing the payload behind +# the package's back is what an interrupted uninstall leaves. +python3 - "$MANIFEST" <<'PY' +import json, sys +p = sys.argv[1] +d = json.load(open(p)) +d["subos_info"]["envs"]["ghost@9.9.9"] = [ + {"var": "GHOST", "op": "set", "value": "${pkgdir}/lib"}] +json.dump(d, open(p, "w"), indent=2) +PY +DOC="$(x self doctor 2>&1)" +grep -qi "subos env orphan" <<<"$DOC" \ + || { echo "$DOC" >&2; fail "doctor missed an orphaned env section"; } +x self doctor --fix >/dev/null 2>&1 +python3 -c ' +import json,sys +envs = json.load(open(sys.argv[1]))["subos_info"]["envs"] +assert "ghost@9.9.9" not in envs, "--fix left the orphan behind" +assert "envfixture@1.0.0" in envs, "--fix took the healthy section too"' "$MANIFEST" \ + || fail "doctor --fix did not repair exactly the orphan" +log " ✓ doctor detects and repairs an orphaned section, and only that" + +# ── 6. uninstall drops the section ────────────────────────────────────── +x remove envfixture -y >/dev/null 2>&1 || x uninstall envfixture -y >/dev/null 2>&1 + +python3 - "$MANIFEST" <<'PY' || exit 1 +import json, sys +info = json.load(open(sys.argv[1]))["subos_info"] +envs = info["envs"] +if "envfixture@1.0.0" in envs: + raise SystemExit(f"uninstall left the section behind: {envs}") +# "envs" itself must stay, as {}. An absent key and an empty one would be two +# states meaning the same thing, and every reader would have to handle both. +if not isinstance(envs, dict): + raise SystemExit(f"envs is no longer an object: {envs!r}") +print("[project-e2e] ✓ section removed, envs kept as {}") +PY + +# And the variable is gone from what a new shell would get. +AFTER="$(x subos use default --cmd 'echo AFTER=${E2E_DRIVERS_PATH:-}' 2>/dev/null)" +grep -q "AFTER=" <<<"$AFTER" \ + || { echo "$AFTER" >&2; fail "the variable is still injected after uninstall"; } +log " ✓ the variable is no longer injected" + +log "E2E subos.env declaration lifecycle: PASS" diff --git a/tests/e2e/subos_env_probe_compat_test.sh b/tests/e2e/subos_env_probe_compat_test.sh new file mode 100755 index 00000000..cbcf0b7f --- /dev/null +++ b/tests/e2e/subos_env_probe_compat_test.sh @@ -0,0 +1,200 @@ +#!/usr/bin/env bash +# E2E: the `subos.env` capability probe, run against a REAL old xlings. +# +# `xim.libxpkg.subos` is a NEW MODULE, and that changes the probe rule the V2 +# spec gives for a new *function* on an existing module. +# +# if xvm.files then ... end -- correct: `xvm` exists on old +# clients, so the field really is nil +# if subos.env then ... end -- WRONG for a new module +# if type(subos.env) == "function" -- correct for a new module +# +# import() answers an unknown module with a permissive proxy stub: every key +# read off it returns a truthy, callable table. So on a client that predates +# the module, `if subos.env then` is TRUE, the recipe takes the new branch, +# calls it, and the call evaporates. The install succeeds, nothing is +# configured, and nothing says so. +# +# Reading the prelude says all this. That is not enough — the same reasoning +# said `xvm.files` was safe, and it was only safe by accident of `xvm` already +# existing. So this runs one recipe through a real released binary and the +# current build and asserts what each observed: +# +# old client → truthiness TRUE, type() FALSE → legacy branch +# new client → truthiness TRUE, type() TRUE → subos.env branch +# +# The first line is the finding. If it ever reads FALSE the rule can be +# relaxed; until then it must stay, and this is what proves it. +# +# XLINGS_OLD_BIN can point at an already-downloaded old binary; otherwise the +# test fetches one, and skips (exit 0) if there is no network. + +set -uo pipefail + +# shellcheck source=./project_test_lib.sh +source "$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/project_test_lib.sh" + +require_fixture_index + +OLD_VERSION="${XLINGS_OLD_VERSION:-2026.8.4.2}" +RUNTIME_DIR="$ROOT_DIR/tests/e2e/runtime/subos_env_probe" +LOCAL_INDEX_DIR="$RUNTIME_DIR/xim-pkgindex" + +cleanup() { rm -rf "$RUNTIME_DIR"; } +trap cleanup EXIT +cleanup +mkdir -p "$RUNTIME_DIR" + +NEW_BIN="$(find_xlings_bin)" + +OLD_BIN="${XLINGS_OLD_BIN:-}" +if [[ -z "$OLD_BIN" ]]; then + TARBALL="$RUNTIME_DIR/old.tar.gz" + URL="https://github.com/openxlings/xlings/releases/download/v${OLD_VERSION}/xlings-${OLD_VERSION}-linux-x86_64.tar.gz" + if ! curl -fsSL --max-time 120 -o "$TARBALL" "$URL"; then + log "SKIP: cannot fetch xlings $OLD_VERSION (no network?)" + exit 0 + fi + tar xzf "$TARBALL" -C "$RUNTIME_DIR" + OLD_BIN="$(find "$RUNTIME_DIR" -type f -name xlings -perm -u+x | head -1)" +fi +[[ -x "$OLD_BIN" ]] || { log "SKIP: no usable old binary"; exit 0; } +log "old client: $("$OLD_BIN" --version 2>&1 | head -1)" +log "new client: $("$NEW_BIN" --version 2>&1 | head -1)" + +# ── one recipe, recording what each client observed ───────────────────── +cp -r "$FIXTURE_INDEX_DIR" "$LOCAL_INDEX_DIR" +printf 'xim_indexrepos = {}\n' > "$LOCAL_INDEX_DIR/xim-indexrepos.lua" +rm -f "$LOCAL_INDEX_DIR/.xlings-index-cache.json" +mkdir -p "$LOCAL_INDEX_DIR/pkgs/s" + +cat > "$LOCAL_INDEX_DIR/pkgs/s/subosprobe.lua" <<'LUA' +package = { + spec = "1", + name = "subosprobe", + description = "Local fixture for tests/e2e/subos_env_probe_compat_test.sh", + authors = {"xlings-ci"}, + licenses = {"MIT"}, + type = "package", + archs = {"x86_64"}, + status = "stable", + categories = {"test-fixture"}, + + xpm = { + linux = { ["1.0.0"] = {} }, + macosx = { ["1.0.0"] = {} }, + windows = { ["1.0.0"] = {} }, + }, +} + +import("xim.libxpkg.pkginfo") +import("xim.libxpkg.xvm") +import("xim.libxpkg.subos") + +function install() + local dir = pkginfo.install_dir() + os.tryrm(dir) + os.mkdir(path.join(dir, "bin")) + os.mkdir(path.join(dir, "drivers")) + io.writefile(path.join(dir, "bin", "subosprobe"), "#!/bin/sh\necho probe\n") + os.exec("chmod +x " .. path.join(dir, "bin", "subosprobe")) + return true +end + +function config() + xvm.add(package.name, { bindir = path.join(pkginfo.install_dir(), "bin") }) + + local dir = pkginfo.install_dir() + -- Both readings, written down. The gap between them IS the finding. + io.writefile(path.join(dir, "TRUTHY"), + tostring(subos.env ~= nil)) + io.writefile(path.join(dir, "TYPED"), + tostring(type(subos.env) == "function")) + + if type(subos.env) == "function" then + io.writefile(path.join(dir, "BRANCH"), "subos.env") + subos.env{ var = "E2E_PROBE_PATH", op = "set", + value = "${pkgdir}/drivers", + binding = package.name .. "@" .. pkginfo.version() } + else + io.writefile(path.join(dir, "BRANCH"), "legacy") + end + return true +end + +function uninstall() + return true +end +LUA + +run_with() { # + ( cd /tmp && env -i HOME="$HOME" PATH=/usr/bin:/bin XLINGS_HOME="$2" "$1" "${@:3}" ) +} + +prepare_home() { # + mkdir -p "$1/subos/default/bin" "$1/data/xim-index-repos" + cat > "$1/.xlings.json" < "$1/data/xim-index-repos/xim-indexrepos.json" +} + +read_marker() { # + local f + f="$(find "$1" -name "$2" -type f 2>/dev/null | head -1)" + [[ -n "$f" ]] && cat "$f" || echo "" +} + +# old: truthy true, typed false, legacy branch +# new: truthy true, typed true, subos.env branch +for pair in "old:$OLD_BIN:true:false:legacy" "new:$NEW_BIN:true:true:subos.env"; do + IFS=: read -r label bin wantTruthy wantTyped wantBranch <<<"$pair" + HOME_DIR="$RUNTIME_DIR/home-$label" + mkdir -p "$HOME_DIR" + cp "$bin" "$HOME_DIR/xlings" + prepare_home "$HOME_DIR" + run_with "$bin" "$HOME_DIR" self init >/dev/null 2>&1 || true + + OUT="$(run_with "$bin" "$HOME_DIR" install subosprobe@1.0.0 -y 2>&1)" + if grep -qiE "unsupported registration node kind|config hook failed" <<<"$OUT"; then + echo "$OUT" >&2 + fail "$label client failed to install — the probe does not gate cleanly" + fi + + gotTruthy="$(read_marker "$HOME_DIR" TRUTHY)" + gotTyped="$(read_marker "$HOME_DIR" TYPED)" + gotBranch="$(read_marker "$HOME_DIR" BRANCH)" + + [[ "$gotTruthy" == "$wantTruthy" ]] || { + echo "$OUT" >&2 + fail "$label: 'subos.env ~= nil' was $gotTruthy, expected $wantTruthy" + } + [[ "$gotTyped" == "$wantTyped" ]] || { + echo "$OUT" >&2 + fail "$label: 'type(subos.env)==function' was $gotTyped, expected $wantTyped" + } + [[ "$gotBranch" == "$wantBranch" ]] || { + echo "$OUT" >&2 + fail "$label client took the '$gotBranch' branch, expected '$wantBranch'" + } + log " ✓ $label → truthy=$gotTruthy typed=$gotTyped branch=$gotBranch" +done + +# The old client must not have silently produced a manifest section, and the +# new one must have. +OLD_MANIFEST="$RUNTIME_DIR/home-old/subos/default/.xlings.json" +NEW_MANIFEST="$RUNTIME_DIR/home-new/subos/default/.xlings.json" + +if [[ -f "$OLD_MANIFEST" ]] && grep -q "E2E_PROBE_PATH" "$OLD_MANIFEST"; then + fail "the old client recorded an env declaration it cannot apply" +fi +grep -q "E2E_PROBE_PATH" "$NEW_MANIFEST" \ + || fail "the new client did not record the declaration" + +log " ✓ only the client that can apply the declaration recorded one" +log "E2E subos.env probe compatibility: PASS" diff --git a/tests/unit/test_subos_manifest.cpp b/tests/unit/test_subos_manifest.cpp new file mode 100644 index 00000000..7c298764 --- /dev/null +++ b/tests/unit/test_subos_manifest.cpp @@ -0,0 +1,345 @@ +// Unit tests for the `subos_info` manifest block (subos slice 1). +// +// Design: .agents/docs/2026-08-05-subos-minimum-design.md +// Landing plan: .agents/docs/2026-08-05-subos-slice1-landing-plan.md +// +// The module deliberately takes paths and a resolver instead of reaching for +// Config, so everything here runs without a home on disk. +#include + +import std; +import xlings.core.subos.manifest; +import xlings.libs.json; + +namespace m = xlings::subos::manifest; +namespace fs = std::filesystem; + +namespace { + +m::Placeholders test_placeholders() { + return m::Placeholders{ + .subosdir = "/x/subos/default", + .home = "/home/u", + .xlings_home = "/x", + .pkgdir_of = [](std::string_view binding) -> fs::path { + if (binding == "compat.mesa@25.0.0") return "/x/pkgs/compat.mesa/25.0.0"; + if (binding == "fontconfig@2.15.0") return "/x/pkgs/fontconfig/2.15.0"; + return {}; // unknown provider → unresolvable, on purpose + }, + }; +} + +nlohmann::json doc_with(const nlohmann::json& envs, + std::string runtime = "glibc@2.39") { + nlohmann::json d; + d["workspace"] = nlohmann::json::object(); + d["subos_info"] = { + {"schema_version", m::SCHEMA_VERSION}, + {"runtime", std::move(runtime)}, + {"envs", envs}, + {"created_at", "2026-08-05T14:23:11Z"}, + {"created_by", "xlings test"}, + }; + return d; +} + +bool has(const std::vector& fs, m::Defect d) { + return std::ranges::any_of(fs, [&](const auto& f) { return f.kind == d; }); +} + +} // namespace + +// ── runtime family ─────────────────────────────────────────────────── + +TEST(SubosManifestRuntime, DerivesFamilyFromThePackageName) { + EXPECT_EQ(m::family_of("glibc@2.39"), "linux-x86_64-glibc"); + EXPECT_EQ(m::family_of("musl@1.2.5"), "linux-x86_64-musl"); + EXPECT_EQ(m::family_of("glibc@2.39", "aarch64"), "linux-aarch64-glibc"); + EXPECT_EQ(m::family_of("wasi-libc@0.1"), "wasm32-wasi"); +} + +// A family that is derived cannot contradict the runtime it came from, which +// is the whole reason it is not a stored field. +TEST(SubosManifestRuntime, UnknownRuntimeIsNamedRatherThanGuessed) { + EXPECT_EQ(m::family_of("something-else@1.0"), "unknown"); + EXPECT_EQ(m::family_of(""), "unknown"); +} + +TEST(SubosManifestRuntime, BindingShapeRequiresBothHalves) { + EXPECT_TRUE(m::is_binding("glibc@2.39")); + EXPECT_FALSE(m::is_binding("glibc")); + EXPECT_FALSE(m::is_binding("glibc@")); + EXPECT_FALSE(m::is_binding("@2.39")); +} + +// ── invariants ─────────────────────────────────────────────────────── + +TEST(SubosManifestValidate, AcceptsAWellFormedBlock) { + EXPECT_TRUE(m::validate_block(doc_with(nlohmann::json::object())).empty()); +} + +TEST(SubosManifestValidate, ReportsAMissingBlock) { + nlohmann::json d; + d["workspace"] = nlohmann::json::object(); + EXPECT_TRUE(has(m::validate_block(d), m::Defect::BlockMissing)); +} + +// An empty envs object is the correct state for a subos with no declarations, +// and must not be confused with a missing one — the whole point of writing {} +// at creation is that no reader has to handle "absent". +TEST(SubosManifestValidate, EmptyEnvsIsValidButAbsentEnvsIsNot) { + EXPECT_TRUE(m::validate_block(doc_with(nlohmann::json::object())).empty()); + + auto d = doc_with(nlohmann::json::object()); + d["subos_info"].erase("envs"); + EXPECT_TRUE(has(m::validate_block(d), m::Defect::EnvsMalformed)); +} + +TEST(SubosManifestValidate, RejectsAMalformedRuntime) { + EXPECT_TRUE(has(m::validate_block(doc_with(nlohmann::json::object(), "glibc")), + m::Defect::RuntimeMalformed)); +} + +TEST(SubosManifestValidate, RejectsAProviderKeyThatIsNotABinding) { + auto d = doc_with({{"compat.mesa", nlohmann::json::array()}}); + EXPECT_TRUE(has(m::validate_block(d), m::Defect::EnvsMalformed)); +} + +TEST(SubosManifestValidate, RejectsADeclarationWithAnOpThisSliceCannotApply) { + auto d = doc_with({{"compat.mesa@25.0.0", nlohmann::json::array({ + {{"var", "PATH"}, {"op", "append"}, {"value", "x"}}, + })}}); + EXPECT_TRUE(has(m::validate_block(d), m::Defect::EnvDeclMalformed)); +} + +TEST(SubosManifestValidate, RejectsAFutureSchemaVersion) { + auto d = doc_with(nlohmann::json::object()); + d["subos_info"]["schema_version"] = m::SCHEMA_VERSION + 1; + EXPECT_TRUE(has(m::validate_block(d), m::Defect::SchemaUnsupported)); +} + +TEST(SubosManifestValidate, ReportsMissingProvenance) { + auto d = doc_with(nlohmann::json::object()); + d["subos_info"]["created_by"] = ""; + EXPECT_TRUE(has(m::validate_block(d), m::Defect::ProvenanceMissing)); +} + +// validate() on disk, including the case that must not be treated as "absent": +// a file that exists but does not parse. Silently reading it as {} would let a +// repair rewrite a document it never managed to read. +TEST(SubosManifestValidate, DistinguishesAbsentFromUnreadable) { + auto dir = fs::temp_directory_path() / "xlings_subos_manifest_validate"; + fs::remove_all(dir); + fs::create_directories(dir); + + EXPECT_TRUE(has(m::validate(dir), m::Defect::ConfigMissing)); + + std::ofstream(m::config_path(dir)) << "{ this is not json"; + EXPECT_TRUE(has(m::validate(dir), m::Defect::ConfigUnreadable)); + + std::ofstream(m::config_path(dir)) << doc_with(nlohmann::json::object()).dump(2); + EXPECT_TRUE(m::validate(dir).empty()); + + fs::remove_all(dir); +} + +TEST(SubosManifestValidate, ReportsAMissingDirectory) { + EXPECT_TRUE(has(m::validate(fs::temp_directory_path() / "xlings_no_such_subos"), + m::Defect::DirMissing)); +} + +// ── declarations ───────────────────────────────────────────────────── + +TEST(SubosManifestEnv, RecordsUnderTheProviderBinding) { + auto d = doc_with(nlohmann::json::object()); + EXPECT_TRUE(m::add_env(d, "compat.mesa@25.0.0", + {"LIBGL_DRIVERS_PATH", "set", "${pkgdir}/lib/dri"})); + ASSERT_TRUE(d["subos_info"]["envs"].contains("compat.mesa@25.0.0")); + EXPECT_EQ(d["subos_info"]["envs"]["compat.mesa@25.0.0"].size(), 1u); +} + +// config() runs again on every dependent package's install. Without this the +// section would grow a duplicate row per rebuild. +TEST(SubosManifestEnv, IsIdempotentOnTheWholeTriple) { + auto d = doc_with(nlohmann::json::object()); + const m::EnvDecl decl{"LIBGL_DRIVERS_PATH", "set", "${pkgdir}/lib/dri"}; + + EXPECT_TRUE(m::add_env(d, "compat.mesa@25.0.0", decl)); + EXPECT_FALSE(m::add_env(d, "compat.mesa@25.0.0", decl)); + EXPECT_EQ(d["subos_info"]["envs"]["compat.mesa@25.0.0"].size(), 1u); + + // A different value for the same variable is a new declaration, not a + // duplicate — the manifest records what was asked for, and resolve() + // decides what wins. + EXPECT_TRUE(m::add_env(d, "compat.mesa@25.0.0", + {"LIBGL_DRIVERS_PATH", "set", "${pkgdir}/lib/other"})); + EXPECT_EQ(d["subos_info"]["envs"]["compat.mesa@25.0.0"].size(), 2u); +} + +TEST(SubosManifestEnv, RemovingAProviderTakesItsWholeSectionAndNothingElse) { + auto d = doc_with(nlohmann::json::object()); + m::add_env(d, "compat.mesa@25.0.0", {"LIBGL_DRIVERS_PATH", "set", "a"}); + m::add_env(d, "fontconfig@2.15.0", {"FONTCONFIG_PATH", "set", "b"}); + + EXPECT_TRUE(m::remove_provider(d, "compat.mesa@25.0.0")); + EXPECT_FALSE(d["subos_info"]["envs"].contains("compat.mesa@25.0.0")); + EXPECT_TRUE(d["subos_info"]["envs"].contains("fontconfig@2.15.0")); + + // Removing what is not there is a no-op, not a failure: uninstall runs for + // packages that never declared anything. + EXPECT_FALSE(m::remove_provider(d, "compat.mesa@25.0.0")); +} + +TEST(SubosManifestEnv, EnvsSurvivesAsAnEmptyObjectAfterTheLastRemoval) { + auto d = doc_with(nlohmann::json::object()); + m::add_env(d, "compat.mesa@25.0.0", {"LIBGL_DRIVERS_PATH", "set", "a"}); + m::remove_provider(d, "compat.mesa@25.0.0"); + + ASSERT_TRUE(d["subos_info"]["envs"].is_object()); + EXPECT_TRUE(d["subos_info"]["envs"].empty()); + EXPECT_TRUE(m::validate_block(d).empty()); +} + +TEST(SubosManifestEnv, FindsStaleProvidersOfTheSamePackageAcrossVersions) { + auto d = doc_with(nlohmann::json::object()); + m::add_env(d, "compat.mesa@25.0.0", {"A", "set", "1"}); + m::add_env(d, "compat.mesa@24.0.0", {"A", "set", "2"}); + m::add_env(d, "fontconfig@2.15.0", {"B", "set", "3"}); + + auto found = m::providers_named(d, "compat.mesa"); + EXPECT_EQ(found.size(), 2u); +} + +// ── placeholders ───────────────────────────────────────────────────── + +TEST(SubosManifestExpand, SubstitutesEveryKnownPlaceholder) { + auto ph = test_placeholders(); + EXPECT_EQ(m::expand("${pkgdir}/lib/dri", "compat.mesa@25.0.0", ph), + "/x/pkgs/compat.mesa/25.0.0/lib/dri"); + EXPECT_EQ(m::expand("${subosdir}/usr", "any@1", ph), "/x/subos/default/usr"); + EXPECT_EQ(m::expand("${home}/.cache", "any@1", ph), "/home/u/.cache"); + EXPECT_EQ(m::expand("${xlings_home}/data", "any@1", ph), "/x/data"); +} + +TEST(SubosManifestExpand, HandlesSeveralPlaceholdersAndPlainText) { + auto ph = test_placeholders(); + EXPECT_EQ(m::expand("a:${home}/b:${xlings_home}/c", "any@1", ph), + "a:/home/u/b:/x/c"); +} + +// The reason unresolvable does not mean empty: "${pkgdir}/lib/dri" collapsing +// to "/lib/dri" is a real path on the host, outside the subos, and a driver +// search would follow it. Leaving the text intact keeps the failure visible. +TEST(SubosManifestExpand, LeavesAnUnresolvableProviderVerbatim) { + auto ph = test_placeholders(); + const auto out = m::expand("${pkgdir}/lib/dri", "never.installed@9.9", ph); + EXPECT_EQ(out, "${pkgdir}/lib/dri"); + EXPECT_TRUE(m::has_unresolved(out)); +} + +TEST(SubosManifestExpand, LeavesUnknownAndUnterminatedPlaceholdersVerbatim) { + auto ph = test_placeholders(); + EXPECT_EQ(m::expand("${nope}/x", "any@1", ph), "${nope}/x"); + EXPECT_EQ(m::expand("${unterminated", "any@1", ph), "${unterminated"); + EXPECT_EQ(m::expand("plain", "any@1", ph), "plain"); +} + +// ── resolution ─────────────────────────────────────────────────────── + +TEST(SubosManifestResolve, ExpandsAndSortsDeclarations) { + auto d = doc_with(nlohmann::json::object()); + m::add_env(d, "compat.mesa@25.0.0", + {"LIBGL_DRIVERS_PATH", "set", "${pkgdir}/lib/dri"}); + m::add_env(d, "compat.mesa@25.0.0", + {"__EGL_VENDOR_LIBRARY_DIRS", "set", "${pkgdir}/share/glvnd"}); + + auto resolved = m::resolve(m::parse(d), test_placeholders()); + ASSERT_EQ(resolved.size(), 2u); + EXPECT_EQ(resolved[0].var, "LIBGL_DRIVERS_PATH"); + EXPECT_EQ(resolved[0].value, "/x/pkgs/compat.mesa/25.0.0/lib/dri"); + EXPECT_EQ(resolved[1].var, "__EGL_VENDOR_LIBRARY_DIRS"); + EXPECT_FALSE(resolved[0].conflicted); +} + +TEST(SubosManifestResolve, PrependsComposeWithoutBeingAConflict) { + auto d = doc_with(nlohmann::json::object()); + m::add_env(d, "compat.mesa@25.0.0", {"XDG_DATA_DIRS", "prepend", "${pkgdir}/share"}); + m::add_env(d, "fontconfig@2.15.0", {"XDG_DATA_DIRS", "prepend", "${pkgdir}/share"}); + + auto resolved = m::resolve(m::parse(d), test_placeholders()); + ASSERT_EQ(resolved.size(), 1u); + EXPECT_EQ(resolved[0].op, "prepend"); + EXPECT_FALSE(resolved[0].conflicted); + // Later provider nearer the front: "the newest thing installed is found + // first". Ordering is by binding, so it does not depend on install history. + EXPECT_EQ(resolved[0].value, + "/x/pkgs/fontconfig/2.15.0/share:/x/pkgs/compat.mesa/25.0.0/share"); + EXPECT_EQ(resolved[0].providers.size(), 2u); +} + +TEST(SubosManifestResolve, TwoSetsOnOneVariableAreReportedAsAConflict) { + auto d = doc_with(nlohmann::json::object()); + m::add_env(d, "compat.mesa@25.0.0", {"MESA_LOADER_DRIVER_OVERRIDE", "set", "a"}); + m::add_env(d, "fontconfig@2.15.0", {"MESA_LOADER_DRIVER_OVERRIDE", "set", "b"}); + + auto resolved = m::resolve(m::parse(d), test_placeholders()); + ASSERT_EQ(resolved.size(), 1u); + EXPECT_TRUE(resolved[0].conflicted); + // Last in binding order wins. The value matters less than the fact that it + // is the same on every machine holding this manifest. + EXPECT_EQ(resolved[0].value, "b"); +} + +TEST(SubosManifestResolve, SetBeatsPrependAndSaysSo) { + auto d = doc_with(nlohmann::json::object()); + m::add_env(d, "compat.mesa@25.0.0", {"XDG_DATA_DIRS", "prepend", "p"}); + m::add_env(d, "fontconfig@2.15.0", {"XDG_DATA_DIRS", "set", "s"}); + + auto resolved = m::resolve(m::parse(d), test_placeholders()); + ASSERT_EQ(resolved.size(), 1u); + EXPECT_EQ(resolved[0].op, "set"); + EXPECT_EQ(resolved[0].value, "s"); + EXPECT_TRUE(resolved[0].conflicted); +} + +// Resolution must be a function of the manifest alone. Two homes holding the +// same bytes have to export the same values, or a shared subos description +// describes nothing. +TEST(SubosManifestResolve, DoesNotDependOnDeclarationOrder) { + auto forwards = doc_with(nlohmann::json::object()); + m::add_env(forwards, "a.pkg@1.0", {"V", "prepend", "one"}); + m::add_env(forwards, "b.pkg@1.0", {"V", "prepend", "two"}); + + auto backwards = doc_with(nlohmann::json::object()); + m::add_env(backwards, "b.pkg@1.0", {"V", "prepend", "two"}); + m::add_env(backwards, "a.pkg@1.0", {"V", "prepend", "one"}); + + auto ph = test_placeholders(); + EXPECT_EQ(m::resolve(m::parse(forwards), ph)[0].value, + m::resolve(m::parse(backwards), ph)[0].value); +} + +TEST(SubosManifestResolve, FlagsAValueItCouldNotExpand) { + auto d = doc_with(nlohmann::json::object()); + m::add_env(d, "never.installed@9.9", {"V", "set", "${pkgdir}/lib"}); + + auto resolved = m::resolve(m::parse(d), test_placeholders()); + ASSERT_EQ(resolved.size(), 1u); + EXPECT_TRUE(resolved[0].unresolved); +} + +// ── creation ───────────────────────────────────────────────────────── + +TEST(SubosManifestBlock, NewBlockSatisfiesItsOwnInvariants) { + nlohmann::json d; + d["workspace"] = nlohmann::json::object(); + d["subos_info"] = m::make_block(m::DEFAULT_RUNTIME, "xlings test"); + + EXPECT_TRUE(m::validate_block(d).empty()); + + auto info = m::parse(d); + EXPECT_EQ(info.schema_version, m::SCHEMA_VERSION); + EXPECT_EQ(info.runtime, m::DEFAULT_RUNTIME); + EXPECT_TRUE(info.envs.empty()); + EXPECT_FALSE(info.created_at.empty()); +} From d4828a5299a990fa69d852fc08c93cb28a7696ed Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Wed, 5 Aug 2026 05:05:53 +0800 Subject: [PATCH 2/4] ci: an inexact mcpp cache restore must not keep its BMIs The mcpp cache is keyed on hashFiles(mcpp.toml, mcpp.lock, .xlings.json) with a `mcpp--` fallback. Changing a dependency misses the exact key, the fallback restores BMIs built against the previous dependency set, and the build fails with mcpplibs.tinyhttps: error: import 'std' has CRC mismatch which reads like a compiler bug and is a cache restored for the wrong inputs. The existing retry does not help -- it was written for a stale index entry, and re-running the same build against the same cache fails identically. Bumping only the version field never triggered it (PR #479 passed); bumping a dependency does. Measured here on mcpplibs.xpkg 0.0.47 -> 0.0.48. Drops only ~/.mcpp/{bmi,build-cache} on an inexact restore. The payload store under registry/data/xpkgs is dependency-independent and stays -- it is what makes restoring the ~800 MB cache worth doing. --- .github/workflows/release.yml | 17 +++++++++++++++++ .github/workflows/xlings-ci-linux-e2e.yml | 17 +++++++++++++++++ .github/workflows/xlings-ci-linux-root.yml | 17 +++++++++++++++++ .github/workflows/xlings-ci-linux.yml | 17 +++++++++++++++++ .github/workflows/xlings-ci-macos.yml | 17 +++++++++++++++++ .github/workflows/xlings-ci-windows.yml | 17 +++++++++++++++++ 6 files changed, 102 insertions(+) diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index b972a780..0943162c 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -50,6 +50,7 @@ jobs: xlings-deps-${{ runner.os }}- - name: Cache mcpp sandbox and project packages + id: mcpp-cache uses: actions/cache@v4 with: path: | @@ -58,6 +59,22 @@ jobs: key: mcpp-${{ runner.os }}-${{ hashFiles('mcpp.toml', 'mcpp.lock', '.xlings.json') }}-${{ env.BOOTSTRAP_XLINGS_VERSION }} restore-keys: | mcpp-${{ runner.os }}- + # An inexact restore (the `restore-keys` fallback) brings BMIs built + # under a different dependency set. Mixing them with anything rebuilt + # here fails as `import 'std' has CRC mismatch` — which reads like a + # compiler bug and is a cache restored for the wrong inputs. Measured on + # PR #480, where `mcpplibs.xpkg` went 0.0.47 → 0.0.48: the fallback + # cache's tinyhttps/cmdline BMIs no longer matched the `std` the fresh + # xpkg build produced. + # + # Only the BMIs go. The payload store under registry/data/xpkgs is + # dependency-independent and is what makes the ~800 MB cache worth + # restoring at all. + - name: Drop stale BMIs when the cache key did not match exactly + if: steps.mcpp-cache.outputs.cache-hit != 'true' + shell: bash + run: | + rm -rf ~/.mcpp/bmi ~/.mcpp/build-cache .mcpp/bmi .mcpp/build-cache - name: Prepare bootstrap package index run: | diff --git a/.github/workflows/xlings-ci-linux-e2e.yml b/.github/workflows/xlings-ci-linux-e2e.yml index 35d40841..4ae256e3 100644 --- a/.github/workflows/xlings-ci-linux-e2e.yml +++ b/.github/workflows/xlings-ci-linux-e2e.yml @@ -58,6 +58,7 @@ jobs: xlings-deps-${{ runner.os }}- - name: Cache mcpp sandbox and project packages + id: mcpp-cache uses: actions/cache@v4 with: path: | @@ -66,6 +67,22 @@ jobs: key: mcpp-${{ runner.os }}-${{ hashFiles('mcpp.toml', 'mcpp.lock', '.xlings.json') }}-${{ env.BOOTSTRAP_XLINGS_VERSION }} restore-keys: | mcpp-${{ runner.os }}- + # An inexact restore (the `restore-keys` fallback) brings BMIs built + # under a different dependency set. Mixing them with anything rebuilt + # here fails as `import 'std' has CRC mismatch` — which reads like a + # compiler bug and is a cache restored for the wrong inputs. Measured on + # PR #480, where `mcpplibs.xpkg` went 0.0.47 → 0.0.48: the fallback + # cache's tinyhttps/cmdline BMIs no longer matched the `std` the fresh + # xpkg build produced. + # + # Only the BMIs go. The payload store under registry/data/xpkgs is + # dependency-independent and is what makes the ~800 MB cache worth + # restoring at all. + - name: Drop stale BMIs when the cache key did not match exactly + if: steps.mcpp-cache.outputs.cache-hit != 'true' + shell: bash + run: | + rm -rf ~/.mcpp/bmi ~/.mcpp/build-cache .mcpp/bmi .mcpp/build-cache - name: Prepare bootstrap package index run: | diff --git a/.github/workflows/xlings-ci-linux-root.yml b/.github/workflows/xlings-ci-linux-root.yml index a7c20ed7..aaa309dc 100644 --- a/.github/workflows/xlings-ci-linux-root.yml +++ b/.github/workflows/xlings-ci-linux-root.yml @@ -55,6 +55,7 @@ jobs: xlings-deps-${{ runner.os }}- - name: Cache mcpp sandbox and project packages + id: mcpp-cache uses: actions/cache@v4 with: path: | @@ -63,6 +64,22 @@ jobs: key: mcpp-${{ runner.os }}-${{ hashFiles('mcpp.toml', 'mcpp.lock', '.xlings.json') }}-${{ env.BOOTSTRAP_XLINGS_VERSION }} restore-keys: | mcpp-${{ runner.os }}- + # An inexact restore (the `restore-keys` fallback) brings BMIs built + # under a different dependency set. Mixing them with anything rebuilt + # here fails as `import 'std' has CRC mismatch` — which reads like a + # compiler bug and is a cache restored for the wrong inputs. Measured on + # PR #480, where `mcpplibs.xpkg` went 0.0.47 → 0.0.48: the fallback + # cache's tinyhttps/cmdline BMIs no longer matched the `std` the fresh + # xpkg build produced. + # + # Only the BMIs go. The payload store under registry/data/xpkgs is + # dependency-independent and is what makes the ~800 MB cache worth + # restoring at all. + - name: Drop stale BMIs when the cache key did not match exactly + if: steps.mcpp-cache.outputs.cache-hit != 'true' + shell: bash + run: | + rm -rf ~/.mcpp/bmi ~/.mcpp/build-cache .mcpp/bmi .mcpp/build-cache - name: Prepare bootstrap package index run: | diff --git a/.github/workflows/xlings-ci-linux.yml b/.github/workflows/xlings-ci-linux.yml index 6e865b1f..f2849d39 100644 --- a/.github/workflows/xlings-ci-linux.yml +++ b/.github/workflows/xlings-ci-linux.yml @@ -60,6 +60,7 @@ jobs: xlings-deps-${{ runner.os }}- - name: Cache mcpp sandbox and project packages + id: mcpp-cache uses: actions/cache@v4 with: path: | @@ -68,6 +69,22 @@ jobs: key: mcpp-${{ runner.os }}-${{ hashFiles('mcpp.toml', 'mcpp.lock', '.xlings.json') }}-${{ env.BOOTSTRAP_XLINGS_VERSION }} restore-keys: | mcpp-${{ runner.os }}- + # An inexact restore (the `restore-keys` fallback) brings BMIs built + # under a different dependency set. Mixing them with anything rebuilt + # here fails as `import 'std' has CRC mismatch` — which reads like a + # compiler bug and is a cache restored for the wrong inputs. Measured on + # PR #480, where `mcpplibs.xpkg` went 0.0.47 → 0.0.48: the fallback + # cache's tinyhttps/cmdline BMIs no longer matched the `std` the fresh + # xpkg build produced. + # + # Only the BMIs go. The payload store under registry/data/xpkgs is + # dependency-independent and is what makes the ~800 MB cache worth + # restoring at all. + - name: Drop stale BMIs when the cache key did not match exactly + if: steps.mcpp-cache.outputs.cache-hit != 'true' + shell: bash + run: | + rm -rf ~/.mcpp/bmi ~/.mcpp/build-cache .mcpp/bmi .mcpp/build-cache - name: Prepare bootstrap package index run: | diff --git a/.github/workflows/xlings-ci-macos.yml b/.github/workflows/xlings-ci-macos.yml index ca885e8b..c7073977 100644 --- a/.github/workflows/xlings-ci-macos.yml +++ b/.github/workflows/xlings-ci-macos.yml @@ -50,6 +50,7 @@ jobs: xlings-deps-${{ runner.os }}- - name: Cache mcpp sandbox and project packages + id: mcpp-cache uses: actions/cache@v4 with: path: | @@ -58,6 +59,22 @@ jobs: key: mcpp-${{ runner.os }}-dt110-${{ hashFiles('mcpp.toml', 'mcpp.lock', '.xlings.json') }}-${{ env.BOOTSTRAP_XLINGS_VERSION }} restore-keys: | mcpp-${{ runner.os }}-dt110- + # An inexact restore (the `restore-keys` fallback) brings BMIs built + # under a different dependency set. Mixing them with anything rebuilt + # here fails as `import 'std' has CRC mismatch` — which reads like a + # compiler bug and is a cache restored for the wrong inputs. Measured on + # PR #480, where `mcpplibs.xpkg` went 0.0.47 → 0.0.48: the fallback + # cache's tinyhttps/cmdline BMIs no longer matched the `std` the fresh + # xpkg build produced. + # + # Only the BMIs go. The payload store under registry/data/xpkgs is + # dependency-independent and is what makes the ~800 MB cache worth + # restoring at all. + - name: Drop stale BMIs when the cache key did not match exactly + if: steps.mcpp-cache.outputs.cache-hit != 'true' + shell: bash + run: | + rm -rf ~/.mcpp/bmi ~/.mcpp/build-cache .mcpp/bmi .mcpp/build-cache - name: Prepare bootstrap package index run: | diff --git a/.github/workflows/xlings-ci-windows.yml b/.github/workflows/xlings-ci-windows.yml index 78bf3078..a57a1cde 100644 --- a/.github/workflows/xlings-ci-windows.yml +++ b/.github/workflows/xlings-ci-windows.yml @@ -50,6 +50,7 @@ jobs: xlings-deps-${{ runner.os }}- - name: Cache mcpp sandbox and project packages + id: mcpp-cache uses: actions/cache@v4 with: path: | @@ -58,6 +59,22 @@ jobs: key: mcpp-${{ runner.os }}-${{ hashFiles('mcpp.toml', 'mcpp.lock', '.xlings.json') }}-${{ env.BOOTSTRAP_XLINGS_VERSION }} restore-keys: | mcpp-${{ runner.os }}- + # An inexact restore (the `restore-keys` fallback) brings BMIs built + # under a different dependency set. Mixing them with anything rebuilt + # here fails as `import 'std' has CRC mismatch` — which reads like a + # compiler bug and is a cache restored for the wrong inputs. Measured on + # PR #480, where `mcpplibs.xpkg` went 0.0.47 → 0.0.48: the fallback + # cache's tinyhttps/cmdline BMIs no longer matched the `std` the fresh + # xpkg build produced. + # + # Only the BMIs go. The payload store under registry/data/xpkgs is + # dependency-independent and is what makes the ~800 MB cache worth + # restoring at all. + - name: Drop stale BMIs when the cache key did not match exactly + if: steps.mcpp-cache.outputs.cache-hit != 'true' + shell: bash + run: | + rm -rf ~/.mcpp/bmi ~/.mcpp/build-cache .mcpp/bmi .mcpp/build-cache - name: Prepare bootstrap package index shell: pwsh From 135c5ecad7cf6721e7622b4a8ee447a4f29dca5f Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Wed, 5 Aug 2026 05:16:55 +0800 Subject: [PATCH 3/4] docs: regenerate the command reference for `subos new --runtime` Generated from cli/spec.cppm; the new flag has to appear there or test_generated_command_reference fails the build. --- docs/generated/command-reference.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/docs/generated/command-reference.md b/docs/generated/command-reference.md index 68d5270c..fa32f6f6 100644 --- a/docs/generated/command-reference.md +++ b/docs/generated/command-reference.md @@ -64,7 +64,7 @@ Manage SubOS environments Create a SubOS -Options: `--storage ` — shared, tmpfs or image; `--image-size ` — Image size; `--from ` — Fork source +Options: `--storage ` — shared, tmpfs or image; `--image-size ` — Image size; `--from ` — Fork source; `--runtime ` — Runtime binding, e.g. glibc@2.39 ## `xlings subos use ` From 3c7ef5ce229cb77e5c9b2410189d1f10455930e0 Mon Sep 17 00:00:00 2001 From: Sunrisepeak Date: Wed, 5 Aug 2026 05:20:55 +0800 Subject: [PATCH 4/4] ci: retire the poisoned mcpp cache keyspace (mcpp-v2-) The guard added in the previous commit only fires on an INEXACT restore, and by the time it existed the damage was already stored: the first run of this PR restored a fallback BMI set, failed, and saved that state under its own exact key. Every later run then got an exact hit on it, skipped the guard, and failed identically. Retiring the key prefix is what discards those entries. The guard is what keeps a poisoned set from being written again -- an inexact restore now drops its BMIs before anything is built, so what gets saved is internally consistent. Payload reuse across dependency changes is unaffected, which is the reason for keeping restore-keys at all. The first run on each platform after this is cold. --- .github/workflows/release.yml | 51 +++++++++++++++++++--- .github/workflows/xlings-ci-linux-e2e.yml | 9 +++- .github/workflows/xlings-ci-linux-root.yml | 9 +++- .github/workflows/xlings-ci-linux.yml | 9 +++- .github/workflows/xlings-ci-macos.yml | 9 +++- .github/workflows/xlings-ci-windows.yml | 9 +++- 6 files changed, 80 insertions(+), 16 deletions(-) diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index 0943162c..d4677112 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -56,9 +56,14 @@ jobs: path: | ~/.mcpp .mcpp - key: mcpp-${{ runner.os }}-${{ hashFiles('mcpp.toml', 'mcpp.lock', '.xlings.json') }}-${{ env.BOOTSTRAP_XLINGS_VERSION }} + # `mcpp-v2-`: the v1 keyspace holds entries saved by runs that + # built against a fallback-restored BMI set and failed. Those are + # poisoned at their exact key, so the guard below -- which only fires + # on an INEXACT restore -- can never reach them. Retiring the prefix + # is what discards them; the guard is what stops it recurring. + key: mcpp-v2-${{ runner.os }}-${{ hashFiles('mcpp.toml', 'mcpp.lock', '.xlings.json') }}-${{ env.BOOTSTRAP_XLINGS_VERSION }} restore-keys: | - mcpp-${{ runner.os }}- + mcpp-v2-${{ runner.os }}- # An inexact restore (the `restore-keys` fallback) brings BMIs built # under a different dependency set. Mixing them with anything rebuilt # here fails as `import 'std' has CRC mismatch` — which reads like a @@ -223,14 +228,31 @@ jobs: xlings-deps-${{ runner.os }}- - name: Cache mcpp sandbox and project packages + id: mcpp-cache uses: actions/cache@v4 with: path: | ~/.mcpp .mcpp - key: mcpp-${{ runner.os }}-${{ hashFiles('mcpp.toml', 'mcpp.lock', '.xlings.json') }}-${{ env.BOOTSTRAP_XLINGS_VERSION }} + # `mcpp-v2-`: the v1 keyspace holds entries saved by runs that + # built against a fallback-restored BMI set and failed. Those are + # poisoned at their exact key, so the guard below -- which only fires + # on an INEXACT restore -- can never reach them. Retiring the prefix + # is what discards them; the guard is what stops it recurring. + key: mcpp-v2-${{ runner.os }}-${{ hashFiles('mcpp.toml', 'mcpp.lock', '.xlings.json') }}-${{ env.BOOTSTRAP_XLINGS_VERSION }} restore-keys: | - mcpp-${{ runner.os }}- + mcpp-v2-${{ runner.os }}- + # An inexact restore brings BMIs built under a different dependency set. + # Mixing them with anything rebuilt here fails as `import 'std' has CRC + # mismatch`, which reads like a compiler bug. Only the BMIs go -- the + # payload store is dependency-independent and is what makes the ~800 MB + # cache worth restoring. + - name: Drop stale BMIs when the cache key did not match exactly + if: steps.mcpp-cache.outputs.cache-hit != 'true' + shell: bash + run: | + rm -rf ~/.mcpp/bmi ~/.mcpp/build-cache .mcpp/bmi .mcpp/build-cache + - name: Prepare bootstrap package index run: | @@ -314,14 +336,31 @@ jobs: xlings-deps-${{ runner.os }}- - name: Cache mcpp sandbox and project packages + id: mcpp-cache uses: actions/cache@v4 with: path: | ~\.mcpp .mcpp - key: mcpp-${{ runner.os }}-${{ hashFiles('mcpp.toml', 'mcpp.lock', '.xlings.json') }}-${{ env.BOOTSTRAP_XLINGS_VERSION }} + # `mcpp-v2-`: the v1 keyspace holds entries saved by runs that + # built against a fallback-restored BMI set and failed. Those are + # poisoned at their exact key, so the guard below -- which only fires + # on an INEXACT restore -- can never reach them. Retiring the prefix + # is what discards them; the guard is what stops it recurring. + key: mcpp-v2-${{ runner.os }}-${{ hashFiles('mcpp.toml', 'mcpp.lock', '.xlings.json') }}-${{ env.BOOTSTRAP_XLINGS_VERSION }} restore-keys: | - mcpp-${{ runner.os }}- + mcpp-v2-${{ runner.os }}- + # An inexact restore brings BMIs built under a different dependency set. + # Mixing them with anything rebuilt here fails as `import 'std' has CRC + # mismatch`, which reads like a compiler bug. Only the BMIs go -- the + # payload store is dependency-independent and is what makes the ~800 MB + # cache worth restoring. + - name: Drop stale BMIs when the cache key did not match exactly + if: steps.mcpp-cache.outputs.cache-hit != 'true' + shell: bash + run: | + rm -rf ~/.mcpp/bmi ~/.mcpp/build-cache .mcpp/bmi .mcpp/build-cache + - name: Prepare bootstrap package index shell: pwsh diff --git a/.github/workflows/xlings-ci-linux-e2e.yml b/.github/workflows/xlings-ci-linux-e2e.yml index 4ae256e3..2806fd07 100644 --- a/.github/workflows/xlings-ci-linux-e2e.yml +++ b/.github/workflows/xlings-ci-linux-e2e.yml @@ -64,9 +64,14 @@ jobs: path: | ~/.mcpp .mcpp - key: mcpp-${{ runner.os }}-${{ hashFiles('mcpp.toml', 'mcpp.lock', '.xlings.json') }}-${{ env.BOOTSTRAP_XLINGS_VERSION }} + # `mcpp-v2-`: the v1 keyspace holds entries saved by runs that + # built against a fallback-restored BMI set and failed. Those are + # poisoned at their exact key, so the guard below -- which only fires + # on an INEXACT restore -- can never reach them. Retiring the prefix + # is what discards them; the guard is what stops it recurring. + key: mcpp-v2-${{ runner.os }}-${{ hashFiles('mcpp.toml', 'mcpp.lock', '.xlings.json') }}-${{ env.BOOTSTRAP_XLINGS_VERSION }} restore-keys: | - mcpp-${{ runner.os }}- + mcpp-v2-${{ runner.os }}- # An inexact restore (the `restore-keys` fallback) brings BMIs built # under a different dependency set. Mixing them with anything rebuilt # here fails as `import 'std' has CRC mismatch` — which reads like a diff --git a/.github/workflows/xlings-ci-linux-root.yml b/.github/workflows/xlings-ci-linux-root.yml index aaa309dc..a4377c62 100644 --- a/.github/workflows/xlings-ci-linux-root.yml +++ b/.github/workflows/xlings-ci-linux-root.yml @@ -61,9 +61,14 @@ jobs: path: | ~/.mcpp .mcpp - key: mcpp-${{ runner.os }}-${{ hashFiles('mcpp.toml', 'mcpp.lock', '.xlings.json') }}-${{ env.BOOTSTRAP_XLINGS_VERSION }} + # `mcpp-v2-`: the v1 keyspace holds entries saved by runs that + # built against a fallback-restored BMI set and failed. Those are + # poisoned at their exact key, so the guard below -- which only fires + # on an INEXACT restore -- can never reach them. Retiring the prefix + # is what discards them; the guard is what stops it recurring. + key: mcpp-v2-${{ runner.os }}-${{ hashFiles('mcpp.toml', 'mcpp.lock', '.xlings.json') }}-${{ env.BOOTSTRAP_XLINGS_VERSION }} restore-keys: | - mcpp-${{ runner.os }}- + mcpp-v2-${{ runner.os }}- # An inexact restore (the `restore-keys` fallback) brings BMIs built # under a different dependency set. Mixing them with anything rebuilt # here fails as `import 'std' has CRC mismatch` — which reads like a diff --git a/.github/workflows/xlings-ci-linux.yml b/.github/workflows/xlings-ci-linux.yml index f2849d39..1f4e444a 100644 --- a/.github/workflows/xlings-ci-linux.yml +++ b/.github/workflows/xlings-ci-linux.yml @@ -66,9 +66,14 @@ jobs: path: | ~/.mcpp .mcpp - key: mcpp-${{ runner.os }}-${{ hashFiles('mcpp.toml', 'mcpp.lock', '.xlings.json') }}-${{ env.BOOTSTRAP_XLINGS_VERSION }} + # `mcpp-v2-`: the v1 keyspace holds entries saved by runs that + # built against a fallback-restored BMI set and failed. Those are + # poisoned at their exact key, so the guard below -- which only fires + # on an INEXACT restore -- can never reach them. Retiring the prefix + # is what discards them; the guard is what stops it recurring. + key: mcpp-v2-${{ runner.os }}-${{ hashFiles('mcpp.toml', 'mcpp.lock', '.xlings.json') }}-${{ env.BOOTSTRAP_XLINGS_VERSION }} restore-keys: | - mcpp-${{ runner.os }}- + mcpp-v2-${{ runner.os }}- # An inexact restore (the `restore-keys` fallback) brings BMIs built # under a different dependency set. Mixing them with anything rebuilt # here fails as `import 'std' has CRC mismatch` — which reads like a diff --git a/.github/workflows/xlings-ci-macos.yml b/.github/workflows/xlings-ci-macos.yml index c7073977..ce0a111b 100644 --- a/.github/workflows/xlings-ci-macos.yml +++ b/.github/workflows/xlings-ci-macos.yml @@ -56,9 +56,14 @@ jobs: path: | ~/.mcpp .mcpp - key: mcpp-${{ runner.os }}-dt110-${{ hashFiles('mcpp.toml', 'mcpp.lock', '.xlings.json') }}-${{ env.BOOTSTRAP_XLINGS_VERSION }} + # `mcpp-v2-`: the v1 keyspace holds entries saved by runs that + # built against a fallback-restored BMI set and failed. Those are + # poisoned at their exact key, so the guard below -- which only fires + # on an INEXACT restore -- can never reach them. Retiring the prefix + # is what discards them; the guard is what stops it recurring. + key: mcpp-v2-${{ runner.os }}-dt110-${{ hashFiles('mcpp.toml', 'mcpp.lock', '.xlings.json') }}-${{ env.BOOTSTRAP_XLINGS_VERSION }} restore-keys: | - mcpp-${{ runner.os }}-dt110- + mcpp-v2-${{ runner.os }}-dt110- # An inexact restore (the `restore-keys` fallback) brings BMIs built # under a different dependency set. Mixing them with anything rebuilt # here fails as `import 'std' has CRC mismatch` — which reads like a diff --git a/.github/workflows/xlings-ci-windows.yml b/.github/workflows/xlings-ci-windows.yml index a57a1cde..b9c6bfc7 100644 --- a/.github/workflows/xlings-ci-windows.yml +++ b/.github/workflows/xlings-ci-windows.yml @@ -56,9 +56,14 @@ jobs: path: | ~\.mcpp .mcpp - key: mcpp-${{ runner.os }}-${{ hashFiles('mcpp.toml', 'mcpp.lock', '.xlings.json') }}-${{ env.BOOTSTRAP_XLINGS_VERSION }} + # `mcpp-v2-`: the v1 keyspace holds entries saved by runs that + # built against a fallback-restored BMI set and failed. Those are + # poisoned at their exact key, so the guard below -- which only fires + # on an INEXACT restore -- can never reach them. Retiring the prefix + # is what discards them; the guard is what stops it recurring. + key: mcpp-v2-${{ runner.os }}-${{ hashFiles('mcpp.toml', 'mcpp.lock', '.xlings.json') }}-${{ env.BOOTSTRAP_XLINGS_VERSION }} restore-keys: | - mcpp-${{ runner.os }}- + mcpp-v2-${{ runner.os }}- # An inexact restore (the `restore-keys` fallback) brings BMIs built # under a different dependency set. Mixing them with anything rebuilt # here fails as `import 'std' has CRC mismatch` — which reads like a