diff --git a/.agents/docs/2026-08-06-subos-architecture-proposal.md b/.agents/docs/2026-08-06-subos-architecture-proposal.md new file mode 100644 index 00000000..46ade205 --- /dev/null +++ b/.agents/docs/2026-08-06-subos-architecture-proposal.md @@ -0,0 +1,896 @@ +# subos 生态:架构级优化方案 + +日期:2026-08-06 +输入:`.agents/docs/2026-08-06-subos-matrix-verification.md` 的 5 个缺陷 +性质:设计提案,待 review 后再实施 + +--- + +## 0. 为什么需要这份文档 + +上一轮验证修掉的 4 个缺陷都是**症状**。把它们和 8 月 5 日那轮的 glibc 崩溃放在一起看,底下是**三个不同的架构问题**,每一个都还会继续产出新的缺陷: + +| 架构问题 | 已产出的缺陷 | 还会产出什么 | +|---|---|---| +| P1 一个问题有多个回答者 | 依赖版本(8-05)、home 在哪里(8-06 四个回答者) | 每加一个"缺省即约定"的契约,就多一批 | +| P2 用进程全局机制满足单库需求 | libc 上 `LD_LIBRARY_PATH` 杀死 shell | 每个 host-link 类包都会重演,且各自答案不同 —— **已找到机制层面的出路并端到端验证,见 §2.3** | +| P3 subos 层没有"恰好一个"的执行点 | 同一 subos 绑定两个 mesa | 任何"装第二个版本"的操作 | + +另有一个横切属性:**沉默成功**——"没发生"和"成功了"输出相同。它不是第四个问题,而是让上面三个都变得难以发现的原因。 + +下面每一节给出:现状实测 → 为什么是架构问题 → 方案与取舍 → 可观测性要求。 + +--- + +## 1. P1:一个问题有多个回答者 + +### 1.1 已确诊两次 + +**"这个依赖是哪个版本?"**(2026-08-05)四个回答者:resolver 的 plan node、libxpkg `pkginfo` 的目录扫描、xvm 的 active version、`elfpatch` 的 `{lib64, lib}` 约定探测。 + +**"xlings home 在哪里?"**(2026-08-06)四个回答者:`XLINGS_HOME` 环境变量、沙箱绑定目标、烘焙进产物的绝对路径、shim 从 argv0 反推的 owner home。 + +两次都是同一个机制:**在默认配置下所有答案恰好相同,所以从未被迫达成一致**。第二个版本 / 第二个 home 出现的那一刻,它们同时分歧。 + +### 1.2 什么让一个问题长出多个回答者 + +共同前提是**契约里写了"缺省即约定"**: + +> `deps_exports` 里没有条目,意味着这个依赖什么都没声明——回退到约定。 + +一句"没有就自己猜"等于授权每个读端各自实现一份猜测。回答者的数量等于读端的数量,而读端会随时间增加。 + +`XLINGS_HOME` 是同一句话的另一种写法:"没设就用 `$HOME/.xlings`"。默认路径把四个独立计算变成了同一个字符串。 + +### 1.3 七条规则,建议提升为规范 + +`2026-08-05-dependency-resolution-single-source.md` 里推导出五条,本轮补上两条。七条都是通用的: + +- **R1 权威记录必须是全量的** —— 每一项都记,不只是"声明了东西的那些"。值通常本来就算出来了,只是被 `break` 扔掉。 +- **R2 约定只在写端应用** —— 读端永远不猜。 +- **R3 删除而非调和** —— 如果改动是**增加**一条路径而不是**移除**一条,它是 workaround。 +- **R4 对产物断言,不对意图断言** —— 在安装时失败,而不是在运行时。 +- **R5 决策必须持久化** —— 需要复现才能查看的决策不叫可追溯。 +- **R6 内部消费者绑定 payload,不绑定视图** —— 见 §1.5(本轮补上)。 +- **R7 闭包完整** —— 关于"需要什么 / 引用了什么"的测量必须覆盖传递闭包,不能只取一个入口。见 AD-14(本轮补上)。 + +**提议 A1**:把这七条写进 `xim-pkgindex/docs/V2/xpackage-spec.md` 的规范正文(目前只在 xlings 的设计文档里),并给每条配一个可执行判据,比如 R3 的判据: + +> 一个修复如果只是让两个独立答案**更可能一致**,它是 workaround。只有**删掉第二个回答者**才是解决。 +> libxpkg 0.0.49 没通过这条(它让扫描取最高版本而不是直接失败,仍与 `pin_target_to_active` 分歧);0.0.50 通过了。 + +**提议 A2**:契约文档里**禁止**"缺省即约定"式措辞。凡是"没有 X 就回退到 Y"的句子,要么改成"X 必须存在"(写端保证全量),要么改成"没有 X 是错误"。 + +### 1.4 下一个还没修的实例 + +**"一个 dlopen 进来的宿主文件,去哪里找它的依赖?"** 今天有三个答案(recipe 的手写表、宿主默认搜索、什么都不做),见 §2。 +§2.3 给出的 interposer 机制把它收敛为一个:**链接期依赖由我们拥有的对象上的 DT_RPATH 回答,运行时 dlopen 由宿主回答**,两者界线可判定。 + +### 1.5 R6:内部消费者绑定 payload,不绑定视图 + +三层模型里,前两层的**消费者不同**,而这一点从来没被写成规则: + +| 层 | 是什么 | 谁应该消费它 | +|---|---|---| +| **payload** `data/xpkgs///…` | 不可变、唯一确定的产物 | **xlings 与 libxpkg 自身**;RPATH / INTERP;`resolved_deps` | +| **subos sysroot / bin / PATH** | 给用户的**选择性视图**:可变、经 shim、受 `xlings use` 影响、可能属于别的 home | 用户,以及用户运行的程序 | +| 每个消费者的 RPATH/INTERP | 安装时冻结的决定 | 动态加载器 | + +> **xlings 自己需要一个工具时,必须解析到 payload。视图只服务用户程序。** + +内部代码去消费第二层,是层级倒置:它把自己的正确性交给一个**用户可以随时改变**的选择。而且视图与 payload 在默认状态下**恰好一致**——又是同一个陷阱。 + +这条规则**吞掉**了先前"借用了另一个 home 的东西"那个说法:那只是视图与 payload 分歧的**一种**方式,不是一个独立类别。 + +#### 违反一:`locate_proot_` 的 PATH 回退 + +已修,但修得不够彻底:当时只拒绝了位于 xlings home 内的候选。按 R6,**整条 PATH 步骤对内部使用都是错的**。真正的 `/usr/bin/proot` 属于**宿主依赖**,按 §2.8 应当是一条**显式声明**加一句可见的提示,而不是一个与 payload 平级的静默候选。 + +**提议 A4**:`locate_proot_` 去掉 PATH 步骤;宿主 proot 作为显式声明的回退保留,并在使用时打印一行说明。 + +#### 违反二:libxpkg 的 `_find_tool` —— 这条更严重 + +`elfpatch` 用它找 **patchelf**,而 patchelf 正是给每一个载荷烙上 INTERP 与 RPATH 的工具。候选顺序: + +``` +1. subos_sysrootdir/bin/ ← 视图 +2. _RUNTIME.bin_dir/ ← 视图 +3. /usr/bin/, /usr/local/bin/ ← 宿主 +4. which ← PATH +``` + +**payload 路径根本不在候选里。** + +实测(`prodhome`): + +| 候选 | 实际是什么 | +|---|---| +| payload `xim-x-patchelf/0.18.0/bin/patchelf` | 唯一确定的文件,**不在列表中** | +| 候选 1 `subos/default/bin/patchelf` | **符号链接指向 xlings 二进制**(shim),exec 时按 `XLINGS_HOME` 与活动 subos 解析 | +| 候选 3 `/usr/bin/patchelf` | 宿主的,**这台机器上存在** | + +于是:**同源不变量所依赖的那把工具,身份由可变视图决定,并且带一条通往宿主的静默回退。** patchelf 各版本在动态段增长策略、`--force-rpath` 语义上都有过实际差异,所以"哪个 patchelf"不是无关紧要的细节——它决定产物长什么样。 + +同一个函数也用于 `readelf`,问题相同。 + +**提议 A3**:`_find_tool` 改为**优先且默认解析 payload**——通过 `pkginfo.resolved_dep()` / `build_dep()` 拿到工具包的载荷目录。视图与宿主降为显式、可见、需声明的回退,而不是排在最前的静默候选。这同时满足 R6 与规则 2。 + +## 2. P2:用进程全局机制满足单库需求 + +### 2.1 现状 + +`nvidia-gl-host-link` 的处境是真实的:NVIDIA vendor 库是宿主的文件(符号链接指向 `/lib/x86_64-linux-gnu/`),**不能**给它打 RPATH——那要改宿主的文件。所以 recipe 把依赖收拢到 `lib/xlings-deps/`,声明到 `LD_LIBRARY_PATH`。 + +`LD_LIBRARY_PATH` 是**进程全局、被每个子进程继承**的。需求是"这一个被 dlopen 的库要找到它的依赖",施加范围却是"这个 subos 里的每一个进程"。这个错配就是 P2。 + +### 2.2 实测:手写表两个方向都错 + +对宿主 NVIDIA 用户态全部文件求 DT_NEEDED,与 recipe 的手写表对照: + +| SONAME | 被几个 nvidia 库 NEED | store 里谁提供 | 在手写表里? | +|---|---|---|---| +| `libc.so.6` | 全部 | glibc | ~~曾在~~ 已移除 | +| `ld-linux-x86-64.so.2` | 全部 | glibc | 否(正确) | +| `libpthread.so.0` / `librt.so.1` / `libdl.so.2` | 多个 | glibc | ✓ | +| **`libm.so.6`** | **16 个**,含 `libnvidia-glcore` | glibc | **✗ 漏** | +| `libX11.so.6` / `libXext.so.6` | 5 个 | libX11 / libXext | ✓ | +| **`libdrm.so.2`** | 有 | libdrm | **✗ 漏** | +| **`libgbm.so.1`** | 有 | mesa | **✗ 漏** | +| **`libgcc_s.so.1`** | 有 | gcc | **✗ 漏** | +| **`libwayland-client/server.so.0`** | 有 | wayland | **✗ 漏** | +| `libxcb.so.1` / `libXau` / `libXdmcp` | **0 个** | libxcb 等 | ✓(理由不同:DT_RUNPATH 不传递) | +| `libcrypto.so.1.1` / `libcrypto.so.3` / `libnvcuvid.so.1` | 有 | 无人提供 | — | + +结论: + +1. **漏了五个**(libm、libdrm、libgbm、libgcc_s、libwayland-*)。它们今天**静默地来自宿主**——正是这个包存在的目的所要关掉的泄漏。 +2. **我上一轮关于 `libm` 的判断是错的**。它不在 `libEGL_nvidia` 的直接 DT_NEEDED 上,但被 16 个 nvidia 库 NEED。上一轮报告里"libm 无人需要"这句话需要更正,已在 §6 记录。 +3. **只有 `libc.so.6` 是"既无用又致命"**:它对每个进程都必然已加载(所以搜索路径永远用不上它),而放上去会杀死宿主二进制。这一条的测量结论不变。 +4. 表里混了**两种理由**:DT_NEEDED 直接需要 vs. DT_RUNPATH 不传递导致的二级需要。`libGLdispatch.so.0` 是第三种(glvnd 分发,应用侧加载)。一张表承载三种语义,是它容易写错的原因。 + +而这个文件自己的注释说过: + +> Enumerated rather than listed: … a fixed list would be a list of one driver release. + +`__nvidia_entries` 遵守了这条,依赖表没有。**同一个文件里的两套标准。** + +### 2.3 找到了机制层面的出路 + +> **术语**:本文档把这个新引入的对象称为 **interposer(插入库)**,不叫 shim。 +> `shim` 在 xlings 里已经有确定含义——`subos//bin/` 下那些指向 xlings 二进制的多调用符号链接。见 AD-6。 + + +先说为什么前一版的推荐(修正表内容 + 加护栏)不合格:按本文档 §1.3 的 **R3** 判据——"如果改动是**增加**一条路径而不是**移除**一条,它是 workaround"——那两条都没有删掉任何回答者。内容修正只是把错的表改对,护栏只是限制损害。`LD_LIBRARY_PATH` 这个进程全局机制仍在。 + +出路来自一条被忽略的 ELF 性质: + +> **DT_RPATH 沿加载链传递,DT_RUNPATH 不传递。** + +这条性质意味着:我们不必修改宿主的文件,也能决定它的依赖去哪里解析——只要在**它和加载器之间放一个我们拥有的对象**,把策略放在那个对象上。 + +#### 合成实验:先证伪 + +构造一个"我们不能修改的宿主 vendor"(无 rpath,NEED 一个只存在于我们目录里的库): + +| 加载方式 | 结果 | +|---|---| +| 直接 dlopen 该 vendor | **失败** —— 依赖找不到 | +| 经过一个我们拥有的 interposer,它带 **DT_RUNPATH** | **失败** | +| 经过同一个 interposer,它带 **DT_RPATH** | **成功**,`LD_DEBUG` 显示依赖从我们的目录解析 | + +RUNPATH 与 RPATH 的对照是关键:两者只差一个 patchelf `--force-rpath`,结果相反。**这条性质是承重的,不是巧合。** +并且 `dlsym(interposer_handle, ...)` 能取到 vendor 的符号——dlsym 搜索句柄的整个依赖树,所以 glvnd 拿到的仍是真 vendor 的入口。 + +#### 真实 NVIDIA 栈:A/B + +同一个探针二进制,同一份 `LD_LIBRARY_PATH`(只含应用自己需要的 X11 目录,**故意不含 glibc**),唯一差别是 vendor JSON 指向谁: + +| JSON 指向 | 结果 | +|---|---| +| 宿主 vendor 本身(今天的机制,去掉 `xlings-deps`) | `DEVICE_COUNT=0` —— vendor 加载不了 | +| 我们的 interposer(DT_RPATH 指向闭包) | `DEVICE_COUNT=1` | + +`LD_DEBUG` 确认:vendor 的 `libdl / libm / libpthread / librt` 从 **我们的 glibc 载荷**解析,而全局搜索路径上没有 glibc。 + +#### 机制的边界(实测,不是推测) + +interposer 组能枚举出设备,但 `eglInitialize` 失败。`strace` 对比工作组与 interposer 组打开的文件,差异是: + +``` +libnvidia-glsi / libnvidia-eglcore / libnvidia-egl-gbm +libnvidia-egl-wayland / libdbus-1 +``` + +vendor 在**运行时按裸 SONAME `dlopen` 自己的兄弟库**。于是边界是: + +> **运行时 dlopen 由发起它的那个对象的 DT_RPATH 服务。** +> 那个对象是我们的,就服务得了;是宿主的文件,就服务不了。 + +(此处原写作"任何 RPATH 机制都服务不了运行时 dlopen"。§2.7 的门禁验证证明那句话 +不准确:调用方对象的 DT_RPATH **确实**参与裸 SONAME 的 dlopen 解析。之所以在这个 +场景里服务不了,是因为发起 dlopen 的是宿主的 vendor 库——我们不能给它打 RPATH。 +这个更正把 GLX 从"做不到"变成"做得到,且不需要全局变量",见 §2.7 V1。) + +#### 边界恰好落在正确的地方 + +那些运行时 dlopen 找的是 **NVIDIA 自己的兄弟库**——宿主的文件,而且**必须**与宿主内核模块匹配。它们本来就该来自宿主。于是问题按同一条线切开: + +| 要找什么 | 由谁提供 | 为什么这是对的 | +|---|---|---| +| vendor 对**我们的**库的 DT_NEEDED | **interposer 的 DT_RPATH** | 作用域是链接链,per-consumer,不向任何其他进程施加任何东西 | +| vendor 运行时 dlopen **它自己的**兄弟库 | 宿主驱动目录放在 `LD_LIBRARY_PATH` | 全是宿主文件,宿主二进制本来就能解析到同一批;我们的库一个都不在上面 | + +这条线正是这个包自己已经画的那条线("`lib/` 是宿主的,`xlings-deps/` 是我们的")。 + +#### 端到端验证 + +`LD_LIBRARY_PATH` 上**只有宿主驱动目录**,`xlings-deps` 完全不参与: + +``` +DEV0_EGL_VENDOR = NVIDIA +DEV0_GL_RENDERER = NVIDIA GeForce RTX 4080/PCIe/SSE2 +DEV0_GL_VERSION = 4.6.0 NVIDIA 550.144.03 +DEV2_GL_RENDERER = llvmpipe (LLVM 20.1.7, 256 bits) ← 软件回退同时可用 +``` + +- 宿主 `/bin/bash` 在同样的 `LD_LIBRARY_PATH` 下正常(那目录里没有我们的任何东西)。 +- interposer 体积 **27KB**(对照:拷贝整套用户态 327MB)。 +- 仍是符号链接指向宿主文件,**用户态/内核模块的版本耦合完整保留**。 +- **不需要安装时的编译器**:已验证用 patchelf 对一个预置空 stub 做 `--add-needed` + `--set-rpath --force-rpath` 即可产出可用的 interposer。 + +### 2.4 这解决了什么,以及为什么它不是 workaround + +1. **`xlings-deps` 整个消失**,两张硬编码表随之消失。interposer 的 DT_RPATH 从已解析依赖推导,而 `resolved_deps` 已经是 xlings 记录的权威记录(R1 已经做过了)。§2.2 漏掉的五个库不需要"补进表里"——表没有了。 +2. **我们的库永远不出现在任何进程全局搜索路径上。** libc 那一类缺陷从"被护栏挡住"变成**结构上不可能**。 +3. **删掉了一个回答者**,而不是增加。这是 R3 意义上的解决。 +4. xlings 侧的护栏保留,但它的角色变了:从"防止损害"变成 **R4 意义上的断言——它应当永远不触发**。触发即说明某个 recipe 又走回了老路。 + +### 2.5 提议 + +**提议 B1**:把这个机制做成 **libxpkg 的公共能力**,而不是 recipe 的私有代码: + +``` +elfpatch.host_link_interposer{ + vendor = "<宿主 vendor 的绝对路径或 SONAME>", + deps = <从 resolved_deps 推导的载荷 libdir 列表>, + out = "<我们 payload 里的 interposer 路径>", + soname = "<需要时,例如 GLX 要求 libGLX_nvidia.so.0>", +} +``` + +`nvidia-gl-host-link` 与 `libcuda-host-link` 共用它——后者实测**既不收拢依赖也不声明 `LD_LIBRARY_PATH`**,今天依赖全部来自宿主,是同一个问题的第二个答案。一个实现,两个消费者。 + +**提议 B2**:`nvidia-gl-host-link` 的 `LD_LIBRARY_PATH` 声明收窄为**只有宿主驱动目录**,并在注释里写明它为什么是安全的(里面没有我们的任何文件)。`xlings-deps` 目录删除。 + +**提议 B3**:规范里写明,任何 `subos.env` 对**会导致代码被载入进程**的变量的声明都是特权操作,需要写明为什么 RPATH 不适用。按 §2.7 的更正,"RPATH 不适用"的真实场景只剩一种:**发起 dlopen 的是宿主自己的文件**(我们不能给它打 RPATH)。已写入 `xpackage-spec.md`。 + +**方案 B(拷贝 327MB + RPATH)正式否决**,理由写进 recipe:它打破用户态与内核模块的版本耦合,而 interposer 用 27KB 拿到了同样的隔离性。 + +### 2.6 interposer 修不了的另一半:被库自己读的搜索变量 + +`LD_LIBRARY_PATH` 不是唯一一个进程全局的搜索变量。`subos.env` 目前声明的四个里,有两个是**同一个形状**: + +- `__EGL_VENDOR_LIBRARY_DIRS` —— libglvnd 自己读 +- `LIBGL_DRIVERS_PATH` —— mesa 自己读 + +它们不经过动态加载器,所以 interposer 的 DT_RPATH 完全够不到,xlings 侧的 libc 护栏也看不见(护栏只检查 loader 读的变量)。 + +**实测。** 一个**宿主**二进制(`INTERP=/lib64/ld-linux-x86-64.so.2`,宿主 loader、宿主 libc),编译时只链接宿主的 `libEGL`: + +| 运行环境 | `GL_RENDERER` | +|---|---| +| 不带 subos 声明 | `NVIDIA GeForce RTX 4080/PCIe/SSE2` | +| 带 subos 声明 | `llvmpipe (LLVM 20.1.7, 256 bits)` | + +`LD_DEBUG` 显示它加载进来的是**我们的** `libm.so.6`(glibc 2.39 载荷)、`libgcc_s`、`libstdc++`、`libxcb`、`libxshmfence`——全都进了一个跑在宿主 libc 上的进程。 + +两个后果: + +1. **规则 1 违反**:宿主的东西依赖了我们的。这台机器宿主 glibc 恰好也是 2.39 所以没崩;换一台旧 glibc 的机器就是 `version 'GLIBC_2.xx' not found`。和 libc 那次是同一个形状,只是慢一拍。 +2. **功能上是静默降级**:宿主程序从硬件加速掉到软件渲染,没有任何提示。用户会认为"进了 subos 之后 GL 变慢了"而查不到原因。 + +**这一条是可以彻底解决的,而且解法就是本文档的主线**:这两个变量存在的唯一目的,是告诉**我们的** GL 栈它自己的驱动在哪里。而 libglvnd 与 mesa **是我们自己构建的**——完全可以把路径**编进产物**(构建时的默认 vendor 目录 / DRI 目录,或 `$ORIGIN` 相对路径),不必经过环境。 + +一旦编进产物: + +- 我们的 GL 栈自己知道去哪里找,不需要任何环境变量; +- 宿主的 libglvnd 用宿主的默认目录,拿到宿主的 vendor——**规则 1 与规则 2 同时成立**; +- `subos.env` 里只剩 `XDG_DATA_DIRS` 这类真正属于"用户可见约定"的变量。 + +**提议 B4**:mesa 与 libglvnd 的构建把 vendor 目录 / DRI 目录设为自身载荷路径,删除这两条 `subos.env` 声明。这比 interposer 更直接——那两个库是我们的,不存在"不能修改宿主文件"的约束,当初用环境变量只是因为没有把"决定应当由产物携带"当成规则。 + +**提议 B5**(已按 AD-3 收窄):xlings 侧的护栏目前只检查**加载器**读的变量(`LD_LIBRARY_PATH` / `LD_PRELOAD`)。扩展到**所有会导致代码被载入进程的变量**——`__EGL_VENDOR_LIBRARY_DIRS`、`LIBGL_DRIVERS_PATH`,以及将来同类的。 + +分界线不是"是不是进程全局",而是"会不会让代码进到别人的进程里": + +| 类别 | 例子 | 处理 | +|---|---|---| +| 导致**代码**被载入 | `LD_LIBRARY_PATH`、`LD_PRELOAD`、`__EGL_VENDOR_LIBRARY_DIRS`、`LIBGL_DRIVERS_PATH` | 值指向我们的载荷时安装期报告 | +| 导致**数据**被找到 | `XDG_DATA_DIRS` | 不管。subos 给默认、用户可覆盖是正常做法(AD-3) | + +### 2.7 门禁验证:已完成(2026-08-06) + +四项都跑了,结论**比 §2.3 当时的推断更好**,并且更正了 §2.3 的一处边界判断。 + +#### 更正:边界不是"运行时 dlopen 服务不了",而是"由发起 dlopen 的那个对象服务" + +§2.3 写的是: + +> DT_RPATH 的传递性覆盖链接期依赖(DT_NEEDED),不覆盖运行时 dlopen。 + +**这句话不准确。** 合成实验(宿主 glibc 13.3.0 工具链,三组对照): + +| 发起 `dlopen("libGLX_probe.so.0")` 的库 | 结果 | +|---|---| +| 无 RPATH / RUNPATH | 失败 | +| 带 **DT_RUNPATH** 指向目标目录 | **成功** | +| 带 **DT_RPATH** 指向目标目录 | **成功** | + +运行时按裸 SONAME 的 dlopen **确实**用调用方对象自己的 DT_RPATH/RUNPATH。 +上一轮之所以观察到"服务不了",是因为当时发起 dlopen 的是**宿主的 vendor 库** +——宿主的文件,我们不能给它打 RPATH。正确的表述是: + +> **运行时 dlopen 由发起它的那个对象的 RPATH 服务。那个对象是我们的就行,是宿主的就不行。** + +(第一次测量用 PATH 上的 `gcc` 跑,解析到了 musl 工具链,三组全失败 —— +musl 的 dlopen 语义不同。这本身是 R6 的一个实例:`gcc` 在 PATH 上是**视图**。) + +#### V1 — GLX:通过,而且不需要任何进程全局变量 + +GLX 没有 vendor JSON。`libGLX.so.0` 用 `libGLX_%s.so.0` 拼出文件名后 +`dlopen`,所以"把 JSON 指向绝对路径"这条路不存在。但按上面的更正,它不需要: + +同一份代码,同一个环境,**`LD_LIBRARY_PATH` 完全没有设置**,唯一差别是发起 +dlopen 的那个库有没有指向我们目录的 DT_RPATH: + +| 进程 | `dlopen("libGLX_nvidia.so.0")` 解析到 | +|---|---| +| **我们的**(dispatcher 带 DT_RPATH → 我们的目录) | `<我们的>/libGLX_nvidia.so.0`,`__glx_Main` 可达 | +| **宿主的**(同样的代码,没有我们的 RPATH) | `/lib/x86_64-linux-gnu/libGLX_nvidia.so.0` | + +**规则 1 与规则 2 同时成立,没有任何全局变量参与。** 条件是我们自己构建的 +`libglvnd` 载荷里的 `libGLX.so.0` 带一条覆盖 interposer 目录的 DT_RPATH —— +它本来就该有,而且是我们的文件。 + +#### V2 — Vulkan ICD:同一个文件,同一个机制 + +实测:`libGLX_nvidia.so.0` **同时**导出 `__glx_Main`、 +`vk_icdGetInstanceProcAddr`、`vk_icdNegotiateLoaderICDInterfaceVersion` —— +GLX vendor 和 Vulkan ICD 是**同一个文件**,这正是 +`/usr/share/vulkan/icd.d/nvidia_icd.json` 里写着 `libGLX_nvidia.so.0` 的原因。 + +所以 interposer 一个文件同时服务两条路径,V3 已证明三个入口点都能透过它取到。 +剩下的只是 ICD JSON 的**发现**方式(`VK_DRIVER_FILES` / `XDG_DATA_DIRS`),那属于 +§2.6 提议 B4 的范畴(把默认目录编进我们自己构建的 loader),与 interposer 无关。 + +#### V3 — dlsym 语义:通过 + +glvnd 与 Vulkan loader 在 vendor 句柄上查的就是上面那三个符号。25 KB 的 +interposer 与真 vendor 对照: + +| 句柄 | `__glx_Main` | `vk_icdGetInstanceProcAddr` | `vk_icdNegotiate…` | +|---|---|---|---| +| 宿主 vendor 本身(对照) | found | found | found | +| 我们的 interposer | found | found | found | + +`dlsym` 搜索句柄的整个依赖树,所以 glvnd 拿到的仍是真 vendor 的入口。 + +#### V4 — stub 分发:AD-12,且生产方式已实测 + +用 **patchelf 一把工具**就能从一个预置空 stub 产出可用的 interposer,安装期 +不需要编译器: + +``` +patchelf --set-soname libGLX_nvidia.so.0 +patchelf --add-needed <宿主 vendor 的绝对路径> +patchelf --set-rpath <闭包目录> --force-rpath +``` + +产物 **25 KB**,仍是指向宿主文件的引用,用户态/内核模块的版本耦合完整保留。 +按 AD-12,预置 stub 作为索引里的一个包分发(每 arch 一份)。 + +#### 门禁结论 + +四项全部通过,B 线可以开工。B1 的能力签名不变,但文档里"运行时 dlopen 够不到" +那句要按上面的更正改写 —— 它决定了 B3 里"RPATH 不适用的真实场景"到底还剩几种: +**只剩一种,即发起 dlopen 的是宿主自己的文件**。 + +### 2.8 规则 2 缺的是执行点,不是意图 + + +> "vendor 的 libm / libdrm / libgbm / libgcc_s / libwayland 为什么要用宿主的?" + +**没有人决定用宿主的。** recipe 的表没列它们,于是没有任何地方提供;动态加载器在我们提供的所有位置都找不到,就落到宿主的默认搜索(`ld.so.cache`、`/lib/x86_64-linux-gnu`)。悄无声息,因为对加载器而言"找到了"就是成功,不问来自哪里。 + +> **宿主是我们没能回答的任何问题的默认答案。** + +这就是为什么规则 2("能不依赖宿主就不依赖")**不能靠意图成立**。它需要一个执行点:让"我们没提供"成为**硬错误**,而不是回退。 + +#### 现状实测 + +对 `prodhome` 的 483 个 ELF 求 DT_NEEDED,统计有多少落到宿主: + +| SONAME | 处数 | 判定 | +|---|---|---| +| `libKSC/libGB/libJIS/libCNS/libJISX0213/libISOIR165` | 20 | **假阳性** —— glibc 自己 `lib/gconv/` 下的模块,扫描没把该目录算作提供方 | +| **`libxml2.so.2`** | 1 | **真漏** —— `wayland` 的载荷 NEED 它,store 里没有 libxml2,解析到宿主的 2.9.14 | + +**483 个 ELF 里只有 1 处真漏。** 我们自己构建的载荷状态其实相当好——nvidia 那五个之所以严重,是因为漏的那个文件是**宿主的 vendor 库**,它不在我们的载荷里,所以任何只扫自己载荷的检查都看不见它。 + +#### 提议 O4:安装期的闭包断言 + +`elfcheck::scan_payload` 已经在做同源断言(§R4)。同一个位置扩展一条: + +> 对刚安装的载荷里每个 ELF,以及**载荷链接进来的每个宿主文件**,解析其 DT_NEEDED 传递闭包。任何会落到宿主的 SONAME,必须在 recipe 的显式清单上;不在清单上就是安装失败。 + +关键是**显式清单**,不是放宽: + +```lua +exports = { + runtime = { + -- 允许落到宿主,并说明为什么。不写在这里的一律是安装错误。 + host_deps = { + "libnvidia-*", -- 必须与宿主内核模块匹配,见 §2.3 + "libdbus-1.so.3", -- 驱动运行时 dlopen,RPATH 够不到 + }, + }, +} +``` + +这样"依赖宿主"从**意外**变成**声明**: + +- 五个漏掉的库会在安装时报错,而不是安静地从宿主拿——它们本来就该由 §2.3 的 interposer 提供; +- `libxml2` 这处会立刻暴露(要么给 wayland 声明 libxml2 依赖,要么写进 `host_deps` 并说明理由); +- 驱动用户态那个**不可解**的洞变成一行有理由的声明,而不是一个没人知道的事实; +- 规则 2 第一次有了可执行判据:**未声明的宿主依赖 = 安装失败**。 + +配套的可观测性(§4 的 D2):安装结束时报告三个数——从我们载荷解析的、按声明落到宿主的、无人提供的。今天只报 "N libraries ✓",而 N 里既有真链接也有静默跳过。 + +--- + +## 3. P3:subos 层没有"恰好一个"的执行点 + +### 3.1 现状 + +三层模型(见验收报告 §3): + +| 层 | 版本数 | +|---|---| +| xpkg store | 多个(设计允许) | +| **subos sysroot** | **恰好一个** | +| 每个消费者的 RPATH/INTERP | 各自一个 | + +实测 `prodhome/default`: + +``` +$ xlings list | grep mesa + ◆ xim:mesa@25.0.7.1 + ◆ xim:mesa@25.0.7 +``` + +subos manifest 里两个 binding 都在,两者都在贡献 `__EGL_VENDOR_LIBRARY_DIRS`(3 项 = nvidia + 两个 mesa),EGL 因此枚举出重复设备。`xlings self doctor` 不报。 + +**中间层的"恰好一个"没有任何地方在执行。** + +### 3.2 为什么这是架构问题而不是一个 bug + +"这个 subos 里有什么"目前有两个记录: + +1. **xvm 注册**(哪些程序/库被绑定) +2. **subos manifest 的 `envs` 段**(哪些 binding 贡献了环境变量) + +安装第二个版本时,两个记录各自追加,没有任何一处执行"替换而非并列"。这是 P1 的又一个实例——只不过这次两个回答者恰好**都答"两个都在"**,所以它们一致,但一致地违反了模型。 + +**规则没有执行点,就不是规则,只是文档。** + +### 3.3 方案 —— 实现时更正了一次(2026-08-06) + +原提议: + +> **C1(执行点)**:安装 `pkg@B` 到已有 `pkg@A` 的 subos 时默认**替换**:解绑 A,绑定 B。 +> **C2(单一记录)**:`envs` 段从绑定集合派生。 +> **C3(可观测)**:doctor 报同一包多个绑定,`--fix` 保留 xvm active 的那个。 + +**C1 那条是错的,写完测出来的。** `install` 往 store 里加,`use` 做选择——这是 +2026.7.31 定下的语义。让安装顺便解绑,等于把 `install` 变成**第二个选择器**:又一个 +回答者,而不是少一个。而且它当场造出了新的分歧——测试里 manifest 写着 2.0.0,xvm 的 +活动版本还是 1.0.0。 + +哪个版本是活的,xvm 已经回答了,而**它的答案就记在同一个文件里**: +`subos//.xlings.json` 同时持有 `workspace`(name → active)和 +`subos_info.envs`(binding → 声明)。所以正确的形状不是"写端多做一件事",而是 +**激活时读 xvm 的答案**。 + +落地为: + +- **C2(真正的修复)**:`manifest::select_effective` —— 只有活动版本那一段贡献环境。 + 另一段**留在记录里**,因为它正是 `xlings use pkg@<旧版>` 能不重装就切回去的原因。 +- **没有活动记录的包,保留全部声明。** 这个默认值是承重的:对一个结果发现并不存在的 + 记录做过滤,会**静默删掉整个包的环境**——正是本文件要防的失败,只是从另一侧来。 + 选它之前先测了:裸 `xvm.add(name)` **确实**会记活动版本,所以这条是"workspace 记录 + 丢了"的抢救路径,不是常态。 +- **C3 报的是"没有活动版本"的那个子集,不是所有重复。** 两个版本里有一个是活动的属于 + 正常,报它会训练用户去删掉让 `use` 能工作的那半边。而"没有活动版本"时每一段都贡献, + 这才是那台机器上一块 GPU 被枚举成两块的状态。 +- **contested 无法修复**——这正是 contested 的定义。所以 `--fix` 不猜,remedy 是 + `xlings use pkg@`,把它变成某个人做的决定。 +- **C1 收窄**为:仅当 xvm 没有答案时,在"人正指名一个版本"的那一刻替换,并打印一行。 + +报告端与修复端共用 `manifest::contested_bindings` 一个函数(`reference_reporter_ +repairer_predicate_drift`)。 + +### 3.4 迁移 + +已有的 home 里可能已经存在多重绑定(prodhome 就是)。C1 上线前 doctor 必须先能报告并修复,否则用户会在下一次安装时遇到一个"突然开始替换"的行为变化而不知道为什么。**顺序:C3 → C1 → C2。** + +--- + +## 4. 横切:沉默成功是这个代码库的默认失败模式 + +### 4.1 实例清单 + +| 现象 | "没发生"与"成功了"如何变得不可区分 | +|---|---| +| doctor 不报双绑定 | 干净的 doctor 输出 = 没有双绑定 **或** doctor 不看这个 | +| `dep_install_dir()` 返回 nil,内层循环整个跳过 | 依赖没提供 = 依赖不需要提供 | +| 手写表漏了 libdrm/libgbm/… | 从宿主拿到了 = 我们提供了 | +| 沙箱不应用 subos.env | 变量为空 = 没有包声明过 **或** 整层被跳过 | +| 隔离 home 借用宿主 proot | 沙箱正常进入 = 用的是这个 home **或** 用的是另一个 home | +| e2e S3 的 skip 分支 | PASS = 测过了 **或** 跳过了整个特性 | + +**实现期又发现四个**(2026-08-06),形状完全一样: + +| 现象 | 两种结果为什么输出相同 | 怎么被发现的 | +|---|---|---| +| `elfpatch._find_tool` 的宿主回退 | 用了 payload 的 patchelf = 用了 `/usr/bin/patchelf`,产物看起来都正常 | 读代码时发现 payload 根本不在候选表里 | +| `slice-real-home.sh` 不重定向索引缓存 | 改了 recipe 生效 = **在读宿主 home 的 recipe** | 同一个改动**做了两遍都没反应**,而且两次都没有任何诊断输出 | +| e2e S12 钉住修复前的拼写 | 断言失败 = 代码坏了 **或** 断言写的是旧行为 | CI:S1–S11 全过、S12 拒绝了**正确**的值 | +| CI 缓存守卫只在非精确恢复时删 BMI | 换 key 前缀"修好了" = 前缀是根因 **或** 换前缀恰好触发了守卫 | 同一 workflow 连续两轮的步骤结论:`success` → 通过,`skipped` → 挂 | + +最后一条值得单独说:它让**三次**独立的"修复"都看起来成立,而每次只管用一轮。这是这个 +失败模式最贵的形态 —— 它不只掩盖缺陷,还伪造出修复成功的证据。 + +`subos_sandbox_test.sh` 的 S3 分支里已经有人意识到了这个问题并写了注释("Reporting PASS +while silently skipping the entire feature under test is how a real regression would reach +a release looking exactly like an unattended laptop")——但那是一个人在一个地方的自觉, +不是机制。 + +**四个里有三个,是靠"同一测量做两遍,结果不一致"发现的。** 这本身就是 O1 的论据: +差集必须是**自动可查**的,而不是靠人恰好做了第二遍。 + +### 4.2 提议 + +> **编号说明**:本节的提议编号为 **O**(observability)。 +> `D1`–`D5` 在验收报告 `2026-08-06-subos-matrix-verification.md` 里指**缺陷**,本文档沿用那个含义,不用于提议。 + + +**提议 O1(机制,不是清单)**:凡是"因为条件不满足所以没做"的分支,输出必须与"做了"不同。 + +这条**不需要新机制**,它是 R1 的推论(AD-10):只要权威记录是**全量**的——每个输入项都必须有一条记录,哪怕标记为 `skipped`——那么 `declared` 与 `recorded` 的差集就是**自动可查**的,不依赖任何人记得写日志。`.xlings-resolution.json` 已经是这个形状,把它推广到每一个遍历声明项的循环即可。 + +评审清单是兜底,不是主要手段:新增的 `if (...) continue` / `if not X then return end` 若发生在遍历声明项的循环里,必须先写记录再 `continue`。 + +**提议 O2**:host-link 类包安装结束时,报告**三个数**:命中我们载荷的、落回宿主的、无人提供的。今天用户只看到 "N libraries ✓",而 N 里既有真链接也有静默跳过。 + +**提议 O3**:`§1.3 R5` 的持久化已经有了 `.xlings-resolution.json` 和 `xlings why`。建议把 host-link 的解析结果也写进同一个文件——"哪个 SONAME 来自哪里"是一个事后必然会被问到的问题,现在需要重建 store 状态才能回答。 + +--- + +## 5. 测试架构 + +三条,都来自本轮实测,都是流程问题不是代码问题。 + +**提议 E1:隔离 home 成为 subos/沙箱测试的默认环境。** +上述四个 home 相关缺陷在默认 `~/.xlings` 下全部无症状。只要测试都在默认 home 或同形路径下跑,这一整类缺陷不可见。**并且被测 home 应当放在一个与 `$HOME` 无共同前缀的路径下**——本轮把它放在 `/tmp` 下,恰好命中了绑定顺序最难的情形(`/tmp` 先被私有化再挂 home),这个偶然应当变成故意。 + +**提议 E2:测试断言写契约,不写实现。** +`subos_sandbox_test.sh` 的 S8 断言 "`~/.xlings` 在沙箱内能看到宿主内容"——那正是 D1 的错误行为。**一个测试不仅可能漏掉缺陷,还可能把缺陷钉死**:修 D1 必须同时改这条断言,而改测试断言在评审里天然可疑。已改写为契约断言(`XLINGS_HOME` 即宿主路径、`PATH[0]` 与之一致、不存在第二个拼写)。 + +**提议 E3:核心模块改动后必须双工具链构建。** +`views::split | ranges::to` 在发布目标 gcc 15.1.0-musl 下编译通过,在默认 gcc 16.1.0 下让整个模块以 "Bad file data" 失败,并指向一个未改动的 TU(`cli.cppm`)。只跑发布目标的构建会让它直接发出去;只跑默认目标则会以为是自己刚改的文件坏了。CI 已有两个目标,但**本地开发循环没有门禁**。 + +--- + +## 6. 对上一轮报告的更正 + +`2026-08-06-subos-matrix-verification.md` §4 里的这句话是错的: + +> `libm` 有 1203 个符号(真正的 ABI 面),但**没人要它**。 + +`libm.so.6` 被 16 个 nvidia 库 NEED,包括核心渲染器 `libnvidia-glcore`。正确的说法是:它不在 `libEGL_nvidia` 的**直接** DT_NEEDED 上,而我当时只看了那一个文件。 + +这不改变 §4 的其余结论(`libc.so.6` 既无用又致命;三个桩库必需),但它改变**方法论上的结论**:逐库测量比逐库推理好,而我做的逐库测量本身取样不足——只取了闭包的一个入口。这正是 §2.2 提议改用闭包推导的直接理由。 + +对 xlings 侧防线的判断不变:防线只排除 `libc.so.6` 和 loader,`libm` 不在其中,所以补上 libm **不需要**改 xlings。 + +--- + +## 7. 架构决策记录(AD-1 ~ AD-14) + +本节记录 review 中由 sunrisepeak 定下的架构决策,连同它们否掉的我的错误判断。 +决策按定下的先后编号,内容按主题排列。 + +### AD-1:subos 同时是编译期与运行期概念,优先级规则是"能直连 payload 就直连" + +我曾把它当成二义性("对二进制是编译期概念,对 alias 是运行期概念")。不是二义性,是**优先级**: + +> 能直接走 payload 的就直接走 payload;走不了的,subos 正常映射。 +> **优先走 payload 是为了稳定性。** + +推论,并且解释了 doctor 那条告警为什么两半都对: + +- 已编译产物的 RPATH/INTERP 冻结在 payload 上 —— 用户切 subos 不改变一个已经构建好的二进制的行为。这就是"稳定性"。 +- alias / shim 在 exec 时跟随活动 subos —— 这是视图按设计工作。 +- 所以 `x86_64-linux-gnu-gcc@16.1.0 records an install-time subos path` 这条告警里,**"执行跟随活动 subos"是正确行为**,而**"记录里存着安装期的 subos 路径"是缺陷** —— 记录本应存 payload 路径或什么都不存。 + +与 §1.5 的 R6 一致:R6 说的是"内部消费者"这一侧,AD-1 说的是**通用优先级**,R6 是它在 xlings 自身代码上的特例。 + +### AD-2:refcount 是删除判据 —— 有引用不删,强制删除必须告警 + +不需要更复杂的机制。 + +告警的范围见 AD-9:只针对 xlings 自己记录的包引用。 + +### AD-3:`XDG_DATA_DIRS` 类变量不属于问题域 + +subos 提供默认值、用户可覆盖,这就是 Linux 的常规做法,没有问题。 + +我把它和 `LD_LIBRARY_PATH` 归为一类是**过度概括**。正确的分界线不是"是不是进程全局",而是: + +| 类别 | 例子 | 为什么危险 / 不危险 | +|---|---|---| +| **导致代码被载入进程** | `LD_LIBRARY_PATH`、`LD_PRELOAD`、`__EGL_VENDOR_LIBRARY_DIRS`、`LIBGL_DRIVERS_PATH` | 把我们的库塞进宿主进程 → ABI 耦合 → 崩溃或静默降级 | +| **导致数据被找到** | `XDG_DATA_DIRS` | 最坏是宿主程序看到我们的一个 `.desktop` 或图标。没有 ABI 面 | +| **决定哪个可执行文件被运行** | `PATH` | 不往已有进程里塞代码,但决定跑的是谁 —— 归 R6 / AD-1 管,不归这道护栏管 | + +§2.6 的提议 B5 已按这条重写:护栏只管第一类。 + +### AD-4:更正 —— glibc 默认搜索路径不是"靠意外维持的承重属性" + +我曾把 ld.so 里烙着 `/home/xlings/.xlings_data/...` 当成"一个承重属性靠构建参数的副作用维持着"。**这个判断是错的**,两点: + +1. 它不是旧产物遗留。这一季刚发布的 **glibc 2.44 烙的是同一个前缀**,是当前构建约定。 +2. 更重要:对一个**可重定位**的包,构建期 `--prefix` 永远不可能等于运行时安装路径。默认搜索路径**必然**指向不存在的位置——这是结构性保证,不是运气。"一切必须靠 RPATH"因此是结构决定的。 + +真正剩下的是**可追溯性问题**:那个字符串泄漏了构建机的 home 布局(`.xlings_data` 是早已废弃的运行时布局)。与任务 #35(libxml2 的 `.pc` 写着构建机)同类,应当统一处理为"产物里不得出现构建机路径,除非是刻意保留的占位前缀"。 + +### AD-5:ld.so 的默认搜索路径保持"必然不存在",并把它显式化 + +**相对路径不是选项。** 那个字符串是编译期常量,被加载器**原样**用于搜索: + +- 写成相对路径 → 相对**进程的当前工作目录**解析。搜索结果随 `cd` 而变,且等价于把 `.` 放进 `PATH`,是安全问题。 +- `$ORIGIN` 在 `DT_RPATH` / `DT_RUNPATH` 里可用,但**内建默认搜索路径不做 `$ORIGIN` 展开**——它不是动态段的 tag。(构建时值得再验一次,但即使可行,上一条已经足以否掉相对路径。) + +**保留一个必然不存在的路径,影响是什么:** + +| 方面 | 影响 | +|---|---| +| 依赖解析 | 默认搜索找不到任何东西 → 一切必须来自 `DT_RPATH`。**这正是规则 2**,不是缺陷 | +| 漏配 RPATH 的产物 | 首次运行**响亮失败**(`cannot open shared object file`),而不是靠宿主库悄悄跑起来 | +| `ld.so.cache` | 永远不命中 → 查找略慢,`ldconfig` 不可用。我们本来就不用 ldconfig | +| 安全 / 行为 | 无负面影响 | + +所以这条路径**应该**不存在。唯一要改的是让它**刻意**而非**偶然**:构建流水线把 `--prefix` 换成一个显式保留的占位前缀,理由写在构建脚本里,产物中不再出现任何构建机的痕迹。 + +与任务 #35(libxml2 的 `.pc` 写着构建机)合并为一条规范:**产物中不得出现构建机路径,除非是刻意保留的占位前缀**;检查方式见 §8「修法」的第 3 条断言。 + +### AD-6:采用 interposer + DT_RPATH,把 deprecated 的风险写进文档,以后再优化 + +**先回答"为什么必须让宿主 vendor 解析到我们的库"**——这个问题的答案也解释了整段历史: + +vendor 被 dlopen 进的那个进程**是我们的**(INTERP 指向我们的 glibc,已经加载了我们的 libc / libX11 / libstdc++)。vendor 的每个 DT_NEEDED 只有三种下场: + +1. **该 SONAME 进程里已经加载了** → 加载器直接复用,自动就是我们的。这部分不需要任何机制。 +2. **没加载,而搜索路径能找到宿主的** → 同一进程里出现两份同名库的不同构建,ABI 混用;而且是规则 2 违反。 +3. **没加载,搜索路径也找不到** → **vendor 加载失败,GPU 直接没有**。 + +我们的 loader 默认搜索路径必然不存在(AD-5),所以第 3 种是默认结局——实测就是 `DEVICE_COUNT=0`。`LD_LIBRARY_PATH` 当初正是为了从第 3 种逃到第 1/2 种而引入的。 + +interposer 的作用是让第 2 类落到**我们的**库上,同时不向任何其他进程施加任何东西。 + +**决策**:采用 interposer + `DT_RPATH`。`DT_RPATH` 已被标记 deprecated 是已知风险,**在设计文档与 recipe 注释中写明**,glibc 目前没有移除迹象;若将来失效,回退路径是 AD-7 的 wrapper 方案。先落地,后优化。 + +命名:这个东西在本文档中称 **interposer(插入库)**,不叫 shim——`shim` 在 xlings 里已经指 `subos//bin/` 下那些指向 xlings 二进制的多调用符号链接,复用会造成混淆。 + +### AD-7:nixGL 的机制本身没错,错的是作用域——这给出一条回退方案 + +我先前把 nixGL 归为"和 xlings 今天一样",不够准确。差别在**作用域**: + +| | 作用域 | 后果 | +|---|---|---| +| nixGL | **包裹单个程序**:`nixGL `,只影响这一个进程树 | 宿主 shell 不受影响 | +| xlings 今天 | `subos.env` 作用于**整个 subos 会话** | 会话里每个宿主二进制都继承,`/bin/bash` 因此 SIGSEGV | + +于是可选方案按优劣排序: + +1. **interposer**(AD-6)——per-consumer,不向任何进程施加东西。最优。 +2. **wrapper**(nixGL 式)——per-program。需要知道哪些程序要用 GL;用户自己编译的 GL 程序拿不到。可作为 interposer 失效时的回退。 +3. **会话级 `LD_LIBRARY_PATH`**——今天的做法,已证明会杀死宿主 shell。**不再使用**。 + +### AD-8:不同硬件的机器不是 xlings 能解决的问题,但可以做成体验 + +驱动用户态必须与宿主内核模块匹配,这是物理约束(§2.3)。**xlings 只能做到尽可能可移植,不能做到完全可移植**——这应当写成边界条件,而不是继续尝试消除。 + +可以改善的是**体验**:让这件事**可见**。例如 subos 进入时或 `doctor` 中报告 + +> 此 subos 的 GL 栈链接到宿主驱动 NVIDIA 550.144.03;在驱动版本不同的机器上不可用。 + +用户于是知道"这个 subos 不可搬到那台机器",而不是搬过去之后遇到一个无从解释的失败。 + +### AD-9:refcount 的告警只针对包引用 + +修正我在 AD-2 里加的那句"告警文案应说清用户自编译产物"——**不需要**。 + +- xlings 自己记录的 refcount > 0,而用户强制删除 → **给 warn**。 +- 用户自己编译的二进制引用了某个 payload → **不需要任何告警**。删掉之后运行时的报错本身就是提示,而且那个报错会告诉用户该装什么依赖。 + +### AD-10:D1 是 R1 的推论,不是独立规则 + +接受。"沉默跳过"不需要新机制:只要权威记录是**全量**的(每个输入项都必须有一条记录,哪怕标记为 skipped),`declared` 与 `recorded` 的差集就是自动可查的,不依赖人记得写日志。`.xlings-resolution.json` 已经是这个形状,推广到每一个遍历声明项的循环即可。 + +### AD-11:占位前缀 + +构建 glibc(以及任何会把 `--prefix` 烙进产物的包)时使用: + +``` +/nonexistent/xlings-use-rpath-not-default-search +``` + +选择理由: + +- **必然不存在**,而且是刻意的。`/nonexistent` 有发行版先例(Debian 用它作系统用户的 home),不会有人误建。 +- **自解释**。下一个读到它的人不需要查文档就知道这是故意的、以及为什么——避免有人"顺手把这个奇怪的路径修好"。 +- 它同时决定 glibc 自身产物的 INTERP。未打补丁的二进制因此 `execve` 报 ENOENT——**响亮失败**,而不是指向宿主 loader 后在 GLIBC_PRIVATE 层静默配错。 + +`--prefix` 与 `DESTDIR` 分离是标准做法,不影响安装布局。 + +### AD-12:interposer 的预置 stub 作为索引里的一个包 + +不由 libxpkg 携带。理由是 AD-1 的"能直连 payload 就直连":做成包之后 + +- 每个 arch 一份,走正常的索引/镜像/校验流程; +- 消费它的 recipe 通过 `pkginfo.resolved_dep()` 拿到**payload 路径**,与 R6 一致; +- 版本可独立演进,不必跟着 libxpkg 发版。 + +### AD-13:驱动耦合的提示出现在两处 + +1. **`xlings self doctor`** —— 主动跑时报告,不打扰日常使用; +2. **安装 host-link 类包时报一次** —— 用户第一次把 subos 绑到宿主驱动的那一刻,正是他需要知道这件事的时刻。 + +不在每次进入 subos 时报——那会变成噪音,而噪音会训练用户忽略它。 + +### AD-14:R7 —— 涉及依赖/引用的测量必须覆盖传递闭包 + +写进规范,与 R1–R6 并列: + +> **R7 闭包完整**:任何关于"需要什么 / 引用了什么"的测量,必须覆盖**传递闭包**,不能只取一个入口。 + +这一条来自本轮两次真实的错误判断,都是同一个原因: + +- 判断"`libm` 没人需要"——只看了 `libEGL_nvidia` 的直接 DT_NEEDED。实际被 16 个 nvidia 库 NEED,含核心渲染器 `libnvidia-glcore`。 +- 推荐"修正表内容 + 加护栏就够了"——没有对整个用户态求闭包,因而没看到表还漏了 `libdrm` / `libgbm` / `libgcc_s` / `libwayland-*`。 + +判据可执行:一份依赖清单如果是**手写**的,它就没有通过 R7;必须是从产物**枚举**出来的。这也解释了为什么 §2.2 的手写表、§8 的 `relocate_files` 清单、§1.5 的 `_find_tool` 候选表是同一个反模式的三个实例。 + +R7 与 R1 的关系:R1 要求记录全量(写下每一项),R7 要求**输入集合本身**是完整的(不漏项)。记录得再全量,输入取样不足一样得出错误结论。 + +## 8. 追查决策时发现的真实缺陷:glibc 的路径重写 +问题从"为什么 `ld.so` 里烙着 `/home/xlings/.xlings_data/...`"开始。答案不是 gcc specs,也不是旧产物: + +- glibc 是**下载预构建产物**,tarball 里带着构建流水线的 `--prefix`(那台机器用的是早已废弃的 `.xlings_data` home 布局); +- recipe **已经知道**这件事,`install()` 末尾有一段重写代码。 + +那段代码有三个问题,叠在一起。 + +### 一、硬编码文件清单 + +```lua +relocate_files = { "lib/libc.so", "lib/libm.a", + "bin/ldd", "bin/tzselect", "bin/xtrace", "bin/sotruss" } +``` + +又是一张"某一次构建的清单"。与 §2.2 里 nvidia 依赖表、§1.5 的 `_find_tool` 候选表同一个反模式。 + +### 二、贪婪且未锚定的模式,把文件改坏了 + +```lua +local path_pattern = "([^%s)]+)/fromsource%-x%-glibc/" .. version .. "/lib" +``` + +`[^%s)]+` 匹配任意非空白、非 `)` 的连续串——**包括变量名和引号**。对照宿主未经改动的 `ldd`: + +```sh +# 宿主(正确) +RTLDLIST="/lib/ld-linux.so.2 /lib64/ld-linux-x86-64.so.2 /libx32/ld-linux-x32.so.2" + +# 我们的(损坏) +.64/ld-linux-x86-64.so.2 ./ld-linux.so.2 .x32/ld-linux-x32.so.2" +``` + +`RTLDLIST="` 被连同路径一起吞掉了。**我们发布的 `ldd` 连 `bash -n` 都过不了**(line 39 语法错误),glibc 2.39 与 2.44 都是如此。`diff` 还显示插入了一行 `unused=`、删掉了一个 `;;`——脚本结构已经坏了。 + +这与 openxlings/xlings#486 是同一个形状:**一次正则改写产出了一个"看起来还行、实际语义已变"的文件,而没有任何东西回头检查**。那次是 Lua 仍然能解析,这次是 shell 已经不能解析——都没被发现。 + +### 三、匹配锚在 `/lib`,所以本职工作也没做完 + +模式要求路径以 `/lib` 结尾,于是 `bin/ldd` 里的 + +``` +TEXTDOMAINDIR=/home/xlings/.xlings_data/.../fromsource-x-glibc/2.44/share/locale +``` + +原封不动。构建机路径**仍然在产物里**——而这正是这段代码存在的唯一目的。 + +### 四、并且报告成功 + +`if count > 0 then io.writefile(...)` ——写了就算成功。"还有残留的构建路径"和"文件被改坏了"两种结果都不产生任何输出。又一次沉默成功。 + +### 修法(与本文档其余部分同一套原则) + +1. **枚举,不要清单**:扫描整个载荷找构建路径,而不是点名六个文件(实测残留在 5 个文件里,其中 4 个**就在清单上**却没被正确处理)。 +2. **锚定路径 token**:匹配一个完整的绝对路径(`/` 开头,到空白或引号为止),保留 `/lib` 之后的尾巴,而不是让 `[^%s)]+` 向左吞。 +3. **改完回头断言**(R4): + - 载荷里不得再出现构建路径; + - 每个被改写的 shell 脚本必须 `bash -n` 通过。 + 两条都不满足即安装失败。 + +第 3 条是关键——它把"改写"从一个**期望**变成一个**可验证的结果**。以上三条都不依赖对 glibc 的了解,可以直接做成 libxpkg 的通用重定位能力,供所有下载预构建产物的 recipe 使用。 + +## 9. 落地顺序与实际状态 + +依赖关系决定顺序,不是优先级。已定的决策见 §7。 +**2026-08-06 更新:第一批已全部落地,B 线门禁已通过。** + +### 第一批 —— 已落地(2026.8.6.1 / libxpkg 0.0.51) + +| 项 | 内容 | 落在哪 | +|---|---|---| +| **#42** | glibc 路径重写:枚举取代清单、锚定路径 token、改完断言(§8) | libxpkg `elfpatch.relocate_build_paths` + index glibc recipe | +| **A3** | `_find_tool` 走 payload(R6 / §1.5),**以及** xlings 侧同源断言的 `command -v patchelf` | libxpkg `pkginfo.tool_payload_dir` + xlings `elfcheck::locate_patchelf` | +| **A4** | `locate_proot_` 去掉 PATH 步骤,宿主 proot 降为具名回退 | xlings | +| **E1/E2** | 隔离 home 成为默认(`$TMPDIR`,与 `$HOME` 无共同前缀)+ `assert_home_is_isolated` | xlings 测试库 | +| **A1/A2/B3** | R1–R7 写进 `xpackage-spec.md`,每条带可执行判据;禁止"缺省即约定"措辞 | index 规范 | +| **B5** | 护栏扩到所有**会导致代码被载入**的变量,默认拒绝 | xlings `manifest::is_privileged_env` | +| **C 线** | 见 §3.3 —— C1 按提议是错的,实际落地为 C2(激活时读 xvm 的答案)+ C3(只报无活动版本的子集) | xlings | +| **AD-11** | 占位前缀 `/nonexistent/xlings-use-rpath-not-default-search`,并在构建后断言它确实烙进去了 | index `build-glibc.sh` | + +顺带修掉的两个:`slice-real-home.sh` 的索引缓存指向真实 home(任何针对 slice 的 recipe +实验其实都在读宿主的索引);CI 的 mcpp cache key 一个 key 多个写入者。 + +### 第二批:B 线门禁已开,实现待做(任务 #55) + +``` + §2.7 四项验证 —— 2026-08-06 全部通过,见 §2.7 + │ 结论比原推断更好:GLX 不需要任何全局变量; + │ GLX vendor 与 Vulkan ICD 是同一个文件 + ▼ + AD-12 interposer stub 作为索引包(每 arch 一份) + ▼ + B1 libxpkg 的 elfpatch.host_link_interposer + ├─→ B2 nvidia-gl-host-link 切换,删除 xlings-deps + └─→ B2' libcuda-host-link 用同一能力 + ▼ + B4 mesa / libglvnd 把 vendor 与 DRI 目录编进产物,删除那两条 subos.env +``` + +**为什么不在本轮**:B1/B2 需要图形栈装好才能端到端验证,而本机的 home 里 mesa / +libglvnd / nvidia-gl-host-link 都没装。门禁验证本身不需要它们(用合成实验 + 宿主的 +vendor 库就够),所以先做了。 + +### 第三批:依赖 R7 的枚举能力 + +- **O4** 安装期闭包断言 + `host_deps` 显式清单(§2.8) +- **O2/O3** 安装报三个数、host-link 解析结果持久化(§4.2) +- **AD-13** 驱动耦合提示:`doctor` + 安装 host-link 包时报一次 +- **AD-2/AD-9** refcount 强制删除的告警 +- **B4 的产物**:需要重新构建 mesa / libglvnd 并重新发布 tarball + +### 另记 + +- 任务 **#53**:`SubosManifest` / `SubosEnvOrphan` / `SubosEnvUnresolved` 是 Error 级 + 但不计入退出码,`healed` 也因此算不出来。本轮只把新增的 `SubosDoubleBinding` 计了进去 + ——把其余几条打开会改变现有 home 的退出码,需要单独验一遍。 + +### 已落地、与本提案不冲突 + +验收报告里那四个已落地的缺陷修复(该文档的 D1–D4)保持不变。§2 的 interposer 机制会让 xlings 侧的 libc 护栏**永远不触发**,但护栏本身作为 R4 断言应当保留——触发即说明某个 recipe 又走回了老路。 diff --git a/.agents/docs/2026-08-06-subos-landing-plan.md b/.agents/docs/2026-08-06-subos-landing-plan.md new file mode 100644 index 00000000..a933a13f --- /dev/null +++ b/.agents/docs/2026-08-06-subos-landing-plan.md @@ -0,0 +1,172 @@ +# subos 架构方案:落地计划与跨仓库依赖 + +日期:2026-08-06 +输入:`.agents/docs/2026-08-06-subos-architecture-proposal.md`(§7 的 AD-1~AD-14 已 review 通过) +性质:执行计划。提案说"做什么和为什么",这份说"谁先谁后、在哪个仓库、怎么验"。 + +--- + +## 0. 四个仓库,一条发布链 + +改动落在四个仓库,顺序不可换——下游的东西不存在时上游的代码没法测: + +``` +openxlings/libxpkg Lua stdlib(recipe 侧能力)+ C++ 执行器 + 0.0.50 → 0.0.51 A3 · #42-generic · O4-scan · B1 + │ + │ merge → git tag → GitHub tarball + │ gtc release publish mcpp-res/xpkg(CN 镜像,字节相同) + │ mcpplibs/mcpp-index pkgs/x/xpkg.lua(GLOBAL+CN+sha256 ×3 平台) + │ publish-artifact.yml → xlings-res/mcpp-index + ▼ +openxlings/xlings 客户端(沙箱/subos/doctor/env 层) + 2026.8.5.3 → 2026.8.6.1 A4 · E1/E2 · C1/C2/C3 · B5 · O2/O3 · AD-13 · AD-2/AD-9 + │ + mcpp.toml: xpkg 0.0.50 → 0.0.51 + │ release.yml → GitHub release + gtc 补 gitcode 资源 + ▼ +openxlings/xim-pkgindex recipe + 规范 + 构建流水线 + │ #42-glibc · A1/A2/B3 · AD-12 · B2/B2' · AD-11-build · B4-build + ▼ +真实验证 隔离 XLINGS_HOME + xlings subos +``` + +**关键约束**(来自 `project_ecosystem_release_chain`): + +- libxpkg 的 mcpp-index 条目发布后有 artifact TTL,`rm -rf ~/.mcpp/registry/data/mcpplibs` 强制刷新; +- xlings 的 `release.yml` 里 `bump-index` 与 `mirror-binaries` **都会吞掉自己的失败**,发布后必须用 GET(不是 HEAD)核对 gitcode 资源,并核对 `latest.ref`; +- 改 `.xlings.json` 的 mcpp pin 要同步 6 个 workflow 的 `XIM_PKGINDEX_REF`。 + +--- + +## 1. 任务分解与依赖图 + +编号沿用提案。`⊘` 表示被门禁挡住。 + +``` + ── 第一批:无前置,四条并行 ─────────────────────────────────────── + + [L1] libxpkg A3 _find_tool 走 payload ─┐ + [L2] libxpkg #42-gen elfpatch.relocate_build_paths(通用重定位) ─┤ + [X1] xlings A4 locate_proot_ 去掉 PATH 步骤 ─┤ 互不依赖 + [X2] xlings E1/E2 隔离 home 成测试默认 + 断言写契约 ─┤ + [P1] index A1/A2/B3 R1–R7 写进 xpackage-spec.md ─┘ + + ── 第二批:依赖第一批的产物 ────────────────────────────────────── + + [L2] ──→ [P2] index #42-glibc glibc recipe 改用通用重定位 + └─ 需要 libxpkg 0.0.51 已发布 + + [X3] xlings C3 doctor 报双绑定 ──→ [X4] C1 单版本执行点 ──→ [X5] C2 envs 派生 + 顺序不可换(§3.4):先能报告,再改行为 + + [X6] xlings B5 护栏扩到 __EGL_VENDOR_LIBRARY_DIRS / LIBGL_DRIVERS_PATH + [X7] xlings O2/O3 安装报三个数 + host-link 解析结果持久化 + [X8] xlings AD-13 驱动耦合提示(doctor + 安装 host-link 时一次) + [X9] xlings AD-2/AD-9 refcount 强删告警 + + ── B 线:门禁在前 ──────────────────────────────────────────────── + + [V] §2.7 四项验证 GLX / Vulkan ICD / dlsym 语义 / stub 分发 + │ ← AD-14 的直接应用:未验完不写代码 + ▼ + [P3] index AD-12 interposer stub 作为索引包 + ▼ + [L3] libxpkg B1 elfpatch.host_link_interposer + ▼ + [P4] index B2 nvidia-gl-host-link 切换,删 xlings-deps + [P5] index B2' libcuda-host-link 用同一能力 + + ── 需要重新构建载荷,本轮只改流水线 ────────────────────────────── + + [P6] index AD-11-build build-glibc.sh 换占位前缀 + [P7] index B4-build mesa/libglvnd 把 vendor/DRI 目录编进产物 + 两者都只在**下一次构建**生效,不改变现有 tarball + + ── 第三批:依赖 R7 的枚举能力 ──────────────────────────────────── + + [L4] libxpkg O4-scan DT_NEEDED 传递闭包枚举 + ▼ + [P8] index O4 host_deps 显式清单 + 安装期闭包断言 +``` + +### 1.1 为什么这个顺序 + +| 边 | 理由 | +|---|---| +| L2 → P2 | recipe 调用的函数必须先存在于已发布的 libxpkg | +| X3 → X4 | 现有 home 里已有双绑定(prodhome 就是)。先能报告再改行为,否则用户遇到"突然开始替换"而无从解释(§3.4) | +| X4 → X5 | `envs` 从绑定集合派生,前提是绑定集合已经是"恰好一个" | +| V → P3 → L3 | AD-14:手写清单不通过 R7。没验证过 GLX/Vulkan 就写 B1,等于再造一张手写表 | +| L4 → P8 | O4 的断言依赖闭包枚举,而闭包枚举本身是 R7 的可执行形式 | + +### 1.2 本轮不做的 + +| 项 | 为什么 | +|---|---| +| AD-11 / B4 的**产物** | 需要重新构建 glibc / mesa / libglvnd 并重新发布 tarball,是独立的构建发布train。本轮只把构建脚本改对,下次构建生效 | +| #33 Intel 显卡、#34 vulkan-loader | 与本方案正交,不在提案范围 | + +--- + +## 2. 单 PR 策略 + +目标是"尽量单 PR 全部实现",但发布链强制三个仓库分三个 PR——libxpkg 不发版,xlings 就没法用新能力。所以是**每仓库一个 PR**,三个 PR 一条线: + +| 仓库 | PR | 版本 | 内容 | +|---|---|---|---| +| libxpkg | [#35](https://github.com/openxlings/libxpkg/pull/35) | 0.0.50 → 0.0.51 | L1 A3 · L2 relocate_build_paths | +| xlings | 本 PR | 2026.8.5.3 → 2026.8.6.1 | A3(第二个站点)· A4 · B5 · C1/C2/C3 · E1/E2 + xpkg pin 0.0.51 | +| mcpp-index | [#164](https://github.com/mcpplibs/mcpp-index/pull/164) | — | xpkg 0.0.51 条目(GLOBAL+CN+sha256 ×3 平台) | +| xim-pkgindex | [#522](https://github.com/openxlings/xim-pkgindex/pull/522) | (无版本号) | #42-glibc · A1/A2/B3 规范 · AD-11 构建前缀 | + +**B 线(B1/B2/AD-12)不在本轮。** §2.7 的四项门禁 2026-08-06 全部通过,结论记录在提案 +§2.7,但实现需要图形栈装好才能端到端验证,而本机的 home 里 mesa / libglvnd / +nvidia-gl-host-link 都没装。见任务 #55。 + +三个 PR 都带完整测试,CI 各自全绿后按链顺序合并、发布。 + +--- + +## 3. 验证策略 + +### 3.1 每一项的验收判据 + +判据必须是**可执行的**,不是"看起来对了"。逐项: + +| 项 | 判据 | +|---|---| +| A3 | 在一个**同时存在** payload patchelf 与 `/usr/bin/patchelf` 的 home 上安装包,`_find_tool` 解析到 payload。宿主候选被用时必须打印一行 | +| #42 | 安装后 `bash -n` 过每个被改写的 shell 脚本;载荷里 `grep -r` 不到构建机路径 | +| A4 | PATH 上放一个假 proot,沙箱仍拒绝它并给出 `xlings install proot` 提示 | +| E1/E2 | 测试 home 在与 `$HOME` 无共同前缀的路径下;S8 断言的是契约不是实现 | +| C3 | 在 prodhome 的副本(双 mesa)上 doctor 报告;`--fix` 与报告共用同一谓词 | +| C1 | 装 `pkg@B` 到已有 `pkg@A` 的 subos,A 解绑、B 绑定、store 里 A 还在 | +| B5 | 声明 `__EGL_VENDOR_LIBRARY_DIRS` 指向我们载荷 → 安装期报告 | +| O2 | host-link 安装结束打印三个数,且三数之和 = 闭包大小 | +| B1/B2 | 端到端:`LD_LIBRARY_PATH` 上只有宿主驱动目录,`GL_RENDERER` 仍是 NVIDIA | + +### 3.2 最终真实验证 + +`.agents/tools/verify-release-lifecycle.sh --bin `。 + +在隔离 `XLINGS_HOME` 下,用**发布产物**(不是 dev build)跑完整生命周期,并断言两件 +一般测试不断言的事: + +1. **被测 home 不是开发者的**,且与 `$HOME` 无共同前缀。shim 会把 `XLINGS_HOME` 改写 + 成拥有它的那个 home,所以"我们跑在隔离 home 里"是一个需要**核对**的断言,不是前提。 +2. **真实 `~/.xlings` 事后逐字节未变**。不是"我们没打算动它",是查过。 + +做成脚本而不是清单,是因为手工执行的清单只测一次;8-06 那四个 home 缺陷里有三个, +是靠同一测量做了两遍、结果不一致才发现的。 + +--- + +## 4. 风险与回退 + +| 风险 | 触发信号 | 回退 | +|---|---|---| +| A3 改变了 patchelf 的身份,产物 RPATH 形态变化 | 同源断言开始报错 | payload 优先可以用一个环境变量关掉,回到旧顺序 | +| #42 的枚举式重写误伤二进制文件 | `bash -n` 或同源断言失败 | 重写只作用于文本文件,二进制走 patchelf | +| C1 的替换语义打破现有用户的并存假设 | doctor 在升级后大面积报告 | C3 先上线一个版本,C1 在下一个版本 | +| B1 的 DT_RPATH 被 glibc 移除 | 未来 glibc 报错 | AD-7 的 wrapper 方案 | +| 发布链两个 job 吞掉失败 | 无信号——必须主动查 | 发布后按 §0 的三项核对 | diff --git a/.agents/docs/2026-08-06-subos-matrix-verification.md b/.agents/docs/2026-08-06-subos-matrix-verification.md new file mode 100644 index 00000000..19e28138 --- /dev/null +++ b/.agents/docs/2026-08-06-subos-matrix-verification.md @@ -0,0 +1,271 @@ +# subos × libc × 图形栈:深度验证报告 + +日期:2026-08-06 +验证对象:xlings 2026.8.5.3 + libxpkg 0.0.50 + xim-pkgindex(当前 main) +验证方式:全部在隔离 `XLINGS_HOME` 中执行,宿主 `~/.xlings` 未被修改(见 §7) + +--- + +## 1. 结论先说 + +矩阵跑通了,但**不是一开始就跑通的**。这一轮验证发现了 5 个缺陷,其中 4 个已修复并提交,1 个记录待办。 + +关键在于:**这 5 个缺陷全都只在隔离 home 下暴露**。在默认 `~/.xlings` 下,每一个都表现为完全正常。这不是巧合——它们共享同一个成因,见 §3。 + +| # | 缺陷 | 默认 home 下的表现 | 状态 | +|---|---|---|---| +| D1 | 沙箱把 xlings home 重映射到 `~/.xlings`,烘焙的绝对路径全部落空 | 两个拼写恰好相同,无症状 | 已修复 `7604bb9` | +| D2 | recipe 把我们的 `libc.so.6` 放上 `LD_LIBRARY_PATH`,宿主二进制在宿主 loader 下崩溃 | 宿主 glibc 版本号相同,恰好不崩 | 已修复(xlings `e486364` + pkgindex recipe) | +| D3 | `locate_proot_` 经 PATH 找到**另一个 home 的 shim**,整个沙箱改道到那个 home | PATH 上的 shim 就属于当前 home,无症状 | 已修复 `0bef205` | +| D4 | `subos use --sandbox` 完全跳过 subos.env 层 | 同一 subos 两种进入方式配置不同,但图形程序常在非沙箱路径使用 | 已修复 `0bef205` | +| D5 | 同一 subos 绑定同一包的两个版本,双方都在贡献环境声明 | EGL 枚举出重复设备,doctor 不报 | **未修复**,见 §5 | + +附带修掉一个构建缺陷:`views::split \| ranges::to` 让 gcc 16 的模块构建以 "Bad file data" 失败并指向一个未改动的 TU(`f523219`)。发布目标用的是 gcc 15.1.0-musl,编译通过——所以它本可以带着这个问题发布。 + +--- + +## 2. 矩阵结果 + +### 2.1 libc 维度 + +同一个隔离 home 里两个 subos,各自装一套工具链,编译同一份 `hello.c`: + +| subos | 编译器解析到 | 产物 INTERP | DT_NEEDED | 运行 | +|---|---|---|---|---| +| `g-world` | `subos/g-world/bin/gcc` | `/data/xpkgs/xim-x-glibc/2.44/lib64/ld-linux-x86-64.so.2` | 1 | ✓ | +| `m-world` | `subos/m-world/bin/gcc` | 无(静态) | 0 | ✓,`env -i` 下也 ✓ | + +两个 subos 里编译器都叫 `gcc`,各自解析到正确的那一个,没有交叉污染。 + +用 g-world 产物自己的 loader 展开依赖: + +``` +libc.so.6 => /data/xpkgs/xim-x-glibc/2.44/lib64/libc.so.6 + /data/xpkgs/xim-x-glibc/2.44/lib64/ld-linux-x86-64.so.2 +``` + +INTERP 与 `libc.so.6` 来自**同一个 payload 目录**——同源不变量在用户自己编译的产物上成立,而不只是在我们安装的载荷上。除 `linux-vdso.so.1`(内核提供)外不涉及宿主。 + +全量扫描两个 home 的 store: + +| home | 通过 | 违反 | 跳过(无 INTERP) | +|---|---|---|---| +| mx1 | 35 | **0** | 365 | +| prodhome | 71 | **0** | 329 | + +musl 侧需要说明:索引里的 musl 只以工具链形式存在,产出静态二进制;整个索引中**只有 `glibc` 声明了 `exports.runtime.loader`**。也就是说 `abi` 字段目前没有第二个 ABI 可供区分——它是为将来准备的,现在还没有被真正行使。 + +### 2.2 图形栈维度 + +在 `prodhome`(mesa + libglvnd + nvidia-gl-host-link + X11/wayland 全栈)中,修复后两种进入方式结果完全一致: + +| 探针 | shell 进入 | `--sandbox proot` | +|---|---|---| +| EGL device 枚举 | DEVICE_COUNT=5 | DEVICE_COUNT=5 | +| 硬件路径 | `NVIDIA GeForce RTX 4080/PCIe/SSE2` | 同 | +| 软件回退 | `llvmpipe (LLVM 20.1.7, 256 bits)` | 同 | +| `LIBGL_DRIVERS_PATH` | 2 项 | 2 项 | +| `__EGL_VENDOR_LIBRARY_DIRS` | 3 项 | 3 项 | + +修复前:沙箱内三个变量**全为空**,GL 程序静默回退到宿主能提供的任何东西(D4)。 + +`MESA: error: ZINK: failed to load libvulkan.so.1` 在两侧都出现,是已知缺口——vulkan-loader 尚未打包(任务 #34),zink 后端因此不可用,不影响 llvmpipe 与 NVIDIA 路径。 + +### 2.3 沙箱后端与时延 + +| 项 | 结果 | +|---|---| +| proot 进入 | 59ms 均值,5/5 成功 | +| bwrap 进入 | 0/5,不可用 | +| 非沙箱 shell 进入 | 52ms 均值 | + +沙箱只比普通进入多约 7ms。 + +bwrap 不可用有**两个独立原因**,任一都足以致其失败: + +1. `kernel.apparmor_restrict_unprivileged_userns=1`(Ubuntu 24+ 默认); +2. 隔离 home 里的 bwrap 是 `-rwxr-xr-x`,而真实 home 里是 `-rwsr-xr-x`。setuid 位需要 root 才能设置,安装钩子在没有 sudo 时设不上。 + +这意味着**在没有 sudo 的机器上,新建的隔离 home 只能用 proot**。proot 基于 ptrace,功能上够用(上面所有测量都是它跑的),但性能与隔离强度都弱于 bwrap。这不是缺陷,是需要写进文档的既有约束。 + +### 2.4 生命周期 + +| 操作 | 耗时 | 结果 | +|---|---|---| +| `subos new` | 80ms | ✓ | +| `subos use --global` | 81ms | ✓ | +| `install`(已缓存) | 166ms | ✓ | +| 来回切换 | 99 / 101ms | ✓ | +| `subos remove`(活动中) | 34ms | ✗ rc=1 —— **正确行为** | +| `subos remove`(切走后) | — | ✓ | + +删除活动 subos 被拒绝,并给出补救命令(`switch first: xlings subos use default`)。这是本轮唯一一个"拒绝得恰到好处"的地方,值得记一笔:它同时报告了规则和出路。 + +--- + +## 3. 五个缺陷的同一个成因 + +D1、D2、D3、D5 是同一个形状的四个实例,也就是 +`.agents/docs/2026-08-05-dependency-resolution-single-source.md` 里那个"一个问题、多个回答者"——只不过这次问的不是"依赖是哪个版本",而是: + +> **xlings home 在哪里?** + +回答者有四个: + +1. `XLINGS_HOME` 环境变量; +2. 沙箱绑定的目标路径(原来是 `/.xlings`); +3. 安装时烘焙进产物的绝对路径(xvm alias target、RPATH、INTERP); +4. shim 从 argv0 反推出的 owner home。 + +**在默认配置下这四个答案是同一个字符串。** 所以它们从未被迫达成一致——它们只是恰好一致。隔离 home 是第一个让它们分开的配置,一分开就同时暴露四个缺陷。 + +这解释了为什么这些缺陷能活到现在:整个测试体系(包括 e2e)几乎都在默认 home 或与默认 home 同形的路径下运行。`subos_sandbox_test.sh` 的 S8 甚至**把错误模型写成了断言**——"`~/.xlings` 能看到宿主内容"——所以它不但没发现 D1,还会阻止别人修。 + +### 修法 + +按同一份设计文档的规则:**删掉多余的回答者,而不是让它们更容易一致。** + +- D1:home 永远绑定在**它自己的绝对路径**上。内外只有一个拼写,烘焙的绝对路径原样有效。 + 曾试过"两个路径都绑"——更糟:bind mount 不会把两条路径合并成一个,`weakly_canonical` 仍看到两个 home,shim 于是报告"与自己冲突"。 +- D3:PATH 回退拒绝任何位于 xlings home 内的候选。真正的 `/usr/bin/proot` 仍然可用。 +- D4:两条进入路径共用同一个 applier,而不是各写一份。 + +D2 是另一个形状,见下节。 + +--- + +## 4. D2:libc 不是可以放上搜索路径的依赖 + +这个值得单独讲,因为它是**同源分裂从设计断言看不见的方向**打进来的。 + +`nvidia-gl-host-link` 的处境是真实的:NVIDIA vendor 库是宿主的文件(符号链接指向 `/lib/x86_64-linux-gnu/`),我们**不能**给它打 RPATH——那要改宿主的文件。所以它需要一个搜索路径,recipe 把依赖收拢到 `lib/xlings-deps/` 并声明到 `LD_LIBRARY_PATH`。 + +问题是 `LD_LIBRARY_PATH` 是**进程全局且被每个子进程继承**的,而 subos 里绝大多数子进程是**宿主二进制、跑在宿主 loader 下**。目录里有我们的 `libc.so.6`,于是: + +``` +$ xlings subos use default --cmd 'echo HI' + ▸ entering subos default + 1229155: __vdso_time +$ echo $? +139 +``` + +`/bin/bash` 在打印任何字符之前就 SIGSEGV 了。宿主 glibc 是 2.39,我们的也是 2.39——**同一个上游版本,只是不同构建**,GLIBC_PRIVATE 就已经对不上。 + +同源断言看不见它,因为**我们安装的东西没有一个是错的**。错的是我们让宿主的东西去用我们的一半。 + +### 逐库测量,而不是逐库推理 + +我第一版修复删掉了整个 glibc 行。那是错的,测量推翻了它: + +| `xlings-deps` 内容 | EGL 枚举出 NVIDIA | `/bin/bash` | +|---|---|---| +| 什么都不放 | ✗ 消失 | ✓ | +| 只放 `libc.so.6` | ✗ 消失 | ✗ SIGSEGV | +| `libm/libpthread/libdl/librt`,不放 libc | ✓ | ✓ | +| `libpthread/librt/libdl`(vendor 的 DT_NEEDED 减去 libc) | ✓ RTX 4080 | ✓ | + +vendor 的 DT_NEEDED 是 `libpthread.so.0, librt.so.1, libc.so.6, libdl.so.2`(外加自家 `libnvidia-glsi`)。`libm` 根本不在上面。 + +- `libc.so.6` **既无用又致命**:vendor 是被 dlopen 进一个已在运行的进程的,它的 libc 早就绑定好了,已加载的 SONAME 不会再去搜索。所以这一项永远不会被用于它声称的目的。 +- `libpthread/librt/libdl` **必需且无害**:自 glibc 2.34 起它们是兼容桩,实现都搬进了 `libc.so.6`,分别只剩 27、13、9 个定义符号。 +- `libm` 有 1203 个符号(真正的 ABI 面)。**这里原本写的是「没人要它」,那是错的**—— + 它不在 `libEGL_nvidia` 的直接 DT_NEEDED 上,但被 16 个 nvidia 库 NEED,含核心渲染器 + `libnvidia-glcore`。我当时只取了闭包的一个入口。更正与影响见 + `2026-08-06-subos-architecture-proposal.md` §6。 + +最终 recipe 保留 glibc 行,只列这三个桩。 + +### 防线也按测量收窄 + +xlings 侧加了一道通用防线:构建环境时,拒绝把含 libc 的目录放上 `LD_LIBRARY_PATH`/`LD_PRELOAD`,点名目录和声明它的包,并保留该变量的其余部分。 + +我最初把整个 glibc 集合都列进"耦合"名单——**那会打断一个刚被测量证明必需的配置**。收窄到 `libc.so.6` 和 `ld-linux*`/`ld-musl*`:这一对的失败方式是"在 main 之前崩溃且不指名任何文件"。不匹配的 `libm` 会响亮地报 `version 'GLIBC_2.38' not found` 并指出文件名——那是可诊断的,不是这道防线要防的东西。 + +E2E-63 覆盖的是这一**类**,不是这个 recipe,并且专门断言桩库不能被误伤。 + +### 一个诚实的局限 + +`LD_LIBRARY_PATH` 只能按**目录**取舍,不能说"这个目录除了某个文件"。所以在尚未更新的载荷上,防线丢掉整个目录,连带丢掉 libX11 等本该保留的库,图形栈降级——直到 recipe 重新发布。 + +这是安全的方向(降级而非崩溃),而且有明确的报告。要做到文件级,就得让 xlings 物化一个过滤后的镜像目录——那会成为"vendor 的依赖在哪里"的第二个回答者,正是本轮在拆的东西。所以按目录取舍是这里的诚实边界,recipe 才是正确的修复位置。 + +--- + +## 5. D5:未修复 —— 一个 subos 绑定同一个包的两个版本 + +`prodhome` 的 `default` subos 里: + +``` +$ xlings list | grep mesa + ◆ xim:mesa@25.0.7.1 + ◆ xim:mesa@25.0.7 +``` + +subos manifest 里两个 binding 都在,两者都在贡献 `__EGL_VENDOR_LIBRARY_DIRS`(3 项 = nvidia + 两个 mesa)。EGL 因此枚举出重复的 llvmpipe 设备。 + +按你定的三层模型:**store 可以有多个版本,subos sysroot 恰好一个**。这里 subos 层拿到了两个,环境层把 store 的"多版本"泄漏进了 subos 的"恰好一个"。 + +目前的实际后果有限(两个 mesa 版本兼容,枚举出重复设备而已)。但如果两个版本的 DRI ABI 不同,选中哪个驱动将取决于目录顺序——一个由安装顺序决定的、没有任何东西报告的结果。 + +**`xlings self doctor` 不报这个。** 又是一次"从未发生"和"成功了"输出相同。 + +没有在本轮修复:它需要在 subos 层强制单版本(安装第二个版本时替换而非并列),牵涉 xvm 注册与 env manifest 两处语义,不适合塞进这批修复里。已作为独立任务记录。 + +--- + +## 6. 已确认成立的性质 + +这些跑通了,并且是**用会失败的方式**验证的: + +- **同源不变量**:两个 store 共 106 个带 INTERP 的 ELF,0 违反;用户自己编译的产物同样成立。 +- **home 隔离**:修复 D3 后,隔离 home 缺少沙箱后端时**拒绝执行**并给出 `xlings install proot`,而不是借用宿主的。 +- **路径同一性**:沙箱内外 home 只有一个拼写,`XLINGS_HOME`、`PATH[0]`、烘焙路径三者一致。E2E 里用 `env -i` 净环境断言,且被测 home 位于 `/tmp` 下——这是绑定顺序最难的情形(`/tmp` 先被私有化,再挂 home),它通过了。 +- **两条进入路径等价**:shell 与 sandbox 的 subos.env 现在逐项一致。 +- **libc 矩阵**:glibc 动态与 musl 静态在同一 home 中共存,互不干扰;musl 产物在 `env -i` 下运行正常。 +- **生命周期**:create/use/install/switch/remove/doctor 全程无残留,删除活动 subos 被正确拒绝。 +- **沙箱私有 /tmp**:宿主 `/tmp` 下的文件在沙箱内不可见(这是预期行为,验证时踩到过一次)。 + +--- + +## 7. 宿主 xlings home 未被破坏 + +按约束核对: + +- 我的第一个提交是 03:07。 +- 宿主 home 的实质性改动(packages、index、`subos/default`,共 1000+ 项)时间戳在 **01:52–02:32**,全部早于此。 +- 我的工作窗口(03:00 之后)内,宿主 home 只有一个文件被动过:`data/xim-pkgindex-local/.xlings-index-cache.json`(一个索引**缓存**)。 +- `data/xpkgs`、`subos/`、`.xlings.json` 三处在 03:00 后改动数为 **0**。 + +即:宿主的包、subos 与配置状态未被本轮验证改动。 + +需要说明的是,`--version` 与 `list` 单独执行不会写入宿主 home(已实测),那个缓存文件的写入没能复现出来,来源未确定。 + +--- + +## 8. 测试体系上的教训 + +三条,都值得改进流程而不只是改代码: + +1. **e2e 把错误模型写成了断言。** `subos_sandbox_test.sh` 的 S8 断言 `~/.xlings` 在沙箱内能看到宿主内容——那正是 D1 的错误行为。一个测试不仅可能漏掉缺陷,还可能把缺陷钉死。已改写为契约断言(`XLINGS_HOME` 即宿主路径、`PATH[0]` 与之一致、不存在第二个拼写)。 + +2. **默认 home 让四个答案恒等。** 只要测试都在默认 home 或同形路径下跑,这一整类缺陷就不可见。**隔离 home 应当成为沙箱与 subos 测试的默认环境,而不是特例。** + +3. **发布目标编译通过 ≠ 代码没问题。** `views::split | ranges::to` 在 gcc 15.1.0-musl(发布目标)下编译通过,在 gcc 16.1.0(默认工具链)下让整个模块以 "Bad file data" 失败,并且指向一个未改动的 TU。只跑发布目标的构建会让它直接发出去。**改动核心模块后必须两个工具链都构建。** + +--- + +## 9. 提交 + +xlings(分支 `fix/sandbox-xlings-home`): + +| commit | 内容 | +|---|---| +| `7604bb9` | D1 —— home 绑定在自己的绝对路径上;rc 模板三份重复收敛为一个写入器;S8 改写 | +| `e486364` | D2 防线 —— 拒绝把 libc 放上全局搜索路径;新增 E2E-63 | +| `0bef205` | D3 + D4 —— 拒绝另一个 home 的 shim;两条进入路径共用 env applier;防线按测量收窄 | +| `f523219` | 构建 —— 去掉让 gcc 16 模块构建失败的 C++23 ranges | + +xim-pkgindex(分支 `docs/resolved-deps-spec`):`nvidia-gl-host-link.lua` —— `xlings-deps` 只保留 `libpthread/librt/libdl`,去掉 `libc.so.6`(致命且无用)与 `libm.so.6`(无人需要且 ABI 面最大)。 + +测试:单测 35 passed / 0 failed(55 个用例);E2E-63 六条断言全过。 diff --git a/.agents/tools/slice-real-home.sh b/.agents/tools/slice-real-home.sh index 007d7754..9a342dc2 100755 --- a/.agents/tools/slice-real-home.sh +++ b/.agents/tools/slice-real-home.sh @@ -190,10 +190,32 @@ import os, pathlib, sys src, dst = sys.argv[1].rstrip('/'), sys.argv[2].rstrip('/') root = pathlib.Path(dst) targets = [root / '.xlings.json'] + sorted(root.glob('subos/*/.xlings.json')) + +# The index cache maps every package name to the ABSOLUTE path of its recipe. +# Left alone it points at the real home, so the slice installs packages by +# reading the real home's recipes -- and a recipe change under test is silently +# not the one being tested. Measured: editing glibc.lua in the slice's own +# data/xim-pkgindex changed nothing at all, twice, with no diagnostic; the +# install kept running the version in ~/.xlings. +# +# That is the trap this whole tool exists to avoid, one level deeper than the +# payload store: an experiment that reports on a home other than the one it +# claims to. Enumerated per directory rather than by a `data/**` glob, because +# `data/xpkgs` is tens of gigabytes and holds JSON belonging to payloads, which +# must keep whatever paths they were installed with. +for d in ('', 'data', 'data/xim-pkgindex', 'data/xim-index-repos'): + base = root / d if d else root + if not base.is_dir(): + continue + targets += sorted(p for p in base.glob('*.json') if p.is_file()) + targets += sorted(p for p in base.glob('.*.json') if p.is_file()) + n = 0 +seen = set() for path in targets: - if not path.is_file() or path.is_symlink(): + if path in seen or not path.is_file() or path.is_symlink(): continue + seen.add(path) text = path.read_text(encoding='utf-8') if src not in text: continue @@ -201,6 +223,23 @@ for path in targets: n += 1 print(f" rewrote {n} state file(s)") +# Assert it, rather than trust the list above. A new state file that nobody +# added to that list would otherwise keep pointing at the real home, and the +# only symptom would be a measurement that quietly describes the wrong home. +missed = [] +for d in ('', 'data', 'data/xim-pkgindex', 'data/xim-index-repos'): + base = root / d if d else root + if not base.is_dir(): + continue + for path in list(base.glob('*.json')) + list(base.glob('.*.json')): + if not path.is_file() or path.is_symlink(): + continue + if src in path.read_text(encoding='utf-8', errors='replace'): + missed.append(str(path)) +if missed: + raise SystemExit("slice-real-home: these still name the real home after " + "repointing: " + ", ".join(missed)) + # The sysroot is made of symlinks INTO the payload store, and `cp -a` copies a # symlink's text verbatim -- so every one of them still points at the real # home. Left that way the slice does not merely lose fidelity, it invents a diff --git a/.agents/tools/verify-release-lifecycle.sh b/.agents/tools/verify-release-lifecycle.sh new file mode 100755 index 00000000..11dcaf2d --- /dev/null +++ b/.agents/tools/verify-release-lifecycle.sh @@ -0,0 +1,175 @@ +#!/usr/bin/env bash +# Run the whole subos lifecycle against a RELEASED xlings, in an isolated home. +# +# Why a script and not a checklist: the four home-related defects of 2026-08-06 +# were all invisible in the default home, and three of them were found only +# because a measurement was taken twice and disagreed. A checklist executed by +# hand takes the measurement once. +# +# Two things this asserts that a normal test does not: +# +# * the home under test is NOT the developer's, and shares no prefix with it. +# A shim rewrites XLINGS_HOME to whichever home owns it, so "we ran in the +# isolated home" is a claim that has to be checked, not assumed. +# * the real ~/.xlings is byte-unchanged afterwards. Not "we did not mean to +# touch it" -- checked. +# +# Usage: +# verify-release-lifecycle.sh --bin [--home ] [--keep] +set -uo pipefail + +BIN="" +HOME_DIR="" +KEEP=0 +while [[ $# -gt 0 ]]; do + case "$1" in + --bin) BIN="$2"; shift 2 ;; + --home) HOME_DIR="$2"; shift 2 ;; + --keep) KEEP=1; shift ;; + *) echo "unknown argument: $1" >&2; exit 2 ;; + esac +done +[[ -n "$BIN" && -x "$BIN" ]] || { echo "usage: --bin " >&2; exit 2; } +BIN="$(cd "$(dirname "$BIN")" && pwd)/$(basename "$BIN")" +HOME_DIR="${HOME_DIR:-${TMPDIR:-/tmp}/xlings-release-verify-$(id -u)}" + +fail() { echo "FAIL: $*" >&2; exit 1; } +ok() { echo " ✓ $*"; } +step() { echo; echo "── $* ─────────────────────────────"; } + +REAL="${HOME%/}/.xlings" +case "$HOME_DIR" in + "$REAL"|"$REAL"/*) fail "the verification home IS the real home: $HOME_DIR" ;; +esac +[[ "$HOME_DIR" == "${HOME%/}"/* ]] \ + && echo " note: shares a prefix with \$HOME — the sandbox bind ordering" \ + "this exercises is the easy one" + +rm -rf "$HOME_DIR"; mkdir -p "$HOME_DIR" +trap '[[ $KEEP == 1 ]] || rm -rf "$HOME_DIR"' EXIT + +# A marker older than anything this run can do, to compare the real store +# against afterwards. +MARKER="$(mktemp)"; trap 'rm -f "$MARKER"' RETURN 2>/dev/null || true + +x() { ( cd /tmp && env -i HOME="$HOME" PATH=/usr/bin:/bin \ + XLINGS_HOME="$HOME_DIR" "$BIN" "$@" ) } + +echo "binary: $BIN" +echo "home: $HOME_DIR" +echo "version: $(x --version 2>&1 | head -1)" + +step "1. the client anchors to the home under test" +x self init >/dev/null 2>&1 || true +GOT="$(x self config 2>&1)" +echo "$GOT" | grep -q "$HOME_DIR" \ + || fail "the client reports a home other than the one under test. +That is the failure this whole script exists to make visible: a shim rewrites +XLINGS_HOME to whichever home owns it, so every measurement taken afterwards +would describe the developer's home while looking exactly like a measurement of +this one. +$GOT" +ok "XLINGS_HOME is honoured" + +step "2. install a package and check the payload" +x install patchelf -y >/dev/null 2>&1 || fail "install patchelf failed" +PE="$(ls "$HOME_DIR"/data/xpkgs/*-x-patchelf/*/bin/patchelf 2>/dev/null | head -1)" +[[ -n "$PE" ]] || fail "no patchelf payload after a successful install" +ok "payload at ${PE#$HOME_DIR/}" + +step "3. glibc's payload carries no build path and its ldd parses" +x install glibc -y >/dev/null 2>&1 || x install xim:glibc -y >/dev/null 2>&1 \ + || fail "install glibc failed" +G="$(ls -d "$HOME_DIR"/data/xpkgs/*-x-glibc/* 2>/dev/null | head -1)" +[[ -n "$G" ]] || fail "no glibc payload" +# TEXT files only, and that distinction is the point rather than a shortcut. +# +# ld.so has the build prefix compiled into it as its default library search +# path, and after AD-11 that is `/nonexistent/xlings-use-rpath-not-default- +# search` -- deliberately a path that cannot exist, so that everything must +# come from DT_RPATH and an unpatched binary fails loudly instead of quietly +# picking up the host's loader. A marker inside a binary is the design; the +# same marker inside a shell script or a .pc file is the defect. +# +# `.xpkg.lua` is the recipe copied in as a record, and it mentions the marker +# in prose. +LEFT="$(grep -rlI "xlings_data" "$G" 2>/dev/null \ + | grep -v '\.xpkg\.lua$' || true)" +if [[ -n "$LEFT" ]]; then + echo "$LEFT" | head -10 >&2 + fail "$(echo "$LEFT" | wc -l) text file(s) in the payload still name the build machine" +fi +ok "no build path in any text file" +bash -n "$G/bin/ldd" 2>/dev/null || fail "the ldd we ship does not parse" +ok "bin/ldd passes bash -n" +grep -q "RTLDLIST=\"" "$G/bin/ldd" || fail "RTLDLIST was swallowed by the rewrite" +ok "RTLDLIST survived the rewrite" + +step "4. a subos of its own, entered, with its environment" +x subos new relverify >/dev/null 2>&1 || fail "subos new failed" +x subos use relverify >/dev/null 2>&1 || true + +# The graphics stack is the real consumer of subos.env, but it is large and not +# present on every machine. glibc is: it registers with xvm, so `subos use` +# has something to resolve and PATH has something to carry. +OUT="$(x subos use relverify --cmd 'echo "IN=[$XLINGS_HOME]"; echo "P0=[${PATH%%:*}]"' 2>&1)" +echo "$OUT" | grep -q "IN=\[$HOME_DIR\]" \ + || fail "inside the subos, XLINGS_HOME is not the home under test. +A session that re-anchors to another home makes every measurement taken inside +it a measurement of the wrong home, while looking exactly like the right one: +$OUT" +ok "subos session anchors to the home under test" + +echo "$OUT" | grep -q "P0=\[$HOME_DIR/subos/relverify/bin\]" \ + || fail "PATH[0] inside the subos is not this subos's bin: +$OUT" +ok "PATH[0] is this subos's bin" + +step "5. one package, one live version" +# Two versions of glibc exist in the index. Installing both must leave the +# environment naming exactly one -- the contract E2E-64 states, checked here +# against a released binary and the real index rather than a fixture. +x install xim:glibc@2.44 -y >/dev/null 2>&1 || true +MAN="$HOME_DIR/subos/relverify/.xlings.json" +if [[ -f "$MAN" ]]; then + DUP="$(python3 - "$MAN" <<'PY' +import json, sys, collections +d = json.load(open(sys.argv[1])) +envs = d.get("subos_info", {}).get("envs", {}) +by = collections.Counter(k.split("@")[0] for k in envs) +ws = d.get("workspace", {}) +bad = [n for n, c in by.items() + if c > 1 and not ws.get(n, {}).get("active")] +print(" ".join(bad)) +PY +)" + [[ -z "$DUP" ]] || fail "a package is bound at several versions with no +active version, so every one of them contributes: $DUP" + ok "no contested binding" +fi + +step "6. doctor, from inside the subos" +OUT="$(x subos use relverify --cmd "$BIN self doctor" 2>&1 || true)" +echo "$OUT" | grep -qiE "double binding|loader/libc split" \ + && fail "doctor reports a structural defect on a freshly built home: +$OUT" +ok "no structural findings from inside the subos" + +step "7. doctor is clean" +OUT="$(x self doctor 2>&1 || true)" +echo "$OUT" | grep -qiE "double binding|env orphan|loader/libc split" \ + && fail "doctor reports a defect on a freshly built home: +$OUT" +ok "no findings" + +step "8. the real home was never written" +if [[ -d "$REAL/data/xpkgs" ]]; then + NEWER="$(find "$REAL/data/xpkgs" -newer "$MARKER" -print -quit 2>/dev/null)" + [[ -z "$NEWER" ]] || fail "the real store changed during this run: $NEWER" + ok "$REAL/data/xpkgs unchanged" +else + ok "no real store on this machine to disturb" +fi + +echo +echo "PASS: release lifecycle" diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index da91bcbf..5394f12f 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -55,22 +55,40 @@ jobs: with: path: | ~/.mcpp + !~/.mcpp/bmi + !~/.mcpp/build-cache .mcpp - # `mcpp-v3-`: retired v2 on 2026-08-06. The key already hashes - # mcpp.toml, so a dependency bump misses it exactly -- but - # `restore-keys` then falls back to the newest v2 entry, whose - # registry carries the mcpp-index snapshot from BEFORE the bump. - # `mcpplibs.xpkg@0.0.50` was published and resolvable from a clean - # home while three CI jobs insisted it did not exist. Retiring the - # prefix is what stops the fallback reaching those entries. - # `mcpp-v3-`: the v1 keyspace holds entries saved by runs that - # built against a fallback-restored BMI set and failed. Those are - # poisoned at their exact key, so the guard below -- which only fires - # on an INEXACT restore -- can never reach them. Retiring the prefix - # is what discards them; the guard is what stops it recurring. - key: mcpp-v3-${{ runner.os }}-${{ hashFiles('mcpp.toml', 'mcpp.lock', '.xlings.json') }}-${{ env.BOOTSTRAP_XLINGS_VERSION }} + !.mcpp/bmi + !.mcpp/build-cache + # BMIs are NOT cached, and are deleted after every restore. That + # is the fix, and it took three prefix retirements to find. + # + # A restored BMI set never works. `import 'std' has CRC mismatch` + # follows, which reads like a compiler bug. What hid it is that the + # old guard wiped the BMIs on an INEXACT restore -- so retiring the + # key prefix forced an inexact restore, fired the guard, and produced + # one clean run. Each retirement therefore looked like a fix and was + # an accident; the next run got an exact hit, skipped the guard, and + # failed again. Read straight off two consecutive runs of THIS + # workflow: the one that passed shows "Drop stale BMIs" as `success`, + # the one that failed shows it `skipped`. + # + # So the wipe is unconditional now, and the BMI directories are + # excluded from the cache as well -- the exclusion is an optimisation + # (do not store bytes we delete on arrival), the unconditional wipe is + # the guarantee, and it cannot depend on exclude-pattern semantics. + # + # What the cache is FOR is `registry/data/xpkgs`: ~800 MB of payloads + # that are toolchain-independent and survive a round trip fine. + # + # The key still names the workflow that produced it. mcpp.toml + # declares both the dev toolchain (gcc@16.1.0) and the release target + # (gcc@15.1.0-musl), so a CI build and a release build hash the same + # -- sharing one key across workflows is wrong on its own terms even + # though it was not what caused this. + key: mcpp-v4-release-${{ runner.os }}-${{ hashFiles('mcpp.toml', 'mcpp.lock', '.xlings.json') }}-${{ env.BOOTSTRAP_XLINGS_VERSION }} restore-keys: | - mcpp-v3-${{ runner.os }}- + mcpp-v4-release-${{ runner.os }}- # An inexact restore (the `restore-keys` fallback) brings BMIs built # under a different dependency set. Mixing them with anything rebuilt # here fails as `import 'std' has CRC mismatch` — which reads like a @@ -82,8 +100,7 @@ jobs: # Only the BMIs go. The payload store under registry/data/xpkgs is # dependency-independent and is what makes the ~800 MB cache worth # restoring at all. - - name: Drop stale BMIs when the cache key did not match exactly - if: steps.mcpp-cache.outputs.cache-hit != 'true' + - name: Drop restored BMIs shell: bash run: | rm -rf ~/.mcpp/bmi ~/.mcpp/build-cache .mcpp/bmi .mcpp/build-cache @@ -249,22 +266,25 @@ jobs: with: path: | ~/.mcpp + !~/.mcpp/bmi + !~/.mcpp/build-cache .mcpp + !.mcpp/bmi + !.mcpp/build-cache # `mcpp-v3-`: the v1 keyspace holds entries saved by runs that # built against a fallback-restored BMI set and failed. Those are # poisoned at their exact key, so the guard below -- which only fires # on an INEXACT restore -- can never reach them. Retiring the prefix # is what discards them; the guard is what stops it recurring. - key: mcpp-v3-${{ runner.os }}-${{ hashFiles('mcpp.toml', 'mcpp.lock', '.xlings.json') }}-${{ env.BOOTSTRAP_XLINGS_VERSION }} + key: mcpp-v4-release-${{ runner.os }}-${{ hashFiles('mcpp.toml', 'mcpp.lock', '.xlings.json') }}-${{ env.BOOTSTRAP_XLINGS_VERSION }} restore-keys: | - mcpp-v3-${{ runner.os }}- + mcpp-v4-release-${{ runner.os }}- # An inexact restore brings BMIs built under a different dependency set. # Mixing them with anything rebuilt here fails as `import 'std' has CRC # mismatch`, which reads like a compiler bug. Only the BMIs go -- the # payload store is dependency-independent and is what makes the ~800 MB # cache worth restoring. - - name: Drop stale BMIs when the cache key did not match exactly - if: steps.mcpp-cache.outputs.cache-hit != 'true' + - name: Drop restored BMIs shell: bash run: | rm -rf ~/.mcpp/bmi ~/.mcpp/build-cache .mcpp/bmi .mcpp/build-cache @@ -366,22 +386,25 @@ jobs: with: path: | ~\.mcpp + !~\.mcpp/bmi + !~\.mcpp/build-cache .mcpp + !.mcpp/bmi + !.mcpp/build-cache # `mcpp-v3-`: the v1 keyspace holds entries saved by runs that # built against a fallback-restored BMI set and failed. Those are # poisoned at their exact key, so the guard below -- which only fires # on an INEXACT restore -- can never reach them. Retiring the prefix # is what discards them; the guard is what stops it recurring. - key: mcpp-v3-${{ runner.os }}-${{ hashFiles('mcpp.toml', 'mcpp.lock', '.xlings.json') }}-${{ env.BOOTSTRAP_XLINGS_VERSION }} + key: mcpp-v4-release-${{ runner.os }}-${{ hashFiles('mcpp.toml', 'mcpp.lock', '.xlings.json') }}-${{ env.BOOTSTRAP_XLINGS_VERSION }} restore-keys: | - mcpp-v3-${{ runner.os }}- + mcpp-v4-release-${{ runner.os }}- # An inexact restore brings BMIs built under a different dependency set. # Mixing them with anything rebuilt here fails as `import 'std' has CRC # mismatch`, which reads like a compiler bug. Only the BMIs go -- the # payload store is dependency-independent and is what makes the ~800 MB # cache worth restoring. - - name: Drop stale BMIs when the cache key did not match exactly - if: steps.mcpp-cache.outputs.cache-hit != 'true' + - name: Drop restored BMIs shell: bash run: | rm -rf ~/.mcpp/bmi ~/.mcpp/build-cache .mcpp/bmi .mcpp/build-cache diff --git a/.github/workflows/xlings-ci-linux-e2e.yml b/.github/workflows/xlings-ci-linux-e2e.yml index 4a3f55e2..cfc197fb 100644 --- a/.github/workflows/xlings-ci-linux-e2e.yml +++ b/.github/workflows/xlings-ci-linux-e2e.yml @@ -63,22 +63,40 @@ jobs: with: path: | ~/.mcpp + !~/.mcpp/bmi + !~/.mcpp/build-cache .mcpp - # `mcpp-v3-`: retired v2 on 2026-08-06. The key already hashes - # mcpp.toml, so a dependency bump misses it exactly -- but - # `restore-keys` then falls back to the newest v2 entry, whose - # registry carries the mcpp-index snapshot from BEFORE the bump. - # `mcpplibs.xpkg@0.0.50` was published and resolvable from a clean - # home while three CI jobs insisted it did not exist. Retiring the - # prefix is what stops the fallback reaching those entries. - # `mcpp-v3-`: the v1 keyspace holds entries saved by runs that - # built against a fallback-restored BMI set and failed. Those are - # poisoned at their exact key, so the guard below -- which only fires - # on an INEXACT restore -- can never reach them. Retiring the prefix - # is what discards them; the guard is what stops it recurring. - key: mcpp-v3-${{ runner.os }}-${{ hashFiles('mcpp.toml', 'mcpp.lock', '.xlings.json') }}-${{ env.BOOTSTRAP_XLINGS_VERSION }} + !.mcpp/bmi + !.mcpp/build-cache + # BMIs are NOT cached, and are deleted after every restore. That + # is the fix, and it took three prefix retirements to find. + # + # A restored BMI set never works. `import 'std' has CRC mismatch` + # follows, which reads like a compiler bug. What hid it is that the + # old guard wiped the BMIs on an INEXACT restore -- so retiring the + # key prefix forced an inexact restore, fired the guard, and produced + # one clean run. Each retirement therefore looked like a fix and was + # an accident; the next run got an exact hit, skipped the guard, and + # failed again. Read straight off two consecutive runs of THIS + # workflow: the one that passed shows "Drop stale BMIs" as `success`, + # the one that failed shows it `skipped`. + # + # So the wipe is unconditional now, and the BMI directories are + # excluded from the cache as well -- the exclusion is an optimisation + # (do not store bytes we delete on arrival), the unconditional wipe is + # the guarantee, and it cannot depend on exclude-pattern semantics. + # + # What the cache is FOR is `registry/data/xpkgs`: ~800 MB of payloads + # that are toolchain-independent and survive a round trip fine. + # + # The key still names the workflow that produced it. mcpp.toml + # declares both the dev toolchain (gcc@16.1.0) and the release target + # (gcc@15.1.0-musl), so a CI build and a release build hash the same + # -- sharing one key across workflows is wrong on its own terms even + # though it was not what caused this. + key: mcpp-v4-ci-linux-e2e-${{ runner.os }}-${{ hashFiles('mcpp.toml', 'mcpp.lock', '.xlings.json') }}-${{ env.BOOTSTRAP_XLINGS_VERSION }} restore-keys: | - mcpp-v3-${{ runner.os }}- + mcpp-v4-ci-linux-e2e-${{ runner.os }}- # An inexact restore (the `restore-keys` fallback) brings BMIs built # under a different dependency set. Mixing them with anything rebuilt # here fails as `import 'std' has CRC mismatch` — which reads like a @@ -90,8 +108,7 @@ jobs: # Only the BMIs go. The payload store under registry/data/xpkgs is # dependency-independent and is what makes the ~800 MB cache worth # restoring at all. - - name: Drop stale BMIs when the cache key did not match exactly - if: steps.mcpp-cache.outputs.cache-hit != 'true' + - name: Drop restored BMIs shell: bash run: | rm -rf ~/.mcpp/bmi ~/.mcpp/build-cache .mcpp/bmi .mcpp/build-cache diff --git a/.github/workflows/xlings-ci-linux-root.yml b/.github/workflows/xlings-ci-linux-root.yml index 3036e106..445a8687 100644 --- a/.github/workflows/xlings-ci-linux-root.yml +++ b/.github/workflows/xlings-ci-linux-root.yml @@ -60,22 +60,40 @@ jobs: with: path: | ~/.mcpp + !~/.mcpp/bmi + !~/.mcpp/build-cache .mcpp - # `mcpp-v3-`: retired v2 on 2026-08-06. The key already hashes - # mcpp.toml, so a dependency bump misses it exactly -- but - # `restore-keys` then falls back to the newest v2 entry, whose - # registry carries the mcpp-index snapshot from BEFORE the bump. - # `mcpplibs.xpkg@0.0.50` was published and resolvable from a clean - # home while three CI jobs insisted it did not exist. Retiring the - # prefix is what stops the fallback reaching those entries. - # `mcpp-v3-`: the v1 keyspace holds entries saved by runs that - # built against a fallback-restored BMI set and failed. Those are - # poisoned at their exact key, so the guard below -- which only fires - # on an INEXACT restore -- can never reach them. Retiring the prefix - # is what discards them; the guard is what stops it recurring. - key: mcpp-v3-${{ runner.os }}-${{ hashFiles('mcpp.toml', 'mcpp.lock', '.xlings.json') }}-${{ env.BOOTSTRAP_XLINGS_VERSION }} + !.mcpp/bmi + !.mcpp/build-cache + # BMIs are NOT cached, and are deleted after every restore. That + # is the fix, and it took three prefix retirements to find. + # + # A restored BMI set never works. `import 'std' has CRC mismatch` + # follows, which reads like a compiler bug. What hid it is that the + # old guard wiped the BMIs on an INEXACT restore -- so retiring the + # key prefix forced an inexact restore, fired the guard, and produced + # one clean run. Each retirement therefore looked like a fix and was + # an accident; the next run got an exact hit, skipped the guard, and + # failed again. Read straight off two consecutive runs of THIS + # workflow: the one that passed shows "Drop stale BMIs" as `success`, + # the one that failed shows it `skipped`. + # + # So the wipe is unconditional now, and the BMI directories are + # excluded from the cache as well -- the exclusion is an optimisation + # (do not store bytes we delete on arrival), the unconditional wipe is + # the guarantee, and it cannot depend on exclude-pattern semantics. + # + # What the cache is FOR is `registry/data/xpkgs`: ~800 MB of payloads + # that are toolchain-independent and survive a round trip fine. + # + # The key still names the workflow that produced it. mcpp.toml + # declares both the dev toolchain (gcc@16.1.0) and the release target + # (gcc@15.1.0-musl), so a CI build and a release build hash the same + # -- sharing one key across workflows is wrong on its own terms even + # though it was not what caused this. + key: mcpp-v4-ci-linux-root-${{ runner.os }}-${{ hashFiles('mcpp.toml', 'mcpp.lock', '.xlings.json') }}-${{ env.BOOTSTRAP_XLINGS_VERSION }} restore-keys: | - mcpp-v3-${{ runner.os }}- + mcpp-v4-ci-linux-root-${{ runner.os }}- # An inexact restore (the `restore-keys` fallback) brings BMIs built # under a different dependency set. Mixing them with anything rebuilt # here fails as `import 'std' has CRC mismatch` — which reads like a @@ -87,8 +105,7 @@ jobs: # Only the BMIs go. The payload store under registry/data/xpkgs is # dependency-independent and is what makes the ~800 MB cache worth # restoring at all. - - name: Drop stale BMIs when the cache key did not match exactly - if: steps.mcpp-cache.outputs.cache-hit != 'true' + - name: Drop restored BMIs shell: bash run: | rm -rf ~/.mcpp/bmi ~/.mcpp/build-cache .mcpp/bmi .mcpp/build-cache diff --git a/.github/workflows/xlings-ci-linux.yml b/.github/workflows/xlings-ci-linux.yml index ad34a9f2..6832b883 100644 --- a/.github/workflows/xlings-ci-linux.yml +++ b/.github/workflows/xlings-ci-linux.yml @@ -65,22 +65,40 @@ jobs: with: path: | ~/.mcpp + !~/.mcpp/bmi + !~/.mcpp/build-cache .mcpp - # `mcpp-v3-`: retired v2 on 2026-08-06. The key already hashes - # mcpp.toml, so a dependency bump misses it exactly -- but - # `restore-keys` then falls back to the newest v2 entry, whose - # registry carries the mcpp-index snapshot from BEFORE the bump. - # `mcpplibs.xpkg@0.0.50` was published and resolvable from a clean - # home while three CI jobs insisted it did not exist. Retiring the - # prefix is what stops the fallback reaching those entries. - # `mcpp-v3-`: the v1 keyspace holds entries saved by runs that - # built against a fallback-restored BMI set and failed. Those are - # poisoned at their exact key, so the guard below -- which only fires - # on an INEXACT restore -- can never reach them. Retiring the prefix - # is what discards them; the guard is what stops it recurring. - key: mcpp-v3-${{ runner.os }}-${{ hashFiles('mcpp.toml', 'mcpp.lock', '.xlings.json') }}-${{ env.BOOTSTRAP_XLINGS_VERSION }} + !.mcpp/bmi + !.mcpp/build-cache + # BMIs are NOT cached, and are deleted after every restore. That + # is the fix, and it took three prefix retirements to find. + # + # A restored BMI set never works. `import 'std' has CRC mismatch` + # follows, which reads like a compiler bug. What hid it is that the + # old guard wiped the BMIs on an INEXACT restore -- so retiring the + # key prefix forced an inexact restore, fired the guard, and produced + # one clean run. Each retirement therefore looked like a fix and was + # an accident; the next run got an exact hit, skipped the guard, and + # failed again. Read straight off two consecutive runs of THIS + # workflow: the one that passed shows "Drop stale BMIs" as `success`, + # the one that failed shows it `skipped`. + # + # So the wipe is unconditional now, and the BMI directories are + # excluded from the cache as well -- the exclusion is an optimisation + # (do not store bytes we delete on arrival), the unconditional wipe is + # the guarantee, and it cannot depend on exclude-pattern semantics. + # + # What the cache is FOR is `registry/data/xpkgs`: ~800 MB of payloads + # that are toolchain-independent and survive a round trip fine. + # + # The key still names the workflow that produced it. mcpp.toml + # declares both the dev toolchain (gcc@16.1.0) and the release target + # (gcc@15.1.0-musl), so a CI build and a release build hash the same + # -- sharing one key across workflows is wrong on its own terms even + # though it was not what caused this. + key: mcpp-v4-ci-linux-${{ runner.os }}-${{ hashFiles('mcpp.toml', 'mcpp.lock', '.xlings.json') }}-${{ env.BOOTSTRAP_XLINGS_VERSION }} restore-keys: | - mcpp-v3-${{ runner.os }}- + mcpp-v4-ci-linux-${{ runner.os }}- # An inexact restore (the `restore-keys` fallback) brings BMIs built # under a different dependency set. Mixing them with anything rebuilt # here fails as `import 'std' has CRC mismatch` — which reads like a @@ -92,8 +110,7 @@ jobs: # Only the BMIs go. The payload store under registry/data/xpkgs is # dependency-independent and is what makes the ~800 MB cache worth # restoring at all. - - name: Drop stale BMIs when the cache key did not match exactly - if: steps.mcpp-cache.outputs.cache-hit != 'true' + - name: Drop restored BMIs shell: bash run: | rm -rf ~/.mcpp/bmi ~/.mcpp/build-cache .mcpp/bmi .mcpp/build-cache diff --git a/.github/workflows/xlings-ci-macos.yml b/.github/workflows/xlings-ci-macos.yml index 9cd0e47c..ec96b948 100644 --- a/.github/workflows/xlings-ci-macos.yml +++ b/.github/workflows/xlings-ci-macos.yml @@ -55,22 +55,40 @@ jobs: with: path: | ~/.mcpp + !~/.mcpp/bmi + !~/.mcpp/build-cache .mcpp - # `mcpp-v3-`: retired v2 on 2026-08-06. The key already hashes - # mcpp.toml, so a dependency bump misses it exactly -- but - # `restore-keys` then falls back to the newest v2 entry, whose - # registry carries the mcpp-index snapshot from BEFORE the bump. - # `mcpplibs.xpkg@0.0.50` was published and resolvable from a clean - # home while three CI jobs insisted it did not exist. Retiring the - # prefix is what stops the fallback reaching those entries. - # `mcpp-v3-`: the v1 keyspace holds entries saved by runs that - # built against a fallback-restored BMI set and failed. Those are - # poisoned at their exact key, so the guard below -- which only fires - # on an INEXACT restore -- can never reach them. Retiring the prefix - # is what discards them; the guard is what stops it recurring. - key: mcpp-v3-${{ runner.os }}-dt110-${{ hashFiles('mcpp.toml', 'mcpp.lock', '.xlings.json') }}-${{ env.BOOTSTRAP_XLINGS_VERSION }} + !.mcpp/bmi + !.mcpp/build-cache + # BMIs are NOT cached, and are deleted after every restore. That + # is the fix, and it took three prefix retirements to find. + # + # A restored BMI set never works. `import 'std' has CRC mismatch` + # follows, which reads like a compiler bug. What hid it is that the + # old guard wiped the BMIs on an INEXACT restore -- so retiring the + # key prefix forced an inexact restore, fired the guard, and produced + # one clean run. Each retirement therefore looked like a fix and was + # an accident; the next run got an exact hit, skipped the guard, and + # failed again. Read straight off two consecutive runs of THIS + # workflow: the one that passed shows "Drop stale BMIs" as `success`, + # the one that failed shows it `skipped`. + # + # So the wipe is unconditional now, and the BMI directories are + # excluded from the cache as well -- the exclusion is an optimisation + # (do not store bytes we delete on arrival), the unconditional wipe is + # the guarantee, and it cannot depend on exclude-pattern semantics. + # + # What the cache is FOR is `registry/data/xpkgs`: ~800 MB of payloads + # that are toolchain-independent and survive a round trip fine. + # + # The key still names the workflow that produced it. mcpp.toml + # declares both the dev toolchain (gcc@16.1.0) and the release target + # (gcc@15.1.0-musl), so a CI build and a release build hash the same + # -- sharing one key across workflows is wrong on its own terms even + # though it was not what caused this. + key: mcpp-v4-ci-macos-${{ runner.os }}-dt110-${{ hashFiles('mcpp.toml', 'mcpp.lock', '.xlings.json') }}-${{ env.BOOTSTRAP_XLINGS_VERSION }} restore-keys: | - mcpp-v3-${{ runner.os }}-dt110- + mcpp-v4-ci-macos-${{ runner.os }}-dt110- # An inexact restore (the `restore-keys` fallback) brings BMIs built # under a different dependency set. Mixing them with anything rebuilt # here fails as `import 'std' has CRC mismatch` — which reads like a @@ -82,8 +100,7 @@ jobs: # Only the BMIs go. The payload store under registry/data/xpkgs is # dependency-independent and is what makes the ~800 MB cache worth # restoring at all. - - name: Drop stale BMIs when the cache key did not match exactly - if: steps.mcpp-cache.outputs.cache-hit != 'true' + - name: Drop restored BMIs shell: bash run: | rm -rf ~/.mcpp/bmi ~/.mcpp/build-cache .mcpp/bmi .mcpp/build-cache diff --git a/.github/workflows/xlings-ci-windows.yml b/.github/workflows/xlings-ci-windows.yml index 27e02a31..b5c3e3c5 100644 --- a/.github/workflows/xlings-ci-windows.yml +++ b/.github/workflows/xlings-ci-windows.yml @@ -55,22 +55,40 @@ jobs: with: path: | ~\.mcpp + !~\.mcpp/bmi + !~\.mcpp/build-cache .mcpp - # `mcpp-v3-`: retired v2 on 2026-08-06. The key already hashes - # mcpp.toml, so a dependency bump misses it exactly -- but - # `restore-keys` then falls back to the newest v2 entry, whose - # registry carries the mcpp-index snapshot from BEFORE the bump. - # `mcpplibs.xpkg@0.0.50` was published and resolvable from a clean - # home while three CI jobs insisted it did not exist. Retiring the - # prefix is what stops the fallback reaching those entries. - # `mcpp-v3-`: the v1 keyspace holds entries saved by runs that - # built against a fallback-restored BMI set and failed. Those are - # poisoned at their exact key, so the guard below -- which only fires - # on an INEXACT restore -- can never reach them. Retiring the prefix - # is what discards them; the guard is what stops it recurring. - key: mcpp-v3-${{ runner.os }}-${{ hashFiles('mcpp.toml', 'mcpp.lock', '.xlings.json') }}-${{ env.BOOTSTRAP_XLINGS_VERSION }} + !.mcpp/bmi + !.mcpp/build-cache + # BMIs are NOT cached, and are deleted after every restore. That + # is the fix, and it took three prefix retirements to find. + # + # A restored BMI set never works. `import 'std' has CRC mismatch` + # follows, which reads like a compiler bug. What hid it is that the + # old guard wiped the BMIs on an INEXACT restore -- so retiring the + # key prefix forced an inexact restore, fired the guard, and produced + # one clean run. Each retirement therefore looked like a fix and was + # an accident; the next run got an exact hit, skipped the guard, and + # failed again. Read straight off two consecutive runs of THIS + # workflow: the one that passed shows "Drop stale BMIs" as `success`, + # the one that failed shows it `skipped`. + # + # So the wipe is unconditional now, and the BMI directories are + # excluded from the cache as well -- the exclusion is an optimisation + # (do not store bytes we delete on arrival), the unconditional wipe is + # the guarantee, and it cannot depend on exclude-pattern semantics. + # + # What the cache is FOR is `registry/data/xpkgs`: ~800 MB of payloads + # that are toolchain-independent and survive a round trip fine. + # + # The key still names the workflow that produced it. mcpp.toml + # declares both the dev toolchain (gcc@16.1.0) and the release target + # (gcc@15.1.0-musl), so a CI build and a release build hash the same + # -- sharing one key across workflows is wrong on its own terms even + # though it was not what caused this. + key: mcpp-v4-ci-windows-${{ runner.os }}-${{ hashFiles('mcpp.toml', 'mcpp.lock', '.xlings.json') }}-${{ env.BOOTSTRAP_XLINGS_VERSION }} restore-keys: | - mcpp-v3-${{ runner.os }}- + mcpp-v4-ci-windows-${{ runner.os }}- # An inexact restore (the `restore-keys` fallback) brings BMIs built # under a different dependency set. Mixing them with anything rebuilt # here fails as `import 'std' has CRC mismatch` — which reads like a @@ -82,8 +100,7 @@ jobs: # Only the BMIs go. The payload store under registry/data/xpkgs is # dependency-independent and is what makes the ~800 MB cache worth # restoring at all. - - name: Drop stale BMIs when the cache key did not match exactly - if: steps.mcpp-cache.outputs.cache-hit != 'true' + - name: Drop restored BMIs shell: bash run: | rm -rf ~/.mcpp/bmi ~/.mcpp/build-cache .mcpp/bmi .mcpp/build-cache diff --git a/mcpp.lock b/mcpp.lock index d92098ed..ace9fc27 100644 --- a/mcpp.lock +++ b/mcpp.lock @@ -31,3 +31,9 @@ version = "0.2.9" source = "index+mcpplibs@0.2.9" hash = "fnv1a:3465dd0bd5d7aa20" +[package."mcpplibs.xpkg"] +namespace = "mcpplibs" +version = "0.0.51" +source = "index+mcpplibs@0.0.51" +hash = "fnv1a:71031031ffb66224" + diff --git a/mcpp.toml b/mcpp.toml index 63fe8055..d1a7d899 100644 --- a/mcpp.toml +++ b/mcpp.toml @@ -1,6 +1,6 @@ [package] name = "xlings" -version = "2026.8.5.3" +version = "2026.8.6.1" description = "Universal package management infrastructure tool with SubOS isolation" license = "Apache-2.0" repo = "https://github.com/openxlings/xlings" @@ -39,7 +39,7 @@ libarchive = "3.8.7" [dependencies.mcpplibs] cmdline = "0.0.2" -xpkg = "0.0.50" +xpkg = "0.0.51" tinyhttps = "0.2.9" capi.lua = "0.0.3" diff --git a/src/core/config.cppm b/src/core/config.cppm index c834ce79..6b118ebc 100644 --- a/src/core/config.cppm +++ b/src/core/config.cppm @@ -13,7 +13,7 @@ import xlings.core.xvm.db; namespace xlings { export struct Info { - static constexpr std::string_view VERSION = "2026.8.5.3"; + static constexpr std::string_view VERSION = "2026.8.6.1"; static constexpr std::string_view REPO = "https://github.com/openxlings/xlings"; }; diff --git a/src/core/elf_same_source.cppm b/src/core/elf_same_source.cppm index 7caec928..5abd3bb6 100644 --- a/src/core/elf_same_source.cppm +++ b/src/core/elf_same_source.cppm @@ -5,6 +5,7 @@ export module xlings.core.elf_same_source; import std; import xlings.platform; import xlings.core.log; +import xlings.core.version_order; // A binary's loader and its libc must come from the same payload. // @@ -106,6 +107,85 @@ inline std::string describe(const Finding& f) { f.binary, f.provider, f.interpPayload, f.rpathPayload); } +// The store root (`.../xpkgs`) containing a path, or empty. +// +// Derived from the scanned directory rather than from XLINGS_HOME: the two +// agree in the default configuration and diverge exactly when it matters -- +// a shim rewrites XLINGS_HOME to the home that owns it, so a helper invoked +// through one sees the real home rather than the isolated one under test. +// The directory being scanned cannot lie about which store it is in. +inline std::string store_root_of(std::string_view p) { + const auto marker = std::string("/xpkgs/"); + auto pos = p.rfind(marker); + if (pos == std::string_view::npos) return {}; + return std::string(p.substr(0, pos + marker.size() - 1)); +} + +// The patchelf that stamped these fields, resolved from the payload. +// +// R6 (see .agents/docs/2026-08-06-subos-architecture-proposal.md §1.5): when +// xlings itself needs a tool it resolves the payload, never the view. This +// used to be `command -v patchelf`, which inside a subos session hits one of +// our shims -- re-entering xlings and anchoring to whichever home owns it -- +// and outside one hits whatever `/usr/bin/patchelf` the machine happens to +// have. patchelf versions differ in how they grow the dynamic segment and in +// `--force-rpath` semantics, so a reader that is not the writer can report a +// mismatch that does not exist, or miss one that does. +// +// Highest installed version, which is the same choice libxpkg's resolver +// makes for an unpinned dependency -- these two have to agree, and agreeing by +// construction is the only way that holds. +// +// Returns empty when there is no payload and no host tool. The caller treats +// that as "unverifiable", not as "clean". +inline std::string locate_patchelf(const std::filesystem::path& scanned) { + namespace fs = std::filesystem; + std::error_code ec; + + auto store = store_root_of(scanned.string()); + if (!store.empty()) { + std::string bestVer, bestPath; + for (auto it = fs::directory_iterator(fs::path(store), ec); + !ec && it != fs::directory_iterator(); it.increment(ec)) { + const auto name = it->path().filename().string(); + if (name != "patchelf" && !name.ends_with("-x-patchelf")) continue; + std::error_code vec; + for (auto vit = fs::directory_iterator(it->path(), vec); + !vec && vit != fs::directory_iterator(); vit.increment(vec)) { + // A separate error_code: sharing the outer loop's would let a + // single unreadable entry set it and end the iteration, which + // reads as "no payload here" and silently falls through to the + // host tool. + std::error_code fec; + auto candidate = vit->path() / "bin" / "patchelf"; + if (!fs::is_regular_file(candidate, fec)) continue; + const auto ver = vit->path().filename().string(); + if (bestVer.empty() + || version_order::compare(ver, bestVer) > 0) { + bestVer = ver; + bestPath = candidate.string(); + } + } + } + if (!bestPath.empty()) return bestPath; + } + + // No payload in this store. Fall back so that an older home stays + // verifiable, and say so -- landing here means the fields are being read + // by a tool that did not write them. + auto [rc, out] = platform::run_command_capture( + "command -v patchelf 2>/dev/null"); + if (rc != 0) return {}; + while (!out.empty() && (out.back() == '\n' || out.back() == '\r')) { + out.pop_back(); + } + if (!out.empty()) { + log::debug("elfcheck: no patchelf payload under {}; reading with {}", + store.empty() ? scanned.string() : store, out); + } + return out; +} + // Every ELF under DIR whose loader and libc come from different payloads. // // Reads the two fields with patchelf, which is the same tool that wrote them @@ -122,23 +202,19 @@ scan_payload(const std::filesystem::path& dir) { std::error_code ec; if (!std::filesystem::is_directory(dir, ec)) return out; - // `command -v`, the same way the rest of this file locates tools. PATH - // already has the subos bin dir prepended by the caller. auto trim = [](std::string v) { while (!v.empty() && (v.back() == '\n' || v.back() == '\r')) v.pop_back(); return v; }; - auto [rcWhich, whichOut] = - platform::run_command_capture("command -v patchelf 2>/dev/null"); - if (rcWhich != 0) return out; - const auto patchelf = trim(whichOut); + const auto patchelf = locate_patchelf(dir); if (patchelf.empty()) return out; - // run_command_capture merges stderr into stdout, and `patchelf` here is - // usually an xlings shim that prints its own advisory lines. Taking the - // whole capture as the value put a log message where a path belongs. The - // value is the last line that looks like one; everything a shim emits is - // bracketed log output, and it comes first. + // run_command_capture merges stderr into stdout. On the payload path + // `patchelf` is the real binary and prints only its answer; on the + // fallback path it can still be one of our shims, which emit their own + // bracketed advisory lines first, and taking the whole capture as the + // value put a log message where a path belongs. The value is therefore the + // last line that looks like one -- correct for both. auto run = [&](const std::string& args) -> std::string { auto [rc, o] = platform::run_command_capture( std::format("\"{}\" {}", patchelf, args)); diff --git a/src/core/subos.cppm b/src/core/subos.cppm index 7bc9d52d..e7399eeb 100644 --- a/src/core/subos.cppm +++ b/src/core/subos.cppm @@ -836,6 +836,102 @@ inline manifest::Placeholders placeholders_for_(const fs::path& subosDir) { }; } +// Is this the name of a library that is welded to a particular ld.so? +// +// ld.so and libc.so.6 are two halves of one build and talk to each other over +// GLIBC_PRIVATE. Pairing halves from different builds does not fail to load -- +// it segfaults before main, naming nothing. +// +// This list is deliberately just those two, and not the rest of the glibc set. +// The argument for a wider list is that libm and friends come from the same +// build; the argument against is measured. nvidia-gl-host-link has to offer +// the vendor library libpthread/librt/libdl, which since glibc 2.34 are +// compatibility stubs with 27, 13 and 9 defined symbols -- their +// implementations moved into libc.so.6 -- and without them the NVIDIA device +// disappears from EGL enumeration entirely. Flagging those would break a +// working configuration to prevent nothing. libm has real surface (1203 +// symbols), but a host binary that picks up a mismatched libm dies with +// "version `GLIBC_2.38' not found" and names the file. Loud and diagnosable +// is not what this guard is for; it is for the failure that names nothing. +inline bool is_loader_coupled_soname_(std::string_view file) { + return file == "libc.so.6" + || file.starts_with("libc.musl-") + || file.starts_with("ld-linux") + || file.starts_with("ld-musl"); +} + +// Refuse to put a libc on a process-global search path. +// +// LD_LIBRARY_PATH and LD_PRELOAD are inherited by every child, and most +// children in a subos are HOST binaries running under the HOST loader. A +// directory of ours holding libc.so.6 hands them our half of a pair whose +// other half is the host's, which is the loader/libc split the rest of this +// codebase exists to prevent -- arrived at from the one direction the +// same-source assertion cannot see, because nothing we installed is wrong. +// `xlings subos use` returned a /bin/bash that died of SIGSEGV before +// printing a character, on a host whose glibc was the same upstream version +// as ours and merely a different build. +// +// Dropping the offending entry rather than the whole variable: the other +// directories on it are usually the point of the declaration, and a package +// that gathers dependencies into one directory has no way to ask for "all of +// these except the libc" today. +inline void drop_loader_coupled_dirs_(std::vector& vars) { + for (auto& v : vars) { + const bool is_preload = v.var == "LD_PRELOAD" + || v.var == "DYLD_INSERT_LIBRARIES"; + if (v.var != "LD_LIBRARY_PATH" && v.var != "DYLD_LIBRARY_PATH" + && !is_preload) continue; + + std::vector kept, dropped; + std::error_code ec; + // Hand-rolled split/join. `views::split | ranges::to` + // reads better and makes gcc 16 fail the whole module with "Bad file + // data" pointing at an unrelated TU; gcc 15 compiles it. Not worth a + // toolchain investigation for four lines. + for (std::size_t pos = 0; pos <= v.value.size(); ) { + const auto sep = v.value.find(':', pos); + const auto len = (sep == std::string::npos ? v.value.size() : sep) - pos; + std::string entry = v.value.substr(pos, len); + pos = (sep == std::string::npos ? v.value.size() : sep) + 1; + if (entry.empty()) { if (sep == std::string::npos) break; continue; } + + bool offends = false; + if (is_preload) { + offends = is_loader_coupled_soname_( + fs::path(entry).filename().string()); + } else if (fs::is_directory(entry, ec)) { + for (const auto& e : platform::dir_entries(entry)) { + if (is_loader_coupled_soname_(e.path().filename().string())) { + offends = true; + break; + } + } + } + (offends ? dropped : kept).push_back(std::move(entry)); + if (sep == std::string::npos) break; + } + if (dropped.empty()) continue; + + v.value.clear(); + for (const auto& k : kept) { + if (!v.value.empty()) v.value += ':'; + v.value += k; + } + for (const auto& d : dropped) { + std::println(stderr, + "[xlings] {} — dropped {} (declared by {}): it holds a libc, " + "and this variable is inherited by host binaries running under " + "the host loader", + v.var, d, + v.providers.empty() ? std::string{"?"} : v.providers.front()); + } + } + std::erase_if(vars, [](const manifest::Resolved& v) { + return v.value.empty() && !v.unresolved; + }); +} + // The variables a subos exports, resolved and ready to apply. // // Empty for a subos with no declarations, which is every subos until a package @@ -844,7 +940,16 @@ inline std::vector subos_env_for_(const std::string& name) { const auto dir = Config::subos_dir(name); auto doc = manifest::read_document(dir); if (!doc) return {}; - return manifest::resolve(manifest::parse(*doc), placeholders_for_(dir)); + // C2: only the providers that can take effect. A package installed at two + // versions keeps both sections -- the dormant one is what lets `xlings use` + // switch back without a reinstall -- but only the active one contributes. + // Which is active is xvm's answer, recorded in this same file; nothing here + // re-derives it. See manifest::select_effective. + auto info = manifest::select_effective(manifest::parse(*doc), + manifest::active_versions(*doc)); + auto vars = manifest::resolve(info, placeholders_for_(dir)); + drop_loader_coupled_dirs_(vars); + return vars; } // UC-2: say what was injected. @@ -872,6 +977,35 @@ inline void report_injected_env_(const std::string& subosName, } } +// Put a subos's declared variables into THIS process's environment. +// +// Shared by the shell-spawn path and the sandbox path. Both hand their +// environment to a child -- `run_shell` to a shell, the sandbox to +// proot/bwrap, which pass it through -- so a variable only one of them sets +// makes the same subos two different environments depending on how it was +// entered. It did: `subos use` had LIBGL_DRIVERS_PATH and +// __EGL_VENDOR_LIBRARY_DIRS, `subos use --sandbox` had neither, and a GL +// program inside the sandbox fell back to whatever the host offered without +// anything saying so. +// +// UC-1 -- a variable already set in this environment is the user's, and `set` +// leaves it alone. `prepend` still contributes, since composing is what +// prepend means. +inline void apply_subos_env_(const std::string& name) { + const auto envVars = subos_env_for_(name); + report_injected_env_(name, envVars); + for (const auto& v : envVars) { + if (v.unresolved) continue; + const auto existing = utils::get_env_or_default(v.var); + if (v.op == manifest::OP_PREPEND) { + platform::set_env_variable( + v.var, existing.empty() ? v.value : v.value + ":" + existing); + } else if (existing.empty()) { + platform::set_env_variable(v.var, v.value); + } + } +} + } // namespace use_detail_ // Internal — not exported. `xlings subos use --global ` and @@ -1063,7 +1197,16 @@ int use_spawn_shell(const std::string& name, EventStream& stream, // M3: `cmd` non-empty switches to non-interactive single-command // execution — `shell -c ` instead of an interactive shell. // Useful for scripts and agent workflows. - if (sandbox) return sandbox::enter(name, stream, sandbox_backend, gpu, cmd); + if (sandbox) { + // Before entering, not inside: the sandbox module cannot import this + // one (this one imports it), and proot/bwrap pass our environment + // through to the shell anyway. The home is bound at its own absolute + // path, so the payload paths in these values mean the same thing on + // both sides of the boundary. + if (auto rc = use_detail_::validate_subos_(name, stream); rc != 0) return rc; + use_detail_::apply_subos_env_(name); + return sandbox::enter(name, stream, sandbox_backend, gpu, cmd); + } warn_storage_dormant_on_shell_(name); if (auto rc = use_detail_::validate_subos_(name, stream); rc != 0) return rc; @@ -1104,20 +1247,7 @@ int use_spawn_shell(const std::string& name, EventStream& stream, // UC-1 -- a variable already set in this environment is the user's, and // `set` leaves it alone. `prepend` still contributes, since composing is // what prepend means. - { - const auto envVars = use_detail_::subos_env_for_(name); - use_detail_::report_injected_env_(name, envVars); - for (const auto& v : envVars) { - if (v.unresolved) continue; - const auto existing = utils::get_env_or_default(v.var); - if (v.op == manifest::OP_PREPEND) { - platform::set_env_variable( - v.var, existing.empty() ? v.value : v.value + ":" + existing); - } else if (existing.empty()) { - platform::set_env_variable(v.var, v.value); - } - } - } + use_detail_::apply_subos_env_(name); nlohmann::json payload; payload["name"] = name; diff --git a/src/core/subos/manifest.cppm b/src/core/subos/manifest.cppm index 65d3808b..0b3d57fe 100644 --- a/src/core/subos/manifest.cppm +++ b/src/core/subos/manifest.cppm @@ -102,6 +102,135 @@ std::string_view binding_name(std::string_view binding) { return at == std::string_view::npos ? binding : binding.substr(0, at); } +std::string_view binding_version(std::string_view binding) { + const auto at = binding.find('@'); + return at == std::string_view::npos ? std::string_view{} + : binding.substr(at + 1); +} + +// ── the subos layer's "exactly one" ───────────────────────────────────── +// +// One package, one version, per subos. The three-layer model has always said +// so -- the xpkg store holds many versions by design, each consumer freezes one +// into its own RPATH/INTERP, and the subos sysroot in between is the layer that +// is supposed to hold exactly one -- but nothing anywhere enforced it. +// +// Measured on a real home: `xlings list` showed mesa@25.0.7 and mesa@25.0.7.1 +// both bound in `default`, both contributing to __EGL_VENDOR_LIBRARY_DIRS, and +// EGL duly enumerated the device twice. `xlings self doctor` said nothing. +// +// This is P1 wearing a different hat. "What is in this subos" has two records +// -- the xvm registration and this manifest's envs section -- and installing a +// second version appends to both. They agree, which is why nothing complained; +// they agree on an answer the model forbids. +// +// ONE function, used by the report and by --fix. Every previous report/repair +// pair in this repo has drifted, and the shape it takes is a finding that +// repairing does not clear, so the predicate lives here rather than in doctor. +struct DuplicateBinding { + std::string name; + std::vector bindings; // every binding for that name, sorted +}; + +std::vector duplicate_bindings(const Info& info) { + std::map> byName; + for (const auto& p : info.envs) { + if (!is_binding(p.binding)) continue; + byName[std::string(binding_name(p.binding))].push_back(p.binding); + } + std::vector out; + for (auto& [name, bindings] : byName) { + if (bindings.size() < 2) continue; + std::ranges::sort(bindings); + bindings.erase(std::ranges::unique(bindings).begin(), bindings.end()); + if (bindings.size() < 2) continue; // one version listed twice is not two + out.push_back({.name = name, .bindings = std::move(bindings)}); + } + return out; +} + +// Which version of each package THIS subos has active. +// +// Read from the same file as the declarations. `subos//.xlings.json` +// holds both `workspace` (name -> active/installed) and `subos_info.envs` +// (binding -> declarations) -- two records of "what is in this subos", in one +// file. That is the whole of P1 in eleven lines of JSON. +std::map active_versions(const nlohmann::json& doc) { + std::map out; + if (!doc.contains("workspace") || !doc["workspace"].is_object()) return out; + for (auto it = doc["workspace"].begin(); it != doc["workspace"].end(); ++it) { + if (!it.value().is_object()) continue; + auto active = it.value().value("active", std::string{}); + if (!active.empty()) out[it.key()] = std::move(active); + } + return out; +} + +// True when `version` is the one `active` names, allowing for the `:` +// form a namespaced install records. +inline bool version_is_active(std::string_view version, std::string_view active) { + if (version == active) return true; + const auto colon = active.find(':'); + return colon != std::string_view::npos && active.substr(colon + 1) == version; +} + +// The providers that actually take effect, out of everything the manifest +// records. +// +// A package installed at two versions keeps BOTH provider sections -- the +// dormant one has to survive so that `xlings use pkg@` restores its +// environment without reinstalling. What must not happen is both contributing +// at once, which is how one GPU came to be enumerated as two. +// +// Which one is live is not a decision made here: xvm already made it, and its +// answer is in this same file. Re-deriving it (highest version? last +// installed?) would be a second answerer to a question that has one -- the +// exact defect this function exists to remove. +// +// A package with NO active version keeps every provider, and that default +// matters more than it looks: filtering on a record that turns out to be absent +// would silently delete a package's whole environment, which is the failure +// mode this file exists to prevent, arrived at from the other side. Measured +// before choosing it -- a bare `xvm.add(name)` does record an active version, +// so this is the salvage path for a manifest whose workspace record was lost +// (pruned, copied between homes, hand-edited), not the common case. Where +// several versions are declared with no active one, nothing can say which is +// meant; that state is reported rather than guessed at. +Info select_effective(const Info& info, + const std::map& active) { + Info out = info; + out.envs.clear(); + for (const auto& p : info.envs) { + if (is_binding(p.binding)) { + const auto it = active.find(std::string(binding_name(p.binding))); + if (it != active.end() + && !version_is_active(binding_version(p.binding), it->second)) { + continue; + } + } + out.envs.push_back(p); + } + return out; +} + +// A package bound at several versions with nothing able to say which is meant. +// +// NOT the same as duplicate_bindings: two versions where one is active is +// ordinary -- the dormant declarations are how `xlings use` can switch back. +// This is the subset that has no active version -- every one of them +// contributes, so the subos exports each variable several times over. Reaching +// it takes a lost workspace record, not an ordinary install. +std::vector +contested_bindings(const Info& info, + const std::map& active) { + std::vector out; + for (auto& dup : duplicate_bindings(info)) { + if (active.contains(dup.name)) continue; + out.push_back(std::move(dup)); + } + return out; +} + // ── invariants ────────────────────────────────────────────────────────── enum class Defect { @@ -407,6 +536,73 @@ bool has_unresolved(std::string_view expanded) { return expanded.find("${") != std::string_view::npos; } +// ── privileged declarations ───────────────────────────────────────────── +// +// A `subos.env` declaration whose value points at OUR payload is one of two +// very different things, and the difference is not "is it process-global": +// +// causes CODE to be loaded LD_LIBRARY_PATH, LD_PRELOAD, +// into someone's process __EGL_VENDOR_LIBRARY_DIRS, LIBGL_DRIVERS_PATH, +// VK_ICD_FILENAMES, and every future variable +// some library invents for finding its plugins +// causes DATA to be found XDG_DATA_DIRS, MANPATH, PKG_CONFIG_PATH +// +// The first class is dangerous because every child of the subos shell inherits +// it, and most of those children are HOST binaries under the HOST loader. +// Measured: a host binary linked against the host's libEGL drops from the +// NVIDIA GPU to llvmpipe under our declarations, and LD_DEBUG shows it loading +// OUR libm, libgcc_s, libstdc++ and libxcb into a process running on the host's +// libc. On this machine the host glibc happened to match; on an older one that +// is `version 'GLIBC_2.xx' not found`. The second class is ordinary — subos +// supplies a default, the user can override it, that is how Linux works (AD-3). +// `PATH` is a third thing again: it does not inject code into an existing +// process, it decides which executable runs, and it is governed by R6/AD-1 +// rather than by this guard. +// +// The list below is the BENIGN one -- named for what it asserts, since PATH is +// on it and PATH plainly does not name only data. The check is default-deny. Listing the +// dangerous set instead would be a hand-written list of "what we happened to +// think of" — the exact anti-pattern R7 names, and the one that already cost us +// five missing entries in nvidia-gl-host-link's dependency table. A variable +// nobody has classified reads as privileged, which fails toward a report. +// +// Adding to this list is a deliberate act: it asserts the variable cannot cause +// code to enter a process. +inline bool never_loads_code(std::string_view var) { + return var == "XDG_DATA_DIRS" || var == "XDG_CONFIG_DIRS" + || var == "XDG_DATA_HOME" || var == "XDG_CONFIG_HOME" + || var == "XDG_CACHE_HOME" || var == "XDG_STATE_HOME" + || var == "MANPATH" || var == "INFOPATH" + || var == "PKG_CONFIG_PATH" || var == "PKG_CONFIG_LIBDIR" + || var == "ACLOCAL_PATH" || var == "TERMINFO" + || var == "FONTCONFIG_PATH" || var == "FONTCONFIG_FILE" + || var == "SSL_CERT_FILE" || var == "SSL_CERT_DIR" + || var == "GIT_SSL_CAINFO" || var == "CURL_CA_BUNDLE" + || var == "LOCPATH" || var == "TZDIR" + || var == "PATH"; // R6/AD-1's business, not this guard's +} + +// A declaration is privileged when it can put code from our payload into a +// process we do not own. +// +// The placeholders are checked as well as an expanded store path, because at +// install time -- the moment this most needs to be reported -- the value has +// not been expanded yet. +// +// `${subosdir}` counts. The subos sysroot is a VIEW onto our payloads, made of +// symlinks into them, so a directory under it on a loader search path delivers +// our libraries just as surely as the store path does. Checking only +// `${pkgdir}` would have let the same declaration through in its other spelling +// -- one hazard with two names, which is the shape this whole review is about. +inline bool is_privileged_env(std::string_view var, std::string_view value) { + if (never_loads_code(var)) return false; + for (const auto* needle : {"${pkgdir}", "${subosdir}", "${xlings_home}", + "/xpkgs/"}) { + if (value.find(needle) != std::string_view::npos) return true; + } + return false; +} + // ── resolution ────────────────────────────────────────────────────────── // One variable as it will actually be exported. diff --git a/src/core/subos/sandbox.cppm b/src/core/subos/sandbox.cppm index 909d4fdd..9a1067c6 100644 --- a/src/core/subos/sandbox.cppm +++ b/src/core/subos/sandbox.cppm @@ -26,6 +26,7 @@ import xlings.core.utils; import xlings.core.xim.commands; // auto_install_backend_ needs cmd_install import xlings.core.xim.compatibility; import xlings.core.subos.gpu; +import xlings.core.xvm.shim; // resolve_owner_home: reject another home's shim // Runtime isolation for a subos: proot/bwrap backends, storage images, GPU // passthrough, and entering an isolated session. @@ -149,24 +150,62 @@ inline constexpr std::string_view kEtcNsswitch = // files that just chain into the host xlings profile; user can edit // these to add their own customizations later (they're sandbox-private // so they won't pollute host). +// $XLINGS_HOME, not $HOME/.xlings. +// +// The sandbox already carries XLINGS_HOME in, correctly. These rc files did +// not read it, so a subos living in a non-default home sourced the DEFAULT +// home's profile and came up with `PATH[0]=$HOME/.xlings/subos//bin` — +// a directory that does not exist there. The shell starts fine and the first +// command reports `cannot execute: required file not found`, naming a path +// the user never asked for. +// +// Two readers of "which home is this", one taking the environment and one +// assuming the default. They agree for everybody using ~/.xlings, which is +// why it went unnoticed. inline constexpr std::string_view kSandboxBashrc = - "# xlings sandbox bashrc — chains to host xlings profile so PATH /\n" - "# prompt pill / XLINGS_BIN are set up. Edit this file to add your\n" + "# xlings sandbox bashrc — chains to the xlings profile of THIS home so\n" + "# PATH / prompt pill / XLINGS_BIN are set up. Edit this file to add your\n" "# own customizations (it's sandbox-private at /home/).\n" - "if [ -r \"$HOME/.xlings/config/shell/xlings-profile.sh\" ]; then\n" - " . \"$HOME/.xlings/config/shell/xlings-profile.sh\"\n" - "fi\n"; + "_xlings_profile=\"${XLINGS_HOME:-$HOME/.xlings}/config/shell/xlings-profile.sh\"\n" + "if [ -r \"$_xlings_profile\" ]; then\n" + " . \"$_xlings_profile\"\n" + "fi\n" + "unset _xlings_profile\n"; inline constexpr std::string_view kSandboxProfile = "# xlings sandbox profile (sourced by sh / bash login shells)\n" "if [ -r \"$HOME/.bashrc\" ]; then\n" " . \"$HOME/.bashrc\"\n" "fi\n"; inline constexpr std::string_view kSandboxFishConfig = - "# xlings sandbox fish config — chains to host xlings profile\n" - "if test -r \"$HOME/.xlings/config/shell/xlings-profile.fish\"\n" - " source \"$HOME/.xlings/config/shell/xlings-profile.fish\"\n" + "# xlings sandbox fish config — chains to the xlings profile of THIS home\n" + "set -l _xlings_home $XLINGS_HOME\n" + "test -n \"$_xlings_home\"; or set _xlings_home \"$HOME/.xlings\"\n" + "if test -r \"$_xlings_home/config/shell/xlings-profile.fish\"\n" + " source \"$_xlings_home/config/shell/xlings-profile.fish\"\n" "end\n"; + +// Every shell's rc for one sandbox home, written from ONE set of templates. +// +// These files existed in three places with three copies of the same text, and +// the copies drifted: fixing the profile path in one left the other two +// pointing at `$HOME/.xlings`, so a subos in a non-default home still sourced +// the wrong profile. Duplication was not the bug, but it is what made the bug +// survive a fix. +inline void write_sandbox_rc_(const fs::path& home_dir) { + auto try_write = [](const fs::path& path, std::string_view body) { + if (fs::exists(path)) return; // never clobber user edits + platform::write_string_to_file(path.string(), std::string(body)); + }; + fs::create_directories(home_dir); + try_write(home_dir / ".bashrc", kSandboxBashrc); + try_write(home_dir / ".zshrc", kSandboxBashrc); + try_write(home_dir / ".profile", kSandboxProfile); + auto fish_dir = home_dir / ".config" / "fish"; + fs::create_directories(fish_dir); + try_write(fish_dir / "config.fish", kSandboxFishConfig); +} + // Initialize the sandbox-specific dirs / templates inside an existing // subos. Idempotent: only writes files that don't yet exist, so a // returning sandbox session won't clobber user customizations and a @@ -231,8 +270,7 @@ void init_sandbox_dirs_(const fs::path& subos_dir, // Seed shell rc files so the xlings profile gets sourced (PATH + // prompt pill). Sandbox-private; user can edit freely. - try_write(user_home / ".bashrc", kSandboxBashrc); - try_write(user_home / ".profile", kSandboxProfile); + write_sandbox_rc_(user_home); auto fish_config_dir = user_home / ".config" / "fish"; fs::create_directories(fish_config_dir); try_write(fish_config_dir / "config.fish", kSandboxFishConfig); @@ -306,11 +344,14 @@ export int unmount_image_(const fs::path& mountpoint) { // Probe order for the proot binary: -// 1. ~/.xlings/data/xpkgs/xim-x-proot//bin/proot (xpkg-managed, -// future once xim:proot ships) -// 2. ~/.xlings/runtimedir/proot (auto-fetch -// cache, populated on first sandbox use) -// 3. PATH-resolved `proot` (system pkg) +// 1. /data/xpkgs/xim-x-proot//bin/proot the PAYLOAD +// 2. /runtimedir/proot auto-fetch cache +// 3. /usr/bin/proot, /usr/local/bin/proot the HOST (reported) +// +// PATH is deliberately absent -- see (3) below. The same rule, in the same +// shape, as libxpkg's elfpatch tool lookup: payload, then a named fallback +// that says so. Both implement R6 (internal consumers bind the payload, not +// the view), .agents/docs/2026-08-06-subos-architecture-proposal.md §1.5. // // Returns the path to a usable proot, or unexpected with a hint string. std::expected @@ -352,27 +393,50 @@ locate_proot_(const fs::path& home_dir) { auto runtime_proot = home_dir / "runtimedir" / "proot"; if (fs::is_regular_file(runtime_proot, ec)) return runtime_proot; - // (3) PATH-resolved - if (auto* path_env = std::getenv("PATH"); path_env && *path_env) { - std::string_view pv = path_env; - std::size_t start = 0; - while (start <= pv.size()) { - auto end = pv.find(':', start); - auto seg = pv.substr(start, end == std::string_view::npos - ? pv.size() - start : end - start); - if (!seg.empty()) { - auto candidate = fs::path(seg) / "proot"; - if (fs::is_regular_file(candidate, ec)) return candidate; - } - if (end == std::string_view::npos) break; - start = end + 1; + // (3) The host's proot, at the two paths a distribution puts it. + // + // There is no PATH step. An earlier version walked PATH and rejected the + // candidates that turned out to be xlings shims; that fixed the symptom + // and left the cause. By R6 the whole step is wrong for internal use: PATH + // is a *view*, it is what the user selected, and letting it decide which + // proot runs makes the sandbox's own identity depend on the environment + // the sandbox was launched from. A shim on PATH is the worst case -- it + // re-enters xlings, anchors to the home that owns it, and re-exports + // XLINGS_HOME to match, so an isolated home with no backend installed + // would silently run against the developer's real home, and every + // measurement taken inside would be of the wrong home while looking + // exactly like a measurement of the right one. + // + // These two paths are different in kind: they are the host's, they are + // named here rather than discovered, and using one is reported. Per §2.8 + // of the architecture proposal, depending on the host is allowed when it + // is DECLARED -- what is not allowed is depending on it by accident. + for (const auto* p : {"/usr/bin/proot", "/usr/local/bin/proot"}) { + auto candidate = fs::path(p); + if (!fs::is_regular_file(candidate, ec)) continue; + // Defensive: a home that installed itself under /usr/local would put + // a shim on one of these paths, and it is still a shim. + if (auto owner = xvm::resolve_owner_home(candidate)) { + log::debug("skipping {}: an xlings shim owned by {}, not the " + "host's proot", candidate.string(), owner->string()); + continue; } + log::warn("using the host's proot ({}) -- no proot payload in {}. " + "Run `xlings install proot` to make this deterministic.", + candidate.string(), home_dir.string()); + return candidate; } - return std::unexpected( - "proot not found. Install via your package manager " - "(e.g. `sudo apt install proot` / `sudo dnf install proot`) " - "or place a proot binary at ~/.xlings/runtimedir/proot"); + // Naming the home rather than "~/.xlings": with an isolated XLINGS_HOME + // the tilde form points somewhere the caller is not using, and a + // reader who follows it lands on the very home this search excluded. + return std::unexpected(std::format( + "proot not found in {}. Run `xlings install proot`, or place a proot " + "binary at {}/runtimedir/proot. The host's proot at /usr/bin/proot is " + "used when present (`sudo apt install proot`), but PATH is not " + "searched: a `proot` on PATH may be an xlings shim, and running it " + "would move the whole session to whichever home owns it.", + home_dir.string(), home_dir.string())); } // ── Unified bind list (shared by proot + bwrap) ────────────────────── @@ -439,8 +503,19 @@ sandbox_binds_(const fs::path& subos_dir, } // tmpfs: home and tmp handled by bwrap --tmpfs (no bind needed) - // xlings shared — always bound regardless of storage mode - binds.push_back({host_xlings_home.string(), user_home + "/.xlings", false}); + // xlings shared — always bound regardless of storage mode, and always + // at its OWN absolute path. + // + // xvm alias targets, RPATH and INTERP are absolute host paths baked at + // install time, and nothing rewrites them on the way in; the home has + // to answer to the same spelling in here that it does outside. Mapping + // it to /.xlings instead strands all of them, and mapping it + // to both makes one directory reachable by two real paths — which a + // bind mount does not collapse, so canonicalising does not save you and + // a shim ends up reporting a conflict with itself. For the default home + // this bind IS /.xlings; that coincidence is the only reason + // the remapped form ever appeared to work. + binds.push_back({host_xlings_home.string(), host_xlings_home.string(), false}); // ── Sandbox: NSS templates ── binds.push_back({(etc / "passwd").string(), "/etc/passwd", false}); @@ -803,30 +878,7 @@ export int enter(const std::string& name, EventStream& stream, if (storage == StorageMode::Shared) { auto user_home_dir = subos_dir / "home" / user; fs::create_directories(user_home_dir); - auto try_write = [](const fs::path& path, std::string_view body) { - if (fs::exists(path)) return; - platform::write_string_to_file(path.string(), std::string(body)); - }; - try_write(user_home_dir / ".bashrc", - "# xlings sandbox bashrc\n" - "if [ -r \"$HOME/.xlings/config/shell/xlings-profile.sh\" ]; then\n" - " . \"$HOME/.xlings/config/shell/xlings-profile.sh\"\n" - "fi\n"); - try_write(user_home_dir / ".profile", - "# xlings sandbox profile\n" - "if [ -r \"$HOME/.bashrc\" ]; then . \"$HOME/.bashrc\"; fi\n"); - auto fish_dir = user_home_dir / ".config" / "fish"; - fs::create_directories(fish_dir); - try_write(fish_dir / "config.fish", - "# xlings sandbox fish config\n" - "if test -r \"$HOME/.xlings/config/shell/xlings-profile.fish\"\n" - " source \"$HOME/.xlings/config/shell/xlings-profile.fish\"\n" - "end\n"); - try_write(user_home_dir / ".zshrc", - "# xlings sandbox zshrc\n" - "if [ -r \"$HOME/.xlings/config/shell/xlings-profile.sh\" ]; then\n" - " . \"$HOME/.xlings/config/shell/xlings-profile.sh\"\n" - "fi\n"); + write_sandbox_rc_(user_home_dir); } #endif @@ -890,30 +942,7 @@ export int enter(const std::string& name, EventStream& stream, if (storage == StorageMode::Image) { auto mp_home = image_mountpoint / user; fs::create_directories(mp_home); - auto try_write = [](const fs::path& path, std::string_view body) { - if (fs::exists(path)) return; - platform::write_string_to_file(path.string(), std::string(body)); - }; - try_write(mp_home / ".bashrc", - "# xlings sandbox bashrc\n" - "if [ -r \"$HOME/.xlings/config/shell/xlings-profile.sh\" ]; then\n" - " . \"$HOME/.xlings/config/shell/xlings-profile.sh\"\n" - "fi\n"); - try_write(mp_home / ".profile", - "# xlings sandbox profile\n" - "if [ -r \"$HOME/.bashrc\" ]; then . \"$HOME/.bashrc\"; fi\n"); - auto fish_dir = mp_home / ".config" / "fish"; - fs::create_directories(fish_dir); - try_write(fish_dir / "config.fish", - "# xlings sandbox fish config\n" - "if test -r \"$HOME/.xlings/config/shell/xlings-profile.fish\"\n" - " source \"$HOME/.xlings/config/shell/xlings-profile.fish\"\n" - "end\n"); - try_write(mp_home / ".zshrc", - "# xlings sandbox zshrc\n" - "if [ -r \"$HOME/.xlings/config/shell/xlings-profile.sh\" ]; then\n" - " . \"$HOME/.xlings/config/shell/xlings-profile.sh\"\n" - "fi\n"); + write_sandbox_rc_(mp_home); } } @@ -950,6 +979,7 @@ export int enter(const std::string& name, EventStream& stream, .code = ErrorCode::NotFound, .message = std::move(bin).error(), .recoverable = false, + .hint = "run: xlings install proot", }); return 1; } @@ -1041,9 +1071,18 @@ export int enter(const std::string& name, EventStream& stream, stream.emit(DataEvent{"subos_entering", payload.dump()}); platform::set_env_variable("HOME", user_home); + // Pin the home explicitly rather than relying on inheritance: it is + // reachable at its own absolute path in here (see sandbox_binds_), and + // every path baked at install time — xvm targets, RPATH, INTERP — + // assumes exactly that spelling. + platform::set_env_variable("XLINGS_HOME", p.homeDir.string()); + // Same spelling as XLINGS_HOME above. A shim decides which home owns it + // by comparing the path it was invoked through against XLINGS_HOME as + // strings; spelling the home two ways here makes it warn about a + // conflict with itself. platform::set_env_variable("PATH", std::format( - "{}/.xlings/subos/{}/bin:{}/.xlings/bin:/usr/local/bin:/usr/bin:/bin", - user_home, name, user_home)); + "{0}/subos/{1}/bin:{0}/bin:/usr/local/bin:/usr/bin:/bin", + p.homeDir.string(), name)); // bwrap `sh -i` prints prompts/job-control warnings into piped CI // commands; keep `-i` only for real terminal sessions. const bool interactive_shell = ::isatty(STDIN_FILENO) == 1; diff --git a/src/core/xim/installer.cppm b/src/core/xim/installer.cppm index 9dc8ca22..93ea3b76 100644 --- a/src/core/xim/installer.cppm +++ b/src/core/xim/installer.cppm @@ -1506,6 +1506,58 @@ bool apply_subos_env_ops_(const std::vector& operations, mf::DEFAULT_RUNTIME, std::format("xlings {}", Info::VERSION)); } + // C1 -- the subos layer's "exactly one", for the case where nothing else + // can supply it. + // + // The store keeps many versions by design; each consumer freezes one into + // its own RPATH; the subos in between holds exactly one. Nothing enforced + // that, so installing a second version appended a second provider section + // and BOTH contributed. Measured: mesa@25.0.7 and mesa@25.0.7.1 both bound + // in one subos, both on __EGL_VENDOR_LIBRARY_DIRS, EGL enumerating the + // device twice, doctor silent. The two records agreed -- on an answer the + // model forbids. + // + // The live version is normally decided by xvm and read back at activation + // (manifest::select_effective), so a package WITH an active version needs + // nothing here: installing a second version leaves the first live until + // `xlings use` says otherwise, which is what install and use have meant + // since 2026.7.31. Superseding here as well would make install a second + // selector -- another answerer to a question that has one, which is the + // defect, not the fix. + // + // What is left is the case where xvm has no answer at all. Measured: a bare + // `xvm.add(name)` DOES record an active version, so an ordinary install + // never reaches this branch -- it covers a manifest whose workspace record + // was lost while its declarations survived (a payload pruned out from under + // it, a subos config copied between homes, a hand edit). There every + // provider contributes, so a second version really does double the + // environment, and the only moment anything can choose is now, while a + // human is naming a version. So: supersede only when nothing else can. + // + // Before the ownership checks below, deliberately: those return false on a + // malformed declaration, and unbinding first would leave the subos holding + // neither version. + std::vector superseded; + { + const auto active = mf::active_versions(*doc); + std::set ownNames; + for (const auto* op : declarations) { + if (!mf::is_binding(op->binding)) continue; + ownNames.insert(std::string(mf::binding_name(op->binding))); + } + for (const auto& name : ownNames) { + if (active.contains(name)) continue; // xvm decides, not install + for (const auto& existing : mf::providers_named(*doc, name)) { + const bool sameBinding = std::ranges::any_of( + declarations, [&](const auto* op) { + return op->binding == existing; + }); + if (sameBinding) continue; // re-install of the same version + superseded.push_back(existing); + } + } + } + bool changed = false; for (const auto* op : declarations) { if (!mf::is_binding(op->binding)) { @@ -1521,9 +1573,44 @@ bool apply_subos_env_ops_(const std::vector& operations, canonical, node.version, op->var, op->binding); return false; } + // B5 / B3: a declaration that can put our code into someone else's + // process is a privileged operation, and install is the only moment a + // human is watching. Reported, not refused -- nvidia-gl-host-link + // legitimately needs one today, and refusing would leave the user with + // no GPU and no way forward. What must not happen is that it lands + // silently: the last one did, and `xlings subos use` returned a + // /bin/bash that died of SIGSEGV before printing a character. + // + // Default-deny by variable name (manifest::names_only_data), so a + // variable nobody has classified reads as privileged. See AD-3. + if (mf::is_privileged_env(op->var, op->value)) { + log::warn("[xim] {}@{} declares {} = {}", canonical, node.version, + op->var, op->value); + log::warn(" This variable can load code from our payload into " + "processes we do not own, including host binaries running " + "under the host loader. Prefer RPATH on the consumer; use " + "this only where RPATH cannot reach (a library that opens " + "its plugins itself), and say why in the recipe."); + } changed |= mf::add_env(*doc, op->binding, {.var = op->var, .op = op->mode, .value = op->value}); } + + // Only now that every declaration passed. Reported at info level, not + // debug: a version silently leaving a subos is exactly the kind of state + // change users later cannot account for. + for (const auto& binding : superseded) { + if (!mf::remove_provider(*doc, binding)) continue; + changed = true; + log::info("[xim] subos '{}' now holds {}@{}; unbound {} (it declares no " + "xvm version, so nothing else could say which is live)", + Config::paths().activeSubos.empty() + ? std::string{"default"} : Config::paths().activeSubos, + canonical, node.version, binding); + log::info(" Its payload is untouched -- the store keeps every " + "installed version. To keep both active, use separate subos."); + } + if (!changed) return true; if (auto findings = mf::validate_block(*doc); !findings.empty()) { diff --git a/src/core/xself/doctor.cppm b/src/core/xself/doctor.cppm index af1dc7e9..df19391b 100644 --- a/src/core/xself/doctor.cppm +++ b/src/core/xself/doctor.cppm @@ -159,6 +159,12 @@ enum class FindingKind { // message naming neither package nor version. Installs cannot produce it // any more; this finds the ones already on disk. LoaderLibcSplit, + // One package bound at two versions in the same subos. The store holds + // many versions by design; the subos in between is the layer that is + // supposed to hold exactly one, and until now nothing enforced it. Both + // versions contribute their env declarations, so a GL stack bound twice + // enumerates its device twice. + SubosDoubleBinding, }; enum class FindingLevel { @@ -498,7 +504,14 @@ std::vector detect_subos_manifest_(const xvm::VersionDB& db, } // D3/D4 — over the resolved set, so both see exactly what activation will. - const auto resolved = mf::resolve(info, mf::Placeholders{ + // + // select_effective first, for the same reason: a dormant provider (a second + // version whose sibling is the active one) contributes nothing, so + // reporting that it "would export an unexpanded path" describes an export + // that will not happen. D2 above deliberately stays over the FULL set -- + // "recorded here but not installed here" is true of a dormant section too. + const auto effective = mf::select_effective(info, mf::active_versions(*doc)); + const auto resolved = mf::resolve(effective, mf::Placeholders{ .subosdir = subosDir, .home = platform::get_home_dir(), .xlings_home = Config::paths().homeDir, @@ -563,6 +576,55 @@ std::vector detect_subos_manifest_(const xvm::VersionDB& db, } } + // D6 — a package bound at several versions with nothing able to say which. + // + // Not every duplicate: two versions where one is active is ordinary, and + // the dormant section is exactly what lets `xlings use pkg@` restore + // an environment without reinstalling. Activation already drops it + // (manifest::select_effective). + // + // What is reportable is the subset with NO active version -- `xvm.add(name)` + // with no version registers a root without one, which is what mesa, + // libglvnd and nvidia-gl-host-link do. For those every provider + // contributes, so a second version really does export every variable twice. + // That is the state that enumerated one GPU as two. + // + // The predicate is mf::contested_bindings, and --fix calls that same + // function rather than an equivalent one written here. Three report/repair + // pairs in this repo have drifted, each showing up as a finding that + // repairing does not clear. + const auto activeInSubos = mf::active_versions(*doc); + for (const auto& dup : mf::contested_bindings(info, activeInSubos)) { + std::string names; + for (const auto& b : dup.bindings) { + if (!names.empty()) names += ", "; + names += b; + } + out.push_back({ + .kind = FindingKind::SubosDoubleBinding, + // Error: the observable effect -- a device enumerated twice, a + // search path with two of everything -- is a wrong result, not a + // cosmetic one. + .level = FindingLevel::Error, + .target = subosName, + .version = dup.name, + .detail = std::format( + "'{}' is bound {} times in subos '{}' ({}) and has no active " + "version, so every one of them contributes", + dup.name, dup.bindings.size(), subosName, names), + // Naming the versions in the remedy: the user is otherwise left + // choosing between two strings with nothing to tell them apart. + .remedy = std::format("xlings use {}@", dup.name, [&] { + std::string vs; + for (const auto& b : dup.bindings) { + if (!vs.empty()) vs += ", "; + vs += std::string(mf::binding_version(b)); + } + return vs; + }()), + }); + } + // D5 — the declared runtime is not installed here. if (mf::is_binding(info.runtime) && !installed(info.runtime)) { out.push_back({ @@ -1526,6 +1588,13 @@ void repair_local_(const DoctorState& st, const Scan& scan, for (const auto& f : scan.findings) if (f.kind == FindingKind::SubosEnvOrphan) orphans.push_back(f.version); + // A contested binding is deliberately NOT repaired here. It is reported + // precisely because nothing can say which version was meant -- the + // package has no active version -- and picking the highest, or the + // newest on disk, would be this codebase's recurring defect rather than + // a repair: a second answerer invented at the read end. Its remedy is + // `xlings use`, which makes the choice a decision someone took. + if (wantsBlock || !orphans.empty()) { auto doc = mf::read_document(p.subosDir); nlohmann::json document = doc ? *doc : nlohmann::json::object(); @@ -1555,6 +1624,7 @@ void repair_local_(const DoctorState& st, const Scan& scan, note(glyph::mark(glyph::bullet, "subos env dropped"), std::format("{} is not installed here", binding)); } + if (changed) { try { platform::write_string_to_file( @@ -2147,6 +2217,16 @@ Counts count_(const Scan& scan) { // every script that wraps it. ++c.broken; break; + case FindingKind::SubosDoubleBinding: + // Counted for the same reason, and for a second one: `healed` + // is computed as before-minus-after over this count, so an + // uncounted finding that --fix repairs reports "healed 0" -- + // repair and report disagreeing about whether anything + // happened. Note that the other Subos* findings are NOT + // counted here today, which is a pre-existing instance of the + // same gap; see task #53. + ++c.broken; + break; } } return c; @@ -2347,6 +2427,13 @@ void render_(const Scan& scan, const RepairReport& repair, bool fix, case FindingKind::SubosEnvOrphan: add(glyph::mark(glyph::failed, "subos env orphan"), f.detail); break; + case FindingKind::SubosDoubleBinding: + // Not collapsed into a count: which versions are bound is the + // finding, and there is at most a handful. + add(glyph::mark(glyph::failed, "subos double binding"), f.detail); + if (!f.remedy.empty()) + add(" " + glyph::mark(glyph::remedy, "run"), f.remedy); + break; case FindingKind::SubosEnvUnresolved: add(glyph::mark(glyph::failed, "subos env unresolved"), f.detail); if (!f.remedy.empty()) diff --git a/tests/e2e/project_test_lib.sh b/tests/e2e/project_test_lib.sh index 44a67f6c..51317c97 100644 --- a/tests/e2e/project_test_lib.sh +++ b/tests/e2e/project_test_lib.sh @@ -65,9 +65,40 @@ require_fixture_index() { [[ -d "$FIXTURE_INDEX_DIR/pkgs" ]] || fail "fixture index repo not found at $FIXTURE_INDEX_DIR" } +# Where a test's isolated XLINGS_HOME lives. +# +# NOT under $ROOT_DIR by default. Four home-related defects in the 2026-08-06 +# review were asymptomatic under `~/.xlings`, and the repo checkout is usually +# under $HOME too -- so a test home there shares a long prefix with the real +# one, and every "did we use the right home?" bug stays invisible in exactly +# the same way. +# +# The 2026-08-06 measurements happened to use a path under /tmp, which put the +# home BELOW a directory the sandbox privatises before binding it. That was an +# accident, and it was the hardest ordering in the bind list. Making it +# deliberate is the point of E1. +# +# E2E_RUNTIME_ROOT overrides it -- CI runners with a small /tmp need that -- +# but the default has to be the awkward path, not the comfortable one. +E2E_RUNTIME_ROOT="${E2E_RUNTIME_ROOT:-${TMPDIR:-/tmp}/xlings-e2e-$(id -u)}" + runtime_home_dir() { local name="$1" - printf '%s\n' "$ROOT_DIR/tests/e2e/runtime/$name" + printf '%s\n' "$E2E_RUNTIME_ROOT/$name" +} + +# Assert the isolation the test relies on, rather than assume it. A home that +# shares a prefix with $HOME cannot distinguish "we used the home under test" +# from "we used the developer's" -- which is the entire class E1 exists for. +assert_home_is_isolated() { + local home_dir="$1" + local real="${HOME%/}/.xlings" + case "$home_dir" in + "$real"|"$real"/*) fail "the test home IS the real home: $home_dir" ;; + esac + [[ "$home_dir" == "${HOME%/}"/* ]] \ + && log " note: test home shares a prefix with \$HOME ($home_dir)" + return 0 } prepare_scenario() { diff --git a/tests/e2e/run_all.sh b/tests/e2e/run_all.sh index 1f707ea4..e5420c2a 100755 --- a/tests/e2e/run_all.sh +++ b/tests/e2e/run_all.sh @@ -112,6 +112,8 @@ TESTS=( "E2E-60 |subos_env_declaration_test.sh||" "E2E-61 |subos_env_probe_compat_test.sh||" "E2E-62 |loader_libc_same_source_test.sh||" + "E2E-63 |subos_env_libc_guard_test.sh||" + "E2E-64 |subos_single_version_test.sh||" ) PASS=0; FAIL=0; SOFTFAIL=0 diff --git a/tests/e2e/subos_env_libc_guard_test.sh b/tests/e2e/subos_env_libc_guard_test.sh new file mode 100755 index 00000000..2695655e --- /dev/null +++ b/tests/e2e/subos_env_libc_guard_test.sh @@ -0,0 +1,196 @@ +#!/usr/bin/env bash +# E2E: a package may not put a libc on a process-global search path. +# +# LD_LIBRARY_PATH is inherited by every child of the subos shell, and most of +# those children are HOST binaries running under the HOST loader. ld.so and +# libc.so.6 are two halves of one build that talk over GLIBC_PRIVATE; handing a +# host binary our half of the pair does not fail to load, it segfaults before +# main and names nothing. +# +# This is how it reached a release: nvidia-gl-host-link gathered its runtime +# dependencies -- including glibc's -- into one directory and declared that +# directory on LD_LIBRARY_PATH, so that the NVIDIA vendor library, which is the +# host's file and so cannot carry an RPATH of ours, could find them. The libc +# in there was never usable for that purpose (the vendor is dlopen'd into a +# process whose libc is long since bound, and an already-loaded SONAME is not +# searched for), but it was inherited by /bin/bash. `xlings subos use` returned +# a shell that died of SIGSEGV before printing a character -- on a host whose +# glibc was the same upstream VERSION as ours, merely a different build. +# +# The recipe was fixed. This test covers the class, not that recipe: xlings +# drops such an entry itself, names it, and keeps the rest of the variable. +# +# What has to hold: +# 1. a declared directory holding libc.so.6 does not reach the environment +# 2. the other directories on the same variable survive -- they are usually +# the point of the declaration +# 3. the drop is reported with the directory AND the package that declared +# it, because a silent drop is indistinguishable from a recipe that never +# declared anything +# 4. a directory with no libc in it is untouched (the guard is not a ban on +# LD_LIBRARY_PATH), and neither is one holding only glibc's 2.34+ +# compatibility stubs, which packages legitimately have to offer + +set -uo pipefail + +# shellcheck source=./project_test_lib.sh +source "$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/project_test_lib.sh" + +require_fixture_index + +RUNTIME_DIR="$(runtime_home_dir subos_env_libc_guard)" +LOCAL_INDEX_DIR="$RUNTIME_DIR/xim-pkgindex" +HOME_DIR="$RUNTIME_DIR/home" + +cleanup() { rm -rf "$RUNTIME_DIR"; } +trap cleanup EXIT +cleanup +mkdir -p "$RUNTIME_DIR" + +assert_home_is_isolated "$HOME_DIR" +BIN="$(find_xlings_bin)" +log "client: $("$BIN" --version 2>&1 | head -1)" + +cp -r "$FIXTURE_INDEX_DIR" "$LOCAL_INDEX_DIR" +printf 'xim_indexrepos = {}\n' > "$LOCAL_INDEX_DIR/xim-indexrepos.lua" +rm -f "$LOCAL_INDEX_DIR/.xlings-index-cache.json" +mkdir -p "$LOCAL_INDEX_DIR/pkgs/l" + +# The fixture declares one variable naming two directories: one that holds a +# libc, one that does not. Both halves of the contract are then observable in +# a single value. +cat > "$LOCAL_INDEX_DIR/pkgs/l/libcguardfixture.lua" <<'LUA' +package = { + spec = "1", + name = "libcguardfixture", + description = "Local fixture for tests/e2e/subos_env_libc_guard_test.sh", + authors = {"xlings-ci"}, + licenses = {"MIT"}, + type = "package", + archs = {"x86_64"}, + status = "stable", + categories = {"test-fixture"}, + xpm = { + linux = { ["1.0.0"] = {} }, + macosx = { ["1.0.0"] = {} }, + windows = { ["1.0.0"] = {} }, + }, +} + +import("xim.libxpkg.pkginfo") +import("xim.libxpkg.subos") + +function install() + local dir = pkginfo.install_dir() + os.tryrm(dir) + -- Contents are never loaded; the guard reads the directory listing, so a + -- file of the right NAME is the whole fixture. Building a real glibc to + -- test a filename check would test the build, not the check. + os.mkdir(path.join(dir, "poisoned")) + io.writefile(path.join(dir, "poisoned", "libc.so.6"), "not a real libc\n") + io.writefile(path.join(dir, "poisoned", "libfixture.so.1"), "\n") + os.mkdir(path.join(dir, "clean")) + io.writefile(path.join(dir, "clean", "libfixture.so.1"), "\n") + -- The glibc 2.34+ compatibility stubs. A package may legitimately have to + -- offer these -- nvidia-gl-host-link does, and without them the NVIDIA + -- device vanishes from EGL enumeration -- and they are nearly empty, + -- their implementations having moved into libc.so.6. A guard that swept + -- up "everything glibc ships" would break that while preventing nothing. + os.mkdir(path.join(dir, "stubs")) + for _, n in ipairs({"libpthread.so.0", "librt.so.1", "libdl.so.2", + "libm.so.6"}) do + io.writefile(path.join(dir, "stubs", n), "\n") + end + return true +end + +function config() + if type(subos.env) == "function" then + local binding = package.name .. "@" .. pkginfo.version() + subos.env{ var = "LD_LIBRARY_PATH", op = "prepend", + value = "${pkgdir}/poisoned:${pkgdir}/clean:${pkgdir}/stubs", + binding = binding } + -- Same directory, a variable the loader does not read. Nothing may be + -- dropped here: the hazard is the variable, not the directory. + subos.env{ var = "E2E_GUARD_CONTROL", op = "set", + value = "${pkgdir}/poisoned", binding = binding } + end + return true +end + +function uninstall() return true end +LUA + +mkdir -p "$HOME_DIR/subos/default/bin" "$HOME_DIR/data/xim-index-repos" +cat > "$HOME_DIR/.xlings.json" < "$HOME_DIR/data/xim-index-repos/xim-indexrepos.json" + +x() { ( cd /tmp && env -i HOME="$HOME" PATH=/usr/bin:/bin \ + XLINGS_HOME="$HOME_DIR" "$BIN" "$@" ) } + +x self init >/dev/null 2>&1 || true + +OUT="$(x install libcguardfixture@1.0.0 -y 2>&1)" \ + || { echo "$OUT" >&2; fail "install failed"; } + +PKGDIR="$HOME_DIR/data/xpkgs/xim-x-libcguardfixture/1.0.0" +[[ -f "$PKGDIR/poisoned/libc.so.6" ]] \ + || fail "fixture did not install: no $PKGDIR/poisoned/libc.so.6" + +# The declaration itself must be recorded unchanged. The guard belongs at the +# point the environment is built, not at the point it is declared: a manifest +# that silently differs from what the recipe said is a second source of truth. +MANIFEST="$HOME_DIR/subos/default/.xlings.json" +grep -q "poisoned" "$MANIFEST" \ + || fail "the manifest did not record the declaration as written: +$(cat "$MANIFEST")" +log " ✓ declaration recorded verbatim in the subos manifest" + +RUN="$(x subos use default --cmd 'echo "LDLP=[$LD_LIBRARY_PATH]"; echo "CTRL=[$E2E_GUARD_CONTROL]"' 2>&1)" + +# 1. the libc directory is gone +echo "$RUN" | grep -q "LDLP=.*$PKGDIR/poisoned" \ + && fail "a directory holding libc.so.6 reached LD_LIBRARY_PATH: +$RUN" +log " ✓ the directory holding libc.so.6 was dropped" + +# 2. the rest of the variable survives +echo "$RUN" | grep -q "LDLP=.*$PKGDIR/clean" \ + || fail "the guard took the whole variable instead of the offending entry: +$RUN" +log " ✓ the other directory on the same variable survived" + +# 3. the drop is reported, with directory and provider +echo "$RUN" | grep -q "dropped $PKGDIR/poisoned" \ + || fail "the drop was silent -- indistinguishable from a recipe that never +declared anything: +$RUN" +echo "$RUN" | grep -q "libcguardfixture@1.0.0" \ + || fail "the report does not name the package that declared it: +$RUN" +log " ✓ reported, naming both the directory and the declaring package" + +# 4. the glibc stubs are not swept up with the libc +echo "$RUN" | grep -q "LDLP=.*$PKGDIR/stubs" \ + || fail "a directory of glibc compatibility stubs was dropped. They are not +libc: their implementations moved into libc.so.6 and what is left is a handful +of symbols. Dropping them breaks packages that must offer them -- the NVIDIA +vendor library names libpthread, librt and libdl, and loses its device without +them -- while preventing no crash: +$RUN" +log " ✓ glibc compatibility stubs (libpthread/librt/libdl/libm) survived" + +# 5. a variable the loader does not read is untouched +echo "$RUN" | grep -q "CTRL=\[$PKGDIR/poisoned\]" \ + || fail "the guard reached a variable the dynamic loader never reads: +$RUN" +log " ✓ non-loader variables naming the same directory are untouched" + +log "PASS: subos env libc guard" diff --git a/tests/e2e/subos_sandbox_test.sh b/tests/e2e/subos_sandbox_test.sh index 5064ab49..b1e4e6ad 100755 --- a/tests/e2e/subos_sandbox_test.sh +++ b/tests/e2e/subos_sandbox_test.sh @@ -29,13 +29,21 @@ set -euo pipefail source "$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/project_test_lib.sh" -RUNTIME_DIR="$ROOT_DIR/tests/e2e/runtime/subos_sandbox" +# Under $TMPDIR, not the checkout. E1: the checkout is normally under $HOME, +# so a home there shares a long prefix with the real one and every "which home +# did we actually use?" defect stays invisible -- which is how all four of the +# 2026-08-06 home defects survived. It also puts the home BELOW a directory the +# sandbox privatises before binding, which is the hardest ordering in the bind +# list and the one S8 exists to pin down. The 2026-08-06 measurements hit that +# ordering by accident; here it is deliberate. +RUNTIME_DIR="$(runtime_home_dir subos_sandbox)" HOME_DIR="$RUNTIME_DIR/home" cleanup() { rm -rf "$RUNTIME_DIR"; } trap cleanup EXIT cleanup +assert_home_is_isolated "$HOME_DIR" XLINGS_BIN="$(find_xlings_bin)" mkdir -p "$HOME_DIR/subos/default/bin" "$HOME_DIR/runtimedir" @@ -188,16 +196,45 @@ sandbox_marker="$HOME_DIR/subos/mybox/home/$USER${marker_file#$HOME}" || fail "S7: sandbox marker not found at $sandbox_marker" log " ✓ host \$HOME unaffected; file landed in /home/$USER/" -# ── S8: ~/.xlings IS host-shared (RW bind override on top of /home) -log "S8: ~/.xlings is the host xlings home, RW shared" -out_xl="$(echo 'ls ~/.xlings/ 2>&1 | tr "\n" " "; echo MARKER; exit' | \ +# ── S8: the xlings home is host-shared AT ITS OWN ABSOLUTE PATH +# +# Not at ~/.xlings. xvm alias targets, RPATH and INTERP are absolute host +# paths baked at install time and nothing rewrites them on the way in, so +# the home has to answer to the same spelling inside as outside. Binding it +# at ~/.xlings instead stranded every one of them; binding it at both made +# one directory reachable by two real paths, which a bind mount does not +# collapse — a shim then reports a conflict with itself. For the default +# home the own-path bind IS ~/.xlings, which is why the remapped form went +# unnoticed until an isolated XLINGS_HOME was used. +log "S8: xlings home is shared at its own absolute path, one spelling" +out_xl="$(echo 'echo "XH=$XLINGS_HOME"; echo "P1=${PATH%%:*}"; ls "$XLINGS_HOME/" 2>&1 | tr "\n" " "; echo MARKER; exit' | \ ( cd /tmp && env -i HOME="$HOME" USER="$USER" SHELL=/bin/sh \ PATH=/usr/bin:/bin XLINGS_HOME="$HOME_DIR" \ timeout 10 "$XLINGS_BIN" subos use mybox --sandbox ) 2>&1 || true)" +abs_home="$(cd "$HOME_DIR" && pwd)" +echo "$out_xl" | grep -q "XH=$abs_home" \ + || fail "S8: XLINGS_HOME inside is not the host home path (expected $abs_home): +$out_xl" +echo "$out_xl" | grep -q "P1=$abs_home/subos/" \ + || fail "S8: PATH[0] is spelled differently from XLINGS_HOME — a shim will +read that as two homes and warn about a conflict with itself: +$out_xl" echo "$out_xl" | grep -q "subos" \ - || fail "S8: ~/.xlings doesn't show host content (expected to see 'subos'): + || fail "S8: the home shows no host content at its own path: $out_xl" -log " ✓ ~/.xlings inside sandbox is host bind (subos/, etc. visible)" +log " ✓ home visible at $abs_home, and PATH agrees with XLINGS_HOME" + +# The second spelling must NOT exist: it is a distinct real path to the same +# directory, and that is what made shim ownership ambiguous. +out_alias="$(echo 'test -d "$HOME/.xlings" && echo SECOND_SPELLING || echo SINGLE_SPELLING; exit' | \ + ( cd /tmp && env -i HOME="$HOME" USER="$USER" SHELL=/bin/sh \ + PATH=/usr/bin:/bin XLINGS_HOME="$HOME_DIR" \ + timeout 10 "$XLINGS_BIN" subos use mybox --sandbox ) 2>&1 || true)" +echo "$out_alias" | grep -q "SINGLE_SPELLING" \ + || fail "S8: the home is also reachable at \$HOME/.xlings — two real paths +to one directory, which canonicalisation does not collapse: +$out_alias" +log " ✓ no second spelling of the home inside the sandbox" # ── S9: /tmp is sandbox-private log "S9: /tmp is sandbox-private (writes don't pollute host /tmp)" @@ -254,15 +291,29 @@ log " ✓ /subos/ marker exists" # xlings profile; non-interactive shells (`bash -c`, scripts) skip the # rc files. Both paths should still see the per-subos bin first because # we set PATH explicitly in env before exec proot. +# +# The expected value is derived from the home UNDER TEST, not written out as +# `/home//.xlings/...`. That literal was this assertion until CI caught +# it: S8's own fix made the home visible at its own absolute path, and S12 was +# left asserting the spelling from before that fix -- so it demanded +# `$HOME/.xlings` from a run whose home is somewhere else entirely. The value +# it rejected was the correct one. +# +# This is E2 in one assertion: a test can not only miss a defect, it can pin +# one, and the pin outlives the fix because changing a test assertion looks +# suspicious in review. Deriving the expectation from $HOME_DIR makes the +# assertion true of any home rather than of the default one. log "S12: PATH front-loads /bin even in non-interactive shell" out_path="$(echo 'echo "PATH:$PATH"; exit' | \ ( cd /tmp && env -i HOME="$HOME" USER="$USER" SHELL=/bin/sh \ PATH=/usr/bin:/bin XLINGS_HOME="$HOME_DIR" \ timeout 10 "$XLINGS_BIN" subos use mybox --sandbox ) 2>&1 || true)" -echo "$out_path" | grep -q "PATH:/home/$USER/.xlings/subos/mybox/bin:" \ - || fail "S12: PATH first segment is NOT /.xlings/subos/mybox/bin: +expect_bin="$(cd "$HOME_DIR" && pwd)/subos/mybox/bin" +echo "$out_path" | grep -q "PATH:$expect_bin:" \ + || fail "S12: PATH's first segment is not this home's subos bin + expected first: $expect_bin $out_path" -log " ✓ PATH starts with /home/$USER/.xlings/subos/mybox/bin" +log " ✓ PATH starts with $expect_bin" # ── S13: seeded shell rc files exist (interactive prompt-pill plumbing) # init_sandbox_dirs_ writes minimal .bashrc / .profile / config.fish diff --git a/tests/e2e/subos_single_version_test.sh b/tests/e2e/subos_single_version_test.sh new file mode 100755 index 00000000..d35800f8 --- /dev/null +++ b/tests/e2e/subos_single_version_test.sh @@ -0,0 +1,234 @@ +#!/usr/bin/env bash +# E2E: one package contributes one version's environment to a subos. +# +# The three-layer model has always said so: the xpkg store holds many versions +# by design, each consumer freezes one into its own RPATH/INTERP, and the subos +# in between is live at exactly one. Nothing enforced the middle line, so +# installing a second version appended a second provider section to the subos +# manifest and BOTH contributed. +# +# Measured on a real home before this test existed: mesa@25.0.7 and +# mesa@25.0.7.1 both bound in `default`, both on __EGL_VENDOR_LIBRARY_DIRS, EGL +# duly enumerating the device twice, and `xlings self doctor` reporting +# nothing. The two records agreed -- on an answer the model forbids. That is +# why "they agree" is never on its own evidence of anything here. +# +# The fix is NOT that a second install unbinds the first. `install` adds to the +# store and `use` selects; making install a second selector would be another +# answerer to a question that already has one. The fix is that activation reads +# xvm's answer, which lives in the same file as the declarations. +# +# What has to hold: +# 1. installing a second version KEEPS both provider sections -- the dormant +# one is what lets `xlings use` switch back without reinstalling +# 2. but only the active version's declarations reach the environment +# 3. `xlings use` on the other version switches which one, with no reinstall +# and no manifest rewrite +# 4. when a package has NO active version, nothing can choose, so a second +# install supersedes at the one moment a human is naming a version +# 5. and a contested state already on disk is REPORTED, with a remedy that +# makes the choice a decision someone took rather than a guess + +set -uo pipefail + +# shellcheck source=./project_test_lib.sh +source "$(cd "$(dirname "${BASH_SOURCE[0]}")" && pwd)/project_test_lib.sh" + +require_fixture_index + +RUNTIME_DIR="$(runtime_home_dir subos_single_version)" +LOCAL_INDEX_DIR="$RUNTIME_DIR/xim-pkgindex" +HOME_DIR="$RUNTIME_DIR/home" + +cleanup() { [[ -n "${E2E_KEEP:-}" ]] || rm -rf "$RUNTIME_DIR"; } +trap cleanup EXIT +cleanup +mkdir -p "$RUNTIME_DIR" + +assert_home_is_isolated "$HOME_DIR" +BIN="$(find_xlings_bin)" +log "client: $("$BIN" --version 2>&1 | head -1)" + +cp -r "$FIXTURE_INDEX_DIR" "$LOCAL_INDEX_DIR" +printf 'xim_indexrepos = {}\n' > "$LOCAL_INDEX_DIR/xim-indexrepos.lua" +rm -f "$LOCAL_INDEX_DIR/.xlings-index-cache.json" +mkdir -p "$LOCAL_INDEX_DIR/pkgs/o" + +# Two versions of one package, each declaring a variable that names its own +# payload. Both must be installable for the test to mean anything: the point is +# not that the second install fails. +# +# xvm-registered WITH a version, the way glibc is. doctor asks xvm whether a +# package is installed here, and activation asks xvm which version is live. +cat > "$LOCAL_INDEX_DIR/pkgs/o/onlyonefixture.lua" <<'LUA' +package = { + spec = "1", + name = "onlyonefixture", + description = "Local fixture for tests/e2e/subos_single_version_test.sh", + authors = {"xlings-ci"}, + licenses = {"MIT"}, + type = "package", + archs = {"x86_64"}, + status = "stable", + categories = {"test-fixture"}, + xpm = { + linux = { ["1.0.0"] = {}, ["2.0.0"] = {} }, + macosx = { ["1.0.0"] = {}, ["2.0.0"] = {} }, + windows = { ["1.0.0"] = {}, ["2.0.0"] = {} }, + }, +} + +import("xim.libxpkg.pkginfo") +import("xim.libxpkg.subos") +import("xim.libxpkg.xvm") + +function install() + local dir = pkginfo.install_dir() + os.tryrm(dir) + os.mkdir(path.join(dir, "share")) + io.writefile(path.join(dir, "share", "version.txt"), pkginfo.version()) + return true +end + +function config() + -- No `binding` field: this IS the package's own node, and a node that + -- binds to itself is rejected (xvm-self-binding). + xvm.add(package.name, { + version = pkginfo.version(), + type = "lib", + bindir = path.join(pkginfo.install_dir(), "share"), + filename = "version.txt", + alias = "version.txt", + }) + if type(subos.env) == "function" then + subos.env{ var = "E2E_ONLYONE_DIRS", op = "prepend", + value = "${pkgdir}/share", + binding = package.name .. "@" .. pkginfo.version() } + end + return true +end + +function uninstall() + xvm.remove(package.name, pkginfo.version()) + return true +end +LUA + +mkdir -p "$HOME_DIR/subos/default/bin" "$HOME_DIR/data/xim-index-repos" +cat > "$HOME_DIR/.xlings.json" < "$HOME_DIR/data/xim-index-repos/xim-indexrepos.json" + +x() { ( cd /tmp && env -i HOME="$HOME" PATH=/usr/bin:/bin \ + XLINGS_HOME="$HOME_DIR" "$BIN" "$@" ) } + +x self init >/dev/null 2>&1 || true + +MANIFEST="$HOME_DIR/subos/default/.xlings.json" + +providers() { + python3 -c " +import json +d=json.load(open('$MANIFEST')) +print(' '.join(sorted(d.get('subos_info',{}).get('envs',{})))) +" 2>/dev/null +} + +# What the variable would actually be, as the subos exports it. +exported() { + x subos use default --cmd "echo \"VAL=[\$$1]\"" 2>&1 \ + | strip_ansi | sed -n 's/.*VAL=\[\(.*\)\].*/\1/p' | tail -1 +} + +# ── 1/2. a second version is recorded but dormant ───────────────────── +OUT="$(x install onlyonefixture@1.0.0 -y 2>&1)" \ + || { echo "$OUT" >&2; fail "install of 1.0.0 failed"; } +[[ "$(providers)" == "onlyonefixture@1.0.0" ]] \ + || fail "after installing 1.0.0 the subos should record it, records: $(providers)" + +OUT="$(x install onlyonefixture@2.0.0 -y 2>&1)" \ + || { echo "$OUT" >&2; fail "install of 2.0.0 failed"; } + +GOT="$(providers)" +[[ "$GOT" == "onlyonefixture@1.0.0 onlyonefixture@2.0.0" ]] \ + || fail "both provider sections must survive an install of a second version -- +the dormant one is what lets \`xlings use\` switch back without reinstalling. +records: '$GOT'" +log " ✓ both versions recorded" + +VAL="$(exported E2E_ONLYONE_DIRS)" +COUNT="$(awk -F: '{print NF}' <<< "$VAL")" +[[ "$COUNT" == "1" ]] \ + || fail "the variable was exported $COUNT times over: '$VAL'. +Two versions recorded means two provider sections; only the ACTIVE one may +contribute. Exporting both is how one GPU came to be enumerated as two." +[[ "$VAL" == *"/xim-x-onlyonefixture/1.0.0/"* ]] \ + || fail "the exported value is not the active version's payload: '$VAL' +(xvm has 1.0.0 active -- install adds to the store, use selects)" +log " ✓ only the active version contributes" + +# ── 3. `use` switches which one, with no reinstall ───────────────────── +OUT="$(x use onlyonefixture@2.0.0 2>&1)" \ + || { echo "$OUT" >&2; fail "use onlyonefixture@2.0.0 failed"; } + +VAL="$(exported E2E_ONLYONE_DIRS)" +[[ "$VAL" == *"/xim-x-onlyonefixture/2.0.0/"* ]] \ + || fail "\`xlings use\` did not change which version's environment is live: +'$VAL'" +COUNT="$(awk -F: '{print NF}' <<< "$VAL")" +[[ "$COUNT" == "1" ]] || fail "still exported $COUNT times over: '$VAL'" +log " ✓ use switched the live version, no reinstall, no manifest rewrite" + +[[ "$(providers)" == "onlyonefixture@1.0.0 onlyonefixture@2.0.0" ]] \ + || fail "switching rewrote the manifest; it should not have to -- the record +of what each version declares is not the record of which one is live" +log " ✓ the manifest was not rewritten" + +# ── 4/5. a contested state, the way a damaged home has it ───────────── +# +# NOT produced by installing: measured, a bare `xvm.add(name)` records an active +# version, so an ordinary install cannot reach this. It takes a manifest whose +# workspace record was lost while its declarations survived -- a payload pruned +# out from under it, a subos config copied between homes, a hand edit. There +# every provider contributes and nothing in the home can say which was meant, +# which is the state that exports every variable twice over. +python3 - "$MANIFEST" <<'PY' +import json, sys +p = sys.argv[1] +d = json.load(open(p)) +for v in ("1.0.0", "2.0.0"): + d["subos_info"]["envs"]["lostws@" + v] = [ + {"var": "E2E_LOSTWS_DIRS", "op": "prepend", "value": "${pkgdir}/share"} + ] +json.dump(d, open(p, "w"), indent=2) +PY + +OUT="$(x self doctor 2>&1 || true)" +echo "$OUT" | strip_ansi | grep -q "double binding" \ + || fail "doctor did not report a package bound at two versions with no active +version, which is the state that exports every variable twice: +$OUT" +echo "$OUT" | strip_ansi | grep -q "xlings use lostws@" \ + || fail "the remedy does not tell the user how to decide. Nothing here CAN +decide -- that is why it is reported rather than repaired -- so the remedy has +to name the choice: +$OUT" +log " ✓ contested binding reported, with a remedy that is a decision" + +# The other package must NOT be reported: two versions with one active is +# ordinary, and the dormant section is the feature, not the defect. +if echo "$OUT" | strip_ansi | grep "double binding" | grep -q "onlyonefixture"; then + fail "doctor reported a package that has an active version. Two versions +where one is active is how \`xlings use\` switches back without a reinstall; +reporting it would train users to delete the thing that makes that work: +$OUT" +fi +log " ✓ a package with an active version is not reported" + +log "PASS: subos single live version per package" diff --git a/tests/unit/test_subos_manifest.cpp b/tests/unit/test_subos_manifest.cpp index 7c298764..f294820e 100644 --- a/tests/unit/test_subos_manifest.cpp +++ b/tests/unit/test_subos_manifest.cpp @@ -343,3 +343,205 @@ TEST(SubosManifestBlock, NewBlockSatisfiesItsOwnInvariants) { EXPECT_TRUE(info.envs.empty()); EXPECT_FALSE(info.created_at.empty()); } + +// ── the subos layer's one live version ─────────────────────────────── +// +// The store holds many versions by design, each consumer freezes one into its +// own RPATH, and the subos in between is live at exactly one. Nothing enforced +// that middle line: installing a second version appended a second provider +// section and BOTH contributed. Measured on a real home as mesa@25.0.7 and +// mesa@25.0.7.1 both on __EGL_VENDOR_LIBRARY_DIRS, EGL enumerating the device +// twice, and doctor silent -- two records agreeing on an answer the model +// forbids. +// +// The fix is not that a second install unbinds the first. `install` adds to +// the store and `use` selects; making install a second selector would be one +// more answerer to a question that already has one. It is that activation +// reads xvm's answer, which is recorded in the same file as the declarations. + +TEST(SubosActiveVersions, ReadsTheWorkspaceFromTheSameDocument) { + auto d = doc_with(nlohmann::json::object()); + d["workspace"]["mesa"] = {{"active", "25.0.7.1"}, + {"installed", {"25.0.7", "25.0.7.1"}}}; + d["workspace"]["nothing"] = {{"installed", {"1.0"}}}; // no active key + + auto active = m::active_versions(d); + EXPECT_EQ(active.size(), 1u); + EXPECT_EQ(active.at("mesa"), "25.0.7.1"); + EXPECT_FALSE(active.contains("nothing")); +} + +TEST(SubosSelectEffective, OnlyTheActiveVersionContributes) { + auto d = doc_with(nlohmann::json::object()); + m::add_env(d, "mesa@25.0.7", {"V", "prepend", "old"}); + m::add_env(d, "mesa@25.0.7.1", {"V", "prepend", "new"}); + d["workspace"]["mesa"] = {{"active", "25.0.7.1"}}; + + auto eff = m::select_effective(m::parse(d), m::active_versions(d)); + ASSERT_EQ(eff.envs.size(), 1u); + EXPECT_EQ(eff.envs[0].binding, "mesa@25.0.7.1"); +} + +// The dormant section is the feature, not the defect: it is what lets +// `xlings use pkg@` restore an environment without a reinstall. +TEST(SubosSelectEffective, TheDormantSectionSurvivesInTheRecord) { + auto d = doc_with(nlohmann::json::object()); + m::add_env(d, "mesa@25.0.7", {"V", "prepend", "old"}); + m::add_env(d, "mesa@25.0.7.1", {"V", "prepend", "new"}); + d["workspace"]["mesa"] = {{"active", "25.0.7.1"}}; + + EXPECT_EQ(m::parse(d).envs.size(), 2u); // the record keeps both + + d["workspace"]["mesa"] = {{"active", "25.0.7"}}; + auto eff = m::select_effective(m::parse(d), m::active_versions(d)); + ASSERT_EQ(eff.envs.size(), 1u); + EXPECT_EQ(eff.envs[0].binding, "mesa@25.0.7"); +} + +// A namespaced install records `:` as the active key. +TEST(SubosSelectEffective, MatchesTheVersionTailOfANamespacedActiveKey) { + auto d = doc_with(nlohmann::json::object()); + m::add_env(d, "mesa@25.0.7", {"V", "prepend", "old"}); + m::add_env(d, "mesa@25.0.7.1", {"V", "prepend", "new"}); + d["workspace"]["mesa"] = {{"active", "local:25.0.7"}}; + + auto eff = m::select_effective(m::parse(d), m::active_versions(d)); + ASSERT_EQ(eff.envs.size(), 1u); + EXPECT_EQ(eff.envs[0].binding, "mesa@25.0.7"); +} + +// The direction this must fail in. Filtering on a record that turns out to be +// absent would silently delete a package's whole environment -- the same +// failure this file exists to prevent, arrived at from the other side. +TEST(SubosSelectEffective, NoActiveRecordKeepsEverything) { + auto d = doc_with(nlohmann::json::object()); + m::add_env(d, "mesa@25.0.7", {"V", "prepend", "old"}); + m::add_env(d, "mesa@25.0.7.1", {"V", "prepend", "new"}); + // no workspace entry for mesa at all + + auto eff = m::select_effective(m::parse(d), m::active_versions(d)); + EXPECT_EQ(eff.envs.size(), 2u); +} + +TEST(SubosDuplicateBindings, OneVersionPerPackageIsNotADuplicate) { + auto d = doc_with(nlohmann::json::object()); + m::add_env(d, "mesa@25.0.7", {"V", "prepend", "a"}); + m::add_env(d, "fontconfig@2.15.0", {"W", "prepend", "b"}); + + EXPECT_TRUE(m::duplicate_bindings(m::parse(d)).empty()); +} + +TEST(SubosDuplicateBindings, TwoVersionsOfOnePackageAreFound) { + auto d = doc_with(nlohmann::json::object()); + m::add_env(d, "mesa@25.0.7", {"V", "prepend", "a"}); + m::add_env(d, "mesa@25.0.7.1", {"V", "prepend", "b"}); + m::add_env(d, "fontconfig@2.15.0", {"W", "prepend", "c"}); + + auto dups = m::duplicate_bindings(m::parse(d)); + ASSERT_EQ(dups.size(), 1u); + EXPECT_EQ(dups[0].name, "mesa"); + ASSERT_EQ(dups[0].bindings.size(), 2u); + EXPECT_EQ(dups[0].bindings[0], "mesa@25.0.7"); + EXPECT_EQ(dups[0].bindings[1], "mesa@25.0.7.1"); +} + +// A package whose name is a prefix of another's must not merge with it. +// "mesa" and "mesa-utils" are two packages; a substring match would report a +// duplicate that does not exist, and a repair would unbind a live package. +TEST(SubosDuplicateBindings, NamesAreComparedWhole) { + auto d = doc_with(nlohmann::json::object()); + m::add_env(d, "mesa@25.0.7", {"V", "prepend", "a"}); + m::add_env(d, "mesa-utils@9.0.0", {"W", "prepend", "b"}); + + EXPECT_TRUE(m::duplicate_bindings(m::parse(d)).empty()); +} + +// What doctor reports is the SUBSET with no active version -- the state where +// every provider contributes and nothing in the home can say which was meant. +// Reporting every duplicate would train users to delete the dormant sections +// that make `xlings use` work. +TEST(SubosContestedBindings, ADuplicateWithAnActiveVersionIsNotContested) { + auto d = doc_with(nlohmann::json::object()); + m::add_env(d, "mesa@25.0.7", {"V", "prepend", "a"}); + m::add_env(d, "mesa@25.0.7.1", {"V", "prepend", "b"}); + d["workspace"]["mesa"] = {{"active", "25.0.7.1"}}; + + EXPECT_TRUE(m::contested_bindings(m::parse(d), m::active_versions(d)).empty()); +} + +TEST(SubosContestedBindings, ADuplicateWithNoActiveVersionIsContested) { + auto d = doc_with(nlohmann::json::object()); + m::add_env(d, "mesa@25.0.7", {"V", "prepend", "a"}); + m::add_env(d, "mesa@25.0.7.1", {"V", "prepend", "b"}); + + auto contested = m::contested_bindings(m::parse(d), m::active_versions(d)); + ASSERT_EQ(contested.size(), 1u); + EXPECT_EQ(contested[0].name, "mesa"); +} + +// ── privileged declarations (B5) ───────────────────────────────────── +// +// Default-deny by variable NAME. Listing the dangerous variables instead would +// be a hand-written list of what we happened to think of -- the anti-pattern +// R7 names, and the one that already cost five missing entries in +// nvidia-gl-host-link's dependency table. + +TEST(SubosPrivilegedEnv, TheLoaderVariablesArePrivileged) { + EXPECT_TRUE(m::is_privileged_env("LD_LIBRARY_PATH", "${pkgdir}/lib")); + EXPECT_TRUE(m::is_privileged_env("LD_PRELOAD", "${pkgdir}/lib/libx.so")); +} + +// These bypass the dynamic loader entirely -- libglvnd and mesa read them +// themselves -- so no RPATH mechanism can reach them and the loader-only guard +// never saw them. Measured: a HOST binary linked against the host's libEGL +// drops from the NVIDIA GPU to llvmpipe under these declarations, loading our +// libm and libstdc++ into a process running on the host's libc. +TEST(SubosPrivilegedEnv, VariablesLibrariesReadThemselvesAreAlsoPrivileged) { + EXPECT_TRUE(m::is_privileged_env("__EGL_VENDOR_LIBRARY_DIRS", + "${pkgdir}/share/glvnd/egl_vendor.d")); + EXPECT_TRUE(m::is_privileged_env("LIBGL_DRIVERS_PATH", "${pkgdir}/lib/dri")); +} + +// The point of default-deny: a variable nobody has classified reads as +// privileged. If this test ever needs changing, someone has decided a new +// variable is safe -- which is a decision, and belongs in names_only_data. +TEST(SubosPrivilegedEnv, AnUnclassifiedVariableReadsAsPrivileged) { + EXPECT_TRUE(m::is_privileged_env("SOME_FUTURE_PLUGIN_PATH", + "${pkgdir}/lib/plugins")); +} + +// AD-3: the line is "causes code to be loaded" vs "causes data to be found", +// not "is it process-global". subos supplying a default that the user can +// override is how Linux works. +TEST(SubosPrivilegedEnv, DataVariablesAreNotPrivileged) { + EXPECT_FALSE(m::is_privileged_env("XDG_DATA_DIRS", "${pkgdir}/share")); + EXPECT_FALSE(m::is_privileged_env("MANPATH", "${pkgdir}/share/man")); + EXPECT_FALSE(m::is_privileged_env("PKG_CONFIG_PATH", "${pkgdir}/lib/pkgconfig")); +} + +// The same hazard in its other spelling. The subos sysroot is a VIEW onto our +// payloads, made of symlinks into them, so a directory under it on a loader +// search path delivers our libraries just as surely as the store path does. +// Checking only ${pkgdir} would let one hazard through under two names -- the +// shape this whole review is about. +TEST(SubosPrivilegedEnv, TheSubosViewCountsAsOurPayload) { + EXPECT_TRUE(m::is_privileged_env("LD_LIBRARY_PATH", "${subosdir}/lib")); + EXPECT_TRUE(m::is_privileged_env("LD_LIBRARY_PATH", "${xlings_home}/lib")); + EXPECT_TRUE(m::is_privileged_env("LD_LIBRARY_PATH", + "/home/u/.xlings/data/xpkgs/xim-x-a/1/lib")); +} + +// PATH is a third category: it does not inject code into a running process, it +// decides which executable runs. That is R6/AD-1's business, not this guard's. +TEST(SubosPrivilegedEnv, PathIsGovernedElsewhere) { + EXPECT_FALSE(m::is_privileged_env("PATH", "${pkgdir}/bin")); +} + +// A value pointing outside our store cannot put OUR code anywhere. The host's +// own driver directory on LD_LIBRARY_PATH is the case that has to stay +// available: it is the one thing an interposer cannot cover, because the +// driver dlopen's its siblings by bare SONAME at runtime. +TEST(SubosPrivilegedEnv, AValueOutsideOurStoreIsNotPrivileged) { + EXPECT_FALSE(m::is_privileged_env("LD_LIBRARY_PATH", + "/usr/lib/x86_64-linux-gnu")); +}