diff --git a/.agents/docs/2026-08-08-compiler-as-capability-implementation-plan.md b/.agents/docs/2026-08-08-compiler-as-capability-implementation-plan.md new file mode 100644 index 00000000..ac50799c --- /dev/null +++ b/.agents/docs/2026-08-08-compiler-as-capability-implementation-plan.md @@ -0,0 +1,148 @@ +# 编译器是能力 —— 跨仓实施计划 + +> **For agentic workers:** 逐 task 执行,每个 task 自带 red→green→commit。**T1 是所有后续 task 的前提**(没有它,后面每一步的观测都不可信)。 + +**Goal:** 让 mcpp 停止依赖编译器的安装期配置,把「用哪份 libc」收敛成一个权威;顺带把已经污染的产物清干净,并让多 subos 构建真正可用。 + +**设计文档:** +- `.agents/docs/2026-08-07-xlings-as-runtime-substrate-design.md`(运行时底座;S1/S3 与撤销 `c_runtime` 的理由) +- `.agents/docs/2026-08-08-payload-version-and-contract-drift-design.md`(本计划的主体) + +**Tech Stack:** C++23 modules,gtest(`tests/unit/`),shell e2e(`tests/e2e/`),Lua(mcpp-index)。 + +## Global Constraints + +- **不改 payload**。`patchelf_walk`(让编译器自己能跑)保留;**写 payload 内配置的一律删掉**。 +- **权威唯一**:`--runtime` → `[xlings] subos` 的 `subos_info.runtime` → **报错,不猜**。删掉「扫目录挑一个」。 +- **平台差异下沉** `src/platform/`,`build/` 里不写 `#ifdef`。 +- **每一处「注释描述行为」必须有对应断言**(R-C)。本轮已发现三处注释与代码不符。 +- **跨仓 wire format 的 fixture 取自写入方**(R-A),不得手写。 +- **一条测试若为覆盖某路径而存在,必须先断言走了那条路径**(R-B)。 +- **不新增任何让产物链宿主 libc 的开关**(8-07 §5.2 已否)。 + +--- + +### T1: 三处「注释撒谎」先钉住 —— 后续观测的地基 + +**Files:** `tests/unit/test_linkmodel.cpp`(新断言)、`src/xlings.cppm`、`src/toolchain/post_install.cppm`(改注释) + +本轮发现三处注释描述了代码没有的行为。**先让它们不能再撒谎**,否则后面每一步都可能在错误的前提上推理。 + +- [ ] **Step 1** 写失败单测:`find_sibling_tool` 的注释说「first (highest)」,断言**两个版本目录时返回的是排序后最高的那个** +- [ ] **Step 2** 跑,确认 FAIL(今天返回 readdir 第一个) +- [ ] **Step 3** 二选一并使注释与代码一致:①真的排序;②改注释为「未定义顺序」并让调用方不得依赖。**本计划选 ②**——T2 会让它不再被用于选版本 +- [ ] **Step 4** 另两处:`fixup_gcc_specs` 的「Idempotent」(跨 home 不幂等)、`execute.cppm` reader 的「旧缓存视为 miss」(T5 修)——先把注释改成事实 +- [ ] **Step 5** Commit + +--- + +### T2: 权威降级链 —— 没有权威就不进 PayloadFirst + +**Files:** `src/toolchain/probe.cppm`、`src/xlings/subos_info.cppm`、`src/build/prepare.cppm` +**Test:** `tests/unit/test_toolchain_probe.cpp`(新) + +**Interfaces:** +```cpp +// probe 不再自己选版本;版本由权威给出 +std::optional probe_payload_paths( + const std::filesystem::path& compilerBin, + std::string_view runtimeBinding); // "glibc@2.39";空 ⇒ 不进 PayloadFirst +``` + +- [ ] **Step 1** 单测:给定 `glibc@2.39` 且 home 内有 2.39/2.44 两份 ⇒ **必须返回 2.39**;给定空 binding ⇒ **返回 nullopt**(不猜) +- [ ] **Step 2** 跑,FAIL +- [ ] **Step 3** 实现;`prepare.cppm` 按 `--runtime` → `[xlings] subos` 的 `subos_info.runtime` 顺序解析,末级不猜 +- [ ] **Step 4** 跑,PASS +- [ ] **Step 5** Commit + +--- + +### T3: fingerprint 加入 runtime 轴 —— 多 subos 的前置 + +**Files:** `src/toolchain/fingerprint.cppm`、`tests/unit/test_fingerprint.cpp` + +不加这条,切 subos 不改 fingerprint ⇒ `target//` 被复用 ⇒ 里面是对另一份 glibc 编译的对象。**比今天更坏。** + +- [ ] **Step 1** 单测:两个只有 `runtimeBinding` 不同的输入 ⇒ **fingerprint 必须不同** +- [ ] **Step 2/3/4** red → 加字段 → green +- [ ] **Step 5** Commit + +--- + +### T4: GCC 也显式发 loader/rpath + clean specs + +**Files:** `src/toolchain/linkmodel.cppm`、`src/build/flags.cppm`、`src/toolchain/post_install.cppm`(删 `fixup_gcc_specs`) +**Test:** `tests/unit/test_linkmodel.cpp`、`tests/e2e/201_gcc_clean_specs.sh`(新) + +**已实测**:`-Wl,--dynamic-linker` 压得过 specs;`-specs=<无 `+` 定义>` 是替换;`g++ -dumpspecs` 给内建 specs(`/tmp` 路径 0 条);真实构建(模块 + `import std` + 共享库)RUNPATH 68→2、死路径 0、可运行。 + +- [ ] **Step 1** e2e:gcc 构建的产物 **RUNPATH 中不得有 `/tmp/tmp.`**,且 interpreter 是权威指定的那份,且能跑 +- [ ] **Step 2** 跑,FAIL(今天有 34+ 条) +- [ ] **Step 3** ①去掉 `linkmodel` 的 `if (clangDriver)` 门;②构建期生成 clean specs 到 **build dir**,`-specs=` 传入;③**删掉 `fixup_gcc_specs`**(保留 `patchelf_walk`) +- [ ] **Step 4** 跑,PASS +- [ ] **Step 5** Commit + +--- + +### T5: 并回 #377 的两处修复 + +**Files:** `src/xlings/subos_info.cppm`、`src/build/execute.cppm`、`tests/unit/test_subos_info.cpp`、`tests/e2e/200_subos_env_reaches_program.sh` + +已在 `fix/fast-run-stale-cache-subos` 分支上完成并 18/18 CI 通过,**原样带过来**: + +- [ ] **Step 1** `git cherry-pick` 该分支的三个 commit(wire format / 旧缓存 / e2e fixture) +- [ ] **Step 2** 跑单测 + e2e,确认仍绿 +- [ ] **Step 3** Commit(或保留 cherry-pick 的原始提交) + +--- + +### T6: D4 沙箱 xlings 版本比对 + D5 陈旧 sysroot + +**Files:** `src/fallback/xlings_binary.cppm`、`src/doctor.cppm`、`src/fallback/probe_sysroot.cppm` + +- [ ] **Step 1** 单测/e2e:vendored xlings 版本**低于 pin** ⇒ 被替换;**高于 pin** ⇒ 不动 +- [ ] **Step 2** red +- [ ] **Step 3** 实现;doctor 增加两条 finding:①xlings 低于 pin;②`--sysroot` 指向当前项目之外 +- [ ] **Step 4** green +- [ ] **Step 5** Commit + +--- + +### T7: 文档 + 版本 + PR + +- [ ] `docs/03-toolchains.md`:权威降级链、`[xlings] subos`、多 subos 构建 +- [ ] `docs/05-mcpp-toml.md`:`[xlings] subos` 与 runtime 的关系 +- [ ] 中文版同步 +- [ ] `MCPP_VERSION` / `mcpp.toml` → `2026.8.8.2`;`kXlingsVersion` → 最新;`check_version_pins.sh` 通过 +- [ ] 全量 `mcpp test` + 关键 e2e +- [ ] 开 PR + +--- + +### T8: mcpp-index 图形栈重新落地(D1 修好之后) + +**Files（mcpp-index）:** `pkgs/c/compat.glx-runtime.lua`、`pkgs/c/compat.glfw.lua` + +- [ ] **Step 1** 新增 e2e/验证:**装 `xim:graphics` 前后**,各跑一次与图形无关的成员,断言产物 `PT_INTERP` 与 `readelf -V` 的 glibc 符号上界**不变**。这是上次事故真正缺失的那个测试 +- [ ] **Step 2** 重新应用 `f44e896` 的两个文件改动(deps 放平台层) +- [ ] **Step 3** CI 全绿后合入 + +--- + +## Self-Review + +| 设计文档条目 | Task | +|---|---| +| 8-08 §3.1 原则(编译器当能力) | T4 | +| 8-08 §3.2 权威降级链 / D1 | T2 | +| 8-08 §3.5① 去掉 clangDriver 门 | T4 | +| 8-08 §3.5② clean specs 进 build dir | T4 | +| 8-08 §3.5③ 删 `fixup_gcc_specs` / D6 | T4 | +| 8-08 §3.6 多 subos + fingerprint 前置 | T2 + T3 | +| 8-08 D4 沙箱 xlings / D5 sysroot | T6 | +| 8-08 D2 wire format / D3 旧缓存 | T5 | +| 8-08 §4.2 R-A/R-B/R-C | T1(R-C)+ 各 task 的测试形态 | +| 8-07 §1.5 图形栈迁移 | T8 | +| 8-07 §3-S2 不加链宿主 libc 的开关 | Global Constraints(不做) | + +**未覆盖且是有意的**:8-07 §7「xlings 落盘 exports」—— 8-08 已说明它不再是关键路径(权威改为 `subos_info.runtime`)。Q6(交叉/musl/MinGW 下 `-specs=` 替换)在 T4 的 e2e 里只覆盖 native;**其余平台列为 PR 中显式声明的未验项**,不假装验过。 diff --git a/.agents/docs/2026-08-08-payload-version-and-contract-drift-design.md b/.agents/docs/2026-08-08-payload-version-and-contract-drift-design.md new file mode 100644 index 00000000..0ebe4dcd --- /dev/null +++ b/.agents/docs/2026-08-08-payload-version-and-contract-drift-design.md @@ -0,0 +1,379 @@ +# 载荷版本与契约漂移:四个缺陷,一条线 + +**日期**:2026-08-08 +**性质**:设计提案,待 review。本文不改代码。 +**触发**:2026.8.8.1 发布当天暴露的四个缺陷,其中三个是我在这一轮自己引入或自己漏掉的。 + +**关联**: +- `.agents/docs/2026-08-07-xlings-as-runtime-substrate-design.md`(本轮的上游设计) +- mcpp#352 / #375、mcpp-index#179(已 revert,PR#180)、mcpp#377 +- xlings `.agents/docs/2026-08-06-subos-architecture-proposal.md` 的 R1–R7 + +--- + +## 0. TL;DR + +四个缺陷,**同一条线**: + +> **mcpp 对 xlings 的每一项事实,都是「自己猜一个」而不是「读一个」;而它猜的那个,和它烙进产物的那个,是两个独立的答案。** + +| # | 缺陷 | 谁引入 | 现状 | +|---|---|---|---| +| **D1** | 载荷探测取**目录序第一个**(注释说是最高,代码没排序),与 gcc 烙定的不是同一份 | 早已存在,本轮触发 | **main 曾变红,已 revert** | +| **D2** | subos_info **wire format 读错**,功能完全不工作 | 本轮我引入 | PR#377 已修 | +| **D3** | 升级后**复用升级前缓存**,修复静默不生效 | 本轮我引入 | PR#377 已修 | +| **D4** | 沙箱 xlings **永不升级**,且 mcpp 打印偏差却不作为 | 早已存在 | **未修** | + +D1 和 D4 是同一件事的两面:**mcpp 把一个可变的外部世界当成了装机时的快照**。D2 和 D3 是另一件事的两面:**我用与实现同源的理解去写测试,于是测试证明的是「我和我自己一致」。** + +--- + +## 1. D1:载荷探测与 gcc 烙定的版本是两个独立答案 + +### 1.1 实测 + +`compat.glx-runtime` 加了一条 `xim:graphics` 依赖(为修 #352)。`mesa` 声明: + +```lua +"xim:glibc@>=2.38", -- 下界,不是钉死 +``` + +解析器装了 **glibc 2.44**,与既有的 2.39 并存。然后: + +``` +.../xim-x-glibc/2.39/lib64/libc.so.6: version `GLIBC_2.42' not found + (required by <测试二进制>) +``` + +**在 `asio-module`、`core` 上炸 —— 与图形毫无关系的包。** + +### 1.2 机制 + +`probe.cppm` 调 `find_sibling_tool(compilerBin, "glibc")`。那个函数的注释说它取最高版本 —— **代码没有排序**: + +```cpp +// Return the first (highest) version dir that exists. +for (auto& v : std::filesystem::directory_iterator(root, ec)) { + if (v.is_directory(ec)) return v.path(); +} +``` + +`directory_iterator` 是无序的,所以真实行为是**取 readdir 碰巧给出的第一个**。本机实测顺序是 `2.44, 2.39`,于是选中 2.44 —— 但**换台机器、装个别的包、甚至删一个目录,结果都可能变**。这是本文档记录的第三处「注释描述了代码没有的行为」(另两处见 §3.3)。 + +于是: + +| | 谁决定 | 结果 | +|---|---|---| +| **编译/链接** 对着哪份 glibc | mcpp 的探测(readdir 第一个) | **2.44**(本机;换台机可能不同) | +| **运行时** PT_INTERP / specs | gcc 载荷装机时 elfpatch 烙定 | **2.39** | + +两个独立的答案,**在只装一份 glibc 时恒相同**,所以从未被迫达成一致 —— 这正是 xlings 侧 R1/R2 诊断出的 P1「一个问题多个回答者」,原样出现在 mcpp。 + +### 1.2b 为什么「新 glibc 向后兼容」救不了这一格 + +glibc 的兼容性是**单向**的:对 2.39 构建的产物在 2.44 上跑 ✅;对 2.44 构建的在 2.39 上跑 ❌。本次报错正是第二种 —— 加载了 2.39 的 `libc.so.6`,而二进制要 `GLIBC_2.42` 的符号。**问题不是运行时太老,是编译期跑到前面去了而运行期没跟上。** + +### 1.2c 参照系:发行版、Nix、和 mcpp 现在在哪 + +| | 发行版(Arch / Fedora) | Nix / xlings | **mcpp 今天** | +|---|---|---|---| +| 同时几份 glibc | **1** | 多份并存 | 多份并存 | +| 路径 | **无版本** `/lib64/ld-linux-x86-64.so.2` | 带版本 `…/2.44/lib64/…` | 带版本 | +| 升级 | **就地替换同一个文件** | 在旁边装一份新的 | 在旁边装一份新的 | +| 编译期与运行期的一致性 | **构造上不可能分歧** | 每个消费者冻结自己的选择 | **两个独立决定者** ❌ | + +滚动发行版之所以能一直换 glibc 而不炸,是因为**编译侧和运行侧是同一个文件**;新 glibc 保留全部旧符号版本,所以既有二进制照跑。 + +**Arch 上唯一会出这个错的场景,与本次同型:部分升级**(`pacman -Sy pkg` 不带 `-u`)—— 装了一个对着更新 glibc 构建的包,而 glibc 没跟着升。Arch 明文禁止部分升级,理由一模一样。 + +**结论:mcpp 用了 Nix 的布局(多版本并存 + 版本化路径),但没有兑现那个布局的前提 —— 每个消费者的选择必须只有一个决定者。** 这不是「要不要支持多 glibc」的问题,而是既有布局的前提没被满足。 + +### 1.2d xlings 早就预见了,而一条下界绕开了它 + +`glibc.lua` 明写 `latest` 故意停在 2.39: + +> Moving `latest` to 2.44 would be safe for the same reason — but it would also **silently change which glibc every existing home resolves to**, and that is a decision to make deliberately rather than as a side effect of adding a version. + +而 `mesa` 的 `xim:glibc@>=2.38` **绕开了 `latest` 指针**,把 2.44 直接拉了进来。一个刻意做出的「不动 latest」的决定,被一条依赖的下界从侧面推翻。 + +**这一条值得同步给 xlings**:范围解析取最高满足者,与 `latest` 别名承载的「这个 home 该用哪一份」是两个不同的意图,而前者会静默压过后者。是否该让范围解析在 `latest` 满足区间时优先取 `latest`,是 xlings 侧的开放问题。 + +### 1.3 判据:它不是「装了新 glibc」的问题 + +**任何**让 home 里多出一份更高版本 glibc 的操作都会触发它 —— 一条新依赖、一次 `xlings install`、一个新包。**这个雷一直在,只是这次我踩了。** + +### 1.4 方案 + +| | A. 探测改为「与 gcc 烙定的那份一致」 | B. 探测取最高但校验一致性,不一致就报错 | C. 维持现状 | +|---|---|---|---| +| 编译/运行是否可能分裂 | **不可能** | 可能,但会被拦下 | 会,且静默 | +| 需要什么输入 | gcc 烙定的 glibc 版本 | 同左 | — | +| 多 glibc 场景 | 天然正确(跟工具链走) | 需要人工介入 | 坏 | +| 结果是否确定 | ✅ | ✅ | ❌ **随目录序变** | + +**推荐 A,B 作为过渡期的护栏。** + +「gcc 烙定的是哪份」不需要猜:`post_install.cppm` 已经有 `extract baked loader from clang cfg` 的能力(扫 `/ld-linux-`),而 gcc 侧的 specs 里也写着。**更好的做法是让 xlings 落盘**(见 §5),但即使不落盘,从 gcc 自己的产物反推也比「取最高」正确 —— 因为它反推的正是运行期会用的那一份。 + +**必须同时做的一件事**:`mcpp doctor` 增加一条 —— *home 里有多份 glibc 且编译期选中的不是 gcc 烙定的那份*。这是 D1 唯一的可观测入口;没有它,下一次触发同样只会表现为一个与 glibc 无关的包炸掉。 + +--- + +## 2. D4:沙箱 xlings 永不升级,而 mcpp 知道却不说 + +### 2.1 实测 + +``` +$ mcpp self env +xlings binary = /home/speak/.mcpp/registry/bin/xlings ← 2026.8.2.1 +xlings pinned = 2026.8.6.3 +``` + +**mcpp 把两个数字并排打印出来,不作任何评价。** + +`fallback/xlings_binary.cppm:22`: + +```cpp +if (std::filesystem::exists(destBin)) return destBin; // 无任何版本检查 +``` + +### 2.2 两种布局,只有一种会更新 + +| 布局 | xlings 从哪来 | 升级 mcpp 时 | +|---|---|---| +| **tarball 相对**(`MCPP_HOME=<解包目录>/registry`) | release 自带的 `registry/bin/xlings` | ✅ 随之更新 | +| **默认 home**(`MCPP_HOME=~/.mcpp`) | 首次 `self init` 时 acquire 的那份 | ❌ **永不更新** | + +CI 走第一种(所以 CI 一直是新的),开发机与普通用户走第二种。**「我记得已经修过」和「实测是旧的」都对,只是说的不是同一种布局。** + +### 2.3 后果:整条修复链断在这里 + +`#352` 需要四环:mcpp 读 `subos_info` → index 拉 `xim:graphics` → **graphics 把声明写进 subos** → subos 有 `subos_info`。第三环需要沙箱 xlings ≥ 2026.8.5.1;`graphics.lua` 显式探测 `type(subos.env) ~= "function"`,**老客户端上声明被静默丢弃**。 + +实测本机沙箱 xlings 2026.8.2.1 的二进制里,`subos_info` 这个字符串出现 **0 次**。 + +### 2.4 方案 + +| | A. 版本低于 pin 就自动替换 | B. doctor 报告 + 一条显式命令 | C. 每次启动检查 | +|---|---|---|---| +| 无感 | ✅ | ❌ 用户要动手 | ✅ | +| 风险 | 换掉用户正在用的 xlings | 低 | 每次 exec 多一次版本读 | +| 与「mcpp 不接管 subos 状态」的边界 | ⚠️ 擦边 | 一致 | ⚠️ | + +**推荐 A + B**:acquire 时比对版本,低于 pin 则替换(这是 mcpp 自己 vendored 的那份,不是用户的系统 xlings,替换它不越界);同时 doctor 把偏差列为一条 finding。 + +**A 的判据必须是「低于 pin」而不是「不等于 pin」** —— 用户手动放了一份更新的进去,不该被降级。 + +> **⚠️ 这条要小心**:替换沙箱 xlings 会改变依赖解析的行为。参考 `index-floor-must-degrade` 的教训——**发布数据不得让程序失效**。所以替换后必须能回退,且 doctor 要能说出「现在用的是哪一份、为什么」。 + +--- + +## 3. D2/D3:测试证明的是「我和我自己一致」 + +### 3.1 D2 —— wire format 读错,功能完全不工作 + +盘上真实的 `envs` 是**以 binding 为键的对象**;我的 reader 期望**数组**。`is_array()` 为假 ⇒ 循环一次不执行 ⇒ 对着真实 subos **一个变量都不应用**。 + +**10 个单测 + 1 条 e2e 一路全绿**,因为每一份 fixture 都是我按同一个臆想手写的。 + +### 3.2 根因不是笔误,是方法 + +> **用与实现同源的理解去造 fixture,抓不出对 wire format 的误解。只有取自写入方的 fixture 能抓。** + +已加 `RealXlingsCapture`(真实输出逐字副本),parser 改为从 xlings 的 reader **誊写**而非建模 —— 这顺带抓到第二处分歧:xlings 丢弃 `op` 不是 `set`/`prepend` 的声明,我原来全收。 + +### 3.3 D3 —— 自证断言,以及注释撒的谎 + +我原本的 e2e 只跑一次 `mcpp run`,PASS —— 而它存在的目的是覆盖**缓存快路径**,那次根本没走。加上「这次必须走快路径,否则报错」之后,立刻暴露快路径丢环境变量,**同一条断言接着又抓到第二个**(缓存行读写顺序不匹配)。 + +更难看的是:reader 的注释**声称**它把旧缓存当 miss —— 那个检查从来没写过。**注释描述了代码没有的行为,比漏掉更糟:它让下一个读代码的人不去查。** + +### 3.4 提议:三条可执行规则 + +- **R-A 跨仓 wire format 的 fixture 必须取自写入方**,不得手写。形式:golden 文件 + 注明来源与采集时间。 +- **R-B 一条测试如果存在的目的是覆盖某条路径,必须先断言「这次确实走了那条路径」。** 否则它可能一直 PASS 而零覆盖。 +- **R-C 注释若描述一条行为,该行为必须有对应断言。** 「reader 把旧缓存当 miss」这句话本身就该是一个测试名。 + +--- + +## 4. 四个缺陷的公共上游 + +``` +D1 载荷版本 ── 猜(取最高) ─┐ +D4 xlings ── 快照(装机时那份) ─┼─→ 把可变的外部世界当成不变的 +D2 wire fmt ── 猜(臆想格式) ─┤ +D3 缓存契约 ── 猜(注释当实现) ─┘ +``` + +**D1/D4 是「外部世界会变,而我记的是装机那一刻」;D2/D3 是「外部契约有确定形状,而我记的是我以为的形状」。** + +两者的解药是同一句话:**凡是别人拥有的事实,读它,并且用它自己的产物来验证你读对了。** + +--- + +## 5. 与上游设计的关系:这轮验证了什么、推翻了什么 + +`2026-08-07-xlings-as-runtime-substrate-design.md` 的核心论点是「mcpp 把 xlings 当目录约定而不是可查询的运行时」。**这一轮四个缺陷全部是那个论点的实例**,而且其中两个是我一边写着那份设计一边犯的 —— 这本身是最强的证据。 + +它也**修正**了那份设计的一处轻描淡写:§7 把「xlings 落盘 exports」排在 P1,理由是「A 降级路径足以工作」。**D1 证明那不成立** —— 没有权威落盘,mcpp 只能靠目录扫描猜一个,而猜出来的那个与 gcc 烙定的那个是两个独立答案。**`exports` 落盘应升到 P0**,且内容要包含「gcc 这份载荷烙定的 glibc 是哪一份」。 + +--- + +## 6. 分期 + +**P0(main 已红过,先止血)** +1. ✅ revert mcpp-index#179(PR#180) +2. ✅ PR#377:wire format + 旧缓存(D2/D3) +3. **D1 护栏**:`mcpp doctor` 报告「多份 glibc 且编译期选中的不是 gcc 烙定的那份」 + +**P1(让 #352 真正到达用户)** +4. **D4**:acquire 时版本比对 + 替换,doctor 报告偏差 +5. **D1 正解**:载荷探测改为跟随 gcc 烙定的版本 +6. 之后才重新落 mcpp-index 的图形栈迁移 + +**P2** +7. xlings 侧落盘 `exports`(含 gcc↔glibc 的绑定),mcpp 改读它 +8. R-A/R-B/R-C 写进贡献规范 + +--- + +## 7. 明确不做 + +- **不在 recipe 里回避 D1**(比如把 mesa 的 glibc 下界改成钉死)。那是把引擎缺陷摊给包作者,而且下一个用下界的包会再犯 —— 与 `aarch64-native-gcc-payload-is-musl` 那次「归因成包缺 arch 轴是错的」同型。 +- **不让 mcpp 接管 subos 生命周期**。D4 的方案只替换 mcpp 自己 vendored 的那份二进制,不碰用户的系统 xlings、不管理 subos。 +- **不追 glibc 版本**。D1 的正解是「跟随工具链烙定的那一份」,不是「总用最新」。 + +--- + +## 8. 开放问题 + +- **Q1** D1 的「gcc 烙定的是哪份 glibc」:从 gcc 产物反推(specs / PT_INTERP),还是等 xlings 落盘?前者能立刻做,后者更正确 —— 是否两者都要,反推作为降级? +- **Q2** D4 的替换时机:`self init` 时?每次 acquire 时?还是只在 doctor --fix?自动替换与「用户手动放了一份新的」如何共存(判据「低于 pin 才换」是否足够)? +- **Q3** 多 glibc 是 xlings 明确要支持的方向(`subos new --runtime glibc@2.44`)。D1 修好之后,mcpp 对「一个 home 里多份 glibc」的正式立场是什么 —— 跟随工具链,还是跟随 subos 的 `runtime`?后者更符合上游设计的 S1,但需要 D4 先修。 +- **Q4** 本轮 revert 掉的图形栈迁移,重新落地时如何验证「装这个栈不会影响 home 的其余部分」?**这一条是这次事故真正缺失的那个测试。** 一个可执行形状:装栈前后各跑一次与图形无关的成员(`asio-module`、`core`),断言产物的 `PT_INTERP` 与 `readelf -V` 的 glibc 符号版本上界不变。 +- **Q5**(xlings 侧,§1.2d)范围解析取最高满足者,压过了 `latest` 别名承载的「这个 home 该用哪一份」。是否该让 `latest` 在满足区间时优先?这决定了「加一个新版本进索引」是不是一个安全操作。 + +--- + +## 9. 实施回写(2026-08-08,PR #378) + +方案落地过程中,有三条是设计当时没看见的,其中第一条**修正了本方案自己的一个假设**。 + +### 9.1 兼容回落读的是「我刚删掉的机制的产物」 + +§3.5 写的兼容路径是:subos 不自述时,读工具链自身烙入的值 —— gcc 的 `specs`、 +clang 的 `.cfg`。这在**已有机器**上成立,因为那些文件是**旧 fixup 写出来的**,还留在 +盘上。 + +在**全新安装**的机器上不成立:mcpp 已经不写它们了,所以它们不存在。链条是 +`无 binding → 无 payloadPaths → 无 --dynamic-linker → 产物拿宿主 loader`,hermetic +检查如实报出 `/lib64/ld-linux-x86-64.so.2 (outside the sandbox)`。 + +**每台开发机都还留着那些文件,所以本机验证永远是绿的,CI 才红。** 这条应当写进判据: + +> 删掉一个机制时,先列出谁在读它的产物。兼容回落尤其危险 —— 它读的往往正是旧机制 +> 留下的东西。 + +补的第三个来源是**编译器自身的 `PT_INTERP`**:由 patchelf 走查写入(该机制仍在运行), +指向的正是这套工具链对齐的那个 glibc payload,与 specs 曾经持有的是同一个事实,但 +**不依赖 mcpp 写过任何东西**。自己读 ELF 头而不用 `patchelf --print-interpreter`—— +本函数在 prepare 期运行,那里不保证 patchelf 已解析。 + +验证方式是把 `specs` 文件真的挪开再跑,而不是读代码。 + +### 9.2 归属谓词加在了一个到不了的函数里 + +§3.5 的 D5 把 `remap_xlings_baked_sysroot` 的判据从「存在」改为「归属」。但调用方 +`probe_sysroot` 的**第一步**是「存在且可用 ⇒ 直接 return」,只有在路径**不存在**时才 +往下走到 remap —— 而该函数存在的全部意义,正是处理「存在、可用、却属于别人」。 + +本机实测:这个仓库的 gcc 报的 sysroot 指向 `xim-pkgindex-fromsource` 下的 +`.xlings/subos/default`,每次构建都在吃另一个仓库的头文件。修法是把归属问在可用性 +**之前**;「外来但可用」保留为最后兜底,因为没有 registry subos 可映射的机器上,取 +「什么都不要」会直接坏掉。 + +### 9.3 生成的 spec 放在了缓存能删的目录里 + +`--no-cache` 会 `remove_all(outputDir)`,而 spec 是 prepare 期写进那个目录的 ⇒ **每次 +冷构建 + gcc 必挂**。修法不是调顺序,而是把「ninja 跑之前该文件必须存在」当作**不变量 +在消费点持有** —— 这样也扛得住用户手动 `rm -rf target`。 + +值得记的是它**怎么被发现的**:e2e 201 是专为这套机制写的测试,但它只跑暖构建,而暖 +构建下文件从上一次留着。**是全量本地 e2e 扫出来的,不是定向子集。** + +### 9.4 链接行去重 + +C-runtime 的那组 flags 同时经 `link_toolchain_flags` 与 `payload_ld` 发出,整组在链接行 +上出现两次。对正确性无害,但链接行有 128KiB 硬上限(`MAX_ARG_STRLEN`),真实 +workspace 已经用掉 43%。`payload_ld` 现在只服务 clang-with-cfg 的 PayloadFirst,其余组合 +本就已经有了。 + +### 9.5 一条跨平台低级错误 + +`rel.native().rfind("..", 0)` 在 Windows 上**编译不过**(`native()` 是 `wstring`),所有 +Windows job 直接挂;而且它在能编译的地方语义也错 —— 名叫 `..cache` 的目录并不是逃逸。 +包含关系是**路径分量**的问题,`path_is_under` 按分量问。 + +### 9.6 「拒绝」的边界:一个 payload 是答案,两个才是问题 + +§3.2 把判据写成「没有权威就拒绝走 payload-first」。落地后发现这条**收得过紧**。 + +所有兼容来源(gcc 的 `specs`、clang 的 `.cfg`、编译器自身的 `PT_INTERP`)都是**某个更早 +的机制写出来的东西**,而一台机器可以合法地一个都没有。这时按 §3.2 的字面判据就该拒绝, +产物于是拿宿主 loader —— 比「猜错版本」更糟,因为它连沙箱都出去了。 + +真正的轴不是「mcpp 能不能去看 payload 目录」,而是**有没有得选**: + +- **恰好一个 glibc payload** ⇒ 没有选择可言。它是这套工具链产出的任何产物唯一可能绑定的 + 运行时,拒绝等于拒答一个只有一个答案的问题。 +- **两个或以上** ⇒ 沉默。这正是事故本身的形状,必须由 subos 来定。 + +这与被移除的旧规则的区别是决定性的:旧规则在**有得选**的时候按 `readdir` 顺序选,而且 +一直到装进第二个 payload 之前都看起来是对的。 + +另外,记录下来的 loader 路径可能指向 subos **视图**(`/subos/default/lib/ld-linux-…`) +而不是 payload —— 视图路径里根本没有版本段。解析前先 canonical 化(R6:产物绑 payload, +不绑可变视图)。 + +验证方式:把 gcc 的 `specs` 挪开**并且**把 gcc 自身的 `PT_INTERP` 改指宿主 loader,即三条 +兼容来源全部失效,binding 仍解析得出,产物仍拿 payload loader。 + +### 9.7 兼容来源是**记录**,不是权威 + +§9.6 补上单例回落后 CI 依然红,日志直说了原因: + +``` +probe: runtime 'glibc@2.39' is not installed in this home +``` + +那台机器上只有一个 glibc payload:**2.44**;而兼容记录烙的是 **2.39**。 + +三条兼容来源(`specs`、`.cfg`、`PT_INTERP`)都是**过去某个时刻的记录** —— 装机时写的、 +fixup 时写的 —— 而 payload 被替换(升级、回收)时**没有任何东西回头去改它们**。§3.2 的 +「精确匹配、绝不回落」于是如实地拒绝了 2.39,结果是没有 payloadPaths、没有 +`--dynamic-linker`、产物拿宿主 loader。 + +**判据要补一句:一条记录只有在它指的东西还在,才算数。** + +现在每条记录都对「实际安装了哪些 payload」核对一遍: + +- 记录指的 payload **还在** ⇒ 采信(即使装了多个,它就是产物会加载的那个) +- 记录指的 payload **不在了** ⇒ 跳过,继续往下(单例回落往往能答) +- 什么都答不上来 ⇒ 沉默 + +这条与 §9.6 是同一件事的两面:**§9.6 说「没得选就不算猜」,§9.7 说「化石不算权威」。** +两条都是在收窄「拒绝」的适用范围 —— 拒绝该留给**真正有歧义**的情况。 + +### 9.8 musl 目标不碰 glibc payload + +`sysroot_mode` 里补 loader / lib 目录时漏了 musl 护栏。musl 的 sysroot 是自包含的,而 +同时被探测到的 glibc payload 属于**宿主**工具链 —— 把它放进链接路径后,ld 会把 glibc 的 +静态 `libc.a` 拉进一次 musl 链接: + +``` +undefined reference to `_DYNAMIC' (dl-reloc-static-pie.o) +hidden symbol `_DYNAMIC' isn't defined +``` + +同一函数里紧接着的 header 分支**本来就有** `is_musl_target` 判断 —— 那句话对库和 loader +一样成立,只是当时没写上。由 e2e 103 抓到,同样是全量跑扫出来的。 diff --git a/.github/actions/bootstrap-mcpp/action.yml b/.github/actions/bootstrap-mcpp/action.yml index 813d3210..2e171056 100644 --- a/.github/actions/bootstrap-mcpp/action.yml +++ b/.github/actions/bootstrap-mcpp/action.yml @@ -25,7 +25,7 @@ inputs: # `package.name`, so one of the two was simply unreachable — and which one # depended on the machine, which is why CI failed on `compat:lua` on # Windows and `mcpplibs.capi:lua` on Linux. Never pin below that. - default: '2026.8.6.3' + default: '2026.8.7.1' cache-target: description: also restore/save target/ (build artifacts + BMIs) required: false diff --git a/.github/actions/setup-macos-llvm/action.yml b/.github/actions/setup-macos-llvm/action.yml index 682d72bc..d00ad2e5 100644 --- a/.github/actions/setup-macos-llvm/action.yml +++ b/.github/actions/setup-macos-llvm/action.yml @@ -15,7 +15,7 @@ inputs: # Floor imposed by the index, not a routine bump — see # .github/actions/bootstrap-mcpp/action.yml for why 0.4.69 is required # (two packages named `lua` in one repo need openxlings/xlings#381). - default: '2026.8.6.3' + default: '2026.8.7.1' runs: using: composite diff --git a/.github/workflows/bootstrap-macos.yml b/.github/workflows/bootstrap-macos.yml index e2e7ba29..f607854d 100644 --- a/.github/workflows/bootstrap-macos.yml +++ b/.github/workflows/bootstrap-macos.yml @@ -17,7 +17,7 @@ jobs: # Dormant (workflow_dispatch only), but kept in step with the rest — # check_version_pins.sh holds it there. Floor: 0.4.69, below which the # index cannot resolve two packages that share a short name. - XLINGS_VERSION: '2026.8.6.3' + XLINGS_VERSION: '2026.8.7.1' steps: - uses: actions/checkout@v4 diff --git a/.github/workflows/ci-fresh-install.yml b/.github/workflows/ci-fresh-install.yml index 9c53eeba..503c8695 100644 --- a/.github/workflows/ci-fresh-install.yml +++ b/.github/workflows/ci-fresh-install.yml @@ -152,7 +152,7 @@ jobs: env: XLINGS_NON_INTERACTIVE: '1' run: | - curl -fsSL https://raw.githubusercontent.com/openxlings/xlings/main/tools/other/quick_install.sh | bash -s v2026.8.6.3 + curl -fsSL https://raw.githubusercontent.com/openxlings/xlings/main/tools/other/quick_install.sh | bash -s v2026.8.7.1 echo "$HOME/.xlings/subos/current/bin" >> "$GITHUB_PATH" - name: Install mcpp and config mirror @@ -292,7 +292,7 @@ jobs: - name: Install xlings + mcpp run: | - curl -fsSL https://raw.githubusercontent.com/openxlings/xlings/main/tools/other/quick_install.sh | bash -s v2026.8.6.3 + curl -fsSL https://raw.githubusercontent.com/openxlings/xlings/main/tools/other/quick_install.sh | bash -s v2026.8.7.1 # Deliberately NOT writing to $GITHUB_PATH here. On container # images that declare no PATH in their config (opensuse/ # tumbleweed), appending a single dir to GITHUB_PATH makes the @@ -363,7 +363,7 @@ jobs: # (older ones carry minos=15 and refuse to start). # v0.4.51+: in-process sha256 — this image has no sha256sum # binary, so pinned fetches failed before it. - curl -fsSL https://raw.githubusercontent.com/openxlings/xlings/main/tools/other/quick_install.sh | bash -s v2026.8.6.3 + curl -fsSL https://raw.githubusercontent.com/openxlings/xlings/main/tools/other/quick_install.sh | bash -s v2026.8.7.1 echo "$HOME/.xlings/subos/current/bin" >> "$GITHUB_PATH" - name: Install mcpp and config mirror diff --git a/.github/workflows/ci-linux-e2e.yml b/.github/workflows/ci-linux-e2e.yml index 790dbfd7..01996456 100644 --- a/.github/workflows/ci-linux-e2e.yml +++ b/.github/workflows/ci-linux-e2e.yml @@ -78,6 +78,16 @@ jobs: "$MCPP" self config # Pin the global default so test 28 (default-toolchain path) gets a # deterministic GNU answer instead of an auto-install pick. + # + # Installed explicitly, not assumed. `toolchain default` requires the + # toolchain to be present, and nothing in this job puts it there: + # building mcpp above uses whatever first-run picks (llvm). It worked + # only because the sandbox cache happened to carry a gcc some earlier + # run had installed -- so the moment the cache key changed (an xlings + # pin bump moves the whole prefix), every e2e shard failed with + # `gcc@16.1.0 is not installed` before running a single test. + # Warm runs re-install nothing; this costs a lookup. + "$MCPP" toolchain install gcc 16.1.0 "$MCPP" toolchain default gcc@16.1.0 # Warm musl once so fresh-home e2e tests inherit the payload. "$MCPP" toolchain install gcc 16.1.0-musl @@ -123,7 +133,7 @@ jobs: - name: Bootstrap xlings + released mcpp run: | - curl -fsSL https://raw.githubusercontent.com/openxlings/xlings/main/tools/other/quick_install.sh | bash -s v2026.8.6.3 + curl -fsSL https://raw.githubusercontent.com/openxlings/xlings/main/tools/other/quick_install.sh | bash -s v2026.8.7.1 export PATH="$HOME/.xlings/subos/current/bin:$PATH" xlings update xlings install mcpp -y -g diff --git a/.github/workflows/cross-build-test.yml b/.github/workflows/cross-build-test.yml index b9f0d053..51bdf9fe 100644 --- a/.github/workflows/cross-build-test.yml +++ b/.github/workflows/cross-build-test.yml @@ -118,7 +118,7 @@ jobs: # release assets were uploaded in a broken state (records present, # blobs missing → 404 on GET); re-uploaded clean. The stale-INDEX # half is handled by the marker-clear below. - XLINGS_VERSION: '2026.8.6.3' + XLINGS_VERSION: '2026.8.7.1' run: | tarball="xlings-${XLINGS_VERSION}-linux-x86_64.tar.gz" curl -fsSL -o "/tmp/${tarball}" \ @@ -255,7 +255,7 @@ jobs: - name: Bootstrap mcpp via xlings env: XLINGS_NON_INTERACTIVE: '1' - XLINGS_VERSION: '2026.8.6.3' + XLINGS_VERSION: '2026.8.7.1' run: | tarball="xlings-${XLINGS_VERSION}-linux-x86_64.tar.gz" curl -fsSL -o "/tmp/${tarball}" \ diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index d8327669..b9be200b 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -96,7 +96,7 @@ jobs: # Pin xlings to a known-good version. The upstream install # script always grabs `latest` (no version override), so we # download + self-install manually to avoid broken releases. - XLINGS_VERSION: '2026.8.6.3' + XLINGS_VERSION: '2026.8.7.1' run: | if [ ! -x "$HOME/.xlings/subos/default/bin/xlings" ]; then tarball="xlings-${XLINGS_VERSION}-linux-x86_64.tar.gz" @@ -288,7 +288,7 @@ jobs: - name: Bootstrap mcpp via xlings env: XLINGS_NON_INTERACTIVE: '1' - XLINGS_VERSION: '2026.8.6.3' + XLINGS_VERSION: '2026.8.7.1' run: | tarball="xlings-${XLINGS_VERSION}-linux-x86_64.tar.gz" curl -fsSL -o "/tmp/${tarball}" \ @@ -358,11 +358,11 @@ jobs: # below are pinned to the same version as XLINGS_VERSION; they are # NOT interpolated from it, so check_version_pins.sh scans for them # explicitly (they were absent from the old lock-step comment). - XLA="xlings-2026.8.6.3-linux-aarch64.tar.gz" + XLA="xlings-2026.8.7.1-linux-aarch64.tar.gz" if curl -fsSL -o "/tmp/$XLA" \ - "https://github.com/openxlings/xlings/releases/download/v2026.8.6.3/$XLA"; then + "https://github.com/openxlings/xlings/releases/download/v2026.8.7.1/$XLA"; then tar -xzf "/tmp/$XLA" -C /tmp - XLBIN=$(find /tmp/xlings-2026.8.6.3-linux-aarch64 -path '*/bin/xlings' -type f | head -1) + XLBIN=$(find /tmp/xlings-2026.8.7.1-linux-aarch64 -path '*/bin/xlings' -type f | head -1) if [ -n "$XLBIN" ]; then mkdir -p "$STAGING/$WRAPPER/registry/bin" cp "$XLBIN" "$STAGING/$WRAPPER/registry/bin/xlings" @@ -440,7 +440,7 @@ jobs: - name: Bootstrap mcpp via xlings env: XLINGS_NON_INTERACTIVE: '1' - XLINGS_VERSION: '2026.8.6.3' + XLINGS_VERSION: '2026.8.7.1' run: | if [ ! -x "$HOME/.xlings/subos/default/bin/xlings" ]; then WORK=$(mktemp -d) @@ -622,7 +622,7 @@ jobs: shell: bash env: XLINGS_NON_INTERACTIVE: '1' - XLINGS_VERSION: '2026.8.6.3' + XLINGS_VERSION: '2026.8.7.1' run: | # Captured before the `cd` below, in POSIX form: this step never # returns to the workspace, and GITHUB_WORKSPACE is a backslash diff --git a/CHANGELOG.md b/CHANGELOG.md index 6c262d43..2bfee02a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -3,6 +3,71 @@ > 本文件追踪 `mcpp-community/mcpp` 公开仓的版本演进。 > 格式参考 [Keep a Changelog](https://keepachangelog.com/zh-CN/1.1.0/)。 +## [2026.8.8.2] — 2026-08-08 + +编译器交付的是**能力**,不是配置。设计与实测证据见 +`.agents/docs/2026-08-08-payload-version-and-contract-drift-design.md`。 + +### 修复 + +- **产物加载哪个 glibc,现在有权威,不再靠目录顺序。** + + payload-first 的构建要链接到某个具体 glibc。旧规则是问目录"那个 glibc"、 + 取 `readdir` 的第一项——只装了一个时它永远对,所以从没有东西逼它对。一条带 + `xim:glibc@>=2.38` 的依赖就足以装进第二个:编译侧取了 2.44,而产物的 + interpreter(装机时冻结在 gcc specs 里)仍是 2.39,于是二进制引用 + `GLIBC_2.42` 符号却跑在没有它的运行时上,报错还落在与那条依赖无关的包上。 + + 现在由 subos 自述的 runtime 作答(`[xlings] subos` 可指定),没有答案就**拒绝** + 走 payload-first,而不是挑一个。该绑定计入工具链指纹(11 字段),两次只在 + runtime 上不同的构建不再共用缓存。旧 subos 回落到工具链自身烙入的值。 + +- **不再改写 GCC 的 `specs`,产物也不再带别人机器的死路径。** + + 旧的 specs 重写用单路径 needle 配双路径 replacement,每个跑过它的 home 都漏下 + 一条:一台开发机产出的每个 gcc 产物里都有 **68** 条陈旧 `RUNPATH`,全指向已删除 + 的 `mktemp` 目录。改为 `-specs=` 一份逐构建生成的干净文件(`-dumpspecs` 取内建 + `*link:`,去掉 loader/rpath),loader 与 rpath 由 mcpp 显式写在链接行上。逐构建、 + 不需写权限,继承来的只读 payload 也能用。e2e `201_gcc_no_specs_pollution.sh` + 断言的是产物而非 specs 文件。 + +- **落后于 pin 的 vendored xlings 会被替换——但只在替换品确实更新时。** + + `acquire_xlings_binary` 过去见到文件存在就返回,于是一个 home 会永远留着它第一次 + 获取的 xlings(实测 2026.8.2.1 对 pin 2026.8.6.3)。`subos_info` 是 2026.8.5.1 + 才有的,所以那台机器上 subos 的自述一直被一个太旧的客户端丢弃。修复的第一版直接 + 删了重取,结果把 2026.8.2.1 换成了系统的 0.4.51——更旧,且同样没有那个特性;现在 + 先给替换品定价再动手。 + +- **`--sysroot` 的判据从"存在"改为"归属"。** gcc 把 + `--sysroot=<...>/.xlings/subos/default` 当字符串烙进去,而一台机器上有很多同名 + 目录,所以那条烙入的路径经常存在、却属于另一个 checkout(实测:mcpp 里的构建解析 + 到了无关仓库下的 sysroot)。 + +### 其他 + +- `mcpp self doctor` 新增两条检查:vendored xlings 落后于 pin;sysroot 既不属于本 + mcpp home 也不属于当前项目。 +- xlings pin 提升到 2026.8.7.1。 + +## [2026.8.8.1] — 2026-08-07 + +xlings 作为运行时底座:subos 环境到达程序,以及 self-contained 的 +`/proc/self/exe` 陷阱(#352、#375)。设计见 +`.agents/docs/2026-08-07-xlings-as-runtime-substrate-design.md`。 + +### 修复 + +- **subos 声明的环境变量现在真的到达被运行的程序(#352)。** 之前 mcpp 读的是自己 + 想象出来的 `subos_info.envs` 线格式(数组),而 xlings 写的是以 binding 为键的 + 对象,于是整条修复是空转的。现在按 xlings 实际写出的格式解析,并用逐字取自真实 + xlings 输出的 fixture 钉住。 +- **升级后的 mcpp 会重放升级前的缓存,导致 subos 环境丢失。** 快路径命中时不带 + subos 环境;该缺陷是靠一条"必须走快路径"的自证断言抓到的,而那条断言随即又抓到 + 第二个:缓存行写在 `profile=` 之前却在 `cacheMode=` 之后解析。 +- **self-contained 打包的 `/proc/self/exe` 陷阱(#375)。** 新增 `MCPP_BUNDLE_DIR` + 契约。 + ## [2026.8.7.1] — 2026-08-07 两处「模型比生态少一层」。设计与实测证据见 `.agents/docs/2026-08-07-windows-resources-and-version-identity-design.md`。 diff --git a/docs/05-mcpp-toml.md b/docs/05-mcpp-toml.md index d72c7900..999ce66c 100644 --- a/docs/05-mcpp-toml.md +++ b/docs/05-mcpp-toml.md @@ -966,6 +966,17 @@ build needs (`make`/`cmake`/`protoc`/…), pin tool versions per project, or set build-time env vars — without hand-editing `.xlings.json`. `[toolchain]` (§2.7) remains the ergonomic shorthand for the compiler; `[xlings.workspace]` is the general form. +`subos` carries a second meaning on Linux: it selects **which C runtime the +build binds against**. A subos describes its own runtime (xlings 2026.8.5.1+), +and mcpp takes that as the authority rather than looking around for a libc — so +`subos = "el8"` and `subos = "trixie"` in two projects on one machine produce +artifacts targeting each one's glibc, with the compile side and the run side +guaranteed to agree. The binding is part of the toolchain fingerprint, so +switching it rebuilds rather than reusing the other subos' objects. A subos +older than that (or none at all) falls back to whatever the toolchain itself +was installed against; `mcpp build -v` prints which of the two happened. See +docs/08-toolchain-internals.md §2.1. + ### 2.14 Host tools from a dependency (mcpp 2026.8.5.1+) A package can build a binary its consumers need *at build time* — `protoc`, a diff --git a/docs/08-toolchain-internals.md b/docs/08-toolchain-internals.md index f4bdcb1b..6a2eb4d4 100644 --- a/docs/08-toolchain-internals.md +++ b/docs/08-toolchain-internals.md @@ -15,6 +15,9 @@ resolve payload (xim:gcc / xim:llvm / xim:musl-gcc xpkg under the sandbox) ▼ ensure_post_install_fixup() ← idempotent convergence (marker-gated) ▼ +resolve runtime binding ← which libc the artifact will load (§2.1) — an + answer, not a search + ▼ detect / probe ← triple, sysroot, payload paths (glibc, linux-headers) ▼ ToolchainLinkModel (single resolver for the C-library axis) @@ -65,12 +68,44 @@ is resolved/auto-installed via the xlings backend into the sandbox |---|---| | `targetTriple` | ` -dumpmachine` | | `sysroot` | `-print-sysroot` (validated: must actually carry libc headers), with a remap fallback for xlings-built GCC whose baked build-time path doesn't exist locally | -| `payloadPaths` | sibling xpkg discovery: glibc payload (`include/` + `lib64|lib/`) and linux-headers payload — the *payload-first* fine-grained sysroot | +| `payloadPaths` | the resolved runtime binding (§2.1) names the glibc payload exactly; linux-headers is still discovered as a sibling. No binding ⇒ no payload-first, by design | | runtime dirs | toolchain-private lib dirs for produced binaries' `-L`/`-rpath` | Note the probe deliberately does **not** mine the clang cfg for `--sysroot` anymore: the cfg is an output of this machinery, not an input (§5). +### 2.1 The runtime binding — which libc, decided once + +A payload-first build links against a specific glibc, and *which* one is a +fact about the environment, not something to be inferred. mcpp resolves it in +order: + +1. `[xlings] subos = ""` — the named subos, a sibling of the active one, + describes its own runtime in the `subos_info` block of its `.xlings.json` + (xlings 2026.8.5.1+). +2. The active subos, same block. +3. *Compatibility.* A subos created before that block existed cannot answer. + The value baked into the toolchain itself then stands in — gcc's specs, + clang's cfg. This is the value the artifact **would** load, so compile side + and run side still agree; it retires itself the moment the subos can speak. + +No binding is a **refusal**, not a default: `CLibMode::PayloadFirst` is +declined rather than picking a libc. + +The rule this replaced asked a directory for "the glibc" and took the first +entry `readdir` yielded. With one glibc installed that is always right, so +nothing forced it to be correct — and a dependency carrying `xim:glibc@>=2.38` +is enough to install a second one. When that happened in mcpp-index, the +compile side took 2.44 while the artifact's interpreter, frozen in gcc's specs +at install time, still named 2.39; binaries referenced `GLIBC_2.42` symbols +against a runtime without them, and the failures surfaced on packages +unrelated to the dependency that pulled the second glibc in. Directory order +is not a decision procedure. + +Because the binding decides what the artifact loads, it is part of the +toolchain fingerprint (11 fields, not 10) — two builds differing only in +runtime must not share a cache entry. + ## 3. The link model (`src/toolchain/linkmodel.cppm`) `ToolchainLinkModel` answers exactly one question — *how do we compile and @@ -86,6 +121,10 @@ CLibMode::PayloadFirst glibc/linux-headers xpkgs found (the normal bundled-LLV -L [+ -rpath + --dynamic-linker for clang] CLibMode::Sysroot a usable --sysroot (GCC include-fixed world, self-contained musl sysroots, the macOS SDK) + link: --sysroot, plus --dynamic-linker and -L/-rpath + for the payload when one is known — the + sysroot says where headers live, not which + loader runs the result CLibMode::None nothing usable — host defaults apply; the hermetic check (§6) rejects that leakage unless an explicit host-library exception is in effect @@ -110,12 +149,14 @@ itself as its first consumer; mcpp can then re-add a reader. ## 4. The unified post-install fixup pipeline (`src/toolchain/post_install.cppm`) -Sandbox payloads are prebuilt ELF trees. Two kinds of paths baked into them -are unknowable at packaging time and must be aligned to the *local* sandbox: -`PT_INTERP`/`RUNPATH` inside binaries, and the loader/rpath lines inside GCC -specs. `ensure_post_install_fixup(cfg, payloadRoot, pkg)` is the **single -entry** for that alignment, called from all three entry paths (explicit -install, default auto-install, manifest auto-install). +Sandbox payloads are prebuilt ELF trees whose baked `PT_INTERP`/`RUNPATH` are +unknowable at packaging time and must be aligned to the *local* sandbox. +`ensure_post_install_fixup(cfg, payloadRoot, pkg)` is the **single entry** for +that alignment, called from all three entry paths (explicit install, default +auto-install, manifest auto-install). + +> This pipeline used to rewrite GCC's `specs` file as well, so that produced +> binaries got a loader and an rpath. That is no longer done, and §5 is why. > Historical note: before 0.0.83 each path remembered — or forgot — its own > subset. The manifest path ran *nothing*, which is how a freshly @@ -146,7 +187,7 @@ mcpp's sight — trust-but-verify is the only reliable semantic. | kind | actions | |---|---| -| `gcc` (glibc) | patchelf walk over the gcc payload **and the shared binutils payload** (PT_INTERP → sandbox loader, RUNPATH → glibc+gcc lib dirs); specs rewrite (baked loader/rpath → payload glibc, specs-grammar-aware — `%{...}` conditionals must never be corrupted) | +| `gcc` (glibc) | patchelf walk over the gcc payload **and the shared binutils payload** (PT_INTERP → sandbox loader, RUNPATH → glibc+gcc lib dirs) — so that *gcc itself* runs. Nothing is written into `specs` | | `llvm` | patchelf walk over `lib/` only (runtime `.so` RUNPATH; `bin/` is left alone to preserve xlings-set RUNPATHs); deterministic cfg regeneration (§5) | | `musl-gcc` | nothing — self-contained sysroot, static world | @@ -162,8 +203,7 @@ mcpp's sight — trust-but-verify is the only reliable semantic. (symlink-inherited from another `MCPP_HOME`) are never patched — their owner already converged them, and patching through the symlink would brick the owner's toolchain. -- Specs rewriting is content-aware (already-aligned specs are skipped). - Extending the same check to the patchelf walk (compare +- Extending content-awareness to the patchelf walk (compare `--print-interpreter`/`--print-rpath` before writing, so an already-aligned payload converges with **zero writes**) is a known follow-up. - The long-term direction is for the *installer* (xlings) to own all @@ -171,11 +211,42 @@ mcpp's sight — trust-but-verify is the only reliable semantic. mcpp read-only + verification. The pipeline here is the compatibility layer until then, and the self-healing mechanism for drift either way. -## 5. The clang cfg: direct-invocation support only +## 5. The compiler is a capability, not a configuration + +A payload ships two separable things: the ability to compile, and an opinion +about how to link. mcpp wants the first and supplies the second itself — the +link line is where a build's decisions belong, because it is the thing that +varies per build. Both compilers now follow that rule; only the mechanism +differs. + +**clang** — `--no-default-config` on every mcpp invocation. + +**gcc** — `-specs=` with a generated file. ` -dumpspecs` prints the +**built-in** specs (unaffected by anything on disk); mcpp extracts the `*link:` +body, drops the loader and rpath lines from it, and writes the result to the +build directory. A `-specs=` file without a leading `+` *replaces* the rule it +names, so the payload's own opinion is overridden without the payload being +touched. Two consequences worth stating: it is per-build, so a second project +on the same machine is unaffected; and it needs no write access to the +toolchain, so an inherited or read-only payload works. + +Removing gcc's baked `*link:` also removes what it provided. mcpp therefore +supplies `--dynamic-linker` and every rpath entry explicitly on the link +line — the loader, the glibc lib dir, and gcc's own `lib64` (libgcc_s). Each +of those was found by removing the specs and watching what broke. + +Why not keep rewriting `specs`? Because the file is shared and the value is +per-build. The rewrite had a single-path needle and a two-path replacement, so +every home that ever ran it left one entry behind: **68** stale `RUNPATH` +entries, all naming deleted `mktemp` directories, in every gcc artifact one +developer machine produced. Nothing detected them, because a dead RUNPATH +entry costs only search time. e2e `201_gcc_no_specs_pollution.sh` asserts on +the artifact, not the specs file — what a user ships is what matters. + +### 5.1 The clang cfg `bin/clang++.cfg` exists so that direct invocations of the bundled -`clang++` (outside mcpp) get a working, hermetic compiler configuration. mcpp's own builds never read it -(`--no-default-config` always). The fixup pipeline **regenerates** it +`clang++` (outside mcpp) get a working, hermetic compiler configuration. The fixup pipeline **regenerates** it deterministically from the link model — same payload ⇒ byte-identical cfg on every machine and install path — rather than line-patching whatever an install produced. On Linux that means CRT discovery (`-B`), payload loader + diff --git a/docs/zh/05-mcpp-toml.md b/docs/zh/05-mcpp-toml.md index 971cf450..c8c109b0 100644 --- a/docs/zh/05-mcpp-toml.md +++ b/docs/zh/05-mcpp-toml.md @@ -724,6 +724,13 @@ OPENBLAS_NUM_THREADS = "1" host 工具(`make`/`cmake`/`protoc`…)、按项目固定工具版本、或设构建期环境变量——无需手改 `.xlings.json`。`[toolchain]`(§2.7)仍是编译器的便捷简写;`[xlings.workspace]` 是其通用形式。 +在 Linux 上 `subos` 还有第二重含义:它决定**构建绑定到哪个 C 运行时**。subos 会自述其 +runtime(xlings 2026.8.5.1 起),mcpp 以此为权威,而不是去四处找一个 libc——所以同一台机器 +上 `subos = "el8"` 与 `subos = "trixie"` 两个项目各自产出面向自己 glibc 的产物,且编译期与 +运行期保证一致。该绑定计入工具链指纹,故切换它会重新构建,而不会复用另一个 subos 的目标文件。 +更旧的 subos(或没有 subos)则回落到工具链自身安装时所对应的运行时;`mcpp build -v` 会打印 +实际走了哪一条。参见 docs/08-toolchain-internals.md §2.1。 + ### 2.14 依赖产出的 host 工具(mcpp 2026.8.5.1+) 一个包能构建出消费者在**构建期**需要的二进制 —— `protoc`、`grpc_cpp_plugin`、 diff --git a/docs/zh/08-toolchain-internals.md b/docs/zh/08-toolchain-internals.md index 4a61a326..b6b41b83 100644 --- a/docs/zh/08-toolchain-internals.md +++ b/docs/zh/08-toolchain-internals.md @@ -14,6 +14,8 @@ mcpp.toml [toolchain] / 全局默认 / `mcpp toolchain install` ▼ ensure_post_install_fixup() ← 幂等收敛(marker 闸门) ▼ +解析 runtime binding ← 产物将加载哪个 libc(§2.1)——是答案,不是搜索 + ▼ detect / probe ← triple、sysroot、payload 路径(glibc、linux-headers) ▼ ToolchainLinkModel(C 库轴的唯一解析器) @@ -56,9 +58,34 @@ xlings 后端解析/自动安装到沙箱 |---|---| | `targetTriple` | ` -dumpmachine` | | `sysroot` | `-print-sysroot`(校验必须真带 libc 头);xlings 构建的 GCC 烙的是构建机路径,有 remap 回退 | -| `payloadPaths` | 兄弟 xpkg 发现:glibc payload(`include/` + `lib64|lib/`)与 linux-headers payload——即 *payload 优先* 的细粒度 sysroot | +| `payloadPaths` | 由解析出的 runtime binding(§2.1)**精确指名** glibc payload;linux-headers 仍按兄弟 xpkg 发现。没有 binding 就不走 payload-first——这是设计,不是缺陷 | | 运行库目录 | 工具链私有 lib 目录,用于产物的 `-L`/`-rpath` | +### 2.1 runtime binding:绑哪个 libc,只决定一次 + +payload-first 的构建会链接到某个具体的 glibc,而**是哪一个**是关于环境的事实, +不该靠推断。mcpp 按顺序解析: + +1. `[xlings] subos = ""` —— 该 subos(活动 subos 的兄弟)在自己 + `.xlings.json` 的 `subos_info` 块里自述 runtime(xlings 2026.8.5.1 起)。 +2. 活动 subos,同一个块。 +3. *兼容路径。* 在该块出现之前创建的 subos 无法作答,此时以工具链自身烙入的值 + 顶上——gcc 的 specs、clang 的 cfg。这正是产物**将会**加载的那个值,故编译期 + 与运行期仍然一致;一旦 subos 能作答,这条自动退场。 + +没有 binding 是**拒绝**而不是取默认值:`CLibMode::PayloadFirst` 会被放弃,而不是 +去挑一个 libc。 + +被它取代的旧规则是:向某个目录问"那个 glibc",然后拿 `readdir` 吐出的第一项。 +只装了一个 glibc 时它永远正确,所以从来没有东西逼它正确——而一条带 +`xim:glibc@>=2.38` 的依赖就足以装进第二个。这在 mcpp-index 上真实发生过:编译侧 +取了 2.44,而产物的 interpreter(装机时冻结在 gcc specs 里)仍指向 2.39;二进制 +引用了 `GLIBC_2.42` 符号,却跑在没有这些符号的运行时上,且报错落在与"拉进第二个 +glibc 的那条依赖"毫无关系的包上。目录顺序不是决策依据。 + +因为 binding 决定产物加载什么,它计入工具链指纹(11 个字段,不是 10 个)——只在 +runtime 上不同的两次构建绝不能共用同一条缓存。 + 注意:probe 已**不再**从 clang cfg 挖 `--sysroot`——cfg 是这套机制的输出, 不是输入(见 §5)。 @@ -95,12 +122,15 @@ CLibMode::None 无可用来源——落宿主默认;除非显式允许 ## 4. 统一 post-install fixup 管线(`src/toolchain/post_install.cppm`) -沙箱 payload 是预编译 ELF 树。有两类打包期不可能预知、必须对齐到**本机沙箱**的 -路径:二进制里的 `PT_INTERP`/`RUNPATH`,以及 GCC specs 里的 loader/rpath 行。 +沙箱 payload 是预编译 ELF 树,其烙入的 `PT_INTERP`/`RUNPATH` 在打包期不可能预知, +必须对齐到**本机沙箱**。 `ensure_post_install_fixup(cfg, payloadRoot, pkg)` 是这次对齐的**唯一入口**, 由三条入口路径(显式 install、默认 auto-install、manifest auto-install)共同 调用。 +> 本管线曾同时重写 GCC 的 `specs` 文件,以便产物拿到 loader 和 rpath。现已不再 +> 这样做,原因见 §5。 + > 历史注记:0.0.83 之前各路径各自记得——或忘记——自己那份 fixup。manifest > 路径什么都不跑:刚 auto-install 的 llvm 因此保留着陈旧、随装机环境漂移的 > cfg(issue #195);gcc 也曾因此产出找不到 `stdlib.h` 的沙箱。"用哪条命令装的" @@ -125,7 +155,7 @@ glibc、payload 从别的 home 继承而来)发生在 mcpp 视野之外,trust-bu | kind | 动作 | |---|---| -| `gcc`(glibc)| 对 gcc payload **及共享的 binutils payload** 做 patchelf 遍历(PT_INTERP → 沙箱 loader,RUNPATH → glibc+gcc lib);specs 重写(烙入的 loader/rpath → payload glibc,**必须感知 specs 语法**——`%{...}` 条件块绝不能被破坏)| +| `gcc`(glibc)| 对 gcc payload **及共享的 binutils payload** 做 patchelf 遍历(PT_INTERP → 沙箱 loader,RUNPATH → glibc+gcc lib)——目的是让 *gcc 自己* 能跑起来。不往 `specs` 里写任何东西 | | `llvm` | 只遍历 `lib/`(运行库 `.so` 的 RUNPATH;`bin/` 不碰,保留 xlings 设置的 RUNPATH);确定性再生 cfg(§5)| | `musl-gcc` | 无——自包含 sysroot,静态世界 | @@ -137,17 +167,43 @@ glibc、payload 从别的 home 继承而来)发生在 mcpp 视野之外,trust-bu `rename` 让新内容拿新 inode,活进程保有旧 inode。 - **所有权护栏。** 解析到本 home registry 之外的 payload(从别的 `MCPP_HOME` symlink 继承)一律不碰——属主已收敛过,隔着 symlink 改写会毁掉属主的工具链。 -- specs 重写是内容感知的(已对齐即跳过)。把同样的检查扩展到 patchelf 遍历 +- 把内容感知扩展到 patchelf 遍历 (写前比对 `--print-interpreter`/`--print-rpath`,已对齐的 payload **零写入** 收敛)是已知的后续项。 - 长期方向:全部写入由**安装器**(xlings)持有——装机时、以及 payload 进入 新 home 时——mcpp 退为只读 + 校验。本管线在那之前是兼容层,也是双向漂移的 自愈机制。 -## 5. clang cfg:仅服务直接调用场景 +## 5. 编译器是能力,不是配置 + +一个 payload 交付两样可分离的东西:编译的**能力**,以及关于如何链接的**主张**。 +mcpp 要前者,后者自己给——链接行才是构建决策该待的地方,因为它是逐次构建而变的 +那一个。现在两个编译器都遵循这条规则,只是机制不同。 + +**clang** —— mcpp 每次调用都带 `--no-default-config`。 + +**gcc** —— 用生成的文件走 `-specs=`。` -dumpspecs` 打印的是**内建** +specs(不受磁盘上任何文件影响);mcpp 从中取出 `*link:` 的正文,删掉其中的 loader +与 rpath 行,把结果写进构建目录。不带前导 `+` 的 `-specs=` 文件是**替换**它所命名的 +规则,于是 payload 自己的主张被覆盖,而 payload 本身分毫未动。两条推论值得写明: +它是逐构建的,所以同机器上的另一个项目不受影响;它不需要对工具链的写权限,所以 +继承来的、或只读的 payload 都能用。 + +删掉 gcc 烙入的 `*link:`,也就删掉了它提供的东西。因此 mcpp 在链接行上显式补齐 +`--dynamic-linker` 与每一条 rpath——loader、glibc lib 目录、以及 gcc 自己的 +`lib64`(libgcc_s)。这三条都是把 specs 拿掉后、看什么先坏掉而逐一找出来的。 + +为什么不继续重写 `specs`?因为那个文件是共享的,而值是逐构建的。旧的重写用的是 +单路径的 needle 配双路径的 replacement,于是每个跑过它的 home 都会漏下一条:一台 +开发机产出的**每一个** gcc 产物里都带着 **68** 条陈旧 `RUNPATH`,全部指向已被删除的 +`mktemp` 目录。没有任何东西发现它们,因为一条死 RUNPATH 只多花一点搜索时间。e2e +`201_gcc_no_specs_pollution.sh` 断言的是**产物**而不是 specs 文件——用户最终交付的 +是产物。 + +### 5.1 clang cfg `bin/clang++.cfg` 的职责是:直接调用打包内 `clang++`(不经由 mcpp)时, -获得可用且 hermetic 的编译器配置。mcpp 自己的构建从不读它(永远 `--no-default-config`)。 +获得可用且 hermetic 的编译器配置。 fixup 管线从链接模型**确定性再生**它——同一 payload ⇒ 任何机器、任何安装路径 产出字节一致的 cfg——而不是对装机产物做行级补丁。Linux 上内容为:CRT 发现 (`-B`)、payload loader + rpath、lld/compiler-rt/libunwind、C++ 驱动附加 diff --git a/mcpp.toml b/mcpp.toml index d2a6dbb6..60309c3d 100644 --- a/mcpp.toml +++ b/mcpp.toml @@ -1,6 +1,6 @@ [package] name = "mcpp" -version = "2026.8.8.1" +version = "2026.8.8.2" description = "Modern C++ build & package management tool" license = "Apache-2.0" authors = ["mcpp-community"] diff --git a/src/build/execute.cppm b/src/build/execute.cppm index b4cb5c1e..b572f967 100644 --- a/src/build/execute.cppm +++ b/src/build/execute.cppm @@ -18,6 +18,7 @@ import mcpp.build.ninja; import mcpp.bmi_cache; import mcpp.manifest; import mcpp.modgraph.scanner; +import mcpp.toolchain.post_install; import mcpp.toolchain.stdmod; import mcpp.xlings; import mcpp.xlings.subos_info; @@ -68,6 +69,11 @@ struct BuildCacheEntry { // property and would otherwise be unknowable on the fast path -- which // has no toolchain to derive it from. std::string subosDir; + // Was the line present at all? An EMPTY subosDir is a legitimate answer + // (a system toolchain outside the xpkgs store has no subos), so it cannot + // stand in for "this cache predates the field" -- and those two need + // opposite treatment: the first runs, the second must rebuild once. + bool subosRecorded = false; // The resolved profile this entry was built for. Entries used to be keyed // by target triple alone, and the fast paths only refuse to run when an // EXPLICIT --profile/--dev/--release is passed — so a bare `mcpp build` @@ -154,7 +160,8 @@ std::vector read_build_cache(const std::filesystem::path& proje // using the cache at all -- the program would work once and then // silently stop finding its runtime data. if (haveNextLine && line.starts_with("subos=")) { - e.subosDir = line.substr(6); + e.subosDir = line.substr(6); + e.subosRecorded = true; haveNextLine = static_cast(std::getline(f, line)); } // Optional profile line. Same back-compat contract as the two blocks @@ -201,7 +208,8 @@ void write_build_cache(const std::filesystem::path& projectRoot, // Insert at front (MRU). BuildCacheEntry newEntry{targetTriple, outputDir.string(), ninjaProgram, fingerprintHex, runtimeEnvKey, runtimeEnvValue, std::move(runTargets), - runEnvKey, runEnvValue, subosDir, profile, cacheMode}; + runEnvKey, runEnvValue, subosDir, /*subosRecorded=*/true, + profile, cacheMode}; entries.insert(entries.begin(), std::move(newEntry)); // Trim to LRU capacity. @@ -369,6 +377,23 @@ export int run_build_plan(BuildContext& ctx, bool verbose, bool no_cache, std::filesystem::remove_all(ctx.outputDir, ec); } + // The generated `*link:` spec lives in the output directory, and the line + // above is allowed to delete that directory. prepare wrote the file before + // this point, so a cold build reached ninja with a link command naming a + // spec that no longer existed -- `g++: fatal error: cannot read spec file`, + // on every `--no-cache` build with gcc. + // + // Regenerating here rather than reordering: the invariant worth holding is + // "the spec exists when ninja runs", and stating it as an invariant + // survives the next thing that clears target/ (a user with `rm -rf`, for + // one). The write is idempotent, so the warm path costs one stat. + if (!ctx.plan.gccCleanSpecs.empty()) { + std::error_code ec; + if (!std::filesystem::exists(ctx.plan.gccCleanSpecs, ec)) + ctx.plan.gccCleanSpecs = mcpp::toolchain::write_clean_link_specs( + ctx.tc.binaryPath, ctx.outputDir); + } + auto be = mcpp::build::make_ninja_backend(); // M5.0: print "Inferred" banner when defaults / target inference fired. @@ -774,6 +799,18 @@ std::optional try_fast_run(const std::filesystem::path& projectRoot, ninjaProgram = ninjaProgram.substr(1, ninjaProgram.size() - 2); if (match->runtimeEnvKey.empty()) return std::nullopt; // old cache entry; go through prepare_build once + // Written before this mcpp knew about subos environments (mcpp#352). Taking + // the fast path here would run the program without them -- which is the + // defect this field exists to fix, surviving an upgrade. + // + // It survives it for a long time, too: the fast path's identity is the + // profile, the cache mode and the resource list, and its fingerprint check + // compares a cached entry against ITSELF. Neither notices that a different + // mcpp wrote the entry, so without this line an upgraded mcpp would reuse a + // pre-upgrade build until something else happened to invalidate it. Measured + // on a real upgrade from 2026.8.7.1, not reasoned about. + if (!match->subosRecorded) + return std::nullopt; // predates `subos=`; rebuild once, then it is there // P1: verify fingerprint matches the outputDir basename. if (!match->fingerprint.empty()) { diff --git a/src/build/flags.cppm b/src/build/flags.cppm index de26950e..40fa3fbe 100644 --- a/src/build/flags.cppm +++ b/src/build/flags.cppm @@ -712,9 +712,37 @@ CompileFlags compute_flags(const BuildPlan& plan) { // host's /lib or, on hosts without a system toolchain, passes bare names // that lld cannot open — issue #195), -L/-rpath for -lc/-lm, and the // payload's dynamic linker. + // + // Only for clang-with-cfg, and only in PayloadFirst. Every other + // combination already has these flags: the gcc branch above assigns + // `link_toolchain_flags = lm.link_flags(...)` for any mode but None, and + // the clang branch adds them for Sysroot. Emitting them here as well put + // the whole C-runtime group on the line twice -- harmless to correctness, + // but the link line has a hard 128KiB ceiling (MAX_ARG_STRLEN) that real + // workspaces already spend 43% of. std::string payload_ld; - if (isClangWithCfg && lm.mode == mcpp::toolchain::CLibMode::PayloadFirst) + if (isClangWithCfg + && lm.mode == mcpp::toolchain::CLibMode::PayloadFirst) payload_ld = lm.link_flags(ninjaEsc); + // GCC: replace the payload's patched `*link:` with the pristine one, so + // its accumulated rpath entries do not reach the artifact. Must come + // BEFORE our own -Wl flags is not required (specs are processed by the + // driver, not positionally against -Wl), but keeping it adjacent to the + // payload flags keeps the C-runtime decisions in one place on the line. + // + // Quoted the way include_token quotes: `-specs=` and the path are ONE argv + // word, so the prefix is joined first and the whole token quoted after -- + // quoting only the path would put the opening quote in the wrong place. + // `escape_path` alone is not enough here: it adds ninja's `$` escapes and + // nothing else, while ninja hands the command to `sh -c`. This path is the + // only payload flag that lives under the PROJECT rather than the payload, + // so it is the only one a spaced project directory splits: + // `cannot read spec file '/tmp/tmp.XXX/my'` for a project in `my project` + // (e2e 179, on CI -- no local path here has a space). + if (!plan.gccCleanSpecs.empty()) + payload_ld = " " + shell_quote_arg(escape_ninja_chars( + "-specs=" + plan.gccCleanSpecs.string())) + + payload_ld; std::string link_extra; if (prof.lto) link_extra += " -flto"; diff --git a/src/build/plan.cppm b/src/build/plan.cppm index 0f1daa91..45a30fa9 100644 --- a/src/build/plan.cppm +++ b/src/build/plan.cppm @@ -112,6 +112,11 @@ struct BuildPlan { // (possibly read-only) registry directory, and deriving the path would put // an IDE database there. Empty → projectRoot, the historical default. std::filesystem::path compileDbPath; + // GCC only: a specs file that replaces the pristine `*link:`, so the + // payload's own (patched by every home that ever installed against it) + // cannot inject rpath entries into this build's artifacts. Empty for + // clang, which bypasses its cfg with --no-default-config instead. + std::filesystem::path gccCleanSpecs; std::filesystem::path stdBmiPath; // absolute path to prebuilt std.gcm std::filesystem::path stdObjectPath; // absolute path to prebuilt std.o std::filesystem::path stdCompatBmiPath; // absolute path to prebuilt std.compat.pcm diff --git a/src/build/prepare.cppm b/src/build/prepare.cppm index a622ac8b..0d61b018 100644 --- a/src/build/prepare.cppm +++ b/src/build/prepare.cppm @@ -43,6 +43,8 @@ import mcpp.build.ninja; // make_ninja_backend — driving that sub-build import mcpp.lockfile; import mcpp.config; import mcpp.xlings; +import mcpp.xlings.subos_info; +import mcpp.toolchain.post_install; import mcpp.platform; import mcpp.fetcher; import mcpp.fetcher.progress; @@ -907,9 +909,88 @@ std::string with_index_cause(std::string msg) { msg += "\n" + hint; return msg; } +// The runtime this build targets, in xlings's own spelling ("glibc@2.39"). +// +// A degradation chain with every step explicit, and deliberately NO final +// "otherwise pick something". Payload resolution declines without an +// authority, because the guess it used to make is what let the compile side +// and the artifact's interpreter name different glibc versions -- invisibly, +// until the binary met a library built against the other one. +// +// 1. [xlings] subos -> that subos's subos_info.runtime +// 2. the active subos's subos_info.runtime +// 3. "" -- no authority, therefore no PayloadFirst +// +// `compilerBin` may be empty on the first call: the compiler has not been +// probed yet, and an inherited toolchain resolves its subos from its OWNER +// home. The caller re-resolves once it knows where the compiler is. +std::string resolve_runtime_binding(const mcpp::manifest::Manifest& m, + const std::filesystem::path& compilerBin) { + auto runtime_of = [](const std::filesystem::path& dir) -> std::string { + return mcpp::xlings::subos::read(dir).runtime; + }; + // A declaration can only be honoured while the payload it names is still + // installed. Subos descriptions are written once and the payloads beneath + // them are replaced independently -- upgraded, garbage-collected -- so a + // subos can go on saying `glibc@2.39` on a machine that now has only 2.44. + // + // Taking that at face value is not conservative, it is worse than any + // substitution: the exact-match probe finds nothing, no loader reaches the + // link line, and the artifact ends up on the HOST loader -- outside the + // sandbox entirely. Measured on CI, three rounds running. + // + // So a declaration naming an absent payload is passed over rather than + // returned, and resolution carries on to something that can be honoured. + auto usable = [&](const std::string& binding) { + if (binding.empty()) return false; + if (compilerBin.empty()) return true; // cannot check yet; caller re-asks + if (mcpp::toolchain::probe_payload_paths(compilerBin, binding)) + return true; + mcpp::log::verbose("probe", std::format( + "subos declares runtime {}, but that payload is not installed here " + "— looking for one that is", binding)); + return false; + }; + if (auto active = mcpp::xlings::paths::subos_dir_of(compilerBin)) { + // 1 — the project names a subos; it is a sibling of the active one. + if (!m.xlings.subos.empty()) { + auto named = active->parent_path() / m.xlings.subos; + if (auto r = runtime_of(named); usable(r)) return r; + } + // 2 — whatever is active. + if (auto r = runtime_of(*active); usable(r)) return r; + } + + // 3 — COMPATIBILITY: the value this toolchain already has baked in. + // + // A subos created before xlings grew `subos_info` cannot answer, and that + // is the state of every machine installed before 2026.8.5.1 -- including + // mcpp's own sandbox, whose vendored xlings is never upgraded. Refusing + // there would break every existing user, so the toolchain's own baked + // value stands in. + // + // This is NOT the guess this design removed. The guess picked a version by + // directory order, unrelated to what the artifact would load. This reads + // the value the artifact WILL use -- gcc's specs, clang's cfg -- so the + // invariant that matters, compile side == run side, still holds. It is a + // migration path, and it goes away on its own: once the subos describes + // itself, step 1 or 2 answers first. + if (!compilerBin.empty()) { + if (auto r = mcpp::toolchain::baked_runtime_binding(compilerBin); + !r.empty()) { + mcpp::log::verbose("probe", std::format( + "subos does not describe itself; using the runtime this " + "toolchain was installed against ({}). `xlings self update` " + "and a fresh subos make this authoritative", r)); + return r; + } + } + return {}; +} } // namespace export std::expected + prepare_build(bool print_fingerprint, bool includeDevDeps = false, std::vector extraTargets = {}, @@ -1641,8 +1722,27 @@ prepare_build(bool print_fingerprint, tcOrigin = TcOrigin::FirstRun; } - auto tc = mcpp::toolchain::detect(explicit_compiler); + // The authority for "which libc", resolved before the toolchain is + // probed: payload paths are addresses for a version this names. + auto runtimeBinding = resolve_runtime_binding(*m, {}); + auto tc = mcpp::toolchain::detect(explicit_compiler, runtimeBinding); if (!tc) return std::unexpected(tc.error().message); + // The first pass has no compiler, so it can neither discover a binding + // that only the compiler's own home knows (an inherited toolchain resolves + // into its owner) nor check that a declared one is still installed. Both + // failures look identical from here: no payload paths. Re-ask with the + // compiler in hand whenever that is what happened. + // + // The old condition was `runtimeBinding.empty()`, which meant a subos + // naming a payload that had since been upgraded away won on the first pass + // and the second pass never ran. + if (!tc->payloadPaths) { + auto fromCompiler = resolve_runtime_binding(*m, tc->binaryPath); + if (!fromCompiler.empty() && fromCompiler != runtimeBinding) { + tc = mcpp::toolchain::detect(explicit_compiler, fromCompiler); + if (!tc) return std::unexpected(tc.error().message); + } + } // ── Targeting the MSVC ABI without a usable MSVC ───────────────────── // @@ -1737,7 +1837,7 @@ prepare_build(bool print_fingerprint, tcSpec = std::string(pins::kFirstRunWinGnu); tcOrigin = TcOrigin::FirstRun; - tc = mcpp::toolchain::detect(explicit_compiler); + tc = mcpp::toolchain::detect(explicit_compiler, runtimeBinding); if (!tc) return std::unexpected(tc.error().message); } @@ -4902,6 +5002,14 @@ prepare_build(bool print_fingerprint, if (!planResult) return std::unexpected(planResult.error()); ctx.plan = std::move(*planResult); ctx.plan.compileDbPath = workRoot / "compile_commands.json"; + // GCC: a clean `*link:` for this build, so the payload's specs cannot + // inject other homes' rpath entries into the artifact. AFTER the plan is + // moved in — an earlier assignment was silently overwritten by that move, + // which produced a generated file that nothing ever passed to the driver. + // Generated here rather than in compute_flags, which runs twice per build. + if (tc->compiler == mcpp::toolchain::CompilerId::GCC) + ctx.plan.gccCleanSpecs = mcpp::toolchain::write_clean_link_specs( + tc->binaryPath, ctx.outputDir); // ── Declared build-graph nodes → the plan ─────────────────────────────── // diff --git a/src/config.cppm b/src/config.cppm index 129a7419..cd49375a 100644 --- a/src/config.cppm +++ b/src/config.cppm @@ -600,7 +600,8 @@ std::expected load_or_init( // 6. Acquire xlings binary if needed if (cfg.xlingsBinaryMode == "bundled") { - auto xbin = mcpp::fallback::acquire_xlings_binary(cfg.xlingsBinary, quiet); + auto xbin = mcpp::fallback::acquire_xlings_binary( + cfg.xlingsBinary, quiet, kXlingsPinnedVersion); if (!xbin) return std::unexpected(ConfigError{xbin.error()}); } else if (cfg.xlingsBinaryMode == "system") { auto sysPath = mcpp::platform::fs::which( diff --git a/src/doctor.cppm b/src/doctor.cppm index 034c1d02..3dc1ce88 100644 --- a/src/doctor.cppm +++ b/src/doctor.cppm @@ -14,6 +14,8 @@ import mcpp.bmi_cache.maintenance; import mcpp.build.prepare; import mcpp.build.plan; import mcpp.config; +import mcpp.fallback.probe_sysroot; +import mcpp.fallback.xlings_binary; import mcpp.fallback.install_integrity; import mcpp.fetcher.progress; import mcpp.home; @@ -69,6 +71,7 @@ export std::vector parse_readelf_runpath(std::string_view dump) { // `mcpp self env`. export int env_report() { auto cfg = mcpp::config::load_or_init(/*quiet=*/false, mcpp::fetcher::make_bootstrap_progress_callback()); + if (!cfg) { mcpp::ui::error(cfg.error().message); return 4; } mcpp::config::print_env(*cfg); @@ -98,6 +101,7 @@ export int doctor_report() { err(std::format("toolchain detection failed: {}", tc.error().message)); } else { ok(std::format("{} at {}", tc->label(), tc->binaryPath.string())); + } // Windows: report the system MSVC (msvc@system). Absence is a warning, @@ -204,6 +208,28 @@ export int doctor_report() { mcpp::ui::status("Checking", "registry"); auto cfg = mcpp::config::load_or_init(/*quiet=*/false, mcpp::fetcher::make_bootstrap_progress_callback()); + + // Whose sysroot is this? gcc bakes `--sysroot=<...>/subos/default` + // at build time, and that path is a string, not a reference -- it + // keeps naming wherever the compiler was built no matter which + // project it now serves. A developer machine has many directories by + // that name, so the baked one frequently EXISTS while belonging to an + // unrelated checkout, and headers then come from a tree this build + // never declared. Existence is not ownership. + if (tc && cfg) { + std::error_code cwdEc; + auto project = std::filesystem::current_path(cwdEc); + if (mcpp::fallback::sysroot_is_foreign( + tc->sysroot, (*cfg).registryDir, + cwdEc ? std::filesystem::path{} : project)) + warn(std::format( + "sysroot {} belongs to neither this mcpp home ({}) nor " + "this project — headers would come from a tree nothing " + "here declared. mcpp remaps a baked sysroot when it can " + "find the equivalent under the registry; seeing it here " + "means it could not", + tc->sysroot.string(), (*cfg).registryDir.string())); + } if (!cfg) { err(cfg.error().message); } else { @@ -366,6 +392,33 @@ export int doctor_report() { else if (!sawAny) ok("no installed toolchains to check"); + // The vendored xlings, against the version this mcpp expects. + // + // Nothing else surfaces this. `mcpp self env` prints both numbers and + // says nothing about the gap, and a home that acquired its xlings once + // never revisited it -- so a machine could sit years behind while every + // command looked healthy. What goes missing is silent by nature: + // features mcpp reads FROM xlings (the subos_info block, for one) + // simply never appear, and the code that consumes them degrades + // quietly because a missing block is also a legitimate state. + { + auto have = mcpp::fallback::vendored_xlings_version((*cfg).xlingsBinary); + const auto want = std::string(mcpp::config::kXlingsPinnedVersion); + if (have.empty()) { + warn(std::format("cannot read the vendored xlings version at {}", + (*cfg).xlingsBinary.string())); + } else if (mcpp::fallback::version_is_older(have, want)) { + warn(std::format( + "vendored xlings is {} but this mcpp expects {} — features " + "mcpp reads from xlings may be silently absent (the subos " + "self-description arrived in 2026.8.5.1). It is replaced " + "automatically on the next `mcpp self init`", + have, want)); + } else { + ok(std::format("vendored xlings {} (pinned {})", have, want)); + } + } + // Dangling symlinks under registry/subos/default/lib — these point // into xim payload lib dirs; a removed package leaves them broken. auto subosLib = (*cfg).xlingsHome() / "subos" / "default" / "lib"; diff --git a/src/fallback/probe_sysroot.cppm b/src/fallback/probe_sysroot.cppm index ee3db9ab..dc91adb9 100644 --- a/src/fallback/probe_sysroot.cppm +++ b/src/fallback/probe_sysroot.cppm @@ -17,17 +17,54 @@ import mcpp.log; export namespace mcpp::fallback { -// When GCC reports a sysroot ending in "subos/default" that doesn't exist -// on the current machine (baked build-time path), remap it to the -// equivalent sysroot relative to the compiler's own xpkgs directory. +// Does `child` sit under `anchor`? +// +// Spelled with path components rather than by inspecting the relative path's +// text. `native()` is a wstring on Windows, so the obvious `rfind("..", 0)` +// does not even compile there -- and where it does compile it is subtly wrong, +// since a directory genuinely named `..cache` starts with those two +// characters without escaping anything. +bool path_is_under(const std::filesystem::path& child, + const std::filesystem::path& anchor) { + if (anchor.empty() || child.empty()) return false; + auto rel = child.lexically_relative(anchor); + if (rel.empty()) return false; + static const std::filesystem::path kUp{".."}; + return *rel.begin() != kUp; +} + +// When GCC reports a baked "subos/default" sysroot that does not belong to +// THIS toolchain's home, remap it to the equivalent sysroot under the +// compiler's own xpkgs tree. +// +// The predicate used to be "does not exist", which is the wrong axis. A +// developer machine has more than one project-local `.xlings/subos/default`, +// and gcc's baked path can name one of them: it exists, so the remap was +// skipped and every build in an unrelated project inherited another +// project's sysroot. Measured -- a build in mcpp resolved +// `--sysroot=/.xlings/subos/default`. +// +// Existence says nothing about ownership. What matters is whether the path is +// under the same registry as the compiler, and a path that is not gets +// remapped whether or not something happens to be there. std::optional remap_xlings_baked_sysroot(std::string_view reportedPath, const std::filesystem::path& compilerBin) { if (reportedPath.empty()) return std::nullopt; if (!reportedPath.ends_with("subos/default")) return std::nullopt; - if (std::filesystem::exists(std::string(reportedPath))) return std::nullopt; - if (auto xpkgs = mcpp::xlings::paths::xpkgs_from_compiler(compilerBin)) { + auto xpkgsOpt = mcpp::xlings::paths::xpkgs_from_compiler(compilerBin); + if (xpkgsOpt) { + // Owned by this toolchain's registry? Then it is the right answer. + auto registry = xpkgsOpt->parent_path().parent_path(); + std::error_code ec; + const bool inside = path_is_under( + std::filesystem::path(std::string(reportedPath)), registry); + if (inside && std::filesystem::exists(std::string(reportedPath), ec)) + return std::nullopt; + } + + if (auto xpkgs = std::move(xpkgsOpt)) { // xpkgs is /data/xpkgs -> registry = xpkgs/../.. auto registrySysroot = xpkgs->parent_path().parent_path() / "subos" / "default"; @@ -37,6 +74,40 @@ remap_xlings_baked_sysroot(std::string_view reportedPath, return std::nullopt; } +// Does this sysroot belong to the same registry as the compiler that reported +// it? Callers need this BEFORE deciding whether a usable sysroot is +// acceptable: usability and ownership are independent, and a path can pass the +// first while failing the second. +bool sysroot_is_owned(std::string_view reportedPath, + const std::filesystem::path& compilerBin) { + if (reportedPath.empty()) return false; + auto xpkgs = mcpp::xlings::paths::xpkgs_from_compiler(compilerBin); + // No registry to compare against -- nothing to contradict, so accept. + if (!xpkgs) return true; + return path_is_under(std::filesystem::path(std::string(reportedPath)), + xpkgs->parent_path().parent_path()); +} + +// Is this sysroot foreign -- neither this mcpp home's registry nor a tree +// belonging to the project being built? +// +// The hazard is specific and was measured: gcc bakes `--sysroot=<...>/.xlings/ +// subos/default` as a STRING at build time, and a developer machine has many +// directories by that name. The baked one therefore frequently EXISTS while +// belonging to an unrelated checkout, and headers silently come from a tree +// this build never declared. remap_xlings_baked_sysroot repairs the case it +// can see; this predicate is what reports the case it cannot. +// +// Both anchors are required. Registry alone would flag every legitimate +// project-local tree; project alone would flag every payload sysroot. +bool sysroot_is_foreign(const std::filesystem::path& sysroot, + const std::filesystem::path& registryRoot, + const std::filesystem::path& projectRoot) { + if (sysroot.empty()) return false; + return !path_is_under(sysroot, registryRoot) + && !path_is_under(sysroot, projectRoot); +} + // Parse a Clang .cfg file alongside the compiler binary for --sysroot=. std::optional parse_clang_cfg_sysroot(const std::filesystem::path& compilerBin) { diff --git a/src/fallback/xlings_binary.cppm b/src/fallback/xlings_binary.cppm index 5a91812d..43f4ce56 100644 --- a/src/fallback/xlings_binary.cppm +++ b/src/fallback/xlings_binary.cppm @@ -6,7 +6,9 @@ // 3. Fail with user-facing instructions module; +#include #include +#include export module mcpp.fallback.xlings_binary; @@ -17,9 +19,77 @@ export namespace mcpp::fallback { // Try to acquire (copy) the xlings binary to destBin. // Returns destBin on success or an error string. +// The version already vendored at destBin, or empty when it cannot be read. +std::string vendored_xlings_version(const std::filesystem::path& bin); + +// The version the acquisition chain WOULD install, without installing it. +// Empty when nothing is available. Replacing a vendored binary is only an +// improvement when this is newer than what is already there. +std::string candidate_source_version(); + +// True when `have` is strictly older than `want`, comparing dot-separated +// numeric components. Anything unparseable answers false -- a version this +// code does not understand is not evidence of being behind. +bool version_is_older(std::string_view have, std::string_view want); + +// Try to acquire (copy) the xlings binary to destBin. +// +// `pinnedVersion` is the version this mcpp expects. A vendored binary OLDER +// than the pin is replaced; one that is newer or equal is left alone. +// +// This used to return early on mere existence, with no version comparison at +// all, and nothing else ever revisited the file. A home created once kept +// whatever xlings it first acquired forever -- measured at 2026.8.2.1 against +// a pin of 2026.8.6.3, with `mcpp self env` printing both numbers side by side +// and saying nothing about the gap. The cost is not cosmetic: features mcpp +// reads from xlings simply do not appear. The subos_info block arrived in +// 2026.8.5.1, so on that machine the graphics packages' declarations were +// discarded by a client too old to have the API, and mcpp#352's fix could +// never take effect. +// +// Strictly-older, not not-equal: a user who put a newer xlings there on +// purpose must not be downgraded by an mcpp that happens to pin an older one. std::expected -acquire_xlings_binary(const std::filesystem::path& destBin, bool quiet = false) { - if (std::filesystem::exists(destBin)) return destBin; +acquire_xlings_binary(const std::filesystem::path& destBin, bool quiet = false, + std::string_view pinnedVersion = {}) { + if (std::filesystem::exists(destBin)) { + auto have = vendored_xlings_version(destBin); + if (pinnedVersion.empty() || have.empty() + || !version_is_older(have, pinnedVersion)) + return destBin; + + // Behind the pin -- but replacing is only an improvement if what we + // would put there is actually newer. The acquisition chain below ends + // at whatever `which xlings` finds, and on a machine whose system + // xlings is ANCIENT that is a downgrade dressed up as an update. + // + // Learned by doing it: this code first deleted the vendored binary and + // re-acquired, which replaced 2026.8.2.1 with the system's 0.4.51 -- + // older still, and equally missing the feature the check exists to + // restore. Look before leaping. + auto candidate = candidate_source_version(); + if (candidate.empty() || !version_is_older(have, candidate)) { + // stderr, not stdout. This is a remark about the environment, + // not output of the command that happens to be running -- and + // `mcpp test --json` promises every stdout line is NDJSON, a + // promise this line broke the moment a machine fell behind the + // pin (e2e 155). + if (!quiet) + std::println(stderr, + "{:>12} vendored xlings {} is older than the " + "pinned {}, but no newer source is available " + "(keeping it; run `xlings self update`)", + "Note", have, pinnedVersion); + return destBin; + } + if (!quiet) + std::println(stderr, + "{:>12} vendored xlings {} -> {} (pinned {})", + "Updating", have, candidate, pinnedVersion); + std::error_code rec; + std::filesystem::remove(destBin, rec); + // fall through and re-acquire + } std::error_code ec; std::filesystem::create_directories(destBin.parent_path(), ec); @@ -82,4 +152,75 @@ acquire_xlings_binary(const std::filesystem::path& destBin, bool quiet = false) (destBin.parent_path().parent_path() / "config.toml").string())); } + +std::string vendored_xlings_version(const std::filesystem::path& bin) { + std::error_code ec; + if (!std::filesystem::exists(bin, ec)) return {}; + auto r = mcpp::platform::process::capture(std::format( + "{} --version 2>/dev/null", mcpp::platform::shell::quote(bin.string()))); + if (r.exit_code != 0) return {}; + // Output carries ANSI colour; take the first dotted-numeric run. + std::string out; + for (std::size_t i = 0; i < r.output.size(); ++i) { + if (r.output[i] == '\x1b') { // skip CSI + while (i < r.output.size() && r.output[i] != 'm') ++i; + continue; + } + out += r.output[i]; + } + std::size_t i = 0; + while (i < out.size()) { + if (std::isdigit(static_cast(out[i]))) { + auto j = i; + while (j < out.size() + && (std::isdigit(static_cast(out[j])) + || out[j] == '.')) ++j; + auto cand = out.substr(i, j - i); + if (cand.find('.') != std::string::npos) return cand; + i = j; + } else ++i; + } + return {}; +} + +bool version_is_older(std::string_view have, std::string_view want) { + auto parts = [](std::string_view v) { + std::vector out; + std::size_t i = 0; + while (i <= v.size()) { + auto dot = v.find('.', i); + auto seg = v.substr(i, dot == std::string_view::npos + ? std::string_view::npos : dot - i); + long long n = 0; + auto [p, e] = std::from_chars(seg.data(), seg.data() + seg.size(), n); + if (e != std::errc{}) return std::vector{}; + out.push_back(n); + if (dot == std::string_view::npos) break; + i = dot + 1; + } + return out; + }; + auto a = parts(have), b = parts(want); + if (a.empty() || b.empty()) return false; // unparseable is not "behind" + for (std::size_t i = 0; i < std::max(a.size(), b.size()); ++i) { + long long x = i < a.size() ? a[i] : 0; + long long y = i < b.size() ? b[i] : 0; + if (x != y) return x < y; + } + return false; +} + + +std::string candidate_source_version() { + if (const char* e = std::getenv("MCPP_VENDORED_XLINGS"); e && *e) { + std::error_code ec; + if (std::filesystem::exists(std::filesystem::path(e), ec)) + return vendored_xlings_version(std::filesystem::path(e)); + } + if (auto sys = mcpp::platform::fs::which( + std::string("xlings") + std::string(mcpp::platform::exe_suffix))) + return vendored_xlings_version(*sys); + return {}; +} + } // namespace mcpp::fallback diff --git a/src/toolchain/detect.cppm b/src/toolchain/detect.cppm index c3079e01..e71acfe9 100644 --- a/src/toolchain/detect.cppm +++ b/src/toolchain/detect.cppm @@ -16,7 +16,8 @@ export namespace mcpp::toolchain { // Detect toolchain. If explicit_compiler is given, use that binary path // directly. Otherwise fall back to $CXX, then PATH g++. std::expected -detect(const std::filesystem::path& explicit_compiler = {}); +detect(const std::filesystem::path& explicit_compiler = {}, + std::string_view runtimeBinding = {}); // Compatibility helper for older call sites/tests: GCC std module lookup now // lives in the GCC provider. @@ -33,7 +34,8 @@ std::optional find_std_module_source( } std::expected -detect(const std::filesystem::path& explicit_compiler) { +detect(const std::filesystem::path& explicit_compiler, + std::string_view runtimeBinding) { auto bin_r = probe_compiler_binary(explicit_compiler); if (!bin_r) return std::unexpected(bin_r.error()); @@ -111,7 +113,8 @@ detect(const std::filesystem::path& explicit_compiler) { // Probe fine-grained payload paths from sibling xpkgs (glibc, linux-headers). // When available, flags are assembled from these paths instead of --sysroot. - tc.payloadPaths = probe_payload_paths(tc.binaryPath); + tc.runtimeBinding = std::string(runtimeBinding); + tc.payloadPaths = probe_payload_paths(tc.binaryPath, tc.runtimeBinding); // For GCC: ensure the probed sysroot has complete headers by symlinking // missing content (linux kernel headers, glibc) from payload xpkgs. diff --git a/src/toolchain/fingerprint.cppm b/src/toolchain/fingerprint.cppm index 59e6562c..beb3f75a 100644 --- a/src/toolchain/fingerprint.cppm +++ b/src/toolchain/fingerprint.cppm @@ -1,11 +1,18 @@ -// mcpp.toolchain.fingerprint — 10-field fingerprint that gates BMI cache safety. +// mcpp.toolchain.fingerprint — 11-field fingerprint that gates BMI cache safety. // -// Per docs/06-toolchain-and-fingerprint.md, the fingerprint MUST cover: +// Per docs/08-toolchain-internals.md, the fingerprint MUST cover: // 1. compiler id 2. compiler version // 3. compiler driver identity 4. target triple // 5. stdlib id+version 6. C++ standard // 7. compile flags hash 8. mcpp version // 9. dependency lock hash 10. std module BMI hash +// 11. runtime binding (which libc this targets) +// +// Field 11 is the prerequisite for building one project under two subos. The +// target triple says `x86_64-linux-gnu` for both glibc 2.39 and 2.44, so +// without it switching subos leaves `target//` untouched: ninja replays a +// graph whose objects were compiled against the other libc, the link +// succeeds, and the result references symbols the interpreter cannot provide. // // MVP uses FNV-1a 64-bit (deterministic, collision-rare-enough for cache // invalidation); spec calls for SHA-256 but for "did anything change?" @@ -35,7 +42,7 @@ struct FingerprintInputs { struct Fingerprint { std::string hex; // 16 hex chars - std::array parts; // each field's stringified form + std::array parts; // each field's stringified form }; std::string hash_file(const std::filesystem::path& p); // returns 16 hex @@ -108,6 +115,7 @@ Fingerprint compute_fingerprint(const FingerprintInputs& in) { fp.parts[7] = std::string(MCPP_VERSION); fp.parts[8] = in.dependencyLockHash; fp.parts[9] = in.stdBmiHash; + fp.parts[10] = tc.runtimeBinding; // Combine all parts deterministically. std::uint64_t h = 0xcbf29ce484222325ull; diff --git a/src/toolchain/linkmodel.cppm b/src/toolchain/linkmodel.cppm index 45195abc..1bf69f8e 100644 --- a/src/toolchain/linkmodel.cppm +++ b/src/toolchain/linkmodel.cppm @@ -72,7 +72,7 @@ struct ToolchainLinkModel { std::vector systemIncludes; // Rendering knobs derived from the toolchain at resolve time. - bool clangDriver = false; // clang: -isystem + rpath/loader on link + bool clangDriver = false; // clang: -isystem headers; gcc: -idirafter // gcc: -idirafter (…#include_next), -B/-L only bool clangWithCfg = false; // sibling .cfg exists (bundled LLVM) @@ -107,15 +107,45 @@ struct ToolchainLinkModel { std::vector out; if (mode == CLibMode::Sysroot) { out.push_back("--sysroot=" + esc(sysroot)); + // Sysroot mode still needs the interpreter named explicitly when + // we know it. mcpp replaces gcc's `*link:` to stop the payload's + // accumulated rpath entries reaching artifacts, and the pristine + // spec it restores carries the HOST's default loader — so removing + // the addressing without supplying a replacement would silently + // hand every artifact /lib64/ld-linux. The hermeticity check + // catches that, which is how it was found. + if (!loader.empty()) + out.push_back("-Wl,--dynamic-linker=" + esc(loader)); + // ...and the runtime search path, for the same reason. Replacing + // `*link:` removes the rpath the payload's specs used to inject + // along with the loader; supplying one without the other produces + // a binary whose interpreter is right and whose libm is not found. + for (auto& dir : libDirs) { + out.push_back("-L" + esc(dir)); + out.push_back("-Wl,-rpath," + esc(dir)); + } return out; } if (mode != CLibMode::PayloadFirst) return out; if (!crtDir.empty()) out.push_back("-B" + esc(crtDir)); for (auto& dir : libDirs) { out.push_back("-L" + esc(dir)); - if (clangDriver) out.push_back("-Wl,-rpath," + esc(dir)); + out.push_back("-Wl,-rpath," + esc(dir)); } - if (clangDriver && !loader.empty()) + // Emitted for BOTH drivers. + // + // GCC used to be left to its specs here, on the theory that the + // install-time fixup owned the loader. That made the RUN side a + // per-toolchain-install decision while the COMPILE side stayed + // per-build, and the two named different glibc versions the moment a + // second one was installed. + // + // mcpp already refuses to depend on clang's install-time cfg + // (`--no-default-config`, "reproducible builds, no dependence on the + // install-time-generated cfg"). The same reasoning had simply never + // been applied to gcc. Measured: an explicit -Wl,--dynamic-linker + // overrides what the specs inject. + if (!loader.empty()) out.push_back("-Wl,--dynamic-linker=" + esc(loader)); return out; } @@ -290,20 +320,69 @@ ToolchainLinkModel resolve_link_model(const Toolchain& tc) { // cross-compile resolves by what it builds FOR. if (is_msvc_target(tc) || is_mingw_target(tc)) return lm; + // The compiler's OWN runtime lives beside it, not in the C library: + // libgcc_s.so.1 for GCC. A produced binary links it whether or not the + // build ever mentions it, so its directory has to be on the artifact's + // RUNPATH -- and gcc's patched specs used to put it there, which is + // exactly why removing that rewrite has to put it back. + // + // Both modes need this, and only one got it at first. PayloadFirst + // artifacts came out with a single RUNPATH entry (the glibc payload) and + // resolved libgcc_s.so.1 from the HOST -- or, on a machine without one, + // not at all: `error while loading shared libraries: libgcc_s.so.1`. This + // developer machine has a usable sysroot, so every local build took the + // other branch and the gap only surfaced on CI (e2e 29). + // + // Not for musl (self-contained sysroot, static world) and not for clang, + // which brings compiler-rt and libunwind instead. + auto add_compiler_runtime_dir = [&] { + if (is_musl_target(tc) || lm.clangDriver || tc.binaryPath.empty()) return; + std::error_code lec; + auto gccLib = tc.binaryPath.parent_path().parent_path() / "lib64"; + if (std::filesystem::exists(gccLib, lec)) + lm.libDirs.push_back(gccLib); + }; + auto payload_first = [&] { auto& pp = *tc.payloadPaths; lm.mode = CLibMode::PayloadFirst; lm.crtDir = pp.glibcLib; lm.libDirs.push_back(pp.glibcLib); + add_compiler_runtime_dir(); lm.systemIncludes.push_back(pp.glibcInclude); if (!pp.linuxInclude.empty()) lm.systemIncludes.push_back(pp.linuxInclude); - if (lm.clangDriver) - lm.loader = resolve_loader(pp.glibcLib, tc.targetTriple); + // Resolved for both drivers now that both emit it. + lm.loader = resolve_loader(pp.glibcLib, tc.targetTriple); }; auto sysroot_mode = [&](const std::filesystem::path& root) { lm.mode = CLibMode::Sysroot; lm.sysroot = root; + // Name the interpreter even here: see link_tokens. The payload is the + // address (R6 — artifacts bind the payload, never the mutable view), + // and it is known whenever a runtime binding resolved. + // + // The pair mirrors exactly what fixup_gcc_specs used to bake in + // (`:`): the C library, and the compiler's own + // runtime (libgcc_s, libstdc++). Emitting it per build is the whole + // point — the baked copy was a per-toolchain-install decision that no + // longer matched the per-build one, and it accumulated one dead entry + // per home that ever touched the shared payload. + // + // Never for a musl target. Its sysroot is self-contained -- it brings + // its own libc, CRT and loader -- and the glibc payload here belongs + // to the HOST toolchain, which merely happens to be probed alongside. + // Putting it on the link path let ld pull glibc's static libc.a into a + // musl link: `undefined reference to _DYNAMIC`, `hidden symbol + // _DYNAMIC isn't defined`, from dl-reloc-static-pie.o. The musl branch + // below already says this about headers; it holds for libraries and + // the loader too. + if (!is_musl_target(tc) + && tc.payloadPaths && !tc.payloadPaths->glibcLib.empty()) { + lm.loader = resolve_loader(tc.payloadPaths->glibcLib, tc.targetTriple); + lm.libDirs.push_back(tc.payloadPaths->glibcLib); + } + add_compiler_runtime_dir(); // Supplement kernel headers when the sysroot lacks them (glibc's // local_lim.h needs ). Self-contained musl sysroots // ship their own; a cross target must not see host-arch headers. diff --git a/src/toolchain/model.cppm b/src/toolchain/model.cppm index 71d3883c..97a95e71 100644 --- a/src/toolchain/model.cppm +++ b/src/toolchain/model.cppm @@ -17,6 +17,10 @@ struct EnvVar { std::string value; }; +// The runtime this toolchain builds AGAINST, in xlings's spelling +// ("glibc@2.39"). Resolved before payload probing, from `--runtime` or the +// active subos's `subos_info.runtime`. Empty means no authority was found, +// and payload resolution then declines rather than guessing. struct PayloadPaths { std::filesystem::path glibcInclude; // glibc headers (features.h, bits/) std::filesystem::path glibcLib; // glibc runtime (libc.so, crt*.o, ld-linux) @@ -29,6 +33,16 @@ struct Toolchain { std::filesystem::path binaryPath; std::string driverIdent; // normalized --version output std::string targetTriple; // "x86_64-linux-gnu" + // The runtime this toolchain builds AGAINST, in xlings's own spelling + // ("glibc@2.39"). Resolved BEFORE payload probing, from `--runtime` or + // the active subos's `subos_info.runtime`. + // + // Empty is a refusal, not a default: payload resolution declines rather + // than picking a libc by directory order. That guess is what let the + // compile side and the artifact's interpreter name different glibc + // versions, with nothing in the resulting binary looking wrong until it + // loaded a library built against the other one. + std::string runtimeBinding; std::string stdlibId; // "libstdc++" std::string stdlibVersion; std::filesystem::path stdModuleSource; // bits/std.cc / std.cppm diff --git a/src/toolchain/post_install.cppm b/src/toolchain/post_install.cppm index a40b6fe0..3230273d 100644 --- a/src/toolchain/post_install.cppm +++ b/src/toolchain/post_install.cppm @@ -158,11 +158,50 @@ export void patchelf_walk(const std::filesystem::path& dir, // - Dynamically detects the baked-in loader path from the specs file // - Replaces it with the sandbox glibc payload's loader // - Replaces the rpath with : -// Idempotent — skips if already pointing at the correct glibc. +// NOT idempotent across homes -- and that is a defect, not a caveat. The +// needle is one path (the baked glibc dir) while the replacement is two +// (glibc + gccLib), so a second home patching the same shared payload leaves +// the first home's gccLib entry behind: it never appears in any later needle. +// One stale rpath per run, forever. Measured at 68 on a developer machine, +// all of them gccLib, none glibc -- a distribution only this mechanism +// produces. See 2026-08-08-payload-version-and-contract-drift-design.md §2.1; +// this function is slated for deletion, with the flags moving to the build. // Extract the baked-in glibc loader path (".../ld-linux-.so.N") from a // gcc specs file. xim bakes the installing user's XLINGS_HOME into specs at // install time, so the DIR varies per machine, and the loader NAME varies // per arch — detect both instead of hardcoding either. +// The runtime binding a toolchain was INSTALLED against, read back out of +// what mcpp itself wrote: gcc's specs, clang's cfg. +// +// Compatibility only. The authority is the subos's `subos_info.runtime`; this +// answers for machines whose subos predates that block, and it answers with +// the version the artifact will actually load -- so compile and run still +// agree, which is the invariant that matters. +// +// Returns "glibc@" or empty. The version comes from the payload path +// (.../xim-x-glibc//lib64), which is the only place it is written down. +export std::string baked_runtime_binding(const std::filesystem::path& compilerBin); + +// Write a specs file that REPLACES gcc's `*link:` with the pristine built-in +// one, and return its path (empty when not applicable or on failure). +// +// Why this exists: `-rpath` accumulates. mcpp emitting its own does not remove +// what the payload's specs already inject, and that file has been patched by +// every home that ever installed against this shared payload -- one stale +// entry per run, forever. Replacing the spec is the only way to stop them +// reaching a user's artifact, and it works on machines already polluted. +// +// The pristine `*link:` comes from `g++ -dumpspecs`, which prints the specs +// COMPILED INTO the binary and is unaffected by the file on disk. So the +// original is always recoverable and the payload is never touched -- an +// earlier plan to reinstall the toolchain was unnecessary. +// +// `-specs=` whose definition does not begin with `+` replaces rather +// than appends; both properties were measured before this was written. +export std::filesystem::path write_clean_link_specs( + const std::filesystem::path& compilerBin, + const std::filesystem::path& outputDir); + export std::string detect_baked_loader(const std::string& specsContent) { // Path-character whitelist. Specs embed loader paths inside %-spec // syntax (`%{mmusl:...;:/baked/dir/ld-linux-x86-64.so.2}`), so scanning @@ -196,65 +235,30 @@ export std::string detect_baked_loader(const std::string& specsContent) { return ""; } -void fixup_gcc_specs(const std::filesystem::path& gccPkgRoot, - const std::filesystem::path& glibcLibDir, - const std::filesystem::path& gccLibDir, - const std::filesystem::path& fenceRoot) -{ - // #273 fence: a payload reached through a symlink is a foreign - // installation — its specs file must not be rewritten either. - if (escapes_containment(gccPkgRoot, fenceRoot)) { - mcpp::log::verbose("toolchain", - "fixup_gcc_specs: skip (payload outside sandbox, #273 fence)"); - return; - } - std::filesystem::path specsParent; - std::error_code ec; - for (auto it = std::filesystem::directory_iterator(gccPkgRoot / "lib" / "gcc", ec); - !ec && it != std::filesystem::directory_iterator{}; it.increment(ec)) { - if (it->is_directory(ec)) { specsParent = it->path(); break; } - } - if (specsParent.empty()) return; - - auto loaderReplacement = resolve_loader(glibcLibDir, /*targetTriple=*/{}).string(); - if (loaderReplacement.empty()) return; - auto rpathReplacement = std::format("{}:{}", - glibcLibDir.string(), - gccLibDir.string()); - - auto replace_all = [](std::string& s, std::string_view needle, - std::string_view rep) - { - for (std::size_t pos = 0; - (pos = s.find(needle, pos)) != std::string::npos;) { - s.replace(pos, needle.size(), rep); - pos += rep.size(); - } - }; - - for (auto& sub : std::filesystem::directory_iterator(specsParent)) { - auto specs = sub.path() / "specs"; - if (!std::filesystem::exists(specs)) continue; - - std::ifstream is(specs); - std::stringstream ss; ss << is.rdbuf(); - std::string content = ss.str(); - - auto bakedLoader = detect_baked_loader(content); - if (bakedLoader.empty()) continue; - auto bakedDir = std::filesystem::path(bakedLoader).parent_path().string(); - // Already pointing at the right place — no fixup needed. - if (bakedDir == glibcLibDir.string()) continue; - - // Order matters: replace the full loader file path first so the - // shorter dir pattern doesn't eat its prefix. - replace_all(content, bakedLoader, loaderReplacement); - replace_all(content, bakedDir, rpathReplacement); +// fixup_gcc_specs is GONE. +// +// It wrote the loader and the rpath into the gcc payload's `specs` at install +// time. Both are now emitted per build by mcpp.toolchain.linkmodel, which +// removes an answerer rather than adding a path: +// +// - the RUN side was a per-toolchain-install decision while the COMPILE +// side was per-build, and they named different glibc versions the moment +// a second one was installed; +// - the substitution had a one-path needle and a two-path replacement, so +// every home that patched the SHARED payload left one entry behind. 68 on +// a developer machine, all pointing at deleted mktemp directories, in +// every gcc artifact it produced. +// +// The payload is no longer written to at all, which restores what R6 asks of +// it: immutable, and independent of any particular home. A build that needs +// a clean `*link:` gets one generated into its own build directory (see +// write_clean_link_specs) -- the pristine spec comes from `g++ -dumpspecs`, +// which prints what is compiled into the binary and is therefore always +// recoverable. +// +// patchelf_walk stays: making the compiler itself runnable is a different +// question from what the compiler produces. - std::ofstream os(specs); - os << content; - } -} // Regenerate the clang driver cfg files after the LLVM payload landed in the // sandbox. The cfg xlings authored at install time is a per-machine, @@ -271,7 +275,7 @@ void fixup_gcc_specs(const std::filesystem::path& gccPkgRoot, export void fixup_clang_cfg(const std::filesystem::path& payloadRoot, const std::filesystem::path& glibcLibDir, const std::filesystem::path& fenceRoot) { - // #273 fence — same rule as fixup_gcc_specs above. + // #273 fence — same rule the removed gcc specs fixup used. if (escapes_containment(payloadRoot, fenceRoot)) { mcpp::log::verbose("toolchain", "fixup_clang_cfg: skip (payload outside sandbox, #273 fence)"); @@ -416,9 +420,6 @@ void gcc_post_install_fixup(const mcpp::config::GlobalConfig& cfg, } patchelf_walk(payloadRoot, loader, rpath, patchelfBin, fence); - mcpp::log::verbose("toolchain", "gcc fixup: fixup_gcc_specs"); - fixup_gcc_specs(payloadRoot, glibcLibDir, gccLibDir, - containment_root(cfg.registryDir)); } else { mcpp::ui::warning( "could not locate sandbox glibc/gcc/patchelf paths; " @@ -528,4 +529,216 @@ export void ensure_post_install_fixup(const mcpp::config::GlobalConfig& cfg, os << expected.dump(2) << "\n"; } + +// PT_INTERP of an ELF, read directly. +// +// Not via `patchelf --print-interpreter`: this runs during prepare, where +// patchelf is not guaranteed to be resolved, and a dependency on an external +// tool for four fields of a header is a dependency that will be missing on +// exactly the machine that needs the answer. +std::string read_elf_interp(const std::filesystem::path& bin) { + std::ifstream is(bin, std::ios::binary); + if (!is) return {}; + unsigned char ident[16]{}; + is.read(reinterpret_cast(ident), sizeof ident); + if (!is || ident[0] != 0x7f || ident[1] != 'E' + || ident[2] != 'L' || ident[3] != 'F') return {}; + const bool is64 = ident[4] == 2; + const bool le = ident[5] == 1; + if (!is64 || !le) return {}; // the only shape payloads ship + + auto u16 = [&](std::streamoff off) -> std::uint16_t { + is.seekg(off); unsigned char b[2]{}; + is.read(reinterpret_cast(b), 2); + return static_cast(b[0] | (b[1] << 8)); + }; + auto u64 = [&](std::streamoff off) -> std::uint64_t { + is.seekg(off); unsigned char b[8]{}; + is.read(reinterpret_cast(b), 8); + std::uint64_t v = 0; + for (int i = 7; i >= 0; --i) v = (v << 8) | b[i]; + return v; + }; + + const auto phoff = u64(0x20); + const auto phentsize = u16(0x36); + const auto phnum = u16(0x38); + if (!is || phoff == 0 || phentsize < 0x38 || phnum == 0) return {}; + + constexpr std::uint32_t kPtInterp = 3; + for (std::uint16_t i = 0; i < phnum; ++i) { + const auto ph = static_cast(phoff) + + static_cast(i) * phentsize; + is.seekg(ph); + unsigned char t[4]{}; + is.read(reinterpret_cast(t), 4); + if (!is) return {}; + const std::uint32_t type = + static_cast(t[0]) | (t[1] << 8) + | (t[2] << 16) | (static_cast(t[3]) << 24); + if (type != kPtInterp) continue; + const auto offset = u64(ph + 0x08); + const auto filesz = u64(ph + 0x20); + if (filesz == 0 || filesz > 4096) return {}; + std::string str(static_cast(filesz), '\0'); + is.seekg(static_cast(offset)); + is.read(str.data(), static_cast(filesz)); + if (!is) return {}; + if (auto z = str.find('\0'); z != std::string::npos) str.resize(z); + return str; + } + return {}; +} + +std::string baked_runtime_binding(const std::filesystem::path& compilerBin) { + if (compilerBin.empty()) return {}; + std::error_code ec; + + auto from_text = [](const std::string& text) -> std::string { + auto loader = detect_baked_loader(text); + if (loader.empty()) return {}; + // The recorded path may name a subos VIEW (`/subos/default/lib/ + // ld-linux-...`), which is a symlink into the payload. R6: artifacts + // bind the payload, never the mutable view -- so resolve to the + // payload before reading a version off it. A view path has no version + // component at all, and parsing one yields nothing. + std::error_code cec; + if (auto real = std::filesystem::canonical(loader, cec); !cec) + loader = real.string(); + // .../xim-x-glibc//lib64/ld-linux-... — the version is the + // grandparent of the lib dir. + auto dir = std::filesystem::path(loader).parent_path(); // lib64 + auto ver = dir.parent_path().filename().string(); // + auto pkg = dir.parent_path().parent_path().filename().string(); + if (ver.empty() || pkg.find("glibc") == std::string::npos) return {}; + return "glibc@" + ver; + }; + + // Which glibc payloads are actually installed. Every source below is + // checked against this, because all of them are RECORDS of a past state: + // specs and cfg were written at some install, PT_INTERP at some fixup. A + // payload can be replaced afterwards -- upgraded, garbage-collected -- and + // the record keeps naming what used to be there. + // + // That is not hypothetical. CI resolved `glibc@2.39` from a record while + // the only payload on disk was 2.44, so the exact-match probe correctly + // refused, no payload paths resolved, and the artifact fell through to the + // host loader. A fossil that names something no longer present is not an + // authority; it is just old. + std::vector installed; + if (auto xpkgs = mcpp::xlings::paths::xpkgs_from_compiler(compilerBin)) { + for (auto it = std::filesystem::directory_iterator(*xpkgs / "xim-x-glibc", ec); + !ec && it != std::filesystem::directory_iterator{}; it.increment(ec)) { + if (!it->is_directory(ec)) continue; + auto v = it->path().filename().string(); + if (!v.empty() && v.front() != '.') installed.push_back(v); + } + } + auto still_there = [&](const std::string& binding) { + if (binding.empty()) return false; + const auto at = binding.find('@'); + if (at == std::string::npos) return false; + auto ver = binding.substr(at + 1); + return std::ranges::find(installed, ver) != installed.end(); + }; + + auto read_file = [&](const std::filesystem::path& p) -> std::string { + if (!std::filesystem::exists(p, ec)) return {}; + std::ifstream is(p); + std::stringstream ss; ss << is.rdbuf(); + return ss.str(); + }; + + // clang: the sibling .cfg. + auto cfg = compilerBin.parent_path() + / (compilerBin.stem().string() + ".cfg"); + if (auto r = from_text(read_file(cfg)); still_there(r)) return r; + + // gcc: lib/gcc///specs, one level of globbing each. + auto gccRoot = compilerBin.parent_path().parent_path(); + for (auto t = std::filesystem::directory_iterator(gccRoot / "lib" / "gcc", ec); + !ec && t != std::filesystem::directory_iterator{}; t.increment(ec)) { + for (auto v = std::filesystem::directory_iterator(t->path(), ec); + !ec && v != std::filesystem::directory_iterator{}; v.increment(ec)) { + if (auto r = from_text(read_file(v->path() / "specs")); still_there(r)) + return r; + } + } + + // The compiler's OWN interpreter. + // + // The two sources above are files mcpp used to write, and mcpp no longer + // writes them -- so on a machine where the toolchain was installed after + // that change they simply are not there, and the compatibility path that + // depends on them answers nothing. That is not hypothetical: it is what + // turned CI red while every existing developer machine, which still has + // the files from before, stayed green. + // + // PT_INTERP is produced by the patchelf walk, which still runs on every + // install, and it names the glibc payload this toolchain was aligned to -- + // the same fact the specs held, from a mechanism that has not gone away. + if (auto r = from_text(read_elf_interp(compilerBin)); still_there(r)) + return r; + + // Last: the installed payload set, but ONLY when it is a singleton. + // + // This is not the rule this design removed. That rule asked a directory + // for "the glibc" and took whatever `readdir` yielded first -- a CHOICE, + // made by something with no bearing on what the artifact would load, and + // silently wrong the moment a second payload appeared. Where exactly one + // payload is installed there is no choice to make: it is the only glibc + // any artifact from this toolchain could bind, and declining would refuse + // a question that has a single answer. + // + // Two or more and this stays silent. That is the case the incident was, + // and it is the case the subos must answer. + if (installed.size() == 1) { + mcpp::log::verbose("probe", std::format( + "runtime binding glibc@{} — the only glibc payload installed, so " + "there is nothing to choose between", installed[0])); + return "glibc@" + installed[0]; + } + if (installed.size() > 1) + mcpp::log::verbose("probe", std::format( + "{} glibc payloads installed and nothing still present declares " + "which one this build binds; declining rather than picking", + installed.size())); + return {}; +} + + +std::filesystem::path write_clean_link_specs( + const std::filesystem::path& compilerBin, + const std::filesystem::path& outputDir) +{ + if (compilerBin.empty() || outputDir.empty()) return {}; + std::error_code ec; + auto out = outputDir / "mcpp-clean-link.specs"; + // Idempotent: the pristine spec is a property of the compiler binary, so + // regenerating it every build would spawn a process for a constant. + if (std::filesystem::exists(out, ec)) return out; + + auto r = mcpp::platform::process::capture(std::format( + "{} -dumpspecs 2>/dev/null", + mcpp::platform::shell::quote(compilerBin.string()))); + if (r.exit_code != 0 || r.output.empty()) return {}; + + // `*link:` is a section header on its own line; its body is the next line. + constexpr std::string_view kHead = "*link:\n"; + auto pos = r.output.find(kHead); + if (pos == std::string::npos) return {}; + auto bodyStart = pos + kHead.size(); + auto bodyEnd = r.output.find('\n', bodyStart); + if (bodyEnd == std::string::npos) return {}; + auto body = r.output.substr(bodyStart, bodyEnd - bodyStart); + if (body.empty()) return {}; + + std::filesystem::create_directories(outputDir, ec); + std::ofstream os(out); + if (!os) return {}; + os << "*link:\n" << body << "\n\n"; + os.close(); + return out; +} + } // namespace mcpp::toolchain diff --git a/src/toolchain/probe.cppm b/src/toolchain/probe.cppm index 5331c12e..35fb26b6 100644 --- a/src/toolchain/probe.cppm +++ b/src/toolchain/probe.cppm @@ -50,11 +50,15 @@ std::filesystem::path probe_sysroot(const std::filesystem::path& compilerBin, const std::string& envPrefix); -// Probe fine-grained sysroot paths from sibling xpkgs payloads. -// Returns populated PayloadPaths if glibc xpkg found; linux-headers -// may be empty if not available. +// Resolve the payload paths for an EXPLICIT runtime binding ("glibc@2.39"). +// +// The version is named by the caller's authority, never searched for. An +// empty binding returns nullopt: declining PayloadFirst is correct, guessing +// is what split the compile and run halves apart (see §3.2 of +// 2026-08-08-payload-version-and-contract-drift-design.md). std::optional -probe_payload_paths(const std::filesystem::path& compilerBin); +probe_payload_paths(const std::filesystem::path& compilerBin, + std::string_view runtimeBinding); // Ensure sysroot directory has complete headers by symlinking from // payload xpkgs. Called when GCC's probed sysroot exists but may @@ -287,21 +291,48 @@ probe_sysroot(const std::filesystem::path& compilerBin, mcpp::platform::null_redirect)); if (r) { auto s = trim_line(*r); - if (!s.empty() && std::filesystem::exists(s)) { - if (usable(s)) return s; + + // A usable sysroot that belongs to somebody else is still somebody + // else's. gcc records this path as a string when it is built and + // reports it forever after; on a machine with several checkouts the + // recorded one routinely exists AND carries headers, so accepting it + // on usability alone hands the build another project's tree. Measured + // right here: this repo's gcc reported a sysroot under an unrelated + // one, and every build took its headers. + // + // The ownership test lives inside remap_xlings_baked_sysroot, which is + // exactly why the order matters -- the early return below reached it + // only when the path was missing, so the case the remap exists for was + // the one case it never saw. + const bool ownedByThisHome = + !s.empty() && mcpp::fallback::sysroot_is_owned(s, compilerBin); + + if (ownedByThisHome && usable(s)) return s; + if (!s.empty() && std::filesystem::exists(s) && !usable(s)) mcpp::log::debug("probe", std::format( "sysroot '{}' exists but lacks usr/include/stdlib.h — ignoring", s)); - } // GCC bakes the build-time sysroot into the binary. For xlings-built // GCC this is a path like /.xlings/subos/default that - // doesn't exist on the user's machine. Remap via fallback module. + // doesn't exist on the user's machine -- or exists and belongs to a + // different one. Remap via fallback module. if (auto remapped = mcpp::fallback::remap_xlings_baked_sysroot(s, compilerBin)) { if (usable(*remapped)) return *remapped; mcpp::log::debug("probe", std::format( "remapped sysroot '{}' lacks usr/include/stdlib.h — ignoring", remapped->string())); } + + // Last resort: a foreign but usable sysroot beats no sysroot. This is + // the pre-existing behaviour, kept for machines that have no registry + // subos to remap to -- it is a worse answer, not a wrong one, and + // taking nothing here would break them outright. + if (!s.empty() && std::filesystem::exists(s) && usable(s)) { + mcpp::log::verbose("probe", std::format( + "using sysroot '{}', which is outside this toolchain's " + "registry — no equivalent found under it", s)); + return std::filesystem::path(s); + } } // 2. macOS fallback: use xcrun to discover the SDK path. @@ -326,19 +357,72 @@ probe_sysroot(const std::filesystem::path& compilerBin, return {}; } +// Resolve `@` to the payload ROOT for that exact version. +// +// The version is named, never searched: the caller has an authority (the +// resolved runtime binding) and this turns it into an address. Two payload +// roots are tried in the order the rest of the file uses -- the compiler's own +// siblings first, then the active home -- because an inherited or symlinked +// compiler resolves into its owner home while the active home may be the one +// that just installed the payload. +std::optional +payload_root_for_binding(const std::filesystem::path& compilerBin, + std::string_view binding) { + const auto at = binding.find('@'); + if (at == std::string_view::npos || at == 0 || at + 1 >= binding.size()) + return std::nullopt; + const auto name = binding.substr(0, at); + const auto version = std::string(binding.substr(at + 1)); + + std::error_code ec; + // Compiler siblings: <...>/xpkgs/xim-x-/ + if (auto xpkgs = mcpp::xlings::paths::xpkgs_from_compiler(compilerBin)) { + auto root = *xpkgs / std::format("xim-x-{}", name) / version; + if (std::filesystem::exists(root, ec)) return root; + } + // Active home. + if (auto xpkgs = mcpp::xlings::paths::active_home_xpkgs()) { + auto root = *xpkgs / std::format("xim-x-{}", name) / version; + if (std::filesystem::exists(root, ec)) return root; + } + return std::nullopt; +} + +// `runtimeBinding` is the AUTHORITY -- `--runtime`, else the active subos's +// `subos_info.runtime`. Empty is a refusal, not a licence to guess: a guess +// here is how the compile side and the artifact's interpreter came to name +// different glibc versions, and nothing about the resulting binary looks +// wrong until it loads a library built against the other one. std::optional -probe_payload_paths(const std::filesystem::path& compilerBin) { - namespace paths = mcpp::xlings::paths; +probe_payload_paths(const std::filesystem::path& compilerBin, + std::string_view runtimeBinding) { + if (runtimeBinding.empty()) { + mcpp::log::verbose("probe", + "no runtime binding resolved — declining PayloadFirst rather than " + "picking a libc by directory order"); + return std::nullopt; + } + // Only a libc family has a payload of this shape. `macos_sdk`/`ucrt` are + // resolved elsewhere, and a musl target uses the sysroot mode. + if (!runtimeBinding.starts_with("glibc@")) { + mcpp::log::verbose("probe", std::format( + "runtime '{}' is not a glibc payload — no PayloadFirst", + std::string(runtimeBinding))); + return std::nullopt; + } - // Find glibc xpkg (required). Compiler siblings first; fall back to the - // ACTIVE home registry — an inherited/symlinked compiler resolves into - // its owner home, while the active home may own (or have just installed) - // the sysroot payloads. - auto glibc = paths::find_sibling_tool(compilerBin, "glibc"); - if (!glibc) glibc = paths::find_home_tool("glibc", "include/features.h"); - if (!glibc) return std::nullopt; + auto glibc = payload_root_for_binding(compilerBin, runtimeBinding); + if (!glibc) { + mcpp::log::verbose("probe", std::format( + "runtime '{}' is not installed in this home", + std::string(runtimeBinding))); + return std::nullopt; + } + + namespace paths = mcpp::xlings::paths; - // Glibc layout: /include/ + /lib64/ (or lib/). + // Layout WITHIN the chosen payload. This convention stays: it answers + // "where inside", not "which one". auto glibcInclude = *glibc / "include"; if (!std::filesystem::exists(glibcInclude / "features.h")) return std::nullopt; diff --git a/src/version.cppm b/src/version.cppm index bd258c4a..7e8caed1 100644 --- a/src/version.cppm +++ b/src/version.cppm @@ -31,6 +31,6 @@ import std; export namespace mcpp { -inline constexpr std::string_view MCPP_VERSION = "2026.8.8.1"; +inline constexpr std::string_view MCPP_VERSION = "2026.8.8.2"; } // namespace mcpp diff --git a/src/xlings.cppm b/src/xlings.cppm index e05f9457..3423e506 100644 --- a/src/xlings.cppm +++ b/src/xlings.cppm @@ -44,7 +44,7 @@ namespace pinned { // in lock-step by hand; that list was already missing both composite // actions, which is how CI's sandbox sat on 0.4.30 unnoticed while // everything else had moved on. Don't reintroduce a hand-maintained list. - inline constexpr std::string_view kXlingsVersion = "2026.8.6.3"; + inline constexpr std::string_view kXlingsVersion = "2026.8.7.1"; inline constexpr std::string_view kNasmVersion = "3.02"; } @@ -716,7 +716,16 @@ find_sibling_tool(const std::filesystem::path& compilerBin, std::error_code ec; if (!std::filesystem::exists(root, ec)) return std::nullopt; - // Return the first (highest) version dir that exists. + // The first version dir readdir yields. ORDER IS UNDEFINED -- this does + // NOT return the highest, and the comment that said so was wrong for as + // long as it stood. With two versions installed the answer varies by + // filesystem, so a caller that needs a SPECIFIC version must name it; + // see probe.cppm, where the version comes from the resolved runtime + // binding rather than from this scan. + // + // A test asserting the old "highest" claim was attempted and removed: it + // passed or failed by directory order, which makes both outcomes + // uninformative. for (auto& v : std::filesystem::directory_iterator(root, ec)) { if (v.is_directory(ec)) return v.path(); } diff --git a/src/xlings/subos_info.cppm b/src/xlings/subos_info.cppm index 0054fb2b..41aaac55 100644 --- a/src/xlings/subos_info.cppm +++ b/src/xlings/subos_info.cppm @@ -131,28 +131,39 @@ Info read(const std::filesystem::path& subosDir) { if (auto v = it->find("runtime"); v != it->end() && v->is_string()) info.runtime = v->get(); - if (auto envs = it->find("envs"); envs != it->end() && envs->is_array()) { - for (auto const& p : *envs) { - if (!p.is_object()) continue; + // `envs` is an OBJECT keyed by binding, whose values are arrays of + // declarations: + // + // "envs": { "mesa@25.0.7.1": [ {"var":…,"op":…,"value":…}, … ], … } + // + // Transcribed from xlings's own reader (core/subos/manifest.cppm), not + // from a model of it. The first version of this file expected an array of + // {binding, decls} objects — a shape xlings never writes — and its tests + // hand-wrote JSON in that same invented shape, so both agreed and both + // were wrong. Against a real subos the loop simply never ran and every + // variable came back unset, silently. That is why the fixture below is a + // verbatim capture of real output rather than something composed here. + if (auto envs = it->find("envs"); envs != it->end() && envs->is_object()) { + for (auto e = envs->begin(); e != envs->end(); ++e) { + if (!e.value().is_array()) continue; Provider prov; - if (auto b = p.find("binding"); b != p.end() && b->is_string()) - prov.binding = b->get(); - if (auto ds = p.find("decls"); ds != p.end() && ds->is_array()) { - for (auto const& d : *ds) { - if (!d.is_object()) continue; - EnvDecl e; - if (auto x = d.find("var"); x != d.end() && x->is_string()) - e.var = x->get(); - if (auto x = d.find("op"); x != d.end() && x->is_string()) - e.op = x->get(); - if (auto x = d.find("value"); x != d.end() && x->is_string()) - e.value = x->get(); - // A declaration with no variable name is not a partial - // declaration to be guessed at — it is malformed input, - // and the right thing is to leave it out rather than - // invent a name for it. - if (!e.var.empty()) prov.decls.push_back(std::move(e)); - } + prov.binding = e.key(); + for (auto const& d : e.value()) { + if (!d.is_object()) continue; + EnvDecl decl; + if (auto x = d.find("var"); x != d.end() && x->is_string()) + decl.var = x->get(); + if (auto x = d.find("op"); x != d.end() && x->is_string()) + decl.op = x->get(); + if (auto x = d.find("value"); x != d.end() && x->is_string()) + decl.value = x->get(); + // xlings drops a declaration whose var is empty or whose op is + // neither "set" nor "prepend". Matched exactly: a reader that + // is more permissive than the writer will one day apply + // something the writer considers malformed. + if (decl.var.empty()) continue; + if (decl.op != "set" && decl.op != "prepend") continue; + prov.decls.push_back(std::move(decl)); } info.providers.push_back(std::move(prov)); } diff --git a/tests/e2e/200_subos_env_reaches_program.sh b/tests/e2e/200_subos_env_reaches_program.sh index 81da7302..4ce376df 100755 --- a/tests/e2e/200_subos_env_reaches_program.sh +++ b/tests/e2e/200_subos_env_reaches_program.sh @@ -14,6 +14,12 @@ # know what any of these variables mean -- it carries whatever the subos # declares -- and a test naming LIBGL_DRIVERS_PATH would quietly suggest # otherwise. +# +# The JSON below is xlings's REAL shape: `envs` is an object keyed by binding, +# whose values are arrays of declarations. The first version of this test wrote +# an array of {binding, decls} -- a shape xlings never produces -- and it +# passed, because the reader had been written from the same misunderstanding. +# Do not "simplify" this structure; it is a wire format, not a convenience. set -euo pipefail TMP=$(mktemp -d) @@ -28,9 +34,9 @@ mkdir -p "$subos/usr/lib/dri" cat > "$subos/.xlings.json" <<'EOF' { "workspace": {}, "subos_info": { "schema_version": 1, "runtime": "glibc@2.39", - "envs": [ { "binding": "probe@1", "decls": [ + "envs": { "probe@1": [ { "var": "MCPP_E2E_PROBE", "op": "prepend", - "value": "${subosdir}/usr/lib/dri" } ] } ] } } + "value": "${subosdir}/usr/lib/dri" } ] } } } EOF cd "$TMP" @@ -106,6 +112,33 @@ echo "$out2" | grep -q 'PROBE=(unset)' || { exit 1 } +# 2b. A cache written before this mcpp knew about subos environments must NOT +# be replayed by the fast path. Simulated by stripping the field, which is +# exactly what an older mcpp's cache looks like. +# +# Without this the fix survives an upgrade in name only: the fast path's +# identity is the profile, the cache mode and the resource list, and its +# fingerprint check compares a cached entry against itself -- so nothing +# notices that a different mcpp wrote it, and an upgraded mcpp would keep +# running the pre-upgrade build with no subos environment at all. +cache="$TMP/hello/target/.build_cache" +[ -f "$cache" ] || { echo "no build cache to age"; exit 1; } +grep -q '^subos=' "$cache" || { echo "cache has no subos= line to strip"; exit 1; } +grep -v '^subos=' "$cache" > "$cache.old" && mv "$cache.old" "$cache" +aged=$(MCPP_SUBOS_DIR="$subos" "$MCPP" run 2>&1) || { + echo "run against an aged cache failed:"; echo "$aged"; exit 1; } +echo "$aged" | grep -q 'Resolving toolchain' || { + echo "an aged cache was replayed by the fast path — the subos environment" + echo " would be missing for every run after an upgrade:" + echo "$aged" + exit 1 +} +echo "$aged" | grep -q "PROBE=$subos/usr/lib/dri" || { + echo "the rebuild after an aged cache did not apply the environment:" + echo "$aged"; exit 1; } +grep -q '^subos=' "$cache" || { + echo "the rebuild did not record subos= , so every later run repeats it"; exit 1; } + # 3. A subos with no self-description degrades quietly and still runs. This is # the state of every subos created before xlings grew the block, so it must # not be an error. diff --git a/tests/e2e/201_gcc_no_specs_pollution.sh b/tests/e2e/201_gcc_no_specs_pollution.sh new file mode 100755 index 00000000..20499ad4 --- /dev/null +++ b/tests/e2e/201_gcc_no_specs_pollution.sh @@ -0,0 +1,173 @@ +#!/usr/bin/env bash +# requires: gcc elf +# 201_gcc_no_specs_pollution.sh — a gcc artifact must carry only the RUNPATH +# entries this build asked for. +# +# mcpp used to write the loader and rpath into the gcc payload's `specs` file +# at install time, and the link command said nothing about either. Two +# consequences, both measured on a developer machine before this test existed: +# +# 1. the compile side (-L, chosen per build) and the run side (specs, frozen +# at toolchain install) could name different glibc versions, and did as +# soon as a second one was installed; +# 2. the substitution that wrote those specs had a one-path needle and a +# two-path replacement, so every home that ever patched the SHARED payload +# left one entry behind -- 68 of them, all pointing at deleted mktemp +# directories, in every gcc artifact the machine produced. +# +# The assertion is deliberately about the artifact, not about the specs file: +# what matters is what a user ends up shipping. A RUNPATH entry under /tmp is +# never something a build asked for. +set -euo pipefail + +SELF_DIR=$(cd "$(dirname "$0")" && pwd) # captured before any cd +TMP=$(mktemp -d) +trap 'rm -rf "$TMP"' EXIT +export MCPP_HOME=$HOME/.mcpp + +cd "$TMP" +"$MCPP" new prog > /dev/null +cd prog +cat >> mcpp.toml <<'EOF' + +[toolchain] +linux = "gcc@16.1.0" +EOF + +"$MCPP" build > "$TMP/build.log" 2>&1 || { cat "$TMP/build.log"; echo "build failed"; exit 1; } + +# A cold build, because the generated spec lives in target/ and a cold build +# deletes target/. Written the obvious way -- generate during prepare, consume +# during the build -- gcc got a `-specs=` naming a file that had been removed +# in between, and EVERY `--no-cache` build failed with `cannot read spec file`. +# The warm path above cannot see that: the file is still there from last time. +"$MCPP" build --no-cache > "$TMP/cold.log" 2>&1 || { + cat "$TMP/cold.log"; echo "cold build failed"; exit 1; } + +bin=$(find target -type f -name prog -path '*/bin/*' | head -1) +[[ -n "$bin" ]] || { echo "no binary produced"; exit 1; } + +# `readelf` may be an xvm shim needing a per-tool pin; use the payload's own. +readelf=$(ls "$MCPP_HOME"/registry/data/xpkgs/xim-x-binutils/*/bin/readelf 2>/dev/null | head -1) +[[ -x "$readelf" ]] || { echo "SKIP: no payload readelf to inspect with"; exit 0; } + +runpath=$("$readelf" -d "$bin" 2>/dev/null | grep -iE 'RUNPATH|RPATH' | sed 's/.*\[\(.*\)\]/\1/' || true) + +# 1. No path under a temp directory. These can only come from a specs file that +# some other home patched; no build ever asks for one. +if echo "$runpath" | tr ':' '\n' | grep -q '^/tmp/'; then + echo "the artifact carries RUNPATH entries from other homes' temp dirs:" + echo "$runpath" | tr ':' '\n' | grep '^/tmp/' | head -5 + echo " ...total $(echo "$runpath" | tr ':' '\n' | grep -c '^/tmp/')" + echo "These come from the payload's specs, which mcpp must no longer write." + exit 1 +fi + +# 2. Every entry must exist. A RUNPATH naming a directory that is not there is +# a cost paid on every symbol resolution for nothing, and a reliable sign +# that something wrote a path belonging to a different machine state. +missing=0 +while IFS= read -r d; do + [[ -z "$d" ]] && continue + [[ "$d" == \$ORIGIN* ]] && continue + [[ -d "$d" ]] || { echo " RUNPATH entry does not exist: $d"; missing=$((missing+1)); } +done <<< "$(echo "$runpath" | tr ':' '\n')" +[[ "$missing" -eq 0 ]] || { echo "$missing dead RUNPATH entries"; exit 1; } + +# 3. The interpreter is the payload's, and the program runs. +file_out=$(file "$bin") +echo "$file_out" | grep -q 'interpreter .*xim-x-glibc' || { + echo "PT_INTERP is not the payload's glibc:"; echo "$file_out"; exit 1; } +out=$("$bin" 2>&1) || { echo "binary did not run: $out"; exit 1; } +echo "$out" | grep -q 'Hello' || { echo "unexpected output: $out"; exit 1; } + +# 4. Nothing loads from the host. +# +# "Does it run" is not this question, and answering the wrong one is how a +# real gap survived: with the compiler's own lib dir missing from RUNPATH, +# artifacts resolved libgcc_s.so.1 from /lib and ran perfectly on any machine +# that has a host toolchain -- which is every developer machine. It failed +# only on CI's throw-away home, and only as `error while loading shared +# libraries`, with no indication of what had gone missing or why. +# +# Asking where each library actually came from makes the same defect visible +# everywhere, host toolchain or not. +# Only lines whose LEFT side is a bare soname count. ldd resolves the program +# interpreter by running the host's loader, so it always reports the +# interpreter as `/ld-linux-x86-64.so.2 => /lib64/ld-linux-x86-64.so.2` +# -- a statement about ldd, not about the artifact, and a false positive that +# failed this very check on CI. A real dependency has no slash on the left. +host_lib_lines() { + ldd "$1" 2>/dev/null \ + | grep -E '^[[:space:]]*[^/[:space:]]+ => +(/lib|/usr/lib|/lib64|/usr/lib64)/' \ + || true +} + +if command -v ldd > /dev/null 2>&1; then + host_libs=$(host_lib_lines "$bin") + if [[ -n "$host_libs" ]]; then + echo "the artifact loads libraries from the host:" + echo "$host_libs" | sed 's/^/ /' + echo "Every shared library a payload build needs is in the payload;" + echo "reaching outside it means a directory is missing from RUNPATH." + exit 1 + fi +fi + +# ── The same assertions again, in the OTHER link mode ──────────────────── +# +# RUNPATH is built by two separate paths -- CLibMode::Sysroot and +# CLibMode::PayloadFirst -- and a machine only ever takes one of them. This +# one has a usable sysroot, so everything above ran through Sysroot mode and +# PayloadFirst went untested here for as long as this file has existed. +# +# It was not academic: PayloadFirst omitted the compiler's own lib dir, so +# artifacts resolved libgcc_s.so.1 from the host. Invisible on any machine +# with a host toolchain; on CI's throw-away home it was `error while loading +# shared libraries` with no further explanation. +# +# A home with payloads but no subos usually has no sysroot to find, which is +# what selects PayloadFirst -- but "usually" is the honest word: where gcc +# reports a baked sysroot that happens to exist (another checkout on the same +# machine), mcpp accepts it as a last resort and this leg runs Sysroot mode a +# second time instead. Which is what it does on the machine these words were +# written on. +# +# So the deterministic guard for this invariant is the unit test +# (tests/unit/test_link_model_runtime_dirs.cpp), which names the mode rather +# than arranging for it. This leg is still worth running: it covers a second +# home shape end to end, and on CI -- where the baked path does not exist -- +# it is the real thing. +export MCPP_HOME="$TMP/payload-first-home" +MCPP_INHERIT_CONFIG=0 MCPP_INHERIT_SUBOS=0 \ + source "$SELF_DIR/_inherit_toolchain.sh" + +cd "$TMP" +"$MCPP" new pf > /dev/null +cd pf +cat >> mcpp.toml <<'EOF' + +[toolchain] +linux = "gcc@16.1.0" +EOF + +"$MCPP" build > "$TMP/pf.log" 2>&1 || { + cat "$TMP/pf.log"; echo "payload-first build failed"; exit 1; } + +pfbin=$(find target -type f -name pf -path '*/bin/*' | head -1) +[[ -n "$pfbin" ]] || { echo "no payload-first binary produced"; exit 1; } + +pf_host=$(host_lib_lines "$pfbin") +if [[ -n "$pf_host" ]]; then + echo "the payload-first artifact loads libraries from the host:" + echo "$pf_host" | sed 's/^/ /' + echo "RUNPATH was:" + "$readelf" -d "$pfbin" 2>/dev/null | grep -iE 'RUNPATH|RPATH' | sed 's/^/ /' + exit 1 +fi + +"$pfbin" > /dev/null 2>&1 || { + echo "payload-first binary did not run: $("$pfbin" 2>&1 | head -2)"; exit 1; } + +echo "PASS: gcc artifacts carry only the RUNPATH the build asked for and load" +echo " nothing from the host — in both sysroot and payload-first modes" diff --git a/tests/e2e/29_toolchain_partial_versions.sh b/tests/e2e/29_toolchain_partial_versions.sh index cf87ab1e..43823b77 100755 --- a/tests/e2e/29_toolchain_partial_versions.sh +++ b/tests/e2e/29_toolchain_partial_versions.sh @@ -89,7 +89,28 @@ file "$binary" | grep -q 'statically linked' && { echo "first-run default unexpectedly produced a static binary (glibc default expected)" exit 1 } -"$binary" >/dev/null 2>&1 || { echo "first-run binary did not run"; exit 1; } +# Say WHY when it does not. "did not run" alone is unactionable: an ELF fails +# to start for exactly a few reasons -- a missing interpreter, an unresolvable +# RUNPATH entry, a symbol the runtime does not have -- and which one it is is +# the whole answer. This assertion sat here reporting none of them while CI +# failed on it and every local run passed. +if ! out=$("$binary" 2>&1); then + rc=$? + echo "first-run binary did not run (exit $rc): $out" + interp=$(file "$binary" | sed -n 's/.*interpreter \([^,]*\).*/\1/p') + echo " PT_INTERP: ${interp:-}" + [[ -n "$interp" ]] && { [[ -e "$interp" ]] && echo " exists" \ + || echo " MISSING"; } + rp=$(readelf -d "$binary" 2>/dev/null \ + | sed -n 's/.*R\(UN\)\?PATH.*\[\(.*\)\]/\2/p') + echo " RUNPATH: ${rp:-}" + while IFS= read -r d; do + [[ -z "$d" ]] && continue + [[ -d "$d" ]] && echo " ok $d" || echo " MISSING $d" + done <<< "$(echo "$rp" | tr ':' '\n')" + ldd "$binary" 2>&1 | sed 's/^/ ldd: /' | head -12 + exit 1 +fi # Second build should be silent on toolchain — no re-install banner. "$MCPP" build > "$TMP/secondrun.log" 2>&1 || { diff --git a/tests/unit/test_fingerprint.cpp b/tests/unit/test_fingerprint.cpp index f6e10143..1fa185ce 100644 --- a/tests/unit/test_fingerprint.cpp +++ b/tests/unit/test_fingerprint.cpp @@ -24,6 +24,44 @@ FingerprintInputs baseline() { return in; } + +// Two builds that differ ONLY in which libc they target must not share a +// fingerprint -- and therefore must not share `target//`. +// +// Without this, switching subos leaves the build directory untouched: ninja +// sees the same graph, the objects inside were compiled against the other +// glibc, and the link succeeds. What comes out references symbols the +// interpreter cannot provide, and nothing along the way said anything. This +// is the prerequisite for building the same project under two subos, not a +// refinement of it. +TEST(Fingerprint, RuntimeBindingIsPartOfTheIdentity) { + mcpp::toolchain::FingerprintInputs a; + a.toolchain.compiler = mcpp::toolchain::CompilerId::GCC; + a.toolchain.version = "16.1.0"; + a.toolchain.targetTriple = "x86_64-linux-gnu"; + a.toolchain.runtimeBinding = "glibc@2.39"; + + auto b = a; + b.toolchain.runtimeBinding = "glibc@2.44"; + + EXPECT_NE(mcpp::toolchain::compute_fingerprint(a).hex, + mcpp::toolchain::compute_fingerprint(b).hex) + << "same toolchain, same triple, different libc — sharing target// " + "would replay objects compiled against the other one"; +} + +// ...and an unset binding is its own identity, not a collision with any +// particular version. +TEST(Fingerprint, EmptyRuntimeBindingIsDistinct) { + mcpp::toolchain::FingerprintInputs a; + a.toolchain.targetTriple = "x86_64-linux-gnu"; + a.toolchain.runtimeBinding = "glibc@2.39"; + auto b = a; + b.toolchain.runtimeBinding.clear(); + EXPECT_NE(mcpp::toolchain::compute_fingerprint(a).hex, + mcpp::toolchain::compute_fingerprint(b).hex); +} + } // namespace TEST(Fingerprint, DeterministicForSameInputs) { diff --git a/tests/unit/test_link_model_runtime_dirs.cpp b/tests/unit/test_link_model_runtime_dirs.cpp new file mode 100644 index 00000000..c38b46b8 --- /dev/null +++ b/tests/unit/test_link_model_runtime_dirs.cpp @@ -0,0 +1,122 @@ +// An artifact must find every library it needs inside the sandbox — in BOTH +// link modes. +// +// RUNPATH is assembled by two separate paths, CLibMode::Sysroot and +// CLibMode::PayloadFirst, and any given machine only ever takes one of them. +// The compiler's own runtime (libgcc_s.so.1) lives beside the compiler, not in +// the C library, and a produced binary links it whether or not the build ever +// mentions it. gcc's patched specs used to supply that directory; removing the +// specs rewrite therefore has to supply it explicitly — and at first only the +// Sysroot path did. +// +// The consequence was invisible where it was written: on a machine with a host +// toolchain the artifact simply resolved libgcc_s.so.1 from /lib and ran. It +// failed only in a throw-away home on CI, as `error while loading shared +// libraries` with nothing said about what was missing. +// +// Asserted here rather than in e2e on purpose. Which mode an e2e reaches +// depends on whether the machine happens to have a usable sysroot — this one +// does, so an e2e leg written for PayloadFirst quietly ran Sysroot twice and +// passed with the defect in place. The link model takes its mode from its +// input, so a test can name the mode instead of hoping for it. + +#include + +import std; +import mcpp.toolchain.linkmodel; +import mcpp.toolchain.model; + +namespace tc = mcpp::toolchain; + +namespace { + +void touch(const std::filesystem::path& p) { + std::filesystem::create_directories(p.parent_path()); + std::ofstream(p) << "x"; +} + +// A payload tree with a glibc beside a gcc, as xlings lays them out. +struct Payload { + std::filesystem::path root, compiler, glibcLib, gccLib, sysroot; + Payload() { + root = std::filesystem::temp_directory_path() + / std::format("mcpp_linkmodel_{}", std::random_device{}()); + auto xpkgs = root / "data" / "xpkgs"; + compiler = xpkgs / "xim-x-gcc" / "16.1.0" / "bin" / "g++"; + touch(compiler); + gccLib = xpkgs / "xim-x-gcc" / "16.1.0" / "lib64"; + touch(gccLib / "libgcc_s.so.1"); + glibcLib = xpkgs / "xim-x-glibc" / "2.39" / "lib64"; + touch(glibcLib / "libc.so.6"); + touch(glibcLib / "ld-linux-x86-64.so.2"); + touch(xpkgs / "xim-x-glibc" / "2.39" / "include" / "features.h"); + sysroot = root / "subos" / "default"; + touch(sysroot / "usr" / "include" / "stdlib.h"); + } + ~Payload() { std::error_code ec; std::filesystem::remove_all(root, ec); } + + tc::Toolchain toolchain(bool withSysroot) const { + tc::Toolchain t; + t.compiler = tc::CompilerId::GCC; + t.version = "16.1.0"; + t.binaryPath = compiler; + t.targetTriple = "x86_64-linux-gnu"; + t.runtimeBinding = "glibc@2.39"; + tc::PayloadPaths pp; + pp.glibcLib = glibcLib; + pp.glibcInclude = glibcLib.parent_path() / "include"; + t.payloadPaths = pp; + if (withSysroot) t.sysroot = sysroot; + return t; + } +}; + +std::string joined(const tc::ToolchainLinkModel& lm) { + auto id = [](const std::filesystem::path& p) { return p.string(); }; + std::string s; + for (auto& tok : lm.link_tokens(id)) { s += tok; s += ' '; } + return s; +} + +TEST(LinkModelRuntimeDirs, PayloadFirstCarriesTheCompilerRuntime) { + Payload p; + auto lm = tc::resolve_link_model(p.toolchain(/*withSysroot=*/false)); + ASSERT_EQ(lm.mode, tc::CLibMode::PayloadFirst); + auto line = joined(lm); + EXPECT_NE(line.find("-Wl,-rpath," + p.gccLib.string()), std::string::npos) + << "libgcc_s.so.1 lives here and nothing else will find it:\n" << line; +} + +TEST(LinkModelRuntimeDirs, SysrootCarriesTheCompilerRuntime) { + Payload p; + auto lm = tc::resolve_link_model(p.toolchain(/*withSysroot=*/true)); + ASSERT_EQ(lm.mode, tc::CLibMode::Sysroot); + auto line = joined(lm); + EXPECT_NE(line.find("-Wl,-rpath," + p.gccLib.string()), std::string::npos) + << line; +} + +// The C library's own directory, in both modes, for the same reason. +TEST(LinkModelRuntimeDirs, BothModesCarryTheCLibrary) { + Payload p; + for (bool withSysroot : {false, true}) { + auto line = joined(tc::resolve_link_model(p.toolchain(withSysroot))); + EXPECT_NE(line.find("-Wl,-rpath," + p.glibcLib.string()), + std::string::npos) + << "withSysroot=" << withSysroot << "\n" << line; + } +} + +// And the interpreter. A sysroot says where headers live; it says nothing +// about which loader runs the result, and gcc's `*link:` no longer answers. +TEST(LinkModelRuntimeDirs, BothModesNameTheInterpreter) { + Payload p; + for (bool withSysroot : {false, true}) { + auto line = joined(tc::resolve_link_model(p.toolchain(withSysroot))); + EXPECT_NE(line.find("--dynamic-linker=" + p.glibcLib.string()), + std::string::npos) + << "withSysroot=" << withSysroot << "\n" << line; + } +} + +} // namespace diff --git a/tests/unit/test_linkmodel.cpp b/tests/unit/test_linkmodel.cpp index 4b7eb1cd..a4f3b5c2 100644 --- a/tests/unit/test_linkmodel.cpp +++ b/tests/unit/test_linkmodel.cpp @@ -180,7 +180,7 @@ TEST(LinkModel, GccSysrootSupplementsMissingKernelHeaders) { EXPECT_NE(lm.compile_flags(ident).find("-isystem"), std::string::npos); } -TEST(LinkModel, GccPayloadUsesIdirafterAndNoLoader) { +TEST(LinkModel, GccPayloadEmitsIdirafterAndItsOwnAddressing) { Tmp dir; auto glibcLib = dir.path / "glibc" / "lib64"; touch(glibcLib / "ld-linux-x86-64.so.2"); @@ -194,12 +194,20 @@ TEST(LinkModel, GccPayloadUsesIdirafterAndNoLoader) { }; auto lm = tc::resolve_link_model(t); EXPECT_EQ(lm.mode, tc::CLibMode::PayloadFirst); + // Headers still differ by driver: libstdc++'s #include_next wrappers need + // -idirafter, and that has not changed. EXPECT_NE(lm.compile_flags(ident).find("-idirafter"), std::string::npos); + + // Addressing no longer differs. GCC used to be left to its install-time + // specs here, which made the RUN side a per-toolchain-install decision + // while the COMPILE side stayed per-build -- and the two named different + // glibc versions as soon as a second one was installed. mcpp already + // refuses to depend on clang's install-time cfg for exactly this reason. auto link = lm.link_flags(ident); EXPECT_NE(link.find(" -B"), std::string::npos); - // GCC's loader/rpath is owned by the specs fixup, not the command line. - EXPECT_EQ(link.find("dynamic-linker"), std::string::npos); - EXPECT_EQ(link.find("-rpath"), std::string::npos); + EXPECT_NE(link.find("--dynamic-linker=" + lm.loader.string()), + std::string::npos); + EXPECT_NE(link.find("-Wl,-rpath," + glibcLib.string()), std::string::npos); } TEST(LinkModel, NothingUsableYieldsNoneAndEmptyFlags) { @@ -212,4 +220,20 @@ TEST(LinkModel, NothingUsableYieldsNoneAndEmptyFlags) { EXPECT_TRUE(lm.link_flags(ident).empty()); } + +// 关于 `find_sibling_tool` 的顺序:**不在这里测**。 +// +// 它的注释曾写着 "Return the first (highest) version dir",而代码只是取 +// directory_iterator 的第一项 —— 没有排序。为它写一条断言的尝试失败了: +// 结果取决于 readdir 顺序,所以那条测试在本机通过、在别的机器上可能失败, +// **通过和失败都不说明问题**。 +// +// 正确的做法不是把不确定性钉住,而是让它不再被用于选版本:版本由权威 +// (`--runtime` / subos 的 `subos_info.runtime`)给出,`find_sibling_tool` +// 只负责定位包根。确定性的判据在 test_toolchain_probe.cpp —— 给定 +// `glibc@2.39`,即使 home 里同时有 2.44,也必须返回 2.39。 +// +// 注释本身已改为陈述事实(顺序未定义,调用方不得依赖)。 +// 设计:.agents/docs/2026-08-08-payload-version-and-contract-drift-design.md §3.2 + } // namespace diff --git a/tests/unit/test_runtime_binding_fallback.cpp b/tests/unit/test_runtime_binding_fallback.cpp new file mode 100644 index 00000000..b61e0d95 --- /dev/null +++ b/tests/unit/test_runtime_binding_fallback.cpp @@ -0,0 +1,132 @@ +// When nothing declares the runtime, how far may mcpp go on its own? +// +// The rule this replaced asked a directory for "the glibc" and took the first +// entry readdir yielded. That is a CHOICE made by something with no bearing on +// what the artifact loads, and it went wrong the moment a dependency's +// `xim:glibc@>=2.38` floor installed a second payload: the compile side took +// 2.44 while the interpreter still named 2.39. +// +// So the axis is not "may mcpp look at the payload directory" -- it is +// "is there anything to choose between". One payload is an answer; two are a +// question only the subos can settle. +// +// This mattered because the compatibility sources (gcc's specs, clang's cfg, +// the compiler's PT_INTERP) are all things some earlier mechanism wrote, and a +// machine can legitimately have none of them. + +#include + +import std; +import mcpp.toolchain.post_install; + +namespace tc = mcpp::toolchain; + +namespace { + +struct Home { + std::filesystem::path root, compiler; + explicit Home(std::initializer_list glibcVersions) { + root = std::filesystem::temp_directory_path() + / std::format("mcpp_binding_{}", std::random_device{}()); + auto xpkgs = root / "data" / "xpkgs"; + compiler = xpkgs / "xim-x-gcc" / "16.1.0" / "bin" / "g++"; + std::filesystem::create_directories(compiler.parent_path()); + std::ofstream(compiler) << "not an elf"; + for (auto v : glibcVersions) + std::filesystem::create_directories( + xpkgs / "xim-x-glibc" / v / "lib64"); + } + ~Home() { std::error_code ec; std::filesystem::remove_all(root, ec); } +}; + +TEST(RuntimeBindingFallback, OnePayloadIsAnAnswer) { + Home h{"2.39"}; + EXPECT_EQ(tc::baked_runtime_binding(h.compiler), "glibc@2.39"); +} + +// The incident, in one assertion. Two payloads and nothing declaring which: +// silence, not a coin flip. +TEST(RuntimeBindingFallback, TwoPayloadsIsSilence) { + Home h{"2.39", "2.44"}; + EXPECT_EQ(tc::baked_runtime_binding(h.compiler), ""); +} + +TEST(RuntimeBindingFallback, NoPayloadIsSilence) { + Home h{}; + EXPECT_EQ(tc::baked_runtime_binding(h.compiler), ""); +} + +// A compiler outside any xpkgs tree (a system gcc) has no payload set to read, +// and must not acquire one from somewhere else. +TEST(RuntimeBindingFallback, NonPayloadCompilerGetsNothing) { + EXPECT_EQ(tc::baked_runtime_binding("/usr/bin/g++"), ""); +} + +TEST(RuntimeBindingFallback, EmptyPathIsSilence) { + EXPECT_EQ(tc::baked_runtime_binding({}), ""); +} + +// A record that names a payload which is no longer installed. +// +// specs, cfg and PT_INTERP are all RECORDS of a past state: written at some +// install or some fixup, and never revisited when the payload underneath is +// replaced. CI hit exactly this -- a record said glibc@2.39 while the only +// payload on disk was 2.44 -- and because the probe matches exactly and never +// falls back, nothing resolved, no loader reached the link line, and the +// artifact took the host's. A fossil naming something absent is not an +// authority; it is just old, and the resolution must carry on past it. +struct HomeWithSpecs { + std::filesystem::path root, compiler; + HomeWithSpecs(std::initializer_list glibcVersions, + std::string_view specsNames) { + root = std::filesystem::temp_directory_path() + / std::format("mcpp_fossil_{}", std::random_device{}()); + auto xpkgs = root / "data" / "xpkgs"; + auto gccRoot = xpkgs / "xim-x-gcc" / "16.1.0"; + compiler = gccRoot / "bin" / "g++"; + std::filesystem::create_directories(compiler.parent_path()); + std::ofstream(compiler) << "not an elf"; + for (auto v : glibcVersions) + std::filesystem::create_directories(xpkgs / "xim-x-glibc" / v / "lib64"); + auto sd = gccRoot / "lib" / "gcc" / "x86_64-linux-gnu" / "16.1.0"; + std::filesystem::create_directories(sd); + std::ofstream(sd / "specs") + << "*link:\n%{!static:--dynamic-linker " << xpkgs.string() + << "/xim-x-glibc/" << specsNames << "/lib64/ld-linux-x86-64.so.2}\n"; + } + ~HomeWithSpecs() { std::error_code ec; std::filesystem::remove_all(root, ec); } +}; + +TEST(RuntimeBindingFallback, RecordNamingAnAbsentPayloadIsSkipped) { + HomeWithSpecs h{{"2.44"}, "2.39"}; + // Not "glibc@2.39": that payload is gone. The singleton rule answers. + EXPECT_EQ(tc::baked_runtime_binding(h.compiler), "glibc@2.44"); +} + +// Linux only, and not as an exemption -- the mechanism does not exist +// elsewhere. detect_baked_loader parses GCC specs, whose loader paths are +// POSIX and absolute, and its character whitelist deliberately excludes `\` +// and `:` so that a `%{...}` spec body is never swallowed. A Windows temp +// directory is `C:\Users\...`, so this fixture cannot be spelled there at +// all: the scan stops at the first backslash and the result does not start +// with `/`. Rightly so -- there are no glibc payloads or ld-linux loaders on +// Windows for it to find. +// +// The other cases in this file are platform-neutral (they exercise the +// payload set, not the parser) and keep running everywhere. +TEST(RuntimeBindingFallback, RecordNamingAPresentPayloadWins) { + if constexpr (!std::filesystem::path::preferred_separator || + std::filesystem::path::preferred_separator != '/') + GTEST_SKIP() << "GCC specs loader paths are POSIX; no such record here"; + HomeWithSpecs h{{"2.39", "2.44"}, "2.39"}; + // Two installed, so the singleton rule cannot answer -- but the record + // names one that IS there, and that is what the artifact would load. + EXPECT_EQ(tc::baked_runtime_binding(h.compiler), "glibc@2.39"); +} + +TEST(RuntimeBindingFallback, StaleRecordWithNothingToFallBackOnIsSilence) { + HomeWithSpecs h{{"2.39", "2.44"}, "2.28"}; + EXPECT_EQ(tc::baked_runtime_binding(h.compiler), ""); +} + +} // namespace diff --git a/tests/unit/test_subos_info.cpp b/tests/unit/test_subos_info.cpp index 95385778..a7f27af2 100644 --- a/tests/unit/test_subos_info.cpp +++ b/tests/unit/test_subos_info.cpp @@ -37,14 +37,14 @@ TEST(SubosInfo, ReadsRuntimeAndEnvDeclarations) { "subos_info": { "schema_version": 1, "runtime": "glibc@2.39", - "envs": [ - { "binding": "mesa@25.0.7.1", "decls": [ + "envs": { + "mesa@25.0.7.1": [ { "var": "LIBGL_DRIVERS_PATH", "op": "prepend", "value": "${subosdir}/usr/lib/dri" }, { "var": "XDG_DATA_DIRS", "op": "prepend", "value": "${subosdir}/share" } - ]} - ] + ] + } } })"); auto info = su::read(t.dir); @@ -64,8 +64,8 @@ TEST(SubosInfo, ReadsRuntimeAndEnvDeclarations) { TEST(SubosInfo, ResolvesSubosdirPlaceholder) { Tmp t; t.write(R"({"subos_info":{"schema_version":1,"runtime":"glibc@2.39", - "envs":[{"binding":"mesa@1","decls":[ - {"var":"LIBGL_DRIVERS_PATH","op":"prepend","value":"${subosdir}/usr/lib/dri"}]}]}})"); + "envs":{"mesa@1":[ + {"var":"LIBGL_DRIVERS_PATH","op":"prepend","value":"${subosdir}/usr/lib/dri"}]}}})"); auto env = su::resolve_env(su::read(t.dir), t.dir); ASSERT_EQ(env.size(), 1u); EXPECT_EQ(env[0].first, "LIBGL_DRIVERS_PATH"); @@ -81,11 +81,9 @@ TEST(SubosInfo, ResolvesSubosdirPlaceholder) { // EGL vendor directory. `prepend` joins them; it must not drop either. TEST(SubosInfo, PrependJoinsProvidersInOrder) { Tmp t; - t.write(R"({"subos_info":{"schema_version":1,"runtime":"glibc@2.39","envs":[ - {"binding":"a-mesa@1","decls":[ - {"var":"V","op":"prepend","value":"${subosdir}/one"}]}, - {"binding":"b-vendor@1","decls":[ - {"var":"V","op":"prepend","value":"${subosdir}/two"}]}]}})"); + t.write(R"({"subos_info":{"schema_version":1,"runtime":"glibc@2.39","envs":{ + "a-mesa@1":[{"var":"V","op":"prepend","value":"${subosdir}/one"}], + "b-vendor@1":[{"var":"V","op":"prepend","value":"${subosdir}/two"}]}}})"); auto env = su::resolve_env(su::read(t.dir), t.dir); ASSERT_EQ(env.size(), 1u); const auto sep = mcpp::platform::env::path_list_separator(); @@ -98,9 +96,9 @@ TEST(SubosInfo, PrependJoinsProvidersInOrder) { // would otherwise grow the variable without bound. TEST(SubosInfo, PrependDeduplicates) { Tmp t; - t.write(R"({"subos_info":{"schema_version":1,"runtime":"glibc@2.39","envs":[ - {"binding":"a@1","decls":[{"var":"V","op":"prepend","value":"${subosdir}/x"}]}, - {"binding":"b@1","decls":[{"var":"V","op":"prepend","value":"${subosdir}/x"}]}]}})"); + t.write(R"({"subos_info":{"schema_version":1,"runtime":"glibc@2.39","envs":{ + "a@1":[{"var":"V","op":"prepend","value":"${subosdir}/x"}], + "b@1":[{"var":"V","op":"prepend","value":"${subosdir}/x"}]}}})"); auto env = su::resolve_env(su::read(t.dir), t.dir); ASSERT_EQ(env.size(), 1u); // One entry, not two. The de-duplication has to split on the PLATFORM's @@ -114,9 +112,9 @@ TEST(SubosInfo, PrependDeduplicates) { // `set` replaces rather than joins — xlings's own precedence. TEST(SubosInfo, SetReplaces) { Tmp t; - t.write(R"({"subos_info":{"schema_version":1,"runtime":"glibc@2.39","envs":[ - {"binding":"a@1","decls":[{"var":"V","op":"prepend","value":"/one"}]}, - {"binding":"b@1","decls":[{"var":"V","op":"set","value":"/two"}]}]}})"); + t.write(R"({"subos_info":{"schema_version":1,"runtime":"glibc@2.39","envs":{ + "a@1":[{"var":"V","op":"prepend","value":"/one"}], + "b@1":[{"var":"V","op":"set","value":"/two"}]}}})"); auto env = su::resolve_env(su::read(t.dir), t.dir); ASSERT_EQ(env.size(), 1u); EXPECT_EQ(env[0].second, "/two"); @@ -177,4 +175,78 @@ TEST(SubosInfo, FamilyOfMirrorsXlings) { EXPECT_EQ(su::family_of("glibc"), "linux-x86_64-glibc"); } + +// A VERBATIM capture of what a real xlings wrote, after `xlings install +// graphics` on an NVIDIA host. Reformatted for width and nothing else -- keys, +// nesting and spelling are as found on disk. +// +// This test exists because its absence shipped a broken feature. The first +// version of this file hand-wrote every fixture in a shape the reader also +// expected and xlings never produces: `envs` as an array of {binding, decls}. +// Ten tests passed against a format that does not exist, and against a real +// subos the released build applied no variables at all -- silently, because +// "no providers" and "nothing declared" look identical. +// +// A fixture composed from the same understanding as the parser cannot catch +// that. Only one taken from the writer can. +TEST(SubosInfo, RealXlingsCapture) { + Tmp t; + t.write(R"({ + "subos_info": { + "created_at": "2026-08-08T01:40:00Z", + "created_by": "xlings 2026.8.7.1", + "runtime": "glibc@2.39", + "schema_version": 1, + "envs": { + "mesa@25.0.7.1": [ + {"op": "prepend", "value": "${subosdir}/usr/lib/dri", "var": "LIBGL_DRIVERS_PATH"}, + {"op": "prepend", "value": "${subosdir}/share/glvnd/egl_vendor.d", "var": "__EGL_VENDOR_LIBRARY_DIRS"}, + {"op": "prepend", "value": "${subosdir}/share", "var": "XDG_DATA_DIRS"} + ], + "nvidia-gl-host-link@0.1.1": [ + {"op": "prepend", "value": "${subosdir}/share/glvnd/egl_vendor.d", "var": "__EGL_VENDOR_LIBRARY_DIRS"} + ] + } + }, + "workspace": {} + })"); + + auto info = su::read(t.dir); + ASSERT_TRUE(info.present); + EXPECT_EQ(info.runtime, "glibc@2.39"); + ASSERT_EQ(info.providers.size(), 2u); + EXPECT_EQ(info.providers[0].binding, "mesa@25.0.7.1"); + EXPECT_EQ(info.providers[1].binding, "nvidia-gl-host-link@0.1.1"); + + auto env = su::resolve_env(info, t.dir); + ASSERT_EQ(env.size(), 3u) << "all three graphics variables must be produced"; + + std::map byVar; + for (auto& [k, v] : env) byVar[k] = v; + const auto sep = mcpp::platform::env::path_list_separator(); + EXPECT_EQ(byVar["LIBGL_DRIVERS_PATH"], t.dir.string() + "/usr/lib/dri"); + EXPECT_EQ(byVar["XDG_DATA_DIRS"], t.dir.string() + "/share"); + // Both providers name the same vendor directory; de-duplication must + // leave exactly one, or libglvnd sees it twice and enumerates the device + // twice -- which is a defect xlings hit on its own side. + EXPECT_EQ(byVar["__EGL_VENDOR_LIBRARY_DIRS"], + t.dir.string() + "/share/glvnd/egl_vendor.d"); + EXPECT_EQ(byVar["__EGL_VENDOR_LIBRARY_DIRS"].find(sep), std::string::npos); +} + +// xlings drops a declaration whose op it does not recognise. A reader more +// permissive than its writer eventually applies something the writer meant to +// reject, so this asserts the same refusal rather than a tolerant guess. +TEST(SubosInfo, UnknownOpIsDroppedLikeXlingsDrops) { + Tmp t; + t.write(R"({"subos_info":{"schema_version":1,"runtime":"glibc@2.39","envs":{ + "a@1":[{"var":"V","op":"append","value":"/nope"}, + {"var":"W","op":"prepend","value":"/yes"}, + {"var":"","op":"prepend","value":"/no-name"}]}}})"); + auto env = su::resolve_env(su::read(t.dir), t.dir); + ASSERT_EQ(env.size(), 1u); + EXPECT_EQ(env[0].first, "W"); + EXPECT_EQ(env[0].second, "/yes"); +} + } // namespace diff --git a/tests/unit/test_sysroot_ownership.cpp b/tests/unit/test_sysroot_ownership.cpp new file mode 100644 index 00000000..56e1f6be --- /dev/null +++ b/tests/unit/test_sysroot_ownership.cpp @@ -0,0 +1,87 @@ +// A sysroot is trusted because of where it lives, not because it is there. +// +// gcc records `--sysroot=<...>/.xlings/subos/default` as a literal string when +// it is built, and keeps reporting that string in every project it later +// serves. On a machine with several checkouts, that path routinely EXISTS and +// belongs to a different one -- so the original predicate, "does the path +// exist", answered yes and the build silently took another project's headers. +// Measured: a build inside mcpp resolved a sysroot under an unrelated repo. +// +// These assertions pin the axis that replaced it. Nothing here touches the +// filesystem, deliberately: the point is that existence is not consulted. + +#include + +import std; +import mcpp.fallback.probe_sysroot; + +namespace fb = mcpp::fallback; + +namespace { + +const std::filesystem::path kRegistry = "/home/u/.mcpp/registry"; +const std::filesystem::path kProject = "/home/u/work/thisrepo"; + +TEST(SysrootOwnership, PayloadSysrootIsOwned) { + EXPECT_FALSE(fb::sysroot_is_foreign( + kRegistry / "subos" / "default", kRegistry, kProject)); +} + +TEST(SysrootOwnership, ProjectLocalSysrootIsOwned) { + EXPECT_FALSE(fb::sysroot_is_foreign( + kProject / ".xlings" / "subos" / "default", kRegistry, kProject)); +} + +// The case that motivated the predicate: a path that exists, is spelled +// exactly like a legitimate one, and belongs to someone else. +TEST(SysrootOwnership, AnotherCheckoutIsForeign) { + EXPECT_TRUE(fb::sysroot_is_foreign( + "/home/u/work/otherrepo/.xlings/subos/default", kRegistry, kProject)); +} + +TEST(SysrootOwnership, AnotherHomeIsForeign) { + EXPECT_TRUE(fb::sysroot_is_foreign( + "/home/other/.mcpp/registry/subos/default", kRegistry, kProject)); +} + +// Both anchors are load-bearing. Registry alone would condemn every +// project-local tree; project alone would condemn every payload sysroot. +TEST(SysrootOwnership, EitherAnchorSuffices) { + EXPECT_FALSE(fb::sysroot_is_foreign( + kRegistry / "subos" / "default", kRegistry, {})); + EXPECT_FALSE(fb::sysroot_is_foreign( + kProject / ".xlings" / "subos" / "default", {}, kProject)); +} + +// No sysroot is not a foreign sysroot. gcc payloads on this machine report an +// empty `-print-sysroot`, so this is the ordinary case, not an edge one. +TEST(SysrootOwnership, EmptyIsNotForeign) { + EXPECT_FALSE(fb::sysroot_is_foreign({}, kRegistry, kProject)); +} + +// A prefix that merely shares characters is not containment. +TEST(SysrootOwnership, SiblingPrefixIsNotContainment) { + EXPECT_TRUE(fb::sysroot_is_foreign( + "/home/u/work/thisrepo-backup/.xlings/subos/default", + kRegistry, kProject)); +} + +// A directory whose NAME begins with two dots is not an escape. The first +// version compared the relative path's text (`rfind("..", 0)`), which called +// `/home/u/.mcpp/registry/..cache` an escape from the registry -- and did not +// compile at all on Windows, where `native()` is a wstring. Containment is a +// question about path components, so it is asked of components. +TEST(SysrootOwnership, DotDotPrefixedNameIsNotAnEscape) { + EXPECT_FALSE(fb::sysroot_is_foreign( + kRegistry / "..cache" / "subos" / "default", kRegistry, kProject)); +} + +TEST(SysrootOwnership, PathIsUnderIsDirectlyTestable) { + EXPECT_TRUE(fb::path_is_under(kRegistry / "a" / "b", kRegistry)); + EXPECT_TRUE(fb::path_is_under(kRegistry, kRegistry)); + EXPECT_FALSE(fb::path_is_under(kRegistry.parent_path(), kRegistry)); + EXPECT_FALSE(fb::path_is_under(kRegistry, {})); + EXPECT_FALSE(fb::path_is_under({}, kRegistry)); +} + +} // namespace diff --git a/tests/unit/test_toolchain_probe.cpp b/tests/unit/test_toolchain_probe.cpp new file mode 100644 index 00000000..8eff0b4b --- /dev/null +++ b/tests/unit/test_toolchain_probe.cpp @@ -0,0 +1,121 @@ +// Payload resolution: the version comes from an AUTHORITY, never from a scan. +// +// Before this, probe_payload_paths asked find_sibling_tool for "the glibc" +// and took whatever came back -- which is whatever readdir yielded first. With +// one glibc installed that is always right, so nothing ever forced it to be +// correct. Installing a second one (a dependency with a `>=` floor is enough) +// split the build in half: the compile side took one version while the +// artifact's interpreter, written into gcc's specs at install time, still +// named the other. Binaries then referenced GLIBC_2.42 symbols against a 2.39 +// runtime, and the failures landed on packages with nothing to do with the +// dependency that pulled the second glibc in. +// +// These assertions are about determinism, not about preference. "Given +// glibc@2.39, return 2.39" must hold with 2.44 also present and must not +// depend on directory order -- which is why the fixture always installs both. +// +// Design: .agents/docs/2026-08-08-payload-version-and-contract-drift-design.md §3.2 + +#include + +import std; +import mcpp.toolchain.probe; + +namespace { + +namespace tc = mcpp::toolchain; + +struct Tmp { + std::filesystem::path path; + Tmp() { + path = std::filesystem::temp_directory_path() + / std::format("mcpp_probe_test_{}", std::random_device{}()); + std::filesystem::create_directories(path); + } + ~Tmp() { std::error_code ec; std::filesystem::remove_all(path, ec); } +}; + +void touch(const std::filesystem::path& p) { + std::filesystem::create_directories(p.parent_path()); + std::ofstream(p) << "x"; +} + +// A home with TWO glibc payloads. Both are always present: a fixture with one +// cannot tell a resolver that reads the authority from one that guesses. +struct TwoGlibcHome { + Tmp dir; + std::filesystem::path compilerBin; + TwoGlibcHome() { + auto xpkgs = dir.path / "data" / "xpkgs"; + compilerBin = xpkgs / "xim-x-gcc" / "16.1.0" / "bin" / "g++"; + touch(compilerBin); + for (auto v : {"2.39", "2.44"}) { + auto root = xpkgs / "xim-x-glibc" / v; + touch(root / "include" / "features.h"); + touch(root / "lib64" / "libc.so.6"); + touch(root / "lib64" / "ld-linux-x86-64.so.2"); + } + } + std::filesystem::path libOf(std::string_view v) const { + return dir.path / "data" / "xpkgs" / "xim-x-glibc" / std::string(v) / "lib64"; + } +}; + +TEST(PayloadProbe, ResolvesTheBindingExactly) { + TwoGlibcHome h; + auto pp = tc::probe_payload_paths(h.compilerBin, "glibc@2.39"); + ASSERT_TRUE(pp.has_value()); + EXPECT_EQ(pp->glibcLib, h.libOf("2.39")) + << "the authority said 2.39 and 2.44 is also installed; a resolver " + "that scans would be free to return either"; +} + +TEST(PayloadProbe, ResolvesTheOtherBindingExactly) { + TwoGlibcHome h; + auto pp = tc::probe_payload_paths(h.compilerBin, "glibc@2.44"); + ASSERT_TRUE(pp.has_value()); + EXPECT_EQ(pp->glibcLib, h.libOf("2.44")); +} + +// No authority is not "pick something sensible" -- it is a refusal. Guessing +// here is what produced a build whose compile and run halves disagreed, and +// the guess is invisible: every artifact looks normal until one links a +// library built against the other version. +TEST(PayloadProbe, NoBindingIsARefusalNotAGuess) { + TwoGlibcHome h; + EXPECT_FALSE(tc::probe_payload_paths(h.compilerBin, "").has_value()); +} + +// A binding naming a version that is not installed must fail rather than fall +// back to one that is. Falling back would reintroduce the split silently. +TEST(PayloadProbe, MissingVersionDoesNotFallBack) { + TwoGlibcHome h; + EXPECT_FALSE(tc::probe_payload_paths(h.compilerBin, "glibc@2.28").has_value()); +} + +// The binding's NAME matters too: `musl@1.2.5` must not resolve a glibc +// payload just because the version string happens to be parseable. +TEST(PayloadProbe, WrongRuntimeFamilyDoesNotResolveGlibc) { + TwoGlibcHome h; + EXPECT_FALSE(tc::probe_payload_paths(h.compilerBin, "musl@1.2.5").has_value()); +} + +// Layout: lib64 preferred, lib accepted. Kept from the original probe -- a +// payload that ships only `lib` is a real shape, and this is the one piece of +// convention that stays, because it is about WHERE inside a chosen payload, +// not about WHICH payload. +TEST(PayloadProbe, AcceptsLibWhenThereIsNoLib64) { + Tmp dir; + auto xpkgs = dir.path / "data" / "xpkgs"; + auto bin = xpkgs / "xim-x-gcc" / "16.1.0" / "bin" / "g++"; + touch(bin); + auto root = xpkgs / "xim-x-glibc" / "2.39"; + touch(root / "include" / "features.h"); + touch(root / "lib" / "ld-linux-x86-64.so.2"); + + auto pp = tc::probe_payload_paths(bin, "glibc@2.39"); + ASSERT_TRUE(pp.has_value()); + EXPECT_EQ(pp->glibcLib, root / "lib"); +} + +} // namespace diff --git a/tests/unit/test_xlings_version_pin.cpp b/tests/unit/test_xlings_version_pin.cpp new file mode 100644 index 00000000..131c5e8a --- /dev/null +++ b/tests/unit/test_xlings_version_pin.cpp @@ -0,0 +1,66 @@ +// The vendored xlings is replaced when it is BEHIND the pin, and only then. +// +// acquire_xlings_binary used to return on mere existence, so a home kept +// whatever xlings it first acquired forever -- measured at 2026.8.2.1 against +// a pin of 2026.8.6.3, with `mcpp self env` printing both numbers next to each +// other and saying nothing. That is not cosmetic: the subos_info block arrived +// in xlings 2026.8.5.1, so on that machine the graphics packages' declarations +// were discarded by a client too old to have the API, and the fix for +// mcpp#352 could never take effect no matter how current mcpp itself was. +// +// The comparison is STRICTLY older, not "not equal". A user who deliberately +// put a newer xlings there must not be downgraded by an mcpp pinning an older +// one, and an unparseable version is not evidence of being behind. + +#include + +import std; +import mcpp.fallback.xlings_binary; + +namespace fb = mcpp::fallback; + +namespace { + +TEST(XlingsVersionPin, OlderIsBehind) { + EXPECT_TRUE(fb::version_is_older("2026.8.2.1", "2026.8.6.3")); + EXPECT_TRUE(fb::version_is_older("2026.7.31.3", "2026.8.1.1")); + EXPECT_TRUE(fb::version_is_older("2026.8.6", "2026.8.6.1")); +} + +TEST(XlingsVersionPin, EqualIsNotBehind) { + EXPECT_FALSE(fb::version_is_older("2026.8.6.3", "2026.8.6.3")); + // Trailing zeros compare equal, not older. + EXPECT_FALSE(fb::version_is_older("2026.8.6.0", "2026.8.6")); +} + +// The case that decides whether this is safe to run automatically: a user who +// put a NEWER xlings in place keeps it. +TEST(XlingsVersionPin, NewerIsNotDowngraded) { + EXPECT_FALSE(fb::version_is_older("2026.9.1.1", "2026.8.6.3")); + EXPECT_FALSE(fb::version_is_older("2027.1.1.1", "2026.12.31.9")); +} + +// A version this code cannot parse says nothing, and "says nothing" must not +// be read as "is behind" -- that would delete a working binary on a guess. +TEST(XlingsVersionPin, UnparseableIsNotBehind) { + EXPECT_FALSE(fb::version_is_older("dev", "2026.8.6.3")); + EXPECT_FALSE(fb::version_is_older("2026.8.6.3", "")); + EXPECT_FALSE(fb::version_is_older("", "2026.8.6.3")); + EXPECT_FALSE(fb::version_is_older("2026.8.x", "2026.8.6.3")); +} + +// The version scheme changed epochs: xlings went 0.4.x -> YYYY.M.D.N. Both +// live on the same disk, and a home's system xlings may still be a 0.4.x while +// its vendored one is already dated. Getting this backwards is not academic -- +// the first cut of the replace-when-behind logic deleted the vendored binary +// and re-acquired from `which xlings`, which on this developer machine turned +// 2026.8.2.1 into 0.4.51: older still, and equally missing the very feature +// the replacement existed to restore. Being behind the pin justifies looking +// for a replacement; it does not justify accepting whatever turns up. +TEST(XlingsVersionPin, DatedSchemeIsNewerThanTheOldOne) { + EXPECT_FALSE(fb::version_is_older("2026.8.2.1", "0.4.51")); + EXPECT_TRUE(fb::version_is_older("0.4.51", "2026.8.2.1")); + EXPECT_TRUE(fb::version_is_older("0.4.51", "0.4.54")); +} + +} // namespace