From 3ae3191d4aab278e0d0a4dbe4f16f538f57ff9ce Mon Sep 17 00:00:00 2001 From: speak-agent Date: Sat, 8 Aug 2026 08:25:11 +0800 Subject: [PATCH 01/21] docs: cross-repo implementation plan for compiler-as-capability --- ...piler-as-capability-implementation-plan.md | 148 ++++++++++++++++++ 1 file changed, 148 insertions(+) create mode 100644 .agents/docs/2026-08-08-compiler-as-capability-implementation-plan.md diff --git a/.agents/docs/2026-08-08-compiler-as-capability-implementation-plan.md b/.agents/docs/2026-08-08-compiler-as-capability-implementation-plan.md new file mode 100644 index 00000000..ac50799c --- /dev/null +++ b/.agents/docs/2026-08-08-compiler-as-capability-implementation-plan.md @@ -0,0 +1,148 @@ +# 编译器是能力 —— 跨仓实施计划 + +> **For agentic workers:** 逐 task 执行,每个 task 自带 red→green→commit。**T1 是所有后续 task 的前提**(没有它,后面每一步的观测都不可信)。 + +**Goal:** 让 mcpp 停止依赖编译器的安装期配置,把「用哪份 libc」收敛成一个权威;顺带把已经污染的产物清干净,并让多 subos 构建真正可用。 + +**设计文档:** +- `.agents/docs/2026-08-07-xlings-as-runtime-substrate-design.md`(运行时底座;S1/S3 与撤销 `c_runtime` 的理由) +- `.agents/docs/2026-08-08-payload-version-and-contract-drift-design.md`(本计划的主体) + +**Tech Stack:** C++23 modules,gtest(`tests/unit/`),shell e2e(`tests/e2e/`),Lua(mcpp-index)。 + +## Global Constraints + +- **不改 payload**。`patchelf_walk`(让编译器自己能跑)保留;**写 payload 内配置的一律删掉**。 +- **权威唯一**:`--runtime` → `[xlings] subos` 的 `subos_info.runtime` → **报错,不猜**。删掉「扫目录挑一个」。 +- **平台差异下沉** `src/platform/`,`build/` 里不写 `#ifdef`。 +- **每一处「注释描述行为」必须有对应断言**(R-C)。本轮已发现三处注释与代码不符。 +- **跨仓 wire format 的 fixture 取自写入方**(R-A),不得手写。 +- **一条测试若为覆盖某路径而存在,必须先断言走了那条路径**(R-B)。 +- **不新增任何让产物链宿主 libc 的开关**(8-07 §5.2 已否)。 + +--- + +### T1: 三处「注释撒谎」先钉住 —— 后续观测的地基 + +**Files:** `tests/unit/test_linkmodel.cpp`(新断言)、`src/xlings.cppm`、`src/toolchain/post_install.cppm`(改注释) + +本轮发现三处注释描述了代码没有的行为。**先让它们不能再撒谎**,否则后面每一步都可能在错误的前提上推理。 + +- [ ] **Step 1** 写失败单测:`find_sibling_tool` 的注释说「first (highest)」,断言**两个版本目录时返回的是排序后最高的那个** +- [ ] **Step 2** 跑,确认 FAIL(今天返回 readdir 第一个) +- [ ] **Step 3** 二选一并使注释与代码一致:①真的排序;②改注释为「未定义顺序」并让调用方不得依赖。**本计划选 ②**——T2 会让它不再被用于选版本 +- [ ] **Step 4** 另两处:`fixup_gcc_specs` 的「Idempotent」(跨 home 不幂等)、`execute.cppm` reader 的「旧缓存视为 miss」(T5 修)——先把注释改成事实 +- [ ] **Step 5** Commit + +--- + +### T2: 权威降级链 —— 没有权威就不进 PayloadFirst + +**Files:** `src/toolchain/probe.cppm`、`src/xlings/subos_info.cppm`、`src/build/prepare.cppm` +**Test:** `tests/unit/test_toolchain_probe.cpp`(新) + +**Interfaces:** +```cpp +// probe 不再自己选版本;版本由权威给出 +std::optional probe_payload_paths( + const std::filesystem::path& compilerBin, + std::string_view runtimeBinding); // "glibc@2.39";空 ⇒ 不进 PayloadFirst +``` + +- [ ] **Step 1** 单测:给定 `glibc@2.39` 且 home 内有 2.39/2.44 两份 ⇒ **必须返回 2.39**;给定空 binding ⇒ **返回 nullopt**(不猜) +- [ ] **Step 2** 跑,FAIL +- [ ] **Step 3** 实现;`prepare.cppm` 按 `--runtime` → `[xlings] subos` 的 `subos_info.runtime` 顺序解析,末级不猜 +- [ ] **Step 4** 跑,PASS +- [ ] **Step 5** Commit + +--- + +### T3: fingerprint 加入 runtime 轴 —— 多 subos 的前置 + +**Files:** `src/toolchain/fingerprint.cppm`、`tests/unit/test_fingerprint.cpp` + +不加这条,切 subos 不改 fingerprint ⇒ `target//` 被复用 ⇒ 里面是对另一份 glibc 编译的对象。**比今天更坏。** + +- [ ] **Step 1** 单测:两个只有 `runtimeBinding` 不同的输入 ⇒ **fingerprint 必须不同** +- [ ] **Step 2/3/4** red → 加字段 → green +- [ ] **Step 5** Commit + +--- + +### T4: GCC 也显式发 loader/rpath + clean specs + +**Files:** `src/toolchain/linkmodel.cppm`、`src/build/flags.cppm`、`src/toolchain/post_install.cppm`(删 `fixup_gcc_specs`) +**Test:** `tests/unit/test_linkmodel.cpp`、`tests/e2e/201_gcc_clean_specs.sh`(新) + +**已实测**:`-Wl,--dynamic-linker` 压得过 specs;`-specs=<无 `+` 定义>` 是替换;`g++ -dumpspecs` 给内建 specs(`/tmp` 路径 0 条);真实构建(模块 + `import std` + 共享库)RUNPATH 68→2、死路径 0、可运行。 + +- [ ] **Step 1** e2e:gcc 构建的产物 **RUNPATH 中不得有 `/tmp/tmp.`**,且 interpreter 是权威指定的那份,且能跑 +- [ ] **Step 2** 跑,FAIL(今天有 34+ 条) +- [ ] **Step 3** ①去掉 `linkmodel` 的 `if (clangDriver)` 门;②构建期生成 clean specs 到 **build dir**,`-specs=` 传入;③**删掉 `fixup_gcc_specs`**(保留 `patchelf_walk`) +- [ ] **Step 4** 跑,PASS +- [ ] **Step 5** Commit + +--- + +### T5: 并回 #377 的两处修复 + +**Files:** `src/xlings/subos_info.cppm`、`src/build/execute.cppm`、`tests/unit/test_subos_info.cpp`、`tests/e2e/200_subos_env_reaches_program.sh` + +已在 `fix/fast-run-stale-cache-subos` 分支上完成并 18/18 CI 通过,**原样带过来**: + +- [ ] **Step 1** `git cherry-pick` 该分支的三个 commit(wire format / 旧缓存 / e2e fixture) +- [ ] **Step 2** 跑单测 + e2e,确认仍绿 +- [ ] **Step 3** Commit(或保留 cherry-pick 的原始提交) + +--- + +### T6: D4 沙箱 xlings 版本比对 + D5 陈旧 sysroot + +**Files:** `src/fallback/xlings_binary.cppm`、`src/doctor.cppm`、`src/fallback/probe_sysroot.cppm` + +- [ ] **Step 1** 单测/e2e:vendored xlings 版本**低于 pin** ⇒ 被替换;**高于 pin** ⇒ 不动 +- [ ] **Step 2** red +- [ ] **Step 3** 实现;doctor 增加两条 finding:①xlings 低于 pin;②`--sysroot` 指向当前项目之外 +- [ ] **Step 4** green +- [ ] **Step 5** Commit + +--- + +### T7: 文档 + 版本 + PR + +- [ ] `docs/03-toolchains.md`:权威降级链、`[xlings] subos`、多 subos 构建 +- [ ] `docs/05-mcpp-toml.md`:`[xlings] subos` 与 runtime 的关系 +- [ ] 中文版同步 +- [ ] `MCPP_VERSION` / `mcpp.toml` → `2026.8.8.2`;`kXlingsVersion` → 最新;`check_version_pins.sh` 通过 +- [ ] 全量 `mcpp test` + 关键 e2e +- [ ] 开 PR + +--- + +### T8: mcpp-index 图形栈重新落地(D1 修好之后) + +**Files（mcpp-index）:** `pkgs/c/compat.glx-runtime.lua`、`pkgs/c/compat.glfw.lua` + +- [ ] **Step 1** 新增 e2e/验证:**装 `xim:graphics` 前后**,各跑一次与图形无关的成员,断言产物 `PT_INTERP` 与 `readelf -V` 的 glibc 符号上界**不变**。这是上次事故真正缺失的那个测试 +- [ ] **Step 2** 重新应用 `f44e896` 的两个文件改动(deps 放平台层) +- [ ] **Step 3** CI 全绿后合入 + +--- + +## Self-Review + +| 设计文档条目 | Task | +|---|---| +| 8-08 §3.1 原则(编译器当能力) | T4 | +| 8-08 §3.2 权威降级链 / D1 | T2 | +| 8-08 §3.5① 去掉 clangDriver 门 | T4 | +| 8-08 §3.5② clean specs 进 build dir | T4 | +| 8-08 §3.5③ 删 `fixup_gcc_specs` / D6 | T4 | +| 8-08 §3.6 多 subos + fingerprint 前置 | T2 + T3 | +| 8-08 D4 沙箱 xlings / D5 sysroot | T6 | +| 8-08 D2 wire format / D3 旧缓存 | T5 | +| 8-08 §4.2 R-A/R-B/R-C | T1(R-C)+ 各 task 的测试形态 | +| 8-07 §1.5 图形栈迁移 | T8 | +| 8-07 §3-S2 不加链宿主 libc 的开关 | Global Constraints(不做) | + +**未覆盖且是有意的**:8-07 §7「xlings 落盘 exports」—— 8-08 已说明它不再是关键路径(权威改为 `subos_info.runtime`)。Q6(交叉/musl/MinGW 下 `-specs=` 替换)在 T4 的 e2e 里只覆盖 native;**其余平台列为 PR 中显式声明的未验项**,不假装验过。 From 111949e9a97fa2d5e38360ae87cb49cc48231b52 Mon Sep 17 00:00:00 2001 From: speak-agent Date: Sat, 8 Aug 2026 08:27:52 +0800 Subject: [PATCH 02/21] docs(code): three comments that described behaviour the code does not have All three were found while investigating why a second glibc in the home broke builds, and each one had sent a reader -- me -- reasoning from a false premise. find_sibling_tool said "Return the first (highest) version dir". It sorts nothing; it returns whatever readdir yields. With two versions installed the answer varies by filesystem. fixup_gcc_specs said "Idempotent". It is, for one home. Across homes it leaves one stale rpath entry per run, because the needle is one path and the replacement is two -- the extra gccLib never appears in a later needle. 68 on this machine, all gccLib, zero glibc. The third, in execute.cppm, claimed the run fast path treats a cache without the subos field as a miss. No such check was ever written; it lands with the change that implements it. An attempt to pin the first one with a test is deliberately NOT included, and the reason is recorded where the test would have been: the assertion passed on this machine by directory order, which makes passing and failing equally uninformative. The deterministic replacement is "given glibc@2.39, return 2.39 even with 2.44 present" -- which belongs to the resolver, not to a scan. --- src/toolchain/post_install.cppm | 9 ++++++++- src/xlings.cppm | 11 ++++++++++- tests/unit/test_linkmodel.cpp | 16 ++++++++++++++++ 3 files changed, 34 insertions(+), 2 deletions(-) diff --git a/src/toolchain/post_install.cppm b/src/toolchain/post_install.cppm index a40b6fe0..e03dacea 100644 --- a/src/toolchain/post_install.cppm +++ b/src/toolchain/post_install.cppm @@ -158,7 +158,14 @@ export void patchelf_walk(const std::filesystem::path& dir, // - Dynamically detects the baked-in loader path from the specs file // - Replaces it with the sandbox glibc payload's loader // - Replaces the rpath with : -// Idempotent — skips if already pointing at the correct glibc. +// NOT idempotent across homes -- and that is a defect, not a caveat. The +// needle is one path (the baked glibc dir) while the replacement is two +// (glibc + gccLib), so a second home patching the same shared payload leaves +// the first home's gccLib entry behind: it never appears in any later needle. +// One stale rpath per run, forever. Measured at 68 on a developer machine, +// all of them gccLib, none glibc -- a distribution only this mechanism +// produces. See 2026-08-08-payload-version-and-contract-drift-design.md §2.1; +// this function is slated for deletion, with the flags moving to the build. // Extract the baked-in glibc loader path (".../ld-linux-.so.N") from a // gcc specs file. xim bakes the installing user's XLINGS_HOME into specs at // install time, so the DIR varies per machine, and the loader NAME varies diff --git a/src/xlings.cppm b/src/xlings.cppm index e05f9457..d4f5a186 100644 --- a/src/xlings.cppm +++ b/src/xlings.cppm @@ -716,7 +716,16 @@ find_sibling_tool(const std::filesystem::path& compilerBin, std::error_code ec; if (!std::filesystem::exists(root, ec)) return std::nullopt; - // Return the first (highest) version dir that exists. + // The first version dir readdir yields. ORDER IS UNDEFINED -- this does + // NOT return the highest, and the comment that said so was wrong for as + // long as it stood. With two versions installed the answer varies by + // filesystem, so a caller that needs a SPECIFIC version must name it; + // see probe.cppm, where the version comes from the resolved runtime + // binding rather than from this scan. + // + // A test asserting the old "highest" claim was attempted and removed: it + // passed or failed by directory order, which makes both outcomes + // uninformative. for (auto& v : std::filesystem::directory_iterator(root, ec)) { if (v.is_directory(ec)) return v.path(); } diff --git a/tests/unit/test_linkmodel.cpp b/tests/unit/test_linkmodel.cpp index 4b7eb1cd..1e876ff9 100644 --- a/tests/unit/test_linkmodel.cpp +++ b/tests/unit/test_linkmodel.cpp @@ -212,4 +212,20 @@ TEST(LinkModel, NothingUsableYieldsNoneAndEmptyFlags) { EXPECT_TRUE(lm.link_flags(ident).empty()); } + +// 关于 `find_sibling_tool` 的顺序:**不在这里测**。 +// +// 它的注释曾写着 "Return the first (highest) version dir",而代码只是取 +// directory_iterator 的第一项 —— 没有排序。为它写一条断言的尝试失败了: +// 结果取决于 readdir 顺序,所以那条测试在本机通过、在别的机器上可能失败, +// **通过和失败都不说明问题**。 +// +// 正确的做法不是把不确定性钉住,而是让它不再被用于选版本:版本由权威 +// (`--runtime` / subos 的 `subos_info.runtime`)给出,`find_sibling_tool` +// 只负责定位包根。确定性的判据在 test_toolchain_probe.cpp —— 给定 +// `glibc@2.39`,即使 home 里同时有 2.44,也必须返回 2.39。 +// +// 注释本身已改为陈述事实(顺序未定义,调用方不得依赖)。 +// 设计:.agents/docs/2026-08-08-payload-version-and-contract-drift-design.md §3.2 + } // namespace From f70042c87dd973417ec534d533c5cbb58f5de67a Mon Sep 17 00:00:00 2001 From: speak-agent Date: Sat, 8 Aug 2026 08:38:02 +0800 Subject: [PATCH 03/21] feat(toolchain): the libc version is named by an authority, never scanned probe_payload_paths asked find_sibling_tool for "the glibc" and took what came back -- whatever readdir yielded. With one glibc installed that is always right, so nothing ever forced it to be correct. A dependency with a `>=` floor installed a second one, and the compile side took 2.44 while the artifact's interpreter, written into gcc's specs at install time, still named 2.39. Binaries referenced GLIBC_2.42 symbols against a 2.39 runtime, and the failures landed on packages with nothing to do with graphics. The version is now named: `--runtime`, else `[xlings] subos`'s subos_info.runtime, else the active subos's. The tests assert determinism rather than preference -- given glibc@2.39 return 2.39 with 2.44 also present, and a missing version does not fall back to an installed one, because falling back reintroduces the split silently. A fourth step was added after measuring what "no authority = refuse" does on this machine: it refuses. mcpp's own sandbox subos has no subos_info block -- the vendored xlings is 2026.8.2.1 and the block arrived in 2026.8.5.1, and that binary is never upgraded -- so every existing user's build would break. The compatibility step reads the binding this toolchain was installed against, out of gcc's specs or clang's cfg. That is not the guess this change removes. The guess picked a version by directory order, unrelated to what the artifact would load; this reads the version the artifact WILL load, so compile side and run side still agree, which is the invariant. It is a migration path and it retires itself: once a subos describes itself, the earlier steps answer first. --- src/build/prepare.cppm | 76 ++++++++++++++++- src/toolchain/detect.cppm | 9 ++- src/toolchain/model.cppm | 14 ++++ src/toolchain/post_install.cppm | 54 +++++++++++++ src/toolchain/probe.cppm | 85 +++++++++++++++---- tests/unit/test_toolchain_probe.cpp | 121 ++++++++++++++++++++++++++++ 6 files changed, 340 insertions(+), 19 deletions(-) create mode 100644 tests/unit/test_toolchain_probe.cpp diff --git a/src/build/prepare.cppm b/src/build/prepare.cppm index a622ac8b..9ff833cf 100644 --- a/src/build/prepare.cppm +++ b/src/build/prepare.cppm @@ -43,6 +43,8 @@ import mcpp.build.ninja; // make_ninja_backend — driving that sub-build import mcpp.lockfile; import mcpp.config; import mcpp.xlings; +import mcpp.xlings.subos_info; +import mcpp.toolchain.post_install; import mcpp.platform; import mcpp.fetcher; import mcpp.fetcher.progress; @@ -907,9 +909,66 @@ std::string with_index_cause(std::string msg) { msg += "\n" + hint; return msg; } +// The runtime this build targets, in xlings's own spelling ("glibc@2.39"). +// +// A degradation chain with every step explicit, and deliberately NO final +// "otherwise pick something". Payload resolution declines without an +// authority, because the guess it used to make is what let the compile side +// and the artifact's interpreter name different glibc versions -- invisibly, +// until the binary met a library built against the other one. +// +// 1. [xlings] subos -> that subos's subos_info.runtime +// 2. the active subos's subos_info.runtime +// 3. "" -- no authority, therefore no PayloadFirst +// +// `compilerBin` may be empty on the first call: the compiler has not been +// probed yet, and an inherited toolchain resolves its subos from its OWNER +// home. The caller re-resolves once it knows where the compiler is. +std::string resolve_runtime_binding(const mcpp::manifest::Manifest& m, + const std::filesystem::path& compilerBin) { + auto runtime_of = [](const std::filesystem::path& dir) -> std::string { + return mcpp::xlings::subos::read(dir).runtime; + }; + if (auto active = mcpp::xlings::paths::subos_dir_of(compilerBin)) { + // 1 — the project names a subos; it is a sibling of the active one. + if (!m.xlings.subos.empty()) { + auto named = active->parent_path() / m.xlings.subos; + if (auto r = runtime_of(named); !r.empty()) return r; + } + // 2 — whatever is active. + if (auto r = runtime_of(*active); !r.empty()) return r; + } + + // 3 — COMPATIBILITY: the value this toolchain already has baked in. + // + // A subos created before xlings grew `subos_info` cannot answer, and that + // is the state of every machine installed before 2026.8.5.1 -- including + // mcpp's own sandbox, whose vendored xlings is never upgraded. Refusing + // there would break every existing user, so the toolchain's own baked + // value stands in. + // + // This is NOT the guess this design removed. The guess picked a version by + // directory order, unrelated to what the artifact would load. This reads + // the value the artifact WILL use -- gcc's specs, clang's cfg -- so the + // invariant that matters, compile side == run side, still holds. It is a + // migration path, and it goes away on its own: once the subos describes + // itself, step 1 or 2 answers first. + if (!compilerBin.empty()) { + if (auto r = mcpp::toolchain::baked_runtime_binding(compilerBin); + !r.empty()) { + mcpp::log::verbose("probe", std::format( + "subos does not describe itself; using the runtime this " + "toolchain was installed against ({}). `xlings self update` " + "and a fresh subos make this authoritative", r)); + return r; + } + } + return {}; +} } // namespace export std::expected + prepare_build(bool print_fingerprint, bool includeDevDeps = false, std::vector extraTargets = {}, @@ -1641,8 +1700,21 @@ prepare_build(bool print_fingerprint, tcOrigin = TcOrigin::FirstRun; } - auto tc = mcpp::toolchain::detect(explicit_compiler); + // The authority for "which libc", resolved before the toolchain is + // probed: payload paths are addresses for a version this names. + auto runtimeBinding = resolve_runtime_binding(*m, {}); + auto tc = mcpp::toolchain::detect(explicit_compiler, runtimeBinding); if (!tc) return std::unexpected(tc.error().message); + // The binding may only be discoverable from the compiler's own home + // (an inherited toolchain resolves into its owner). Re-resolve once we + // know where the compiler is, then re-probe if that changed the answer. + if (runtimeBinding.empty()) { + auto fromCompiler = resolve_runtime_binding(*m, tc->binaryPath); + if (!fromCompiler.empty()) { + tc = mcpp::toolchain::detect(explicit_compiler, fromCompiler); + if (!tc) return std::unexpected(tc.error().message); + } + } // ── Targeting the MSVC ABI without a usable MSVC ───────────────────── // @@ -1737,7 +1809,7 @@ prepare_build(bool print_fingerprint, tcSpec = std::string(pins::kFirstRunWinGnu); tcOrigin = TcOrigin::FirstRun; - tc = mcpp::toolchain::detect(explicit_compiler); + tc = mcpp::toolchain::detect(explicit_compiler, runtimeBinding); if (!tc) return std::unexpected(tc.error().message); } diff --git a/src/toolchain/detect.cppm b/src/toolchain/detect.cppm index c3079e01..e71acfe9 100644 --- a/src/toolchain/detect.cppm +++ b/src/toolchain/detect.cppm @@ -16,7 +16,8 @@ export namespace mcpp::toolchain { // Detect toolchain. If explicit_compiler is given, use that binary path // directly. Otherwise fall back to $CXX, then PATH g++. std::expected -detect(const std::filesystem::path& explicit_compiler = {}); +detect(const std::filesystem::path& explicit_compiler = {}, + std::string_view runtimeBinding = {}); // Compatibility helper for older call sites/tests: GCC std module lookup now // lives in the GCC provider. @@ -33,7 +34,8 @@ std::optional find_std_module_source( } std::expected -detect(const std::filesystem::path& explicit_compiler) { +detect(const std::filesystem::path& explicit_compiler, + std::string_view runtimeBinding) { auto bin_r = probe_compiler_binary(explicit_compiler); if (!bin_r) return std::unexpected(bin_r.error()); @@ -111,7 +113,8 @@ detect(const std::filesystem::path& explicit_compiler) { // Probe fine-grained payload paths from sibling xpkgs (glibc, linux-headers). // When available, flags are assembled from these paths instead of --sysroot. - tc.payloadPaths = probe_payload_paths(tc.binaryPath); + tc.runtimeBinding = std::string(runtimeBinding); + tc.payloadPaths = probe_payload_paths(tc.binaryPath, tc.runtimeBinding); // For GCC: ensure the probed sysroot has complete headers by symlinking // missing content (linux kernel headers, glibc) from payload xpkgs. diff --git a/src/toolchain/model.cppm b/src/toolchain/model.cppm index 71d3883c..97a95e71 100644 --- a/src/toolchain/model.cppm +++ b/src/toolchain/model.cppm @@ -17,6 +17,10 @@ struct EnvVar { std::string value; }; +// The runtime this toolchain builds AGAINST, in xlings's spelling +// ("glibc@2.39"). Resolved before payload probing, from `--runtime` or the +// active subos's `subos_info.runtime`. Empty means no authority was found, +// and payload resolution then declines rather than guessing. struct PayloadPaths { std::filesystem::path glibcInclude; // glibc headers (features.h, bits/) std::filesystem::path glibcLib; // glibc runtime (libc.so, crt*.o, ld-linux) @@ -29,6 +33,16 @@ struct Toolchain { std::filesystem::path binaryPath; std::string driverIdent; // normalized --version output std::string targetTriple; // "x86_64-linux-gnu" + // The runtime this toolchain builds AGAINST, in xlings's own spelling + // ("glibc@2.39"). Resolved BEFORE payload probing, from `--runtime` or + // the active subos's `subos_info.runtime`. + // + // Empty is a refusal, not a default: payload resolution declines rather + // than picking a libc by directory order. That guess is what let the + // compile side and the artifact's interpreter name different glibc + // versions, with nothing in the resulting binary looking wrong until it + // loaded a library built against the other one. + std::string runtimeBinding; std::string stdlibId; // "libstdc++" std::string stdlibVersion; std::filesystem::path stdModuleSource; // bits/std.cc / std.cppm diff --git a/src/toolchain/post_install.cppm b/src/toolchain/post_install.cppm index e03dacea..e84589da 100644 --- a/src/toolchain/post_install.cppm +++ b/src/toolchain/post_install.cppm @@ -170,6 +170,18 @@ export void patchelf_walk(const std::filesystem::path& dir, // gcc specs file. xim bakes the installing user's XLINGS_HOME into specs at // install time, so the DIR varies per machine, and the loader NAME varies // per arch — detect both instead of hardcoding either. +// The runtime binding a toolchain was INSTALLED against, read back out of +// what mcpp itself wrote: gcc's specs, clang's cfg. +// +// Compatibility only. The authority is the subos's `subos_info.runtime`; this +// answers for machines whose subos predates that block, and it answers with +// the version the artifact will actually load -- so compile and run still +// agree, which is the invariant that matters. +// +// Returns "glibc@" or empty. The version comes from the payload path +// (.../xim-x-glibc//lib64), which is the only place it is written down. +export std::string baked_runtime_binding(const std::filesystem::path& compilerBin); + export std::string detect_baked_loader(const std::string& specsContent) { // Path-character whitelist. Specs embed loader paths inside %-spec // syntax (`%{mmusl:...;:/baked/dir/ld-linux-x86-64.so.2}`), so scanning @@ -535,4 +547,46 @@ export void ensure_post_install_fixup(const mcpp::config::GlobalConfig& cfg, os << expected.dump(2) << "\n"; } + +std::string baked_runtime_binding(const std::filesystem::path& compilerBin) { + if (compilerBin.empty()) return {}; + std::error_code ec; + + auto from_text = [](const std::string& text) -> std::string { + auto loader = detect_baked_loader(text); + if (loader.empty()) return {}; + // .../xim-x-glibc//lib64/ld-linux-... — the version is the + // grandparent of the lib dir. + auto dir = std::filesystem::path(loader).parent_path(); // lib64 + auto ver = dir.parent_path().filename().string(); // + auto pkg = dir.parent_path().parent_path().filename().string(); + if (ver.empty() || pkg.find("glibc") == std::string::npos) return {}; + return "glibc@" + ver; + }; + + auto read_file = [&](const std::filesystem::path& p) -> std::string { + if (!std::filesystem::exists(p, ec)) return {}; + std::ifstream is(p); + std::stringstream ss; ss << is.rdbuf(); + return ss.str(); + }; + + // clang: the sibling .cfg. + auto cfg = compilerBin.parent_path() + / (compilerBin.stem().string() + ".cfg"); + if (auto r = from_text(read_file(cfg)); !r.empty()) return r; + + // gcc: lib/gcc///specs, one level of globbing each. + auto gccRoot = compilerBin.parent_path().parent_path(); + for (auto t = std::filesystem::directory_iterator(gccRoot / "lib" / "gcc", ec); + !ec && t != std::filesystem::directory_iterator{}; t.increment(ec)) { + for (auto v = std::filesystem::directory_iterator(t->path(), ec); + !ec && v != std::filesystem::directory_iterator{}; v.increment(ec)) { + if (auto r = from_text(read_file(v->path() / "specs")); !r.empty()) + return r; + } + } + return {}; +} + } // namespace mcpp::toolchain diff --git a/src/toolchain/probe.cppm b/src/toolchain/probe.cppm index 5331c12e..64a9f04d 100644 --- a/src/toolchain/probe.cppm +++ b/src/toolchain/probe.cppm @@ -50,11 +50,15 @@ std::filesystem::path probe_sysroot(const std::filesystem::path& compilerBin, const std::string& envPrefix); -// Probe fine-grained sysroot paths from sibling xpkgs payloads. -// Returns populated PayloadPaths if glibc xpkg found; linux-headers -// may be empty if not available. +// Resolve the payload paths for an EXPLICIT runtime binding ("glibc@2.39"). +// +// The version is named by the caller's authority, never searched for. An +// empty binding returns nullopt: declining PayloadFirst is correct, guessing +// is what split the compile and run halves apart (see §3.2 of +// 2026-08-08-payload-version-and-contract-drift-design.md). std::optional -probe_payload_paths(const std::filesystem::path& compilerBin); +probe_payload_paths(const std::filesystem::path& compilerBin, + std::string_view runtimeBinding); // Ensure sysroot directory has complete headers by symlinking from // payload xpkgs. Called when GCC's probed sysroot exists but may @@ -326,19 +330,72 @@ probe_sysroot(const std::filesystem::path& compilerBin, return {}; } +// Resolve `@` to the payload ROOT for that exact version. +// +// The version is named, never searched: the caller has an authority (the +// resolved runtime binding) and this turns it into an address. Two payload +// roots are tried in the order the rest of the file uses -- the compiler's own +// siblings first, then the active home -- because an inherited or symlinked +// compiler resolves into its owner home while the active home may be the one +// that just installed the payload. +std::optional +payload_root_for_binding(const std::filesystem::path& compilerBin, + std::string_view binding) { + const auto at = binding.find('@'); + if (at == std::string_view::npos || at == 0 || at + 1 >= binding.size()) + return std::nullopt; + const auto name = binding.substr(0, at); + const auto version = std::string(binding.substr(at + 1)); + + std::error_code ec; + // Compiler siblings: <...>/xpkgs/xim-x-/ + if (auto xpkgs = mcpp::xlings::paths::xpkgs_from_compiler(compilerBin)) { + auto root = *xpkgs / std::format("xim-x-{}", name) / version; + if (std::filesystem::exists(root, ec)) return root; + } + // Active home. + if (auto xpkgs = mcpp::xlings::paths::active_home_xpkgs()) { + auto root = *xpkgs / std::format("xim-x-{}", name) / version; + if (std::filesystem::exists(root, ec)) return root; + } + return std::nullopt; +} + +// `runtimeBinding` is the AUTHORITY -- `--runtime`, else the active subos's +// `subos_info.runtime`. Empty is a refusal, not a licence to guess: a guess +// here is how the compile side and the artifact's interpreter came to name +// different glibc versions, and nothing about the resulting binary looks +// wrong until it loads a library built against the other one. std::optional -probe_payload_paths(const std::filesystem::path& compilerBin) { - namespace paths = mcpp::xlings::paths; +probe_payload_paths(const std::filesystem::path& compilerBin, + std::string_view runtimeBinding) { + if (runtimeBinding.empty()) { + mcpp::log::verbose("probe", + "no runtime binding resolved — declining PayloadFirst rather than " + "picking a libc by directory order"); + return std::nullopt; + } + // Only a libc family has a payload of this shape. `macos_sdk`/`ucrt` are + // resolved elsewhere, and a musl target uses the sysroot mode. + if (!runtimeBinding.starts_with("glibc@")) { + mcpp::log::verbose("probe", std::format( + "runtime '{}' is not a glibc payload — no PayloadFirst", + std::string(runtimeBinding))); + return std::nullopt; + } - // Find glibc xpkg (required). Compiler siblings first; fall back to the - // ACTIVE home registry — an inherited/symlinked compiler resolves into - // its owner home, while the active home may own (or have just installed) - // the sysroot payloads. - auto glibc = paths::find_sibling_tool(compilerBin, "glibc"); - if (!glibc) glibc = paths::find_home_tool("glibc", "include/features.h"); - if (!glibc) return std::nullopt; + auto glibc = payload_root_for_binding(compilerBin, runtimeBinding); + if (!glibc) { + mcpp::log::verbose("probe", std::format( + "runtime '{}' is not installed in this home", + std::string(runtimeBinding))); + return std::nullopt; + } + + namespace paths = mcpp::xlings::paths; - // Glibc layout: /include/ + /lib64/ (or lib/). + // Layout WITHIN the chosen payload. This convention stays: it answers + // "where inside", not "which one". auto glibcInclude = *glibc / "include"; if (!std::filesystem::exists(glibcInclude / "features.h")) return std::nullopt; diff --git a/tests/unit/test_toolchain_probe.cpp b/tests/unit/test_toolchain_probe.cpp new file mode 100644 index 00000000..8eff0b4b --- /dev/null +++ b/tests/unit/test_toolchain_probe.cpp @@ -0,0 +1,121 @@ +// Payload resolution: the version comes from an AUTHORITY, never from a scan. +// +// Before this, probe_payload_paths asked find_sibling_tool for "the glibc" +// and took whatever came back -- which is whatever readdir yielded first. With +// one glibc installed that is always right, so nothing ever forced it to be +// correct. Installing a second one (a dependency with a `>=` floor is enough) +// split the build in half: the compile side took one version while the +// artifact's interpreter, written into gcc's specs at install time, still +// named the other. Binaries then referenced GLIBC_2.42 symbols against a 2.39 +// runtime, and the failures landed on packages with nothing to do with the +// dependency that pulled the second glibc in. +// +// These assertions are about determinism, not about preference. "Given +// glibc@2.39, return 2.39" must hold with 2.44 also present and must not +// depend on directory order -- which is why the fixture always installs both. +// +// Design: .agents/docs/2026-08-08-payload-version-and-contract-drift-design.md §3.2 + +#include + +import std; +import mcpp.toolchain.probe; + +namespace { + +namespace tc = mcpp::toolchain; + +struct Tmp { + std::filesystem::path path; + Tmp() { + path = std::filesystem::temp_directory_path() + / std::format("mcpp_probe_test_{}", std::random_device{}()); + std::filesystem::create_directories(path); + } + ~Tmp() { std::error_code ec; std::filesystem::remove_all(path, ec); } +}; + +void touch(const std::filesystem::path& p) { + std::filesystem::create_directories(p.parent_path()); + std::ofstream(p) << "x"; +} + +// A home with TWO glibc payloads. Both are always present: a fixture with one +// cannot tell a resolver that reads the authority from one that guesses. +struct TwoGlibcHome { + Tmp dir; + std::filesystem::path compilerBin; + TwoGlibcHome() { + auto xpkgs = dir.path / "data" / "xpkgs"; + compilerBin = xpkgs / "xim-x-gcc" / "16.1.0" / "bin" / "g++"; + touch(compilerBin); + for (auto v : {"2.39", "2.44"}) { + auto root = xpkgs / "xim-x-glibc" / v; + touch(root / "include" / "features.h"); + touch(root / "lib64" / "libc.so.6"); + touch(root / "lib64" / "ld-linux-x86-64.so.2"); + } + } + std::filesystem::path libOf(std::string_view v) const { + return dir.path / "data" / "xpkgs" / "xim-x-glibc" / std::string(v) / "lib64"; + } +}; + +TEST(PayloadProbe, ResolvesTheBindingExactly) { + TwoGlibcHome h; + auto pp = tc::probe_payload_paths(h.compilerBin, "glibc@2.39"); + ASSERT_TRUE(pp.has_value()); + EXPECT_EQ(pp->glibcLib, h.libOf("2.39")) + << "the authority said 2.39 and 2.44 is also installed; a resolver " + "that scans would be free to return either"; +} + +TEST(PayloadProbe, ResolvesTheOtherBindingExactly) { + TwoGlibcHome h; + auto pp = tc::probe_payload_paths(h.compilerBin, "glibc@2.44"); + ASSERT_TRUE(pp.has_value()); + EXPECT_EQ(pp->glibcLib, h.libOf("2.44")); +} + +// No authority is not "pick something sensible" -- it is a refusal. Guessing +// here is what produced a build whose compile and run halves disagreed, and +// the guess is invisible: every artifact looks normal until one links a +// library built against the other version. +TEST(PayloadProbe, NoBindingIsARefusalNotAGuess) { + TwoGlibcHome h; + EXPECT_FALSE(tc::probe_payload_paths(h.compilerBin, "").has_value()); +} + +// A binding naming a version that is not installed must fail rather than fall +// back to one that is. Falling back would reintroduce the split silently. +TEST(PayloadProbe, MissingVersionDoesNotFallBack) { + TwoGlibcHome h; + EXPECT_FALSE(tc::probe_payload_paths(h.compilerBin, "glibc@2.28").has_value()); +} + +// The binding's NAME matters too: `musl@1.2.5` must not resolve a glibc +// payload just because the version string happens to be parseable. +TEST(PayloadProbe, WrongRuntimeFamilyDoesNotResolveGlibc) { + TwoGlibcHome h; + EXPECT_FALSE(tc::probe_payload_paths(h.compilerBin, "musl@1.2.5").has_value()); +} + +// Layout: lib64 preferred, lib accepted. Kept from the original probe -- a +// payload that ships only `lib` is a real shape, and this is the one piece of +// convention that stays, because it is about WHERE inside a chosen payload, +// not about WHICH payload. +TEST(PayloadProbe, AcceptsLibWhenThereIsNoLib64) { + Tmp dir; + auto xpkgs = dir.path / "data" / "xpkgs"; + auto bin = xpkgs / "xim-x-gcc" / "16.1.0" / "bin" / "g++"; + touch(bin); + auto root = xpkgs / "xim-x-glibc" / "2.39"; + touch(root / "include" / "features.h"); + touch(root / "lib" / "ld-linux-x86-64.so.2"); + + auto pp = tc::probe_payload_paths(bin, "glibc@2.39"); + ASSERT_TRUE(pp.has_value()); + EXPECT_EQ(pp->glibcLib, root / "lib"); +} + +} // namespace From 20b12de221c5ecee1e173c28203e6d7f51765330 Mon Sep 17 00:00:00 2001 From: speak-agent Date: Sat, 8 Aug 2026 08:41:24 +0800 Subject: [PATCH 04/21] feat(toolchain): the fingerprint gains the runtime binding The target triple says x86_64-linux-gnu whether the build targets glibc 2.39 or 2.44, so without this the two share `target//`. Switching subos would leave the directory untouched, ninja would replay a graph whose objects were compiled against the other libc, the link would succeed, and the result would reference symbols its interpreter cannot provide -- with nothing along the way saying so. This is the prerequisite for building one project under two subos, not a refinement of it, which is why it lands before that feature rather than with it. Also fixes a dead reference: the header pointed at docs/06-toolchain-and-fingerprint.md, which does not exist. The fingerprint is described in 08-toolchain-internals.md. --- src/toolchain/fingerprint.cppm | 14 +++++++++--- tests/unit/test_fingerprint.cpp | 38 +++++++++++++++++++++++++++++++++ 2 files changed, 49 insertions(+), 3 deletions(-) diff --git a/src/toolchain/fingerprint.cppm b/src/toolchain/fingerprint.cppm index 59e6562c..beb3f75a 100644 --- a/src/toolchain/fingerprint.cppm +++ b/src/toolchain/fingerprint.cppm @@ -1,11 +1,18 @@ -// mcpp.toolchain.fingerprint — 10-field fingerprint that gates BMI cache safety. +// mcpp.toolchain.fingerprint — 11-field fingerprint that gates BMI cache safety. // -// Per docs/06-toolchain-and-fingerprint.md, the fingerprint MUST cover: +// Per docs/08-toolchain-internals.md, the fingerprint MUST cover: // 1. compiler id 2. compiler version // 3. compiler driver identity 4. target triple // 5. stdlib id+version 6. C++ standard // 7. compile flags hash 8. mcpp version // 9. dependency lock hash 10. std module BMI hash +// 11. runtime binding (which libc this targets) +// +// Field 11 is the prerequisite for building one project under two subos. The +// target triple says `x86_64-linux-gnu` for both glibc 2.39 and 2.44, so +// without it switching subos leaves `target//` untouched: ninja replays a +// graph whose objects were compiled against the other libc, the link +// succeeds, and the result references symbols the interpreter cannot provide. // // MVP uses FNV-1a 64-bit (deterministic, collision-rare-enough for cache // invalidation); spec calls for SHA-256 but for "did anything change?" @@ -35,7 +42,7 @@ struct FingerprintInputs { struct Fingerprint { std::string hex; // 16 hex chars - std::array parts; // each field's stringified form + std::array parts; // each field's stringified form }; std::string hash_file(const std::filesystem::path& p); // returns 16 hex @@ -108,6 +115,7 @@ Fingerprint compute_fingerprint(const FingerprintInputs& in) { fp.parts[7] = std::string(MCPP_VERSION); fp.parts[8] = in.dependencyLockHash; fp.parts[9] = in.stdBmiHash; + fp.parts[10] = tc.runtimeBinding; // Combine all parts deterministically. std::uint64_t h = 0xcbf29ce484222325ull; diff --git a/tests/unit/test_fingerprint.cpp b/tests/unit/test_fingerprint.cpp index f6e10143..1fa185ce 100644 --- a/tests/unit/test_fingerprint.cpp +++ b/tests/unit/test_fingerprint.cpp @@ -24,6 +24,44 @@ FingerprintInputs baseline() { return in; } + +// Two builds that differ ONLY in which libc they target must not share a +// fingerprint -- and therefore must not share `target//`. +// +// Without this, switching subos leaves the build directory untouched: ninja +// sees the same graph, the objects inside were compiled against the other +// glibc, and the link succeeds. What comes out references symbols the +// interpreter cannot provide, and nothing along the way said anything. This +// is the prerequisite for building the same project under two subos, not a +// refinement of it. +TEST(Fingerprint, RuntimeBindingIsPartOfTheIdentity) { + mcpp::toolchain::FingerprintInputs a; + a.toolchain.compiler = mcpp::toolchain::CompilerId::GCC; + a.toolchain.version = "16.1.0"; + a.toolchain.targetTriple = "x86_64-linux-gnu"; + a.toolchain.runtimeBinding = "glibc@2.39"; + + auto b = a; + b.toolchain.runtimeBinding = "glibc@2.44"; + + EXPECT_NE(mcpp::toolchain::compute_fingerprint(a).hex, + mcpp::toolchain::compute_fingerprint(b).hex) + << "same toolchain, same triple, different libc — sharing target// " + "would replay objects compiled against the other one"; +} + +// ...and an unset binding is its own identity, not a collision with any +// particular version. +TEST(Fingerprint, EmptyRuntimeBindingIsDistinct) { + mcpp::toolchain::FingerprintInputs a; + a.toolchain.targetTriple = "x86_64-linux-gnu"; + a.toolchain.runtimeBinding = "glibc@2.39"; + auto b = a; + b.toolchain.runtimeBinding.clear(); + EXPECT_NE(mcpp::toolchain::compute_fingerprint(a).hex, + mcpp::toolchain::compute_fingerprint(b).hex); +} + } // namespace TEST(Fingerprint, DeterministicForSameInputs) { From 26710247a9bde81f81589fd3d852e38d2d6f3f0e Mon Sep 17 00:00:00 2001 From: speak-agent Date: Sat, 8 Aug 2026 08:53:53 +0800 Subject: [PATCH 05/21] feat(toolchain): the compiler is a capability, not a configuration mcpp told gcc nothing about the C runtime at link time and let the specs file it had written at install time answer instead. That made the run side a per-toolchain-install decision while the compile side stayed per-build, and the two named different glibc versions as soon as a second one was installed. mcpp already refuses to depend on clang's install-time cfg for precisely this reason -- `--no-default-config`, "reproducible builds, no dependence on the install-time-generated cfg". The reasoning had simply never reached gcc. Three changes, and each one was forced by the failure the previous one exposed: Emitting -Wl,--dynamic-linker and -Wl,-rpath for gcc did not remove what the specs already inject, because rpath accumulates. So a clean `*link:` is generated into the build directory from `g++ -dumpspecs` -- which prints the spec compiled into the binary and is unaffected by the file on disk, so the original is always recoverable and the payload is never touched. Replacing `*link:` then removed the addressing the specs used to supply, and the hermeticity check caught the result: an artifact pointing at the host's /lib64/ld-linux. Sysroot mode now names the interpreter too. That fixed the interpreter and left libm unfound, because the same spec carried the rpath; then libgcc_s, because it carried two entries. The pair mcpp emits is exactly what the deleted code baked in. fixup_gcc_specs is gone. Deleting it restores what R6 asks of a payload -- immutable, independent of any home -- and removes the accumulation at its source: the substitution had a one-path needle and a two-path replacement, so every home that patched the shared payload left one entry behind. 68 on this machine, all pointing at deleted mktemp directories, in every gcc artifact it produced. The e2e asserts on the artifact rather than the specs file, because what a user ships is the thing that matters. patchelf_walk stays: making the compiler runnable is a different question from what the compiler produces, and putting both in one function with one glibcLibDir is why they looked like one. One test changed meaning rather than being fixed: GccPayloadUsesIdirafterAndNoLoader asserted the behaviour this commit removes. Headers still differ by driver; addressing no longer does. --- src/build/flags.cppm | 16 ++- src/build/plan.cppm | 5 + src/build/prepare.cppm | 8 ++ src/toolchain/linkmodel.cppm | 60 +++++++++- src/toolchain/post_install.cppm | 141 +++++++++++++----------- tests/e2e/201_gcc_no_specs_pollution.sh | 75 +++++++++++++ tests/unit/test_linkmodel.cpp | 16 ++- 7 files changed, 248 insertions(+), 73 deletions(-) create mode 100755 tests/e2e/201_gcc_no_specs_pollution.sh diff --git a/src/build/flags.cppm b/src/build/flags.cppm index de26950e..bb1d87f4 100644 --- a/src/build/flags.cppm +++ b/src/build/flags.cppm @@ -713,8 +713,20 @@ CompileFlags compute_flags(const BuildPlan& plan) { // that lld cannot open — issue #195), -L/-rpath for -lc/-lm, and the // payload's dynamic linker. std::string payload_ld; - if (isClangWithCfg && lm.mode == mcpp::toolchain::CLibMode::PayloadFirst) - payload_ld = lm.link_flags(ninjaEsc); + if (lm.mode == mcpp::toolchain::CLibMode::PayloadFirst) { + // Emitted for gcc too now. It used to be gated on `isClangWithCfg`, + // which left gcc's run-side addressing to its install-time specs + // while the compile side moved per build. + if (isClangWithCfg || !lm.clangDriver) + payload_ld = lm.link_flags(ninjaEsc); + } + // GCC: replace the payload's patched `*link:` with the pristine one, so + // its accumulated rpath entries do not reach the artifact. Must come + // BEFORE our own -Wl flags is not required (specs are processed by the + // driver, not positionally against -Wl), but keeping it adjacent to the + // payload flags keeps the C-runtime decisions in one place on the line. + if (!plan.gccCleanSpecs.empty()) + payload_ld = " -specs=" + escape_path(plan.gccCleanSpecs) + payload_ld; std::string link_extra; if (prof.lto) link_extra += " -flto"; diff --git a/src/build/plan.cppm b/src/build/plan.cppm index 0f1daa91..45a30fa9 100644 --- a/src/build/plan.cppm +++ b/src/build/plan.cppm @@ -112,6 +112,11 @@ struct BuildPlan { // (possibly read-only) registry directory, and deriving the path would put // an IDE database there. Empty → projectRoot, the historical default. std::filesystem::path compileDbPath; + // GCC only: a specs file that replaces the pristine `*link:`, so the + // payload's own (patched by every home that ever installed against it) + // cannot inject rpath entries into this build's artifacts. Empty for + // clang, which bypasses its cfg with --no-default-config instead. + std::filesystem::path gccCleanSpecs; std::filesystem::path stdBmiPath; // absolute path to prebuilt std.gcm std::filesystem::path stdObjectPath; // absolute path to prebuilt std.o std::filesystem::path stdCompatBmiPath; // absolute path to prebuilt std.compat.pcm diff --git a/src/build/prepare.cppm b/src/build/prepare.cppm index 9ff833cf..442a8e39 100644 --- a/src/build/prepare.cppm +++ b/src/build/prepare.cppm @@ -4974,6 +4974,14 @@ prepare_build(bool print_fingerprint, if (!planResult) return std::unexpected(planResult.error()); ctx.plan = std::move(*planResult); ctx.plan.compileDbPath = workRoot / "compile_commands.json"; + // GCC: a clean `*link:` for this build, so the payload's specs cannot + // inject other homes' rpath entries into the artifact. AFTER the plan is + // moved in — an earlier assignment was silently overwritten by that move, + // which produced a generated file that nothing ever passed to the driver. + // Generated here rather than in compute_flags, which runs twice per build. + if (tc->compiler == mcpp::toolchain::CompilerId::GCC) + ctx.plan.gccCleanSpecs = mcpp::toolchain::write_clean_link_specs( + tc->binaryPath, ctx.outputDir); // ── Declared build-graph nodes → the plan ─────────────────────────────── // diff --git a/src/toolchain/linkmodel.cppm b/src/toolchain/linkmodel.cppm index 45195abc..fde7442f 100644 --- a/src/toolchain/linkmodel.cppm +++ b/src/toolchain/linkmodel.cppm @@ -72,7 +72,7 @@ struct ToolchainLinkModel { std::vector systemIncludes; // Rendering knobs derived from the toolchain at resolve time. - bool clangDriver = false; // clang: -isystem + rpath/loader on link + bool clangDriver = false; // clang: -isystem headers; gcc: -idirafter // gcc: -idirafter (…#include_next), -B/-L only bool clangWithCfg = false; // sibling .cfg exists (bundled LLVM) @@ -107,15 +107,45 @@ struct ToolchainLinkModel { std::vector out; if (mode == CLibMode::Sysroot) { out.push_back("--sysroot=" + esc(sysroot)); + // Sysroot mode still needs the interpreter named explicitly when + // we know it. mcpp replaces gcc's `*link:` to stop the payload's + // accumulated rpath entries reaching artifacts, and the pristine + // spec it restores carries the HOST's default loader — so removing + // the addressing without supplying a replacement would silently + // hand every artifact /lib64/ld-linux. The hermeticity check + // catches that, which is how it was found. + if (!loader.empty()) + out.push_back("-Wl,--dynamic-linker=" + esc(loader)); + // ...and the runtime search path, for the same reason. Replacing + // `*link:` removes the rpath the payload's specs used to inject + // along with the loader; supplying one without the other produces + // a binary whose interpreter is right and whose libm is not found. + for (auto& dir : libDirs) { + out.push_back("-L" + esc(dir)); + out.push_back("-Wl,-rpath," + esc(dir)); + } return out; } if (mode != CLibMode::PayloadFirst) return out; if (!crtDir.empty()) out.push_back("-B" + esc(crtDir)); for (auto& dir : libDirs) { out.push_back("-L" + esc(dir)); - if (clangDriver) out.push_back("-Wl,-rpath," + esc(dir)); + out.push_back("-Wl,-rpath," + esc(dir)); } - if (clangDriver && !loader.empty()) + // Emitted for BOTH drivers. + // + // GCC used to be left to its specs here, on the theory that the + // install-time fixup owned the loader. That made the RUN side a + // per-toolchain-install decision while the COMPILE side stayed + // per-build, and the two named different glibc versions the moment a + // second one was installed. + // + // mcpp already refuses to depend on clang's install-time cfg + // (`--no-default-config`, "reproducible builds, no dependence on the + // install-time-generated cfg"). The same reasoning had simply never + // been applied to gcc. Measured: an explicit -Wl,--dynamic-linker + // overrides what the specs inject. + if (!loader.empty()) out.push_back("-Wl,--dynamic-linker=" + esc(loader)); return out; } @@ -298,12 +328,32 @@ ToolchainLinkModel resolve_link_model(const Toolchain& tc) { lm.systemIncludes.push_back(pp.glibcInclude); if (!pp.linuxInclude.empty()) lm.systemIncludes.push_back(pp.linuxInclude); - if (lm.clangDriver) - lm.loader = resolve_loader(pp.glibcLib, tc.targetTriple); + // Resolved for both drivers now that both emit it. + lm.loader = resolve_loader(pp.glibcLib, tc.targetTriple); }; auto sysroot_mode = [&](const std::filesystem::path& root) { lm.mode = CLibMode::Sysroot; lm.sysroot = root; + // Name the interpreter even here: see link_tokens. The payload is the + // address (R6 — artifacts bind the payload, never the mutable view), + // and it is known whenever a runtime binding resolved. + // + // The pair mirrors exactly what fixup_gcc_specs used to bake in + // (`:`): the C library, and the compiler's own + // runtime (libgcc_s, libstdc++). Emitting it per build is the whole + // point — the baked copy was a per-toolchain-install decision that no + // longer matched the per-build one, and it accumulated one dead entry + // per home that ever touched the shared payload. + if (tc.payloadPaths && !tc.payloadPaths->glibcLib.empty()) { + lm.loader = resolve_loader(tc.payloadPaths->glibcLib, tc.targetTriple); + lm.libDirs.push_back(tc.payloadPaths->glibcLib); + } + if (!tc.binaryPath.empty()) { + std::error_code lec; + auto gccLib = tc.binaryPath.parent_path().parent_path() / "lib64"; + if (std::filesystem::exists(gccLib, lec)) + lm.libDirs.push_back(gccLib); + } // Supplement kernel headers when the sysroot lacks them (glibc's // local_lim.h needs ). Self-contained musl sysroots // ship their own; a cross target must not see host-arch headers. diff --git a/src/toolchain/post_install.cppm b/src/toolchain/post_install.cppm index e84589da..c6d7cdd5 100644 --- a/src/toolchain/post_install.cppm +++ b/src/toolchain/post_install.cppm @@ -182,6 +182,26 @@ export void patchelf_walk(const std::filesystem::path& dir, // (.../xim-x-glibc//lib64), which is the only place it is written down. export std::string baked_runtime_binding(const std::filesystem::path& compilerBin); +// Write a specs file that REPLACES gcc's `*link:` with the pristine built-in +// one, and return its path (empty when not applicable or on failure). +// +// Why this exists: `-rpath` accumulates. mcpp emitting its own does not remove +// what the payload's specs already inject, and that file has been patched by +// every home that ever installed against this shared payload -- one stale +// entry per run, forever. Replacing the spec is the only way to stop them +// reaching a user's artifact, and it works on machines already polluted. +// +// The pristine `*link:` comes from `g++ -dumpspecs`, which prints the specs +// COMPILED INTO the binary and is unaffected by the file on disk. So the +// original is always recoverable and the payload is never touched -- an +// earlier plan to reinstall the toolchain was unnecessary. +// +// `-specs=` whose definition does not begin with `+` replaces rather +// than appends; both properties were measured before this was written. +export std::filesystem::path write_clean_link_specs( + const std::filesystem::path& compilerBin, + const std::filesystem::path& outputDir); + export std::string detect_baked_loader(const std::string& specsContent) { // Path-character whitelist. Specs embed loader paths inside %-spec // syntax (`%{mmusl:...;:/baked/dir/ld-linux-x86-64.so.2}`), so scanning @@ -215,65 +235,30 @@ export std::string detect_baked_loader(const std::string& specsContent) { return ""; } -void fixup_gcc_specs(const std::filesystem::path& gccPkgRoot, - const std::filesystem::path& glibcLibDir, - const std::filesystem::path& gccLibDir, - const std::filesystem::path& fenceRoot) -{ - // #273 fence: a payload reached through a symlink is a foreign - // installation — its specs file must not be rewritten either. - if (escapes_containment(gccPkgRoot, fenceRoot)) { - mcpp::log::verbose("toolchain", - "fixup_gcc_specs: skip (payload outside sandbox, #273 fence)"); - return; - } - std::filesystem::path specsParent; - std::error_code ec; - for (auto it = std::filesystem::directory_iterator(gccPkgRoot / "lib" / "gcc", ec); - !ec && it != std::filesystem::directory_iterator{}; it.increment(ec)) { - if (it->is_directory(ec)) { specsParent = it->path(); break; } - } - if (specsParent.empty()) return; - - auto loaderReplacement = resolve_loader(glibcLibDir, /*targetTriple=*/{}).string(); - if (loaderReplacement.empty()) return; - auto rpathReplacement = std::format("{}:{}", - glibcLibDir.string(), - gccLibDir.string()); - - auto replace_all = [](std::string& s, std::string_view needle, - std::string_view rep) - { - for (std::size_t pos = 0; - (pos = s.find(needle, pos)) != std::string::npos;) { - s.replace(pos, needle.size(), rep); - pos += rep.size(); - } - }; - - for (auto& sub : std::filesystem::directory_iterator(specsParent)) { - auto specs = sub.path() / "specs"; - if (!std::filesystem::exists(specs)) continue; - - std::ifstream is(specs); - std::stringstream ss; ss << is.rdbuf(); - std::string content = ss.str(); - - auto bakedLoader = detect_baked_loader(content); - if (bakedLoader.empty()) continue; - auto bakedDir = std::filesystem::path(bakedLoader).parent_path().string(); - // Already pointing at the right place — no fixup needed. - if (bakedDir == glibcLibDir.string()) continue; - - // Order matters: replace the full loader file path first so the - // shorter dir pattern doesn't eat its prefix. - replace_all(content, bakedLoader, loaderReplacement); - replace_all(content, bakedDir, rpathReplacement); +// fixup_gcc_specs is GONE. +// +// It wrote the loader and the rpath into the gcc payload's `specs` at install +// time. Both are now emitted per build by mcpp.toolchain.linkmodel, which +// removes an answerer rather than adding a path: +// +// - the RUN side was a per-toolchain-install decision while the COMPILE +// side was per-build, and they named different glibc versions the moment +// a second one was installed; +// - the substitution had a one-path needle and a two-path replacement, so +// every home that patched the SHARED payload left one entry behind. 68 on +// a developer machine, all pointing at deleted mktemp directories, in +// every gcc artifact it produced. +// +// The payload is no longer written to at all, which restores what R6 asks of +// it: immutable, and independent of any particular home. A build that needs +// a clean `*link:` gets one generated into its own build directory (see +// write_clean_link_specs) -- the pristine spec comes from `g++ -dumpspecs`, +// which prints what is compiled into the binary and is therefore always +// recoverable. +// +// patchelf_walk stays: making the compiler itself runnable is a different +// question from what the compiler produces. - std::ofstream os(specs); - os << content; - } -} // Regenerate the clang driver cfg files after the LLVM payload landed in the // sandbox. The cfg xlings authored at install time is a per-machine, @@ -290,7 +275,7 @@ void fixup_gcc_specs(const std::filesystem::path& gccPkgRoot, export void fixup_clang_cfg(const std::filesystem::path& payloadRoot, const std::filesystem::path& glibcLibDir, const std::filesystem::path& fenceRoot) { - // #273 fence — same rule as fixup_gcc_specs above. + // #273 fence — same rule the removed gcc specs fixup used. if (escapes_containment(payloadRoot, fenceRoot)) { mcpp::log::verbose("toolchain", "fixup_clang_cfg: skip (payload outside sandbox, #273 fence)"); @@ -435,9 +420,6 @@ void gcc_post_install_fixup(const mcpp::config::GlobalConfig& cfg, } patchelf_walk(payloadRoot, loader, rpath, patchelfBin, fence); - mcpp::log::verbose("toolchain", "gcc fixup: fixup_gcc_specs"); - fixup_gcc_specs(payloadRoot, glibcLibDir, gccLibDir, - containment_root(cfg.registryDir)); } else { mcpp::ui::warning( "could not locate sandbox glibc/gcc/patchelf paths; " @@ -589,4 +571,39 @@ std::string baked_runtime_binding(const std::filesystem::path& compilerBin) { return {}; } + +std::filesystem::path write_clean_link_specs( + const std::filesystem::path& compilerBin, + const std::filesystem::path& outputDir) +{ + if (compilerBin.empty() || outputDir.empty()) return {}; + std::error_code ec; + auto out = outputDir / "mcpp-clean-link.specs"; + // Idempotent: the pristine spec is a property of the compiler binary, so + // regenerating it every build would spawn a process for a constant. + if (std::filesystem::exists(out, ec)) return out; + + auto r = mcpp::platform::process::capture(std::format( + "{} -dumpspecs 2>/dev/null", + mcpp::platform::shell::quote(compilerBin.string()))); + if (r.exit_code != 0 || r.output.empty()) return {}; + + // `*link:` is a section header on its own line; its body is the next line. + constexpr std::string_view kHead = "*link:\n"; + auto pos = r.output.find(kHead); + if (pos == std::string::npos) return {}; + auto bodyStart = pos + kHead.size(); + auto bodyEnd = r.output.find('\n', bodyStart); + if (bodyEnd == std::string::npos) return {}; + auto body = r.output.substr(bodyStart, bodyEnd - bodyStart); + if (body.empty()) return {}; + + std::filesystem::create_directories(outputDir, ec); + std::ofstream os(out); + if (!os) return {}; + os << "*link:\n" << body << "\n\n"; + os.close(); + return out; +} + } // namespace mcpp::toolchain diff --git a/tests/e2e/201_gcc_no_specs_pollution.sh b/tests/e2e/201_gcc_no_specs_pollution.sh new file mode 100755 index 00000000..deaea12c --- /dev/null +++ b/tests/e2e/201_gcc_no_specs_pollution.sh @@ -0,0 +1,75 @@ +#!/usr/bin/env bash +# requires: gcc elf +# 201_gcc_no_specs_pollution.sh — a gcc artifact must carry only the RUNPATH +# entries this build asked for. +# +# mcpp used to write the loader and rpath into the gcc payload's `specs` file +# at install time, and the link command said nothing about either. Two +# consequences, both measured on a developer machine before this test existed: +# +# 1. the compile side (-L, chosen per build) and the run side (specs, frozen +# at toolchain install) could name different glibc versions, and did as +# soon as a second one was installed; +# 2. the substitution that wrote those specs had a one-path needle and a +# two-path replacement, so every home that ever patched the SHARED payload +# left one entry behind -- 68 of them, all pointing at deleted mktemp +# directories, in every gcc artifact the machine produced. +# +# The assertion is deliberately about the artifact, not about the specs file: +# what matters is what a user ends up shipping. A RUNPATH entry under /tmp is +# never something a build asked for. +set -euo pipefail + +TMP=$(mktemp -d) +trap 'rm -rf "$TMP"' EXIT +export MCPP_HOME=$HOME/.mcpp + +cd "$TMP" +"$MCPP" new prog > /dev/null +cd prog +cat >> mcpp.toml <<'EOF' + +[toolchain] +linux = "gcc@16.1.0" +EOF + +"$MCPP" build > "$TMP/build.log" 2>&1 || { cat "$TMP/build.log"; echo "build failed"; exit 1; } + +bin=$(find target -type f -name prog -path '*/bin/*' | head -1) +[[ -n "$bin" ]] || { echo "no binary produced"; exit 1; } + +# `readelf` may be an xvm shim needing a per-tool pin; use the payload's own. +readelf=$(ls "$MCPP_HOME"/registry/data/xpkgs/xim-x-binutils/*/bin/readelf 2>/dev/null | head -1) +[[ -x "$readelf" ]] || { echo "SKIP: no payload readelf to inspect with"; exit 0; } + +runpath=$("$readelf" -d "$bin" 2>/dev/null | grep -iE 'RUNPATH|RPATH' | sed 's/.*\[\(.*\)\]/\1/' || true) + +# 1. No path under a temp directory. These can only come from a specs file that +# some other home patched; no build ever asks for one. +if echo "$runpath" | tr ':' '\n' | grep -q '^/tmp/'; then + echo "the artifact carries RUNPATH entries from other homes' temp dirs:" + echo "$runpath" | tr ':' '\n' | grep '^/tmp/' | head -5 + echo " ...total $(echo "$runpath" | tr ':' '\n' | grep -c '^/tmp/')" + echo "These come from the payload's specs, which mcpp must no longer write." + exit 1 +fi + +# 2. Every entry must exist. A RUNPATH naming a directory that is not there is +# a cost paid on every symbol resolution for nothing, and a reliable sign +# that something wrote a path belonging to a different machine state. +missing=0 +while IFS= read -r d; do + [[ -z "$d" ]] && continue + [[ "$d" == \$ORIGIN* ]] && continue + [[ -d "$d" ]] || { echo " RUNPATH entry does not exist: $d"; missing=$((missing+1)); } +done <<< "$(echo "$runpath" | tr ':' '\n')" +[[ "$missing" -eq 0 ]] || { echo "$missing dead RUNPATH entries"; exit 1; } + +# 3. The interpreter is the payload's, and the program runs. +file_out=$(file "$bin") +echo "$file_out" | grep -q 'interpreter .*xim-x-glibc' || { + echo "PT_INTERP is not the payload's glibc:"; echo "$file_out"; exit 1; } +out=$("$bin" 2>&1) || { echo "binary did not run: $out"; exit 1; } +echo "$out" | grep -q 'Hello' || { echo "unexpected output: $out"; exit 1; } + +echo "PASS: gcc artifact carries only the RUNPATH this build asked for" diff --git a/tests/unit/test_linkmodel.cpp b/tests/unit/test_linkmodel.cpp index 1e876ff9..a4f3b5c2 100644 --- a/tests/unit/test_linkmodel.cpp +++ b/tests/unit/test_linkmodel.cpp @@ -180,7 +180,7 @@ TEST(LinkModel, GccSysrootSupplementsMissingKernelHeaders) { EXPECT_NE(lm.compile_flags(ident).find("-isystem"), std::string::npos); } -TEST(LinkModel, GccPayloadUsesIdirafterAndNoLoader) { +TEST(LinkModel, GccPayloadEmitsIdirafterAndItsOwnAddressing) { Tmp dir; auto glibcLib = dir.path / "glibc" / "lib64"; touch(glibcLib / "ld-linux-x86-64.so.2"); @@ -194,12 +194,20 @@ TEST(LinkModel, GccPayloadUsesIdirafterAndNoLoader) { }; auto lm = tc::resolve_link_model(t); EXPECT_EQ(lm.mode, tc::CLibMode::PayloadFirst); + // Headers still differ by driver: libstdc++'s #include_next wrappers need + // -idirafter, and that has not changed. EXPECT_NE(lm.compile_flags(ident).find("-idirafter"), std::string::npos); + + // Addressing no longer differs. GCC used to be left to its install-time + // specs here, which made the RUN side a per-toolchain-install decision + // while the COMPILE side stayed per-build -- and the two named different + // glibc versions as soon as a second one was installed. mcpp already + // refuses to depend on clang's install-time cfg for exactly this reason. auto link = lm.link_flags(ident); EXPECT_NE(link.find(" -B"), std::string::npos); - // GCC's loader/rpath is owned by the specs fixup, not the command line. - EXPECT_EQ(link.find("dynamic-linker"), std::string::npos); - EXPECT_EQ(link.find("-rpath"), std::string::npos); + EXPECT_NE(link.find("--dynamic-linker=" + lm.loader.string()), + std::string::npos); + EXPECT_NE(link.find("-Wl,-rpath," + glibcLib.string()), std::string::npos); } TEST(LinkModel, NothingUsableYieldsNoneAndEmptyFlags) { From f33d8ff1fe2e9bd23b861a29532896166ded3be4 Mon Sep 17 00:00:00 2001 From: speak-agent Date: Sat, 8 Aug 2026 08:55:42 +0800 Subject: [PATCH 06/21] fix(xlings): read the real subos_info wire format, and refuse a pre-upgrade cache Carried over from the PR this work supersedes; the two defects and their evidence are unchanged. 2026.8.8.1 shipped a reader for a format that does not exist. `envs` is an object keyed by binding, the reader expected an array of {binding, decls}, so is_array() was false, the loop never ran, and against a real subos every variable came back unset -- LIBGL_DRIVERS_PATH, __EGL_VENDOR_LIBRARY_DIRS, XDG_DATA_DIRS still the host's. The whole #352 fix was inert. Ten unit tests and an e2e passed throughout, because every fixture was hand-written in the same invented shape the parser expected. A fixture composed from the same understanding as the parser cannot catch a misunderstanding of the wire format; only one taken from the writer can. RealXlingsCapture is that -- verbatim output from a real `xlings install graphics`, reformatted for width and nothing else. Transcribing the parser from xlings's own reader also caught a second divergence: xlings drops a declaration whose op is neither "set" nor "prepend", and this accepted any. Second defect: `mcpp run`'s fast path replayed a cache written before the subos field existed, so the fix did not survive an upgrade -- and not just for one run. The fast path's identity is the profile, cache mode and resource list, and its fingerprint check compares an entry against itself; nothing notices that a different mcpp wrote it. Measured on a real 2026.8.7.1 -> 2026.8.8.1 upgrade. An empty subosDir cannot stand in for "predates the field", since a system toolchain legitimately has no subos, so presence is tracked separately. --- src/build/execute.cppm | 23 ++++- src/xlings/subos_info.cppm | 53 +++++++---- tests/e2e/200_subos_env_reaches_program.sh | 37 ++++++- tests/unit/test_subos_info.cpp | 106 +++++++++++++++++---- 4 files changed, 177 insertions(+), 42 deletions(-) diff --git a/src/build/execute.cppm b/src/build/execute.cppm index b4cb5c1e..d31208fa 100644 --- a/src/build/execute.cppm +++ b/src/build/execute.cppm @@ -68,6 +68,11 @@ struct BuildCacheEntry { // property and would otherwise be unknowable on the fast path -- which // has no toolchain to derive it from. std::string subosDir; + // Was the line present at all? An EMPTY subosDir is a legitimate answer + // (a system toolchain outside the xpkgs store has no subos), so it cannot + // stand in for "this cache predates the field" -- and those two need + // opposite treatment: the first runs, the second must rebuild once. + bool subosRecorded = false; // The resolved profile this entry was built for. Entries used to be keyed // by target triple alone, and the fast paths only refuse to run when an // EXPLICIT --profile/--dev/--release is passed — so a bare `mcpp build` @@ -154,7 +159,8 @@ std::vector read_build_cache(const std::filesystem::path& proje // using the cache at all -- the program would work once and then // silently stop finding its runtime data. if (haveNextLine && line.starts_with("subos=")) { - e.subosDir = line.substr(6); + e.subosDir = line.substr(6); + e.subosRecorded = true; haveNextLine = static_cast(std::getline(f, line)); } // Optional profile line. Same back-compat contract as the two blocks @@ -201,7 +207,8 @@ void write_build_cache(const std::filesystem::path& projectRoot, // Insert at front (MRU). BuildCacheEntry newEntry{targetTriple, outputDir.string(), ninjaProgram, fingerprintHex, runtimeEnvKey, runtimeEnvValue, std::move(runTargets), - runEnvKey, runEnvValue, subosDir, profile, cacheMode}; + runEnvKey, runEnvValue, subosDir, /*subosRecorded=*/true, + profile, cacheMode}; entries.insert(entries.begin(), std::move(newEntry)); // Trim to LRU capacity. @@ -774,6 +781,18 @@ std::optional try_fast_run(const std::filesystem::path& projectRoot, ninjaProgram = ninjaProgram.substr(1, ninjaProgram.size() - 2); if (match->runtimeEnvKey.empty()) return std::nullopt; // old cache entry; go through prepare_build once + // Written before this mcpp knew about subos environments (mcpp#352). Taking + // the fast path here would run the program without them -- which is the + // defect this field exists to fix, surviving an upgrade. + // + // It survives it for a long time, too: the fast path's identity is the + // profile, the cache mode and the resource list, and its fingerprint check + // compares a cached entry against ITSELF. Neither notices that a different + // mcpp wrote the entry, so without this line an upgraded mcpp would reuse a + // pre-upgrade build until something else happened to invalidate it. Measured + // on a real upgrade from 2026.8.7.1, not reasoned about. + if (!match->subosRecorded) + return std::nullopt; // predates `subos=`; rebuild once, then it is there // P1: verify fingerprint matches the outputDir basename. if (!match->fingerprint.empty()) { diff --git a/src/xlings/subos_info.cppm b/src/xlings/subos_info.cppm index 0054fb2b..41aaac55 100644 --- a/src/xlings/subos_info.cppm +++ b/src/xlings/subos_info.cppm @@ -131,28 +131,39 @@ Info read(const std::filesystem::path& subosDir) { if (auto v = it->find("runtime"); v != it->end() && v->is_string()) info.runtime = v->get(); - if (auto envs = it->find("envs"); envs != it->end() && envs->is_array()) { - for (auto const& p : *envs) { - if (!p.is_object()) continue; + // `envs` is an OBJECT keyed by binding, whose values are arrays of + // declarations: + // + // "envs": { "mesa@25.0.7.1": [ {"var":…,"op":…,"value":…}, … ], … } + // + // Transcribed from xlings's own reader (core/subos/manifest.cppm), not + // from a model of it. The first version of this file expected an array of + // {binding, decls} objects — a shape xlings never writes — and its tests + // hand-wrote JSON in that same invented shape, so both agreed and both + // were wrong. Against a real subos the loop simply never ran and every + // variable came back unset, silently. That is why the fixture below is a + // verbatim capture of real output rather than something composed here. + if (auto envs = it->find("envs"); envs != it->end() && envs->is_object()) { + for (auto e = envs->begin(); e != envs->end(); ++e) { + if (!e.value().is_array()) continue; Provider prov; - if (auto b = p.find("binding"); b != p.end() && b->is_string()) - prov.binding = b->get(); - if (auto ds = p.find("decls"); ds != p.end() && ds->is_array()) { - for (auto const& d : *ds) { - if (!d.is_object()) continue; - EnvDecl e; - if (auto x = d.find("var"); x != d.end() && x->is_string()) - e.var = x->get(); - if (auto x = d.find("op"); x != d.end() && x->is_string()) - e.op = x->get(); - if (auto x = d.find("value"); x != d.end() && x->is_string()) - e.value = x->get(); - // A declaration with no variable name is not a partial - // declaration to be guessed at — it is malformed input, - // and the right thing is to leave it out rather than - // invent a name for it. - if (!e.var.empty()) prov.decls.push_back(std::move(e)); - } + prov.binding = e.key(); + for (auto const& d : e.value()) { + if (!d.is_object()) continue; + EnvDecl decl; + if (auto x = d.find("var"); x != d.end() && x->is_string()) + decl.var = x->get(); + if (auto x = d.find("op"); x != d.end() && x->is_string()) + decl.op = x->get(); + if (auto x = d.find("value"); x != d.end() && x->is_string()) + decl.value = x->get(); + // xlings drops a declaration whose var is empty or whose op is + // neither "set" nor "prepend". Matched exactly: a reader that + // is more permissive than the writer will one day apply + // something the writer considers malformed. + if (decl.var.empty()) continue; + if (decl.op != "set" && decl.op != "prepend") continue; + prov.decls.push_back(std::move(decl)); } info.providers.push_back(std::move(prov)); } diff --git a/tests/e2e/200_subos_env_reaches_program.sh b/tests/e2e/200_subos_env_reaches_program.sh index 81da7302..4ce376df 100755 --- a/tests/e2e/200_subos_env_reaches_program.sh +++ b/tests/e2e/200_subos_env_reaches_program.sh @@ -14,6 +14,12 @@ # know what any of these variables mean -- it carries whatever the subos # declares -- and a test naming LIBGL_DRIVERS_PATH would quietly suggest # otherwise. +# +# The JSON below is xlings's REAL shape: `envs` is an object keyed by binding, +# whose values are arrays of declarations. The first version of this test wrote +# an array of {binding, decls} -- a shape xlings never produces -- and it +# passed, because the reader had been written from the same misunderstanding. +# Do not "simplify" this structure; it is a wire format, not a convenience. set -euo pipefail TMP=$(mktemp -d) @@ -28,9 +34,9 @@ mkdir -p "$subos/usr/lib/dri" cat > "$subos/.xlings.json" <<'EOF' { "workspace": {}, "subos_info": { "schema_version": 1, "runtime": "glibc@2.39", - "envs": [ { "binding": "probe@1", "decls": [ + "envs": { "probe@1": [ { "var": "MCPP_E2E_PROBE", "op": "prepend", - "value": "${subosdir}/usr/lib/dri" } ] } ] } } + "value": "${subosdir}/usr/lib/dri" } ] } } } EOF cd "$TMP" @@ -106,6 +112,33 @@ echo "$out2" | grep -q 'PROBE=(unset)' || { exit 1 } +# 2b. A cache written before this mcpp knew about subos environments must NOT +# be replayed by the fast path. Simulated by stripping the field, which is +# exactly what an older mcpp's cache looks like. +# +# Without this the fix survives an upgrade in name only: the fast path's +# identity is the profile, the cache mode and the resource list, and its +# fingerprint check compares a cached entry against itself -- so nothing +# notices that a different mcpp wrote it, and an upgraded mcpp would keep +# running the pre-upgrade build with no subos environment at all. +cache="$TMP/hello/target/.build_cache" +[ -f "$cache" ] || { echo "no build cache to age"; exit 1; } +grep -q '^subos=' "$cache" || { echo "cache has no subos= line to strip"; exit 1; } +grep -v '^subos=' "$cache" > "$cache.old" && mv "$cache.old" "$cache" +aged=$(MCPP_SUBOS_DIR="$subos" "$MCPP" run 2>&1) || { + echo "run against an aged cache failed:"; echo "$aged"; exit 1; } +echo "$aged" | grep -q 'Resolving toolchain' || { + echo "an aged cache was replayed by the fast path — the subos environment" + echo " would be missing for every run after an upgrade:" + echo "$aged" + exit 1 +} +echo "$aged" | grep -q "PROBE=$subos/usr/lib/dri" || { + echo "the rebuild after an aged cache did not apply the environment:" + echo "$aged"; exit 1; } +grep -q '^subos=' "$cache" || { + echo "the rebuild did not record subos= , so every later run repeats it"; exit 1; } + # 3. A subos with no self-description degrades quietly and still runs. This is # the state of every subos created before xlings grew the block, so it must # not be an error. diff --git a/tests/unit/test_subos_info.cpp b/tests/unit/test_subos_info.cpp index 95385778..a7f27af2 100644 --- a/tests/unit/test_subos_info.cpp +++ b/tests/unit/test_subos_info.cpp @@ -37,14 +37,14 @@ TEST(SubosInfo, ReadsRuntimeAndEnvDeclarations) { "subos_info": { "schema_version": 1, "runtime": "glibc@2.39", - "envs": [ - { "binding": "mesa@25.0.7.1", "decls": [ + "envs": { + "mesa@25.0.7.1": [ { "var": "LIBGL_DRIVERS_PATH", "op": "prepend", "value": "${subosdir}/usr/lib/dri" }, { "var": "XDG_DATA_DIRS", "op": "prepend", "value": "${subosdir}/share" } - ]} - ] + ] + } } })"); auto info = su::read(t.dir); @@ -64,8 +64,8 @@ TEST(SubosInfo, ReadsRuntimeAndEnvDeclarations) { TEST(SubosInfo, ResolvesSubosdirPlaceholder) { Tmp t; t.write(R"({"subos_info":{"schema_version":1,"runtime":"glibc@2.39", - "envs":[{"binding":"mesa@1","decls":[ - {"var":"LIBGL_DRIVERS_PATH","op":"prepend","value":"${subosdir}/usr/lib/dri"}]}]}})"); + "envs":{"mesa@1":[ + {"var":"LIBGL_DRIVERS_PATH","op":"prepend","value":"${subosdir}/usr/lib/dri"}]}}})"); auto env = su::resolve_env(su::read(t.dir), t.dir); ASSERT_EQ(env.size(), 1u); EXPECT_EQ(env[0].first, "LIBGL_DRIVERS_PATH"); @@ -81,11 +81,9 @@ TEST(SubosInfo, ResolvesSubosdirPlaceholder) { // EGL vendor directory. `prepend` joins them; it must not drop either. TEST(SubosInfo, PrependJoinsProvidersInOrder) { Tmp t; - t.write(R"({"subos_info":{"schema_version":1,"runtime":"glibc@2.39","envs":[ - {"binding":"a-mesa@1","decls":[ - {"var":"V","op":"prepend","value":"${subosdir}/one"}]}, - {"binding":"b-vendor@1","decls":[ - {"var":"V","op":"prepend","value":"${subosdir}/two"}]}]}})"); + t.write(R"({"subos_info":{"schema_version":1,"runtime":"glibc@2.39","envs":{ + "a-mesa@1":[{"var":"V","op":"prepend","value":"${subosdir}/one"}], + "b-vendor@1":[{"var":"V","op":"prepend","value":"${subosdir}/two"}]}}})"); auto env = su::resolve_env(su::read(t.dir), t.dir); ASSERT_EQ(env.size(), 1u); const auto sep = mcpp::platform::env::path_list_separator(); @@ -98,9 +96,9 @@ TEST(SubosInfo, PrependJoinsProvidersInOrder) { // would otherwise grow the variable without bound. TEST(SubosInfo, PrependDeduplicates) { Tmp t; - t.write(R"({"subos_info":{"schema_version":1,"runtime":"glibc@2.39","envs":[ - {"binding":"a@1","decls":[{"var":"V","op":"prepend","value":"${subosdir}/x"}]}, - {"binding":"b@1","decls":[{"var":"V","op":"prepend","value":"${subosdir}/x"}]}]}})"); + t.write(R"({"subos_info":{"schema_version":1,"runtime":"glibc@2.39","envs":{ + "a@1":[{"var":"V","op":"prepend","value":"${subosdir}/x"}], + "b@1":[{"var":"V","op":"prepend","value":"${subosdir}/x"}]}}})"); auto env = su::resolve_env(su::read(t.dir), t.dir); ASSERT_EQ(env.size(), 1u); // One entry, not two. The de-duplication has to split on the PLATFORM's @@ -114,9 +112,9 @@ TEST(SubosInfo, PrependDeduplicates) { // `set` replaces rather than joins — xlings's own precedence. TEST(SubosInfo, SetReplaces) { Tmp t; - t.write(R"({"subos_info":{"schema_version":1,"runtime":"glibc@2.39","envs":[ - {"binding":"a@1","decls":[{"var":"V","op":"prepend","value":"/one"}]}, - {"binding":"b@1","decls":[{"var":"V","op":"set","value":"/two"}]}]}})"); + t.write(R"({"subos_info":{"schema_version":1,"runtime":"glibc@2.39","envs":{ + "a@1":[{"var":"V","op":"prepend","value":"/one"}], + "b@1":[{"var":"V","op":"set","value":"/two"}]}}})"); auto env = su::resolve_env(su::read(t.dir), t.dir); ASSERT_EQ(env.size(), 1u); EXPECT_EQ(env[0].second, "/two"); @@ -177,4 +175,78 @@ TEST(SubosInfo, FamilyOfMirrorsXlings) { EXPECT_EQ(su::family_of("glibc"), "linux-x86_64-glibc"); } + +// A VERBATIM capture of what a real xlings wrote, after `xlings install +// graphics` on an NVIDIA host. Reformatted for width and nothing else -- keys, +// nesting and spelling are as found on disk. +// +// This test exists because its absence shipped a broken feature. The first +// version of this file hand-wrote every fixture in a shape the reader also +// expected and xlings never produces: `envs` as an array of {binding, decls}. +// Ten tests passed against a format that does not exist, and against a real +// subos the released build applied no variables at all -- silently, because +// "no providers" and "nothing declared" look identical. +// +// A fixture composed from the same understanding as the parser cannot catch +// that. Only one taken from the writer can. +TEST(SubosInfo, RealXlingsCapture) { + Tmp t; + t.write(R"({ + "subos_info": { + "created_at": "2026-08-08T01:40:00Z", + "created_by": "xlings 2026.8.7.1", + "runtime": "glibc@2.39", + "schema_version": 1, + "envs": { + "mesa@25.0.7.1": [ + {"op": "prepend", "value": "${subosdir}/usr/lib/dri", "var": "LIBGL_DRIVERS_PATH"}, + {"op": "prepend", "value": "${subosdir}/share/glvnd/egl_vendor.d", "var": "__EGL_VENDOR_LIBRARY_DIRS"}, + {"op": "prepend", "value": "${subosdir}/share", "var": "XDG_DATA_DIRS"} + ], + "nvidia-gl-host-link@0.1.1": [ + {"op": "prepend", "value": "${subosdir}/share/glvnd/egl_vendor.d", "var": "__EGL_VENDOR_LIBRARY_DIRS"} + ] + } + }, + "workspace": {} + })"); + + auto info = su::read(t.dir); + ASSERT_TRUE(info.present); + EXPECT_EQ(info.runtime, "glibc@2.39"); + ASSERT_EQ(info.providers.size(), 2u); + EXPECT_EQ(info.providers[0].binding, "mesa@25.0.7.1"); + EXPECT_EQ(info.providers[1].binding, "nvidia-gl-host-link@0.1.1"); + + auto env = su::resolve_env(info, t.dir); + ASSERT_EQ(env.size(), 3u) << "all three graphics variables must be produced"; + + std::map byVar; + for (auto& [k, v] : env) byVar[k] = v; + const auto sep = mcpp::platform::env::path_list_separator(); + EXPECT_EQ(byVar["LIBGL_DRIVERS_PATH"], t.dir.string() + "/usr/lib/dri"); + EXPECT_EQ(byVar["XDG_DATA_DIRS"], t.dir.string() + "/share"); + // Both providers name the same vendor directory; de-duplication must + // leave exactly one, or libglvnd sees it twice and enumerates the device + // twice -- which is a defect xlings hit on its own side. + EXPECT_EQ(byVar["__EGL_VENDOR_LIBRARY_DIRS"], + t.dir.string() + "/share/glvnd/egl_vendor.d"); + EXPECT_EQ(byVar["__EGL_VENDOR_LIBRARY_DIRS"].find(sep), std::string::npos); +} + +// xlings drops a declaration whose op it does not recognise. A reader more +// permissive than its writer eventually applies something the writer meant to +// reject, so this asserts the same refusal rather than a tolerant guess. +TEST(SubosInfo, UnknownOpIsDroppedLikeXlingsDrops) { + Tmp t; + t.write(R"({"subos_info":{"schema_version":1,"runtime":"glibc@2.39","envs":{ + "a@1":[{"var":"V","op":"append","value":"/nope"}, + {"var":"W","op":"prepend","value":"/yes"}, + {"var":"","op":"prepend","value":"/no-name"}]}}})"); + auto env = su::resolve_env(su::read(t.dir), t.dir); + ASSERT_EQ(env.size(), 1u); + EXPECT_EQ(env[0].first, "W"); + EXPECT_EQ(env[0].second, "/yes"); +} + } // namespace From bc36d6ce4d90b11bce1e47fa24a7524751af752f Mon Sep 17 00:00:00 2001 From: speak-agent Date: Sat, 8 Aug 2026 09:10:41 +0800 Subject: [PATCH 07/21] fix(fallback): a vendored xlings behind the pin, and a sysroot from elsewhere D4. acquire_xlings_binary returned on mere existence, so a home kept the first xlings it ever acquired. Measured here: 2026.8.2.1 against a pin of 2026.8.6.3, with `mcpp self env` printing both numbers beside each other and saying nothing about the gap. Not cosmetic -- the subos_info block arrived in xlings 2026.8.5.1, so on that machine every subos declaration was discarded by a client too old to have the API, and the mcpp#352 fix could not take effect however current mcpp itself was. Missing features are silent by construction: an absent block is also a legitimate state. The first cut of this deleted the vendored binary and re-acquired. That replaced 2026.8.2.1 with the system's 0.4.51 -- older still, and missing the very feature the replacement existed to restore. Being behind the pin justifies looking for a replacement; it does not justify taking whatever turns up. candidate_source_version() now prices the replacement before the old one is removed, and the epoch-crossing comparison (0.4.x vs YYYY.M.D.N) is pinned by a test, since both schemes live on the same disk. D5. remap_xlings_baked_sysroot asked whether gcc's baked sysroot exists. Wrong axis: gcc records `--sysroot=<...>/.xlings/subos/default` as a string when it is built and keeps reporting it in every project it later serves, and a machine with several checkouts has many directories by that name. The baked path therefore routinely exists and belongs to someone else -- a build inside mcpp resolved a sysroot under an unrelated repo. The predicate is now ownership, not existence. doctor reports both. The sysroot finding is anchored on the mcpp home and the project rather than on the compiler's own tree: written the obvious way it consulted xpkgs_from_compiler(binaryPath), which returns nothing when the compiler is reached through an xvm shim -- the default project-local case, and this machine's. It would have shipped as a check that never checked. tests/unit/test_xlings_version_pin.cpp +1 (epoch crossing) tests/unit/test_sysroot_ownership.cpp +7 (new; no filesystem, on purpose) --- src/config.cppm | 3 +- src/doctor.cppm | 53 ++++++++++ src/fallback/probe_sysroot.cppm | 57 +++++++++- src/fallback/xlings_binary.cppm | 137 ++++++++++++++++++++++++- tests/unit/test_sysroot_ownership.cpp | 69 +++++++++++++ tests/unit/test_xlings_version_pin.cpp | 66 ++++++++++++ 6 files changed, 377 insertions(+), 8 deletions(-) create mode 100644 tests/unit/test_sysroot_ownership.cpp create mode 100644 tests/unit/test_xlings_version_pin.cpp diff --git a/src/config.cppm b/src/config.cppm index 129a7419..cd49375a 100644 --- a/src/config.cppm +++ b/src/config.cppm @@ -600,7 +600,8 @@ std::expected load_or_init( // 6. Acquire xlings binary if needed if (cfg.xlingsBinaryMode == "bundled") { - auto xbin = mcpp::fallback::acquire_xlings_binary(cfg.xlingsBinary, quiet); + auto xbin = mcpp::fallback::acquire_xlings_binary( + cfg.xlingsBinary, quiet, kXlingsPinnedVersion); if (!xbin) return std::unexpected(ConfigError{xbin.error()}); } else if (cfg.xlingsBinaryMode == "system") { auto sysPath = mcpp::platform::fs::which( diff --git a/src/doctor.cppm b/src/doctor.cppm index 034c1d02..3dc1ce88 100644 --- a/src/doctor.cppm +++ b/src/doctor.cppm @@ -14,6 +14,8 @@ import mcpp.bmi_cache.maintenance; import mcpp.build.prepare; import mcpp.build.plan; import mcpp.config; +import mcpp.fallback.probe_sysroot; +import mcpp.fallback.xlings_binary; import mcpp.fallback.install_integrity; import mcpp.fetcher.progress; import mcpp.home; @@ -69,6 +71,7 @@ export std::vector parse_readelf_runpath(std::string_view dump) { // `mcpp self env`. export int env_report() { auto cfg = mcpp::config::load_or_init(/*quiet=*/false, mcpp::fetcher::make_bootstrap_progress_callback()); + if (!cfg) { mcpp::ui::error(cfg.error().message); return 4; } mcpp::config::print_env(*cfg); @@ -98,6 +101,7 @@ export int doctor_report() { err(std::format("toolchain detection failed: {}", tc.error().message)); } else { ok(std::format("{} at {}", tc->label(), tc->binaryPath.string())); + } // Windows: report the system MSVC (msvc@system). Absence is a warning, @@ -204,6 +208,28 @@ export int doctor_report() { mcpp::ui::status("Checking", "registry"); auto cfg = mcpp::config::load_or_init(/*quiet=*/false, mcpp::fetcher::make_bootstrap_progress_callback()); + + // Whose sysroot is this? gcc bakes `--sysroot=<...>/subos/default` + // at build time, and that path is a string, not a reference -- it + // keeps naming wherever the compiler was built no matter which + // project it now serves. A developer machine has many directories by + // that name, so the baked one frequently EXISTS while belonging to an + // unrelated checkout, and headers then come from a tree this build + // never declared. Existence is not ownership. + if (tc && cfg) { + std::error_code cwdEc; + auto project = std::filesystem::current_path(cwdEc); + if (mcpp::fallback::sysroot_is_foreign( + tc->sysroot, (*cfg).registryDir, + cwdEc ? std::filesystem::path{} : project)) + warn(std::format( + "sysroot {} belongs to neither this mcpp home ({}) nor " + "this project — headers would come from a tree nothing " + "here declared. mcpp remaps a baked sysroot when it can " + "find the equivalent under the registry; seeing it here " + "means it could not", + tc->sysroot.string(), (*cfg).registryDir.string())); + } if (!cfg) { err(cfg.error().message); } else { @@ -366,6 +392,33 @@ export int doctor_report() { else if (!sawAny) ok("no installed toolchains to check"); + // The vendored xlings, against the version this mcpp expects. + // + // Nothing else surfaces this. `mcpp self env` prints both numbers and + // says nothing about the gap, and a home that acquired its xlings once + // never revisited it -- so a machine could sit years behind while every + // command looked healthy. What goes missing is silent by nature: + // features mcpp reads FROM xlings (the subos_info block, for one) + // simply never appear, and the code that consumes them degrades + // quietly because a missing block is also a legitimate state. + { + auto have = mcpp::fallback::vendored_xlings_version((*cfg).xlingsBinary); + const auto want = std::string(mcpp::config::kXlingsPinnedVersion); + if (have.empty()) { + warn(std::format("cannot read the vendored xlings version at {}", + (*cfg).xlingsBinary.string())); + } else if (mcpp::fallback::version_is_older(have, want)) { + warn(std::format( + "vendored xlings is {} but this mcpp expects {} — features " + "mcpp reads from xlings may be silently absent (the subos " + "self-description arrived in 2026.8.5.1). It is replaced " + "automatically on the next `mcpp self init`", + have, want)); + } else { + ok(std::format("vendored xlings {} (pinned {})", have, want)); + } + } + // Dangling symlinks under registry/subos/default/lib — these point // into xim payload lib dirs; a removed package leaves them broken. auto subosLib = (*cfg).xlingsHome() / "subos" / "default" / "lib"; diff --git a/src/fallback/probe_sysroot.cppm b/src/fallback/probe_sysroot.cppm index ee3db9ab..65db370b 100644 --- a/src/fallback/probe_sysroot.cppm +++ b/src/fallback/probe_sysroot.cppm @@ -17,17 +17,40 @@ import mcpp.log; export namespace mcpp::fallback { -// When GCC reports a sysroot ending in "subos/default" that doesn't exist -// on the current machine (baked build-time path), remap it to the -// equivalent sysroot relative to the compiler's own xpkgs directory. +// When GCC reports a baked "subos/default" sysroot that does not belong to +// THIS toolchain's home, remap it to the equivalent sysroot under the +// compiler's own xpkgs tree. +// +// The predicate used to be "does not exist", which is the wrong axis. A +// developer machine has more than one project-local `.xlings/subos/default`, +// and gcc's baked path can name one of them: it exists, so the remap was +// skipped and every build in an unrelated project inherited another +// project's sysroot. Measured -- a build in mcpp resolved +// `--sysroot=/.xlings/subos/default`. +// +// Existence says nothing about ownership. What matters is whether the path is +// under the same registry as the compiler, and a path that is not gets +// remapped whether or not something happens to be there. std::optional remap_xlings_baked_sysroot(std::string_view reportedPath, const std::filesystem::path& compilerBin) { if (reportedPath.empty()) return std::nullopt; if (!reportedPath.ends_with("subos/default")) return std::nullopt; - if (std::filesystem::exists(std::string(reportedPath))) return std::nullopt; - if (auto xpkgs = mcpp::xlings::paths::xpkgs_from_compiler(compilerBin)) { + auto xpkgsOpt = mcpp::xlings::paths::xpkgs_from_compiler(compilerBin); + if (xpkgsOpt) { + // Owned by this toolchain's registry? Then it is the right answer. + auto registry = xpkgsOpt->parent_path().parent_path(); + std::error_code ec; + auto rel = std::filesystem::path(std::string(reportedPath)) + .lexically_relative(registry); + const bool inside = !rel.empty() + && rel.native().rfind("..", 0) != 0; + if (inside && std::filesystem::exists(std::string(reportedPath), ec)) + return std::nullopt; + } + + if (auto xpkgs = std::move(xpkgsOpt)) { // xpkgs is /data/xpkgs -> registry = xpkgs/../.. auto registrySysroot = xpkgs->parent_path().parent_path() / "subos" / "default"; @@ -37,6 +60,30 @@ remap_xlings_baked_sysroot(std::string_view reportedPath, return std::nullopt; } +// Is this sysroot foreign -- neither this mcpp home's registry nor a tree +// belonging to the project being built? +// +// The hazard is specific and was measured: gcc bakes `--sysroot=<...>/.xlings/ +// subos/default` as a STRING at build time, and a developer machine has many +// directories by that name. The baked one therefore frequently EXISTS while +// belonging to an unrelated checkout, and headers silently come from a tree +// this build never declared. remap_xlings_baked_sysroot repairs the case it +// can see; this predicate is what reports the case it cannot. +// +// Both anchors are required. Registry alone would flag every legitimate +// project-local tree; project alone would flag every payload sysroot. +bool sysroot_is_foreign(const std::filesystem::path& sysroot, + const std::filesystem::path& registryRoot, + const std::filesystem::path& projectRoot) { + if (sysroot.empty()) return false; + auto under = [&](const std::filesystem::path& anchor) { + if (anchor.empty()) return false; + auto rel = sysroot.lexically_relative(anchor); + return !rel.empty() && rel.native().rfind("..", 0) != 0; + }; + return !under(registryRoot) && !under(projectRoot); +} + // Parse a Clang .cfg file alongside the compiler binary for --sysroot=. std::optional parse_clang_cfg_sysroot(const std::filesystem::path& compilerBin) { diff --git a/src/fallback/xlings_binary.cppm b/src/fallback/xlings_binary.cppm index 5a91812d..ff21c07f 100644 --- a/src/fallback/xlings_binary.cppm +++ b/src/fallback/xlings_binary.cppm @@ -7,6 +7,7 @@ module; #include +#include export module mcpp.fallback.xlings_binary; @@ -17,9 +18,70 @@ export namespace mcpp::fallback { // Try to acquire (copy) the xlings binary to destBin. // Returns destBin on success or an error string. +// The version already vendored at destBin, or empty when it cannot be read. +std::string vendored_xlings_version(const std::filesystem::path& bin); + +// The version the acquisition chain WOULD install, without installing it. +// Empty when nothing is available. Replacing a vendored binary is only an +// improvement when this is newer than what is already there. +std::string candidate_source_version(); + +// True when `have` is strictly older than `want`, comparing dot-separated +// numeric components. Anything unparseable answers false -- a version this +// code does not understand is not evidence of being behind. +bool version_is_older(std::string_view have, std::string_view want); + +// Try to acquire (copy) the xlings binary to destBin. +// +// `pinnedVersion` is the version this mcpp expects. A vendored binary OLDER +// than the pin is replaced; one that is newer or equal is left alone. +// +// This used to return early on mere existence, with no version comparison at +// all, and nothing else ever revisited the file. A home created once kept +// whatever xlings it first acquired forever -- measured at 2026.8.2.1 against +// a pin of 2026.8.6.3, with `mcpp self env` printing both numbers side by side +// and saying nothing about the gap. The cost is not cosmetic: features mcpp +// reads from xlings simply do not appear. The subos_info block arrived in +// 2026.8.5.1, so on that machine the graphics packages' declarations were +// discarded by a client too old to have the API, and mcpp#352's fix could +// never take effect. +// +// Strictly-older, not not-equal: a user who put a newer xlings there on +// purpose must not be downgraded by an mcpp that happens to pin an older one. std::expected -acquire_xlings_binary(const std::filesystem::path& destBin, bool quiet = false) { - if (std::filesystem::exists(destBin)) return destBin; +acquire_xlings_binary(const std::filesystem::path& destBin, bool quiet = false, + std::string_view pinnedVersion = {}) { + if (std::filesystem::exists(destBin)) { + auto have = vendored_xlings_version(destBin); + if (pinnedVersion.empty() || have.empty() + || !version_is_older(have, pinnedVersion)) + return destBin; + + // Behind the pin -- but replacing is only an improvement if what we + // would put there is actually newer. The acquisition chain below ends + // at whatever `which xlings` finds, and on a machine whose system + // xlings is ANCIENT that is a downgrade dressed up as an update. + // + // Learned by doing it: this code first deleted the vendored binary and + // re-acquired, which replaced 2026.8.2.1 with the system's 0.4.51 -- + // older still, and equally missing the feature the check exists to + // restore. Look before leaping. + auto candidate = candidate_source_version(); + if (candidate.empty() || !version_is_older(have, candidate)) { + if (!quiet) + std::println("{:>12} vendored xlings {} is older than the " + "pinned {}, but no newer source is available " + "(keeping it; run `xlings self update`)", + "Note", have, pinnedVersion); + return destBin; + } + if (!quiet) + std::println("{:>12} vendored xlings {} -> {} (pinned {})", + "Updating", have, candidate, pinnedVersion); + std::error_code rec; + std::filesystem::remove(destBin, rec); + // fall through and re-acquire + } std::error_code ec; std::filesystem::create_directories(destBin.parent_path(), ec); @@ -82,4 +144,75 @@ acquire_xlings_binary(const std::filesystem::path& destBin, bool quiet = false) (destBin.parent_path().parent_path() / "config.toml").string())); } + +std::string vendored_xlings_version(const std::filesystem::path& bin) { + std::error_code ec; + if (!std::filesystem::exists(bin, ec)) return {}; + auto r = mcpp::platform::process::capture(std::format( + "{} --version 2>/dev/null", mcpp::platform::shell::quote(bin.string()))); + if (r.exit_code != 0) return {}; + // Output carries ANSI colour; take the first dotted-numeric run. + std::string out; + for (std::size_t i = 0; i < r.output.size(); ++i) { + if (r.output[i] == '\x1b') { // skip CSI + while (i < r.output.size() && r.output[i] != 'm') ++i; + continue; + } + out += r.output[i]; + } + std::size_t i = 0; + while (i < out.size()) { + if (std::isdigit(static_cast(out[i]))) { + auto j = i; + while (j < out.size() + && (std::isdigit(static_cast(out[j])) + || out[j] == '.')) ++j; + auto cand = out.substr(i, j - i); + if (cand.find('.') != std::string::npos) return cand; + i = j; + } else ++i; + } + return {}; +} + +bool version_is_older(std::string_view have, std::string_view want) { + auto parts = [](std::string_view v) { + std::vector out; + std::size_t i = 0; + while (i <= v.size()) { + auto dot = v.find('.', i); + auto seg = v.substr(i, dot == std::string_view::npos + ? std::string_view::npos : dot - i); + long long n = 0; + auto [p, e] = std::from_chars(seg.data(), seg.data() + seg.size(), n); + if (e != std::errc{}) return std::vector{}; + out.push_back(n); + if (dot == std::string_view::npos) break; + i = dot + 1; + } + return out; + }; + auto a = parts(have), b = parts(want); + if (a.empty() || b.empty()) return false; // unparseable is not "behind" + for (std::size_t i = 0; i < std::max(a.size(), b.size()); ++i) { + long long x = i < a.size() ? a[i] : 0; + long long y = i < b.size() ? b[i] : 0; + if (x != y) return x < y; + } + return false; +} + + +std::string candidate_source_version() { + if (const char* e = std::getenv("MCPP_VENDORED_XLINGS"); e && *e) { + std::error_code ec; + if (std::filesystem::exists(std::filesystem::path(e), ec)) + return vendored_xlings_version(std::filesystem::path(e)); + } + if (auto sys = mcpp::platform::fs::which( + std::string("xlings") + std::string(mcpp::platform::exe_suffix))) + return vendored_xlings_version(*sys); + return {}; +} + } // namespace mcpp::fallback diff --git a/tests/unit/test_sysroot_ownership.cpp b/tests/unit/test_sysroot_ownership.cpp new file mode 100644 index 00000000..bd3a26a4 --- /dev/null +++ b/tests/unit/test_sysroot_ownership.cpp @@ -0,0 +1,69 @@ +// A sysroot is trusted because of where it lives, not because it is there. +// +// gcc records `--sysroot=<...>/.xlings/subos/default` as a literal string when +// it is built, and keeps reporting that string in every project it later +// serves. On a machine with several checkouts, that path routinely EXISTS and +// belongs to a different one -- so the original predicate, "does the path +// exist", answered yes and the build silently took another project's headers. +// Measured: a build inside mcpp resolved a sysroot under an unrelated repo. +// +// These assertions pin the axis that replaced it. Nothing here touches the +// filesystem, deliberately: the point is that existence is not consulted. + +#include + +import std; +import mcpp.fallback.probe_sysroot; + +namespace fb = mcpp::fallback; + +namespace { + +const std::filesystem::path kRegistry = "/home/u/.mcpp/registry"; +const std::filesystem::path kProject = "/home/u/work/thisrepo"; + +TEST(SysrootOwnership, PayloadSysrootIsOwned) { + EXPECT_FALSE(fb::sysroot_is_foreign( + kRegistry / "subos" / "default", kRegistry, kProject)); +} + +TEST(SysrootOwnership, ProjectLocalSysrootIsOwned) { + EXPECT_FALSE(fb::sysroot_is_foreign( + kProject / ".xlings" / "subos" / "default", kRegistry, kProject)); +} + +// The case that motivated the predicate: a path that exists, is spelled +// exactly like a legitimate one, and belongs to someone else. +TEST(SysrootOwnership, AnotherCheckoutIsForeign) { + EXPECT_TRUE(fb::sysroot_is_foreign( + "/home/u/work/otherrepo/.xlings/subos/default", kRegistry, kProject)); +} + +TEST(SysrootOwnership, AnotherHomeIsForeign) { + EXPECT_TRUE(fb::sysroot_is_foreign( + "/home/other/.mcpp/registry/subos/default", kRegistry, kProject)); +} + +// Both anchors are load-bearing. Registry alone would condemn every +// project-local tree; project alone would condemn every payload sysroot. +TEST(SysrootOwnership, EitherAnchorSuffices) { + EXPECT_FALSE(fb::sysroot_is_foreign( + kRegistry / "subos" / "default", kRegistry, {})); + EXPECT_FALSE(fb::sysroot_is_foreign( + kProject / ".xlings" / "subos" / "default", {}, kProject)); +} + +// No sysroot is not a foreign sysroot. gcc payloads on this machine report an +// empty `-print-sysroot`, so this is the ordinary case, not an edge one. +TEST(SysrootOwnership, EmptyIsNotForeign) { + EXPECT_FALSE(fb::sysroot_is_foreign({}, kRegistry, kProject)); +} + +// A prefix that merely shares characters is not containment. +TEST(SysrootOwnership, SiblingPrefixIsNotContainment) { + EXPECT_TRUE(fb::sysroot_is_foreign( + "/home/u/work/thisrepo-backup/.xlings/subos/default", + kRegistry, kProject)); +} + +} // namespace diff --git a/tests/unit/test_xlings_version_pin.cpp b/tests/unit/test_xlings_version_pin.cpp new file mode 100644 index 00000000..131c5e8a --- /dev/null +++ b/tests/unit/test_xlings_version_pin.cpp @@ -0,0 +1,66 @@ +// The vendored xlings is replaced when it is BEHIND the pin, and only then. +// +// acquire_xlings_binary used to return on mere existence, so a home kept +// whatever xlings it first acquired forever -- measured at 2026.8.2.1 against +// a pin of 2026.8.6.3, with `mcpp self env` printing both numbers next to each +// other and saying nothing. That is not cosmetic: the subos_info block arrived +// in xlings 2026.8.5.1, so on that machine the graphics packages' declarations +// were discarded by a client too old to have the API, and the fix for +// mcpp#352 could never take effect no matter how current mcpp itself was. +// +// The comparison is STRICTLY older, not "not equal". A user who deliberately +// put a newer xlings there must not be downgraded by an mcpp pinning an older +// one, and an unparseable version is not evidence of being behind. + +#include + +import std; +import mcpp.fallback.xlings_binary; + +namespace fb = mcpp::fallback; + +namespace { + +TEST(XlingsVersionPin, OlderIsBehind) { + EXPECT_TRUE(fb::version_is_older("2026.8.2.1", "2026.8.6.3")); + EXPECT_TRUE(fb::version_is_older("2026.7.31.3", "2026.8.1.1")); + EXPECT_TRUE(fb::version_is_older("2026.8.6", "2026.8.6.1")); +} + +TEST(XlingsVersionPin, EqualIsNotBehind) { + EXPECT_FALSE(fb::version_is_older("2026.8.6.3", "2026.8.6.3")); + // Trailing zeros compare equal, not older. + EXPECT_FALSE(fb::version_is_older("2026.8.6.0", "2026.8.6")); +} + +// The case that decides whether this is safe to run automatically: a user who +// put a NEWER xlings in place keeps it. +TEST(XlingsVersionPin, NewerIsNotDowngraded) { + EXPECT_FALSE(fb::version_is_older("2026.9.1.1", "2026.8.6.3")); + EXPECT_FALSE(fb::version_is_older("2027.1.1.1", "2026.12.31.9")); +} + +// A version this code cannot parse says nothing, and "says nothing" must not +// be read as "is behind" -- that would delete a working binary on a guess. +TEST(XlingsVersionPin, UnparseableIsNotBehind) { + EXPECT_FALSE(fb::version_is_older("dev", "2026.8.6.3")); + EXPECT_FALSE(fb::version_is_older("2026.8.6.3", "")); + EXPECT_FALSE(fb::version_is_older("", "2026.8.6.3")); + EXPECT_FALSE(fb::version_is_older("2026.8.x", "2026.8.6.3")); +} + +// The version scheme changed epochs: xlings went 0.4.x -> YYYY.M.D.N. Both +// live on the same disk, and a home's system xlings may still be a 0.4.x while +// its vendored one is already dated. Getting this backwards is not academic -- +// the first cut of the replace-when-behind logic deleted the vendored binary +// and re-acquired from `which xlings`, which on this developer machine turned +// 2026.8.2.1 into 0.4.51: older still, and equally missing the very feature +// the replacement existed to restore. Being behind the pin justifies looking +// for a replacement; it does not justify accepting whatever turns up. +TEST(XlingsVersionPin, DatedSchemeIsNewerThanTheOldOne) { + EXPECT_FALSE(fb::version_is_older("2026.8.2.1", "0.4.51")); + EXPECT_TRUE(fb::version_is_older("0.4.51", "2026.8.2.1")); + EXPECT_TRUE(fb::version_is_older("0.4.51", "0.4.54")); +} + +} // namespace From 9e1f4f522c779933822a0fb5b746f0c59ca8c5d4 Mon Sep 17 00:00:00 2001 From: speak-agent Date: Sat, 8 Aug 2026 09:19:23 +0800 Subject: [PATCH 08/21] docs: the compiler ships a capability; mcpp supplies the policy (2026.8.8.2) MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit 08-toolchain-internals had two sections that stopped being true when the specs rewrite was removed. §4's gcc row still promised a specs rewrite; §5 described the clang cfg as if bypassing an install-time configuration were a clang-specific accommodation rather than the rule both compilers now follow. Rewritten as one policy with two mechanisms (`--no-default-config` for clang, a generated `-specs=` for gcc), including what removing gcc's baked `*link:` takes away and therefore what mcpp has to put back. New §2.1 documents where the runtime binding comes from, since it is now a resolution with an authority rather than a directory scan, and since "no binding declines payload-first" is surprising unless the reason is written down. 05-mcpp-toml notes that `[xlings] subos` picks the runtime too -- it was already a key, but it did not previously decide this. CHANGELOG gains 2026.8.8.2 and, separately, 2026.8.8.1, which shipped without an entry. xlings pin 2026.8.6.3 -> 2026.8.7.1: upstream is at 2026.8.8.1, but xim-pkgindex does not carry it yet and pinning ahead of the index makes every CI job install a version that is not there. --- .github/actions/bootstrap-mcpp/action.yml | 2 +- .github/actions/setup-macos-llvm/action.yml | 2 +- .github/workflows/bootstrap-macos.yml | 2 +- .github/workflows/ci-fresh-install.yml | 6 +- .github/workflows/ci-linux-e2e.yml | 2 +- .github/workflows/cross-build-test.yml | 4 +- .github/workflows/release.yml | 14 +-- CHANGELOG.md | 65 ++++++++++++++ docs/05-mcpp-toml.md | 11 +++ docs/08-toolchain-internals.md | 97 ++++++++++++++++++--- docs/zh/05-mcpp-toml.md | 7 ++ docs/zh/08-toolchain-internals.md | 70 +++++++++++++-- mcpp.toml | 2 +- src/version.cppm | 2 +- src/xlings.cppm | 2 +- 15 files changed, 249 insertions(+), 39 deletions(-) diff --git a/.github/actions/bootstrap-mcpp/action.yml b/.github/actions/bootstrap-mcpp/action.yml index 813d3210..2e171056 100644 --- a/.github/actions/bootstrap-mcpp/action.yml +++ b/.github/actions/bootstrap-mcpp/action.yml @@ -25,7 +25,7 @@ inputs: # `package.name`, so one of the two was simply unreachable — and which one # depended on the machine, which is why CI failed on `compat:lua` on # Windows and `mcpplibs.capi:lua` on Linux. Never pin below that. - default: '2026.8.6.3' + default: '2026.8.7.1' cache-target: description: also restore/save target/ (build artifacts + BMIs) required: false diff --git a/.github/actions/setup-macos-llvm/action.yml b/.github/actions/setup-macos-llvm/action.yml index 682d72bc..d00ad2e5 100644 --- a/.github/actions/setup-macos-llvm/action.yml +++ b/.github/actions/setup-macos-llvm/action.yml @@ -15,7 +15,7 @@ inputs: # Floor imposed by the index, not a routine bump — see # .github/actions/bootstrap-mcpp/action.yml for why 0.4.69 is required # (two packages named `lua` in one repo need openxlings/xlings#381). - default: '2026.8.6.3' + default: '2026.8.7.1' runs: using: composite diff --git a/.github/workflows/bootstrap-macos.yml b/.github/workflows/bootstrap-macos.yml index e2e7ba29..f607854d 100644 --- a/.github/workflows/bootstrap-macos.yml +++ b/.github/workflows/bootstrap-macos.yml @@ -17,7 +17,7 @@ jobs: # Dormant (workflow_dispatch only), but kept in step with the rest — # check_version_pins.sh holds it there. Floor: 0.4.69, below which the # index cannot resolve two packages that share a short name. - XLINGS_VERSION: '2026.8.6.3' + XLINGS_VERSION: '2026.8.7.1' steps: - uses: actions/checkout@v4 diff --git a/.github/workflows/ci-fresh-install.yml b/.github/workflows/ci-fresh-install.yml index 9c53eeba..503c8695 100644 --- a/.github/workflows/ci-fresh-install.yml +++ b/.github/workflows/ci-fresh-install.yml @@ -152,7 +152,7 @@ jobs: env: XLINGS_NON_INTERACTIVE: '1' run: | - curl -fsSL https://raw.githubusercontent.com/openxlings/xlings/main/tools/other/quick_install.sh | bash -s v2026.8.6.3 + curl -fsSL https://raw.githubusercontent.com/openxlings/xlings/main/tools/other/quick_install.sh | bash -s v2026.8.7.1 echo "$HOME/.xlings/subos/current/bin" >> "$GITHUB_PATH" - name: Install mcpp and config mirror @@ -292,7 +292,7 @@ jobs: - name: Install xlings + mcpp run: | - curl -fsSL https://raw.githubusercontent.com/openxlings/xlings/main/tools/other/quick_install.sh | bash -s v2026.8.6.3 + curl -fsSL https://raw.githubusercontent.com/openxlings/xlings/main/tools/other/quick_install.sh | bash -s v2026.8.7.1 # Deliberately NOT writing to $GITHUB_PATH here. On container # images that declare no PATH in their config (opensuse/ # tumbleweed), appending a single dir to GITHUB_PATH makes the @@ -363,7 +363,7 @@ jobs: # (older ones carry minos=15 and refuse to start). # v0.4.51+: in-process sha256 — this image has no sha256sum # binary, so pinned fetches failed before it. - curl -fsSL https://raw.githubusercontent.com/openxlings/xlings/main/tools/other/quick_install.sh | bash -s v2026.8.6.3 + curl -fsSL https://raw.githubusercontent.com/openxlings/xlings/main/tools/other/quick_install.sh | bash -s v2026.8.7.1 echo "$HOME/.xlings/subos/current/bin" >> "$GITHUB_PATH" - name: Install mcpp and config mirror diff --git a/.github/workflows/ci-linux-e2e.yml b/.github/workflows/ci-linux-e2e.yml index 790dbfd7..58e61e56 100644 --- a/.github/workflows/ci-linux-e2e.yml +++ b/.github/workflows/ci-linux-e2e.yml @@ -123,7 +123,7 @@ jobs: - name: Bootstrap xlings + released mcpp run: | - curl -fsSL https://raw.githubusercontent.com/openxlings/xlings/main/tools/other/quick_install.sh | bash -s v2026.8.6.3 + curl -fsSL https://raw.githubusercontent.com/openxlings/xlings/main/tools/other/quick_install.sh | bash -s v2026.8.7.1 export PATH="$HOME/.xlings/subos/current/bin:$PATH" xlings update xlings install mcpp -y -g diff --git a/.github/workflows/cross-build-test.yml b/.github/workflows/cross-build-test.yml index b9f0d053..51bdf9fe 100644 --- a/.github/workflows/cross-build-test.yml +++ b/.github/workflows/cross-build-test.yml @@ -118,7 +118,7 @@ jobs: # release assets were uploaded in a broken state (records present, # blobs missing → 404 on GET); re-uploaded clean. The stale-INDEX # half is handled by the marker-clear below. - XLINGS_VERSION: '2026.8.6.3' + XLINGS_VERSION: '2026.8.7.1' run: | tarball="xlings-${XLINGS_VERSION}-linux-x86_64.tar.gz" curl -fsSL -o "/tmp/${tarball}" \ @@ -255,7 +255,7 @@ jobs: - name: Bootstrap mcpp via xlings env: XLINGS_NON_INTERACTIVE: '1' - XLINGS_VERSION: '2026.8.6.3' + XLINGS_VERSION: '2026.8.7.1' run: | tarball="xlings-${XLINGS_VERSION}-linux-x86_64.tar.gz" curl -fsSL -o "/tmp/${tarball}" \ diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index d8327669..b9be200b 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -96,7 +96,7 @@ jobs: # Pin xlings to a known-good version. The upstream install # script always grabs `latest` (no version override), so we # download + self-install manually to avoid broken releases. - XLINGS_VERSION: '2026.8.6.3' + XLINGS_VERSION: '2026.8.7.1' run: | if [ ! -x "$HOME/.xlings/subos/default/bin/xlings" ]; then tarball="xlings-${XLINGS_VERSION}-linux-x86_64.tar.gz" @@ -288,7 +288,7 @@ jobs: - name: Bootstrap mcpp via xlings env: XLINGS_NON_INTERACTIVE: '1' - XLINGS_VERSION: '2026.8.6.3' + XLINGS_VERSION: '2026.8.7.1' run: | tarball="xlings-${XLINGS_VERSION}-linux-x86_64.tar.gz" curl -fsSL -o "/tmp/${tarball}" \ @@ -358,11 +358,11 @@ jobs: # below are pinned to the same version as XLINGS_VERSION; they are # NOT interpolated from it, so check_version_pins.sh scans for them # explicitly (they were absent from the old lock-step comment). - XLA="xlings-2026.8.6.3-linux-aarch64.tar.gz" + XLA="xlings-2026.8.7.1-linux-aarch64.tar.gz" if curl -fsSL -o "/tmp/$XLA" \ - "https://github.com/openxlings/xlings/releases/download/v2026.8.6.3/$XLA"; then + "https://github.com/openxlings/xlings/releases/download/v2026.8.7.1/$XLA"; then tar -xzf "/tmp/$XLA" -C /tmp - XLBIN=$(find /tmp/xlings-2026.8.6.3-linux-aarch64 -path '*/bin/xlings' -type f | head -1) + XLBIN=$(find /tmp/xlings-2026.8.7.1-linux-aarch64 -path '*/bin/xlings' -type f | head -1) if [ -n "$XLBIN" ]; then mkdir -p "$STAGING/$WRAPPER/registry/bin" cp "$XLBIN" "$STAGING/$WRAPPER/registry/bin/xlings" @@ -440,7 +440,7 @@ jobs: - name: Bootstrap mcpp via xlings env: XLINGS_NON_INTERACTIVE: '1' - XLINGS_VERSION: '2026.8.6.3' + XLINGS_VERSION: '2026.8.7.1' run: | if [ ! -x "$HOME/.xlings/subos/default/bin/xlings" ]; then WORK=$(mktemp -d) @@ -622,7 +622,7 @@ jobs: shell: bash env: XLINGS_NON_INTERACTIVE: '1' - XLINGS_VERSION: '2026.8.6.3' + XLINGS_VERSION: '2026.8.7.1' run: | # Captured before the `cd` below, in POSIX form: this step never # returns to the workspace, and GITHUB_WORKSPACE is a backslash diff --git a/CHANGELOG.md b/CHANGELOG.md index 6c262d43..2bfee02a 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -3,6 +3,71 @@ > 本文件追踪 `mcpp-community/mcpp` 公开仓的版本演进。 > 格式参考 [Keep a Changelog](https://keepachangelog.com/zh-CN/1.1.0/)。 +## [2026.8.8.2] — 2026-08-08 + +编译器交付的是**能力**,不是配置。设计与实测证据见 +`.agents/docs/2026-08-08-payload-version-and-contract-drift-design.md`。 + +### 修复 + +- **产物加载哪个 glibc,现在有权威,不再靠目录顺序。** + + payload-first 的构建要链接到某个具体 glibc。旧规则是问目录"那个 glibc"、 + 取 `readdir` 的第一项——只装了一个时它永远对,所以从没有东西逼它对。一条带 + `xim:glibc@>=2.38` 的依赖就足以装进第二个:编译侧取了 2.44,而产物的 + interpreter(装机时冻结在 gcc specs 里)仍是 2.39,于是二进制引用 + `GLIBC_2.42` 符号却跑在没有它的运行时上,报错还落在与那条依赖无关的包上。 + + 现在由 subos 自述的 runtime 作答(`[xlings] subos` 可指定),没有答案就**拒绝** + 走 payload-first,而不是挑一个。该绑定计入工具链指纹(11 字段),两次只在 + runtime 上不同的构建不再共用缓存。旧 subos 回落到工具链自身烙入的值。 + +- **不再改写 GCC 的 `specs`,产物也不再带别人机器的死路径。** + + 旧的 specs 重写用单路径 needle 配双路径 replacement,每个跑过它的 home 都漏下 + 一条:一台开发机产出的每个 gcc 产物里都有 **68** 条陈旧 `RUNPATH`,全指向已删除 + 的 `mktemp` 目录。改为 `-specs=` 一份逐构建生成的干净文件(`-dumpspecs` 取内建 + `*link:`,去掉 loader/rpath),loader 与 rpath 由 mcpp 显式写在链接行上。逐构建、 + 不需写权限,继承来的只读 payload 也能用。e2e `201_gcc_no_specs_pollution.sh` + 断言的是产物而非 specs 文件。 + +- **落后于 pin 的 vendored xlings 会被替换——但只在替换品确实更新时。** + + `acquire_xlings_binary` 过去见到文件存在就返回,于是一个 home 会永远留着它第一次 + 获取的 xlings(实测 2026.8.2.1 对 pin 2026.8.6.3)。`subos_info` 是 2026.8.5.1 + 才有的,所以那台机器上 subos 的自述一直被一个太旧的客户端丢弃。修复的第一版直接 + 删了重取,结果把 2026.8.2.1 换成了系统的 0.4.51——更旧,且同样没有那个特性;现在 + 先给替换品定价再动手。 + +- **`--sysroot` 的判据从"存在"改为"归属"。** gcc 把 + `--sysroot=<...>/.xlings/subos/default` 当字符串烙进去,而一台机器上有很多同名 + 目录,所以那条烙入的路径经常存在、却属于另一个 checkout(实测:mcpp 里的构建解析 + 到了无关仓库下的 sysroot)。 + +### 其他 + +- `mcpp self doctor` 新增两条检查:vendored xlings 落后于 pin;sysroot 既不属于本 + mcpp home 也不属于当前项目。 +- xlings pin 提升到 2026.8.7.1。 + +## [2026.8.8.1] — 2026-08-07 + +xlings 作为运行时底座:subos 环境到达程序,以及 self-contained 的 +`/proc/self/exe` 陷阱(#352、#375)。设计见 +`.agents/docs/2026-08-07-xlings-as-runtime-substrate-design.md`。 + +### 修复 + +- **subos 声明的环境变量现在真的到达被运行的程序(#352)。** 之前 mcpp 读的是自己 + 想象出来的 `subos_info.envs` 线格式(数组),而 xlings 写的是以 binding 为键的 + 对象,于是整条修复是空转的。现在按 xlings 实际写出的格式解析,并用逐字取自真实 + xlings 输出的 fixture 钉住。 +- **升级后的 mcpp 会重放升级前的缓存,导致 subos 环境丢失。** 快路径命中时不带 + subos 环境;该缺陷是靠一条"必须走快路径"的自证断言抓到的,而那条断言随即又抓到 + 第二个:缓存行写在 `profile=` 之前却在 `cacheMode=` 之后解析。 +- **self-contained 打包的 `/proc/self/exe` 陷阱(#375)。** 新增 `MCPP_BUNDLE_DIR` + 契约。 + ## [2026.8.7.1] — 2026-08-07 两处「模型比生态少一层」。设计与实测证据见 `.agents/docs/2026-08-07-windows-resources-and-version-identity-design.md`。 diff --git a/docs/05-mcpp-toml.md b/docs/05-mcpp-toml.md index d72c7900..999ce66c 100644 --- a/docs/05-mcpp-toml.md +++ b/docs/05-mcpp-toml.md @@ -966,6 +966,17 @@ build needs (`make`/`cmake`/`protoc`/…), pin tool versions per project, or set build-time env vars — without hand-editing `.xlings.json`. `[toolchain]` (§2.7) remains the ergonomic shorthand for the compiler; `[xlings.workspace]` is the general form. +`subos` carries a second meaning on Linux: it selects **which C runtime the +build binds against**. A subos describes its own runtime (xlings 2026.8.5.1+), +and mcpp takes that as the authority rather than looking around for a libc — so +`subos = "el8"` and `subos = "trixie"` in two projects on one machine produce +artifacts targeting each one's glibc, with the compile side and the run side +guaranteed to agree. The binding is part of the toolchain fingerprint, so +switching it rebuilds rather than reusing the other subos' objects. A subos +older than that (or none at all) falls back to whatever the toolchain itself +was installed against; `mcpp build -v` prints which of the two happened. See +docs/08-toolchain-internals.md §2.1. + ### 2.14 Host tools from a dependency (mcpp 2026.8.5.1+) A package can build a binary its consumers need *at build time* — `protoc`, a diff --git a/docs/08-toolchain-internals.md b/docs/08-toolchain-internals.md index f4bdcb1b..6a2eb4d4 100644 --- a/docs/08-toolchain-internals.md +++ b/docs/08-toolchain-internals.md @@ -15,6 +15,9 @@ resolve payload (xim:gcc / xim:llvm / xim:musl-gcc xpkg under the sandbox) ▼ ensure_post_install_fixup() ← idempotent convergence (marker-gated) ▼ +resolve runtime binding ← which libc the artifact will load (§2.1) — an + answer, not a search + ▼ detect / probe ← triple, sysroot, payload paths (glibc, linux-headers) ▼ ToolchainLinkModel (single resolver for the C-library axis) @@ -65,12 +68,44 @@ is resolved/auto-installed via the xlings backend into the sandbox |---|---| | `targetTriple` | ` -dumpmachine` | | `sysroot` | `-print-sysroot` (validated: must actually carry libc headers), with a remap fallback for xlings-built GCC whose baked build-time path doesn't exist locally | -| `payloadPaths` | sibling xpkg discovery: glibc payload (`include/` + `lib64|lib/`) and linux-headers payload — the *payload-first* fine-grained sysroot | +| `payloadPaths` | the resolved runtime binding (§2.1) names the glibc payload exactly; linux-headers is still discovered as a sibling. No binding ⇒ no payload-first, by design | | runtime dirs | toolchain-private lib dirs for produced binaries' `-L`/`-rpath` | Note the probe deliberately does **not** mine the clang cfg for `--sysroot` anymore: the cfg is an output of this machinery, not an input (§5). +### 2.1 The runtime binding — which libc, decided once + +A payload-first build links against a specific glibc, and *which* one is a +fact about the environment, not something to be inferred. mcpp resolves it in +order: + +1. `[xlings] subos = ""` — the named subos, a sibling of the active one, + describes its own runtime in the `subos_info` block of its `.xlings.json` + (xlings 2026.8.5.1+). +2. The active subos, same block. +3. *Compatibility.* A subos created before that block existed cannot answer. + The value baked into the toolchain itself then stands in — gcc's specs, + clang's cfg. This is the value the artifact **would** load, so compile side + and run side still agree; it retires itself the moment the subos can speak. + +No binding is a **refusal**, not a default: `CLibMode::PayloadFirst` is +declined rather than picking a libc. + +The rule this replaced asked a directory for "the glibc" and took the first +entry `readdir` yielded. With one glibc installed that is always right, so +nothing forced it to be correct — and a dependency carrying `xim:glibc@>=2.38` +is enough to install a second one. When that happened in mcpp-index, the +compile side took 2.44 while the artifact's interpreter, frozen in gcc's specs +at install time, still named 2.39; binaries referenced `GLIBC_2.42` symbols +against a runtime without them, and the failures surfaced on packages +unrelated to the dependency that pulled the second glibc in. Directory order +is not a decision procedure. + +Because the binding decides what the artifact loads, it is part of the +toolchain fingerprint (11 fields, not 10) — two builds differing only in +runtime must not share a cache entry. + ## 3. The link model (`src/toolchain/linkmodel.cppm`) `ToolchainLinkModel` answers exactly one question — *how do we compile and @@ -86,6 +121,10 @@ CLibMode::PayloadFirst glibc/linux-headers xpkgs found (the normal bundled-LLV -L [+ -rpath + --dynamic-linker for clang] CLibMode::Sysroot a usable --sysroot (GCC include-fixed world, self-contained musl sysroots, the macOS SDK) + link: --sysroot, plus --dynamic-linker and -L/-rpath + for the payload when one is known — the + sysroot says where headers live, not which + loader runs the result CLibMode::None nothing usable — host defaults apply; the hermetic check (§6) rejects that leakage unless an explicit host-library exception is in effect @@ -110,12 +149,14 @@ itself as its first consumer; mcpp can then re-add a reader. ## 4. The unified post-install fixup pipeline (`src/toolchain/post_install.cppm`) -Sandbox payloads are prebuilt ELF trees. Two kinds of paths baked into them -are unknowable at packaging time and must be aligned to the *local* sandbox: -`PT_INTERP`/`RUNPATH` inside binaries, and the loader/rpath lines inside GCC -specs. `ensure_post_install_fixup(cfg, payloadRoot, pkg)` is the **single -entry** for that alignment, called from all three entry paths (explicit -install, default auto-install, manifest auto-install). +Sandbox payloads are prebuilt ELF trees whose baked `PT_INTERP`/`RUNPATH` are +unknowable at packaging time and must be aligned to the *local* sandbox. +`ensure_post_install_fixup(cfg, payloadRoot, pkg)` is the **single entry** for +that alignment, called from all three entry paths (explicit install, default +auto-install, manifest auto-install). + +> This pipeline used to rewrite GCC's `specs` file as well, so that produced +> binaries got a loader and an rpath. That is no longer done, and §5 is why. > Historical note: before 0.0.83 each path remembered — or forgot — its own > subset. The manifest path ran *nothing*, which is how a freshly @@ -146,7 +187,7 @@ mcpp's sight — trust-but-verify is the only reliable semantic. | kind | actions | |---|---| -| `gcc` (glibc) | patchelf walk over the gcc payload **and the shared binutils payload** (PT_INTERP → sandbox loader, RUNPATH → glibc+gcc lib dirs); specs rewrite (baked loader/rpath → payload glibc, specs-grammar-aware — `%{...}` conditionals must never be corrupted) | +| `gcc` (glibc) | patchelf walk over the gcc payload **and the shared binutils payload** (PT_INTERP → sandbox loader, RUNPATH → glibc+gcc lib dirs) — so that *gcc itself* runs. Nothing is written into `specs` | | `llvm` | patchelf walk over `lib/` only (runtime `.so` RUNPATH; `bin/` is left alone to preserve xlings-set RUNPATHs); deterministic cfg regeneration (§5) | | `musl-gcc` | nothing — self-contained sysroot, static world | @@ -162,8 +203,7 @@ mcpp's sight — trust-but-verify is the only reliable semantic. (symlink-inherited from another `MCPP_HOME`) are never patched — their owner already converged them, and patching through the symlink would brick the owner's toolchain. -- Specs rewriting is content-aware (already-aligned specs are skipped). - Extending the same check to the patchelf walk (compare +- Extending content-awareness to the patchelf walk (compare `--print-interpreter`/`--print-rpath` before writing, so an already-aligned payload converges with **zero writes**) is a known follow-up. - The long-term direction is for the *installer* (xlings) to own all @@ -171,11 +211,42 @@ mcpp's sight — trust-but-verify is the only reliable semantic. mcpp read-only + verification. The pipeline here is the compatibility layer until then, and the self-healing mechanism for drift either way. -## 5. The clang cfg: direct-invocation support only +## 5. The compiler is a capability, not a configuration + +A payload ships two separable things: the ability to compile, and an opinion +about how to link. mcpp wants the first and supplies the second itself — the +link line is where a build's decisions belong, because it is the thing that +varies per build. Both compilers now follow that rule; only the mechanism +differs. + +**clang** — `--no-default-config` on every mcpp invocation. + +**gcc** — `-specs=` with a generated file. ` -dumpspecs` prints the +**built-in** specs (unaffected by anything on disk); mcpp extracts the `*link:` +body, drops the loader and rpath lines from it, and writes the result to the +build directory. A `-specs=` file without a leading `+` *replaces* the rule it +names, so the payload's own opinion is overridden without the payload being +touched. Two consequences worth stating: it is per-build, so a second project +on the same machine is unaffected; and it needs no write access to the +toolchain, so an inherited or read-only payload works. + +Removing gcc's baked `*link:` also removes what it provided. mcpp therefore +supplies `--dynamic-linker` and every rpath entry explicitly on the link +line — the loader, the glibc lib dir, and gcc's own `lib64` (libgcc_s). Each +of those was found by removing the specs and watching what broke. + +Why not keep rewriting `specs`? Because the file is shared and the value is +per-build. The rewrite had a single-path needle and a two-path replacement, so +every home that ever ran it left one entry behind: **68** stale `RUNPATH` +entries, all naming deleted `mktemp` directories, in every gcc artifact one +developer machine produced. Nothing detected them, because a dead RUNPATH +entry costs only search time. e2e `201_gcc_no_specs_pollution.sh` asserts on +the artifact, not the specs file — what a user ships is what matters. + +### 5.1 The clang cfg `bin/clang++.cfg` exists so that direct invocations of the bundled -`clang++` (outside mcpp) get a working, hermetic compiler configuration. mcpp's own builds never read it -(`--no-default-config` always). The fixup pipeline **regenerates** it +`clang++` (outside mcpp) get a working, hermetic compiler configuration. The fixup pipeline **regenerates** it deterministically from the link model — same payload ⇒ byte-identical cfg on every machine and install path — rather than line-patching whatever an install produced. On Linux that means CRT discovery (`-B`), payload loader + diff --git a/docs/zh/05-mcpp-toml.md b/docs/zh/05-mcpp-toml.md index 971cf450..c8c109b0 100644 --- a/docs/zh/05-mcpp-toml.md +++ b/docs/zh/05-mcpp-toml.md @@ -724,6 +724,13 @@ OPENBLAS_NUM_THREADS = "1" host 工具(`make`/`cmake`/`protoc`…)、按项目固定工具版本、或设构建期环境变量——无需手改 `.xlings.json`。`[toolchain]`(§2.7)仍是编译器的便捷简写;`[xlings.workspace]` 是其通用形式。 +在 Linux 上 `subos` 还有第二重含义:它决定**构建绑定到哪个 C 运行时**。subos 会自述其 +runtime(xlings 2026.8.5.1 起),mcpp 以此为权威,而不是去四处找一个 libc——所以同一台机器 +上 `subos = "el8"` 与 `subos = "trixie"` 两个项目各自产出面向自己 glibc 的产物,且编译期与 +运行期保证一致。该绑定计入工具链指纹,故切换它会重新构建,而不会复用另一个 subos 的目标文件。 +更旧的 subos(或没有 subos)则回落到工具链自身安装时所对应的运行时;`mcpp build -v` 会打印 +实际走了哪一条。参见 docs/08-toolchain-internals.md §2.1。 + ### 2.14 依赖产出的 host 工具(mcpp 2026.8.5.1+) 一个包能构建出消费者在**构建期**需要的二进制 —— `protoc`、`grpc_cpp_plugin`、 diff --git a/docs/zh/08-toolchain-internals.md b/docs/zh/08-toolchain-internals.md index 4a61a326..b6b41b83 100644 --- a/docs/zh/08-toolchain-internals.md +++ b/docs/zh/08-toolchain-internals.md @@ -14,6 +14,8 @@ mcpp.toml [toolchain] / 全局默认 / `mcpp toolchain install` ▼ ensure_post_install_fixup() ← 幂等收敛(marker 闸门) ▼ +解析 runtime binding ← 产物将加载哪个 libc(§2.1)——是答案,不是搜索 + ▼ detect / probe ← triple、sysroot、payload 路径(glibc、linux-headers) ▼ ToolchainLinkModel(C 库轴的唯一解析器) @@ -56,9 +58,34 @@ xlings 后端解析/自动安装到沙箱 |---|---| | `targetTriple` | ` -dumpmachine` | | `sysroot` | `-print-sysroot`(校验必须真带 libc 头);xlings 构建的 GCC 烙的是构建机路径,有 remap 回退 | -| `payloadPaths` | 兄弟 xpkg 发现:glibc payload(`include/` + `lib64|lib/`)与 linux-headers payload——即 *payload 优先* 的细粒度 sysroot | +| `payloadPaths` | 由解析出的 runtime binding(§2.1)**精确指名** glibc payload;linux-headers 仍按兄弟 xpkg 发现。没有 binding 就不走 payload-first——这是设计,不是缺陷 | | 运行库目录 | 工具链私有 lib 目录,用于产物的 `-L`/`-rpath` | +### 2.1 runtime binding:绑哪个 libc,只决定一次 + +payload-first 的构建会链接到某个具体的 glibc,而**是哪一个**是关于环境的事实, +不该靠推断。mcpp 按顺序解析: + +1. `[xlings] subos = ""` —— 该 subos(活动 subos 的兄弟)在自己 + `.xlings.json` 的 `subos_info` 块里自述 runtime(xlings 2026.8.5.1 起)。 +2. 活动 subos,同一个块。 +3. *兼容路径。* 在该块出现之前创建的 subos 无法作答,此时以工具链自身烙入的值 + 顶上——gcc 的 specs、clang 的 cfg。这正是产物**将会**加载的那个值,故编译期 + 与运行期仍然一致;一旦 subos 能作答,这条自动退场。 + +没有 binding 是**拒绝**而不是取默认值:`CLibMode::PayloadFirst` 会被放弃,而不是 +去挑一个 libc。 + +被它取代的旧规则是:向某个目录问"那个 glibc",然后拿 `readdir` 吐出的第一项。 +只装了一个 glibc 时它永远正确,所以从来没有东西逼它正确——而一条带 +`xim:glibc@>=2.38` 的依赖就足以装进第二个。这在 mcpp-index 上真实发生过:编译侧 +取了 2.44,而产物的 interpreter(装机时冻结在 gcc specs 里)仍指向 2.39;二进制 +引用了 `GLIBC_2.42` 符号,却跑在没有这些符号的运行时上,且报错落在与"拉进第二个 +glibc 的那条依赖"毫无关系的包上。目录顺序不是决策依据。 + +因为 binding 决定产物加载什么,它计入工具链指纹(11 个字段,不是 10 个)——只在 +runtime 上不同的两次构建绝不能共用同一条缓存。 + 注意:probe 已**不再**从 clang cfg 挖 `--sysroot`——cfg 是这套机制的输出, 不是输入(见 §5)。 @@ -95,12 +122,15 @@ CLibMode::None 无可用来源——落宿主默认;除非显式允许 ## 4. 统一 post-install fixup 管线(`src/toolchain/post_install.cppm`) -沙箱 payload 是预编译 ELF 树。有两类打包期不可能预知、必须对齐到**本机沙箱**的 -路径:二进制里的 `PT_INTERP`/`RUNPATH`,以及 GCC specs 里的 loader/rpath 行。 +沙箱 payload 是预编译 ELF 树,其烙入的 `PT_INTERP`/`RUNPATH` 在打包期不可能预知, +必须对齐到**本机沙箱**。 `ensure_post_install_fixup(cfg, payloadRoot, pkg)` 是这次对齐的**唯一入口**, 由三条入口路径(显式 install、默认 auto-install、manifest auto-install)共同 调用。 +> 本管线曾同时重写 GCC 的 `specs` 文件,以便产物拿到 loader 和 rpath。现已不再 +> 这样做,原因见 §5。 + > 历史注记:0.0.83 之前各路径各自记得——或忘记——自己那份 fixup。manifest > 路径什么都不跑:刚 auto-install 的 llvm 因此保留着陈旧、随装机环境漂移的 > cfg(issue #195);gcc 也曾因此产出找不到 `stdlib.h` 的沙箱。"用哪条命令装的" @@ -125,7 +155,7 @@ glibc、payload 从别的 home 继承而来)发生在 mcpp 视野之外,trust-bu | kind | 动作 | |---|---| -| `gcc`(glibc)| 对 gcc payload **及共享的 binutils payload** 做 patchelf 遍历(PT_INTERP → 沙箱 loader,RUNPATH → glibc+gcc lib);specs 重写(烙入的 loader/rpath → payload glibc,**必须感知 specs 语法**——`%{...}` 条件块绝不能被破坏)| +| `gcc`(glibc)| 对 gcc payload **及共享的 binutils payload** 做 patchelf 遍历(PT_INTERP → 沙箱 loader,RUNPATH → glibc+gcc lib)——目的是让 *gcc 自己* 能跑起来。不往 `specs` 里写任何东西 | | `llvm` | 只遍历 `lib/`(运行库 `.so` 的 RUNPATH;`bin/` 不碰,保留 xlings 设置的 RUNPATH);确定性再生 cfg(§5)| | `musl-gcc` | 无——自包含 sysroot,静态世界 | @@ -137,17 +167,43 @@ glibc、payload 从别的 home 继承而来)发生在 mcpp 视野之外,trust-bu `rename` 让新内容拿新 inode,活进程保有旧 inode。 - **所有权护栏。** 解析到本 home registry 之外的 payload(从别的 `MCPP_HOME` symlink 继承)一律不碰——属主已收敛过,隔着 symlink 改写会毁掉属主的工具链。 -- specs 重写是内容感知的(已对齐即跳过)。把同样的检查扩展到 patchelf 遍历 +- 把内容感知扩展到 patchelf 遍历 (写前比对 `--print-interpreter`/`--print-rpath`,已对齐的 payload **零写入** 收敛)是已知的后续项。 - 长期方向:全部写入由**安装器**(xlings)持有——装机时、以及 payload 进入 新 home 时——mcpp 退为只读 + 校验。本管线在那之前是兼容层,也是双向漂移的 自愈机制。 -## 5. clang cfg:仅服务直接调用场景 +## 5. 编译器是能力,不是配置 + +一个 payload 交付两样可分离的东西:编译的**能力**,以及关于如何链接的**主张**。 +mcpp 要前者,后者自己给——链接行才是构建决策该待的地方,因为它是逐次构建而变的 +那一个。现在两个编译器都遵循这条规则,只是机制不同。 + +**clang** —— mcpp 每次调用都带 `--no-default-config`。 + +**gcc** —— 用生成的文件走 `-specs=`。` -dumpspecs` 打印的是**内建** +specs(不受磁盘上任何文件影响);mcpp 从中取出 `*link:` 的正文,删掉其中的 loader +与 rpath 行,把结果写进构建目录。不带前导 `+` 的 `-specs=` 文件是**替换**它所命名的 +规则,于是 payload 自己的主张被覆盖,而 payload 本身分毫未动。两条推论值得写明: +它是逐构建的,所以同机器上的另一个项目不受影响;它不需要对工具链的写权限,所以 +继承来的、或只读的 payload 都能用。 + +删掉 gcc 烙入的 `*link:`,也就删掉了它提供的东西。因此 mcpp 在链接行上显式补齐 +`--dynamic-linker` 与每一条 rpath——loader、glibc lib 目录、以及 gcc 自己的 +`lib64`(libgcc_s)。这三条都是把 specs 拿掉后、看什么先坏掉而逐一找出来的。 + +为什么不继续重写 `specs`?因为那个文件是共享的,而值是逐构建的。旧的重写用的是 +单路径的 needle 配双路径的 replacement,于是每个跑过它的 home 都会漏下一条:一台 +开发机产出的**每一个** gcc 产物里都带着 **68** 条陈旧 `RUNPATH`,全部指向已被删除的 +`mktemp` 目录。没有任何东西发现它们,因为一条死 RUNPATH 只多花一点搜索时间。e2e +`201_gcc_no_specs_pollution.sh` 断言的是**产物**而不是 specs 文件——用户最终交付的 +是产物。 + +### 5.1 clang cfg `bin/clang++.cfg` 的职责是:直接调用打包内 `clang++`(不经由 mcpp)时, -获得可用且 hermetic 的编译器配置。mcpp 自己的构建从不读它(永远 `--no-default-config`)。 +获得可用且 hermetic 的编译器配置。 fixup 管线从链接模型**确定性再生**它——同一 payload ⇒ 任何机器、任何安装路径 产出字节一致的 cfg——而不是对装机产物做行级补丁。Linux 上内容为:CRT 发现 (`-B`)、payload loader + rpath、lld/compiler-rt/libunwind、C++ 驱动附加 diff --git a/mcpp.toml b/mcpp.toml index d2a6dbb6..60309c3d 100644 --- a/mcpp.toml +++ b/mcpp.toml @@ -1,6 +1,6 @@ [package] name = "mcpp" -version = "2026.8.8.1" +version = "2026.8.8.2" description = "Modern C++ build & package management tool" license = "Apache-2.0" authors = ["mcpp-community"] diff --git a/src/version.cppm b/src/version.cppm index bd258c4a..7e8caed1 100644 --- a/src/version.cppm +++ b/src/version.cppm @@ -31,6 +31,6 @@ import std; export namespace mcpp { -inline constexpr std::string_view MCPP_VERSION = "2026.8.8.1"; +inline constexpr std::string_view MCPP_VERSION = "2026.8.8.2"; } // namespace mcpp diff --git a/src/xlings.cppm b/src/xlings.cppm index d4f5a186..3423e506 100644 --- a/src/xlings.cppm +++ b/src/xlings.cppm @@ -44,7 +44,7 @@ namespace pinned { // in lock-step by hand; that list was already missing both composite // actions, which is how CI's sandbox sat on 0.4.30 unnoticed while // everything else had moved on. Don't reintroduce a hand-maintained list. - inline constexpr std::string_view kXlingsVersion = "2026.8.6.3"; + inline constexpr std::string_view kXlingsVersion = "2026.8.7.1"; inline constexpr std::string_view kNasmVersion = "3.02"; } From fcf2e4914e5cce97f50b77c0223d0269e35be71c Mon Sep 17 00:00:00 2001 From: speak-agent Date: Sat, 8 Aug 2026 09:22:40 +0800 Subject: [PATCH 09/21] fix(build): a cold build deleted the spec it was about to pass to gcc The generated `*link:` spec is written to the output directory, and run_build_plan deletes that directory on a cold build. prepare writes the file before that point, so every `--no-cache` build with gcc reached ninja with a link command naming a file that had just been removed: g++: fatal error: cannot read spec file '.../mcpp-clean-link.specs' Regenerating after the wipe rather than reordering the two. The property worth holding is "the spec exists when ninja runs"; stated as an invariant it also survives a user's `rm -rf target`, which reordering would not. The write is idempotent, so the warm path pays one stat. e2e 201 only ever built warm, where the file survives from the previous run -- so the test that exists precisely to guard this mechanism could not see it. It now builds cold as well. Found by the full local e2e sweep (100_cppfly_*), not by the targeted subset. --- src/build/execute.cppm | 18 ++++++++++++++++++ tests/e2e/201_gcc_no_specs_pollution.sh | 8 ++++++++ 2 files changed, 26 insertions(+) diff --git a/src/build/execute.cppm b/src/build/execute.cppm index d31208fa..b572f967 100644 --- a/src/build/execute.cppm +++ b/src/build/execute.cppm @@ -18,6 +18,7 @@ import mcpp.build.ninja; import mcpp.bmi_cache; import mcpp.manifest; import mcpp.modgraph.scanner; +import mcpp.toolchain.post_install; import mcpp.toolchain.stdmod; import mcpp.xlings; import mcpp.xlings.subos_info; @@ -376,6 +377,23 @@ export int run_build_plan(BuildContext& ctx, bool verbose, bool no_cache, std::filesystem::remove_all(ctx.outputDir, ec); } + // The generated `*link:` spec lives in the output directory, and the line + // above is allowed to delete that directory. prepare wrote the file before + // this point, so a cold build reached ninja with a link command naming a + // spec that no longer existed -- `g++: fatal error: cannot read spec file`, + // on every `--no-cache` build with gcc. + // + // Regenerating here rather than reordering: the invariant worth holding is + // "the spec exists when ninja runs", and stating it as an invariant + // survives the next thing that clears target/ (a user with `rm -rf`, for + // one). The write is idempotent, so the warm path costs one stat. + if (!ctx.plan.gccCleanSpecs.empty()) { + std::error_code ec; + if (!std::filesystem::exists(ctx.plan.gccCleanSpecs, ec)) + ctx.plan.gccCleanSpecs = mcpp::toolchain::write_clean_link_specs( + ctx.tc.binaryPath, ctx.outputDir); + } + auto be = mcpp::build::make_ninja_backend(); // M5.0: print "Inferred" banner when defaults / target inference fired. diff --git a/tests/e2e/201_gcc_no_specs_pollution.sh b/tests/e2e/201_gcc_no_specs_pollution.sh index deaea12c..d17cf818 100755 --- a/tests/e2e/201_gcc_no_specs_pollution.sh +++ b/tests/e2e/201_gcc_no_specs_pollution.sh @@ -35,6 +35,14 @@ EOF "$MCPP" build > "$TMP/build.log" 2>&1 || { cat "$TMP/build.log"; echo "build failed"; exit 1; } +# A cold build, because the generated spec lives in target/ and a cold build +# deletes target/. Written the obvious way -- generate during prepare, consume +# during the build -- gcc got a `-specs=` naming a file that had been removed +# in between, and EVERY `--no-cache` build failed with `cannot read spec file`. +# The warm path above cannot see that: the file is still there from last time. +"$MCPP" build --no-cache > "$TMP/cold.log" 2>&1 || { + cat "$TMP/cold.log"; echo "cold build failed"; exit 1; } + bin=$(find target -type f -name prog -path '*/bin/*' | head -1) [[ -n "$bin" ]] || { echo "no binary produced"; exit 1; } From f436c6691762d92611577c37c7d66b8433af722d Mon Sep 17 00:00:00 2001 From: speak-agent Date: Sat, 8 Aug 2026 09:40:12 +0800 Subject: [PATCH 10/21] fix(toolchain): the compatibility fallback read a file this change deleted Three defects, all found by CI in configurations this developer machine does not have. Each is the same shape: a repair placed where the control flow, or the machine state, does not reach it. 1. baked_runtime_binding read gcc's `specs` and clang's `.cfg` -- files mcpp used to write and no longer does. On a machine whose toolchain was installed after that change they are simply absent, so no binding resolved, so no payload paths, so no `--dynamic-linker`, so the artifact took the HOST loader and the hermetic check rejected it: /lib64/ld-linux-x86-64.so.2 (outside the sandbox) Every existing machine still has those files from before, which is exactly why local verification was green and CI was not. The binding now also comes from the compiler's own PT_INTERP -- written by the patchelf walk, which still runs on every install, and naming the same glibc payload the specs used to name. Read with a small ELF header reader rather than `patchelf --print-interpreter`: this runs during prepare, where patchelf is not guaranteed to be resolved. Verified by moving the specs file aside and rebuilding: binding still resolves, artifact still takes the payload loader. 2. probe_sysroot accepted gcc's reported sysroot as soon as it existed and carried headers, and only consulted remap_xlings_baked_sysroot when it did not exist. So the ownership predicate added for that function -- the whole point of which is a sysroot that EXISTS and belongs to someone else -- was never reached in the case it was written for. Measured on this machine: gcc reported a sysroot under an unrelated repo and every build took its headers. Ownership is now asked first; a foreign-but-usable sysroot remains the last resort, since taking nothing would break machines with no registry subos. 3. `rel.native().rfind("..", 0)` does not compile on Windows, where native() is a wstring -- every Windows job failed to build. It was also subtly wrong where it did compile: a directory named `..cache` is not an escape. Containment is a question about path components, so path_is_under asks it of components. Also removes a duplicated flag group: the C-runtime flags were emitted both as link_toolchain_flags and again as payload_ld for gcc. Correct but wasteful, and the link line has a hard 128KiB ceiling that real workspaces already spend 43% of. gcc with specs / gcc without specs / llvm: payload interpreter, exactly one --dynamic-linker, program runs. 65 unit tests, e2e 201/86/200/65/100/28/30. --- src/build/flags.cppm | 18 ++++--- src/fallback/probe_sysroot.cppm | 44 ++++++++++++---- src/toolchain/post_install.cppm | 75 +++++++++++++++++++++++++++ src/toolchain/probe.cppm | 35 +++++++++++-- tests/unit/test_sysroot_ownership.cpp | 18 +++++++ 5 files changed, 169 insertions(+), 21 deletions(-) diff --git a/src/build/flags.cppm b/src/build/flags.cppm index bb1d87f4..82589f58 100644 --- a/src/build/flags.cppm +++ b/src/build/flags.cppm @@ -712,14 +712,18 @@ CompileFlags compute_flags(const BuildPlan& plan) { // host's /lib or, on hosts without a system toolchain, passes bare names // that lld cannot open — issue #195), -L/-rpath for -lc/-lm, and the // payload's dynamic linker. + // + // Only for clang-with-cfg, and only in PayloadFirst. Every other + // combination already has these flags: the gcc branch above assigns + // `link_toolchain_flags = lm.link_flags(...)` for any mode but None, and + // the clang branch adds them for Sysroot. Emitting them here as well put + // the whole C-runtime group on the line twice -- harmless to correctness, + // but the link line has a hard 128KiB ceiling (MAX_ARG_STRLEN) that real + // workspaces already spend 43% of. std::string payload_ld; - if (lm.mode == mcpp::toolchain::CLibMode::PayloadFirst) { - // Emitted for gcc too now. It used to be gated on `isClangWithCfg`, - // which left gcc's run-side addressing to its install-time specs - // while the compile side moved per build. - if (isClangWithCfg || !lm.clangDriver) - payload_ld = lm.link_flags(ninjaEsc); - } + if (isClangWithCfg + && lm.mode == mcpp::toolchain::CLibMode::PayloadFirst) + payload_ld = lm.link_flags(ninjaEsc); // GCC: replace the payload's patched `*link:` with the pristine one, so // its accumulated rpath entries do not reach the artifact. Must come // BEFORE our own -Wl flags is not required (specs are processed by the diff --git a/src/fallback/probe_sysroot.cppm b/src/fallback/probe_sysroot.cppm index 65db370b..dc91adb9 100644 --- a/src/fallback/probe_sysroot.cppm +++ b/src/fallback/probe_sysroot.cppm @@ -17,6 +17,22 @@ import mcpp.log; export namespace mcpp::fallback { +// Does `child` sit under `anchor`? +// +// Spelled with path components rather than by inspecting the relative path's +// text. `native()` is a wstring on Windows, so the obvious `rfind("..", 0)` +// does not even compile there -- and where it does compile it is subtly wrong, +// since a directory genuinely named `..cache` starts with those two +// characters without escaping anything. +bool path_is_under(const std::filesystem::path& child, + const std::filesystem::path& anchor) { + if (anchor.empty() || child.empty()) return false; + auto rel = child.lexically_relative(anchor); + if (rel.empty()) return false; + static const std::filesystem::path kUp{".."}; + return *rel.begin() != kUp; +} + // When GCC reports a baked "subos/default" sysroot that does not belong to // THIS toolchain's home, remap it to the equivalent sysroot under the // compiler's own xpkgs tree. @@ -42,10 +58,8 @@ remap_xlings_baked_sysroot(std::string_view reportedPath, // Owned by this toolchain's registry? Then it is the right answer. auto registry = xpkgsOpt->parent_path().parent_path(); std::error_code ec; - auto rel = std::filesystem::path(std::string(reportedPath)) - .lexically_relative(registry); - const bool inside = !rel.empty() - && rel.native().rfind("..", 0) != 0; + const bool inside = path_is_under( + std::filesystem::path(std::string(reportedPath)), registry); if (inside && std::filesystem::exists(std::string(reportedPath), ec)) return std::nullopt; } @@ -60,6 +74,20 @@ remap_xlings_baked_sysroot(std::string_view reportedPath, return std::nullopt; } +// Does this sysroot belong to the same registry as the compiler that reported +// it? Callers need this BEFORE deciding whether a usable sysroot is +// acceptable: usability and ownership are independent, and a path can pass the +// first while failing the second. +bool sysroot_is_owned(std::string_view reportedPath, + const std::filesystem::path& compilerBin) { + if (reportedPath.empty()) return false; + auto xpkgs = mcpp::xlings::paths::xpkgs_from_compiler(compilerBin); + // No registry to compare against -- nothing to contradict, so accept. + if (!xpkgs) return true; + return path_is_under(std::filesystem::path(std::string(reportedPath)), + xpkgs->parent_path().parent_path()); +} + // Is this sysroot foreign -- neither this mcpp home's registry nor a tree // belonging to the project being built? // @@ -76,12 +104,8 @@ bool sysroot_is_foreign(const std::filesystem::path& sysroot, const std::filesystem::path& registryRoot, const std::filesystem::path& projectRoot) { if (sysroot.empty()) return false; - auto under = [&](const std::filesystem::path& anchor) { - if (anchor.empty()) return false; - auto rel = sysroot.lexically_relative(anchor); - return !rel.empty() && rel.native().rfind("..", 0) != 0; - }; - return !under(registryRoot) && !under(projectRoot); + return !path_is_under(sysroot, registryRoot) + && !path_is_under(sysroot, projectRoot); } // Parse a Clang .cfg file alongside the compiler binary for --sysroot=. diff --git a/src/toolchain/post_install.cppm b/src/toolchain/post_install.cppm index c6d7cdd5..a2d3a8f7 100644 --- a/src/toolchain/post_install.cppm +++ b/src/toolchain/post_install.cppm @@ -530,6 +530,66 @@ export void ensure_post_install_fixup(const mcpp::config::GlobalConfig& cfg, } +// PT_INTERP of an ELF, read directly. +// +// Not via `patchelf --print-interpreter`: this runs during prepare, where +// patchelf is not guaranteed to be resolved, and a dependency on an external +// tool for four fields of a header is a dependency that will be missing on +// exactly the machine that needs the answer. +std::string read_elf_interp(const std::filesystem::path& bin) { + std::ifstream is(bin, std::ios::binary); + if (!is) return {}; + unsigned char ident[16]{}; + is.read(reinterpret_cast(ident), sizeof ident); + if (!is || ident[0] != 0x7f || ident[1] != 'E' + || ident[2] != 'L' || ident[3] != 'F') return {}; + const bool is64 = ident[4] == 2; + const bool le = ident[5] == 1; + if (!is64 || !le) return {}; // the only shape payloads ship + + auto u16 = [&](std::streamoff off) -> std::uint16_t { + is.seekg(off); unsigned char b[2]{}; + is.read(reinterpret_cast(b), 2); + return static_cast(b[0] | (b[1] << 8)); + }; + auto u64 = [&](std::streamoff off) -> std::uint64_t { + is.seekg(off); unsigned char b[8]{}; + is.read(reinterpret_cast(b), 8); + std::uint64_t v = 0; + for (int i = 7; i >= 0; --i) v = (v << 8) | b[i]; + return v; + }; + + const auto phoff = u64(0x20); + const auto phentsize = u16(0x36); + const auto phnum = u16(0x38); + if (!is || phoff == 0 || phentsize < 0x38 || phnum == 0) return {}; + + constexpr std::uint32_t kPtInterp = 3; + for (std::uint16_t i = 0; i < phnum; ++i) { + const auto ph = static_cast(phoff) + + static_cast(i) * phentsize; + is.seekg(ph); + unsigned char t[4]{}; + is.read(reinterpret_cast(t), 4); + if (!is) return {}; + const std::uint32_t type = + static_cast(t[0]) | (t[1] << 8) + | (t[2] << 16) | (static_cast(t[3]) << 24); + if (type != kPtInterp) continue; + const auto offset = u64(ph + 0x08); + const auto filesz = u64(ph + 0x20); + if (filesz == 0 || filesz > 4096) return {}; + std::string str(static_cast(filesz), '\0'); + is.seekg(static_cast(offset)); + is.read(str.data(), static_cast(filesz)); + if (!is) return {}; + if (auto z = str.find('\0'); z != std::string::npos) str.resize(z); + return str; + } + return {}; +} + std::string baked_runtime_binding(const std::filesystem::path& compilerBin) { if (compilerBin.empty()) return {}; std::error_code ec; @@ -568,6 +628,21 @@ std::string baked_runtime_binding(const std::filesystem::path& compilerBin) { return r; } } + + // The compiler's OWN interpreter. + // + // The two sources above are files mcpp used to write, and mcpp no longer + // writes them -- so on a machine where the toolchain was installed after + // that change they simply are not there, and the compatibility path that + // depends on them answers nothing. That is not hypothetical: it is what + // turned CI red while every existing developer machine, which still has + // the files from before, stayed green. + // + // PT_INTERP is produced by the patchelf walk, which still runs on every + // install, and it names the glibc payload this toolchain was aligned to -- + // the same fact the specs held, from a mechanism that has not gone away. + if (auto r = from_text(read_elf_interp(compilerBin)); !r.empty()) + return r; return {}; } diff --git a/src/toolchain/probe.cppm b/src/toolchain/probe.cppm index 64a9f04d..35fb26b6 100644 --- a/src/toolchain/probe.cppm +++ b/src/toolchain/probe.cppm @@ -291,21 +291,48 @@ probe_sysroot(const std::filesystem::path& compilerBin, mcpp::platform::null_redirect)); if (r) { auto s = trim_line(*r); - if (!s.empty() && std::filesystem::exists(s)) { - if (usable(s)) return s; + + // A usable sysroot that belongs to somebody else is still somebody + // else's. gcc records this path as a string when it is built and + // reports it forever after; on a machine with several checkouts the + // recorded one routinely exists AND carries headers, so accepting it + // on usability alone hands the build another project's tree. Measured + // right here: this repo's gcc reported a sysroot under an unrelated + // one, and every build took its headers. + // + // The ownership test lives inside remap_xlings_baked_sysroot, which is + // exactly why the order matters -- the early return below reached it + // only when the path was missing, so the case the remap exists for was + // the one case it never saw. + const bool ownedByThisHome = + !s.empty() && mcpp::fallback::sysroot_is_owned(s, compilerBin); + + if (ownedByThisHome && usable(s)) return s; + if (!s.empty() && std::filesystem::exists(s) && !usable(s)) mcpp::log::debug("probe", std::format( "sysroot '{}' exists but lacks usr/include/stdlib.h — ignoring", s)); - } // GCC bakes the build-time sysroot into the binary. For xlings-built // GCC this is a path like /.xlings/subos/default that - // doesn't exist on the user's machine. Remap via fallback module. + // doesn't exist on the user's machine -- or exists and belongs to a + // different one. Remap via fallback module. if (auto remapped = mcpp::fallback::remap_xlings_baked_sysroot(s, compilerBin)) { if (usable(*remapped)) return *remapped; mcpp::log::debug("probe", std::format( "remapped sysroot '{}' lacks usr/include/stdlib.h — ignoring", remapped->string())); } + + // Last resort: a foreign but usable sysroot beats no sysroot. This is + // the pre-existing behaviour, kept for machines that have no registry + // subos to remap to -- it is a worse answer, not a wrong one, and + // taking nothing here would break them outright. + if (!s.empty() && std::filesystem::exists(s) && usable(s)) { + mcpp::log::verbose("probe", std::format( + "using sysroot '{}', which is outside this toolchain's " + "registry — no equivalent found under it", s)); + return std::filesystem::path(s); + } } // 2. macOS fallback: use xcrun to discover the SDK path. diff --git a/tests/unit/test_sysroot_ownership.cpp b/tests/unit/test_sysroot_ownership.cpp index bd3a26a4..56e1f6be 100644 --- a/tests/unit/test_sysroot_ownership.cpp +++ b/tests/unit/test_sysroot_ownership.cpp @@ -66,4 +66,22 @@ TEST(SysrootOwnership, SiblingPrefixIsNotContainment) { kRegistry, kProject)); } +// A directory whose NAME begins with two dots is not an escape. The first +// version compared the relative path's text (`rfind("..", 0)`), which called +// `/home/u/.mcpp/registry/..cache` an escape from the registry -- and did not +// compile at all on Windows, where `native()` is a wstring. Containment is a +// question about path components, so it is asked of components. +TEST(SysrootOwnership, DotDotPrefixedNameIsNotAnEscape) { + EXPECT_FALSE(fb::sysroot_is_foreign( + kRegistry / "..cache" / "subos" / "default", kRegistry, kProject)); +} + +TEST(SysrootOwnership, PathIsUnderIsDirectlyTestable) { + EXPECT_TRUE(fb::path_is_under(kRegistry / "a" / "b", kRegistry)); + EXPECT_TRUE(fb::path_is_under(kRegistry, kRegistry)); + EXPECT_FALSE(fb::path_is_under(kRegistry.parent_path(), kRegistry)); + EXPECT_FALSE(fb::path_is_under(kRegistry, {})); + EXPECT_FALSE(fb::path_is_under({}, kRegistry)); +} + } // namespace From 7e3fd6624e3070cb180da0b960431287dcee1210 Mon Sep 17 00:00:00 2001 From: speak-agent Date: Sat, 8 Aug 2026 09:42:33 +0800 Subject: [PATCH 11/21] docs: the design doc this branch cites, plus what implementing it corrected MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit The doc lived only on the branch of the closed PR #377, so every reference to it from code comments, docs, and the implementation plan pointed at a file the repository did not contain. New §9 records what landing it changed about the design itself. §9.1 is the one that matters: the compatibility fallback described in §3.5 reads gcc's specs and clang's cfg -- files the same change stops writing. On an existing machine they are still there; on a fresh install they are not, and nothing downstream says so. That asymmetry is why local verification was green while CI was not, and it generalises past this change: when removing a mechanism, enumerate who reads its output. --- ...yload-version-and-contract-drift-design.md | 315 ++++++++++++++++++ 1 file changed, 315 insertions(+) create mode 100644 .agents/docs/2026-08-08-payload-version-and-contract-drift-design.md diff --git a/.agents/docs/2026-08-08-payload-version-and-contract-drift-design.md b/.agents/docs/2026-08-08-payload-version-and-contract-drift-design.md new file mode 100644 index 00000000..f6a89c45 --- /dev/null +++ b/.agents/docs/2026-08-08-payload-version-and-contract-drift-design.md @@ -0,0 +1,315 @@ +# 载荷版本与契约漂移:四个缺陷,一条线 + +**日期**:2026-08-08 +**性质**:设计提案,待 review。本文不改代码。 +**触发**:2026.8.8.1 发布当天暴露的四个缺陷,其中三个是我在这一轮自己引入或自己漏掉的。 + +**关联**: +- `.agents/docs/2026-08-07-xlings-as-runtime-substrate-design.md`(本轮的上游设计) +- mcpp#352 / #375、mcpp-index#179(已 revert,PR#180)、mcpp#377 +- xlings `.agents/docs/2026-08-06-subos-architecture-proposal.md` 的 R1–R7 + +--- + +## 0. TL;DR + +四个缺陷,**同一条线**: + +> **mcpp 对 xlings 的每一项事实,都是「自己猜一个」而不是「读一个」;而它猜的那个,和它烙进产物的那个,是两个独立的答案。** + +| # | 缺陷 | 谁引入 | 现状 | +|---|---|---|---| +| **D1** | 载荷探测取**目录序第一个**(注释说是最高,代码没排序),与 gcc 烙定的不是同一份 | 早已存在,本轮触发 | **main 曾变红,已 revert** | +| **D2** | subos_info **wire format 读错**,功能完全不工作 | 本轮我引入 | PR#377 已修 | +| **D3** | 升级后**复用升级前缓存**,修复静默不生效 | 本轮我引入 | PR#377 已修 | +| **D4** | 沙箱 xlings **永不升级**,且 mcpp 打印偏差却不作为 | 早已存在 | **未修** | + +D1 和 D4 是同一件事的两面:**mcpp 把一个可变的外部世界当成了装机时的快照**。D2 和 D3 是另一件事的两面:**我用与实现同源的理解去写测试,于是测试证明的是「我和我自己一致」。** + +--- + +## 1. D1:载荷探测与 gcc 烙定的版本是两个独立答案 + +### 1.1 实测 + +`compat.glx-runtime` 加了一条 `xim:graphics` 依赖(为修 #352)。`mesa` 声明: + +```lua +"xim:glibc@>=2.38", -- 下界,不是钉死 +``` + +解析器装了 **glibc 2.44**,与既有的 2.39 并存。然后: + +``` +.../xim-x-glibc/2.39/lib64/libc.so.6: version `GLIBC_2.42' not found + (required by <测试二进制>) +``` + +**在 `asio-module`、`core` 上炸 —— 与图形毫无关系的包。** + +### 1.2 机制 + +`probe.cppm` 调 `find_sibling_tool(compilerBin, "glibc")`。那个函数的注释说它取最高版本 —— **代码没有排序**: + +```cpp +// Return the first (highest) version dir that exists. +for (auto& v : std::filesystem::directory_iterator(root, ec)) { + if (v.is_directory(ec)) return v.path(); +} +``` + +`directory_iterator` 是无序的,所以真实行为是**取 readdir 碰巧给出的第一个**。本机实测顺序是 `2.44, 2.39`,于是选中 2.44 —— 但**换台机器、装个别的包、甚至删一个目录,结果都可能变**。这是本文档记录的第三处「注释描述了代码没有的行为」(另两处见 §3.3)。 + +于是: + +| | 谁决定 | 结果 | +|---|---|---| +| **编译/链接** 对着哪份 glibc | mcpp 的探测(readdir 第一个) | **2.44**(本机;换台机可能不同) | +| **运行时** PT_INTERP / specs | gcc 载荷装机时 elfpatch 烙定 | **2.39** | + +两个独立的答案,**在只装一份 glibc 时恒相同**,所以从未被迫达成一致 —— 这正是 xlings 侧 R1/R2 诊断出的 P1「一个问题多个回答者」,原样出现在 mcpp。 + +### 1.2b 为什么「新 glibc 向后兼容」救不了这一格 + +glibc 的兼容性是**单向**的:对 2.39 构建的产物在 2.44 上跑 ✅;对 2.44 构建的在 2.39 上跑 ❌。本次报错正是第二种 —— 加载了 2.39 的 `libc.so.6`,而二进制要 `GLIBC_2.42` 的符号。**问题不是运行时太老,是编译期跑到前面去了而运行期没跟上。** + +### 1.2c 参照系:发行版、Nix、和 mcpp 现在在哪 + +| | 发行版(Arch / Fedora) | Nix / xlings | **mcpp 今天** | +|---|---|---|---| +| 同时几份 glibc | **1** | 多份并存 | 多份并存 | +| 路径 | **无版本** `/lib64/ld-linux-x86-64.so.2` | 带版本 `…/2.44/lib64/…` | 带版本 | +| 升级 | **就地替换同一个文件** | 在旁边装一份新的 | 在旁边装一份新的 | +| 编译期与运行期的一致性 | **构造上不可能分歧** | 每个消费者冻结自己的选择 | **两个独立决定者** ❌ | + +滚动发行版之所以能一直换 glibc 而不炸,是因为**编译侧和运行侧是同一个文件**;新 glibc 保留全部旧符号版本,所以既有二进制照跑。 + +**Arch 上唯一会出这个错的场景,与本次同型:部分升级**(`pacman -Sy pkg` 不带 `-u`)—— 装了一个对着更新 glibc 构建的包,而 glibc 没跟着升。Arch 明文禁止部分升级,理由一模一样。 + +**结论:mcpp 用了 Nix 的布局(多版本并存 + 版本化路径),但没有兑现那个布局的前提 —— 每个消费者的选择必须只有一个决定者。** 这不是「要不要支持多 glibc」的问题,而是既有布局的前提没被满足。 + +### 1.2d xlings 早就预见了,而一条下界绕开了它 + +`glibc.lua` 明写 `latest` 故意停在 2.39: + +> Moving `latest` to 2.44 would be safe for the same reason — but it would also **silently change which glibc every existing home resolves to**, and that is a decision to make deliberately rather than as a side effect of adding a version. + +而 `mesa` 的 `xim:glibc@>=2.38` **绕开了 `latest` 指针**,把 2.44 直接拉了进来。一个刻意做出的「不动 latest」的决定,被一条依赖的下界从侧面推翻。 + +**这一条值得同步给 xlings**:范围解析取最高满足者,与 `latest` 别名承载的「这个 home 该用哪一份」是两个不同的意图,而前者会静默压过后者。是否该让范围解析在 `latest` 满足区间时优先取 `latest`,是 xlings 侧的开放问题。 + +### 1.3 判据:它不是「装了新 glibc」的问题 + +**任何**让 home 里多出一份更高版本 glibc 的操作都会触发它 —— 一条新依赖、一次 `xlings install`、一个新包。**这个雷一直在,只是这次我踩了。** + +### 1.4 方案 + +| | A. 探测改为「与 gcc 烙定的那份一致」 | B. 探测取最高但校验一致性,不一致就报错 | C. 维持现状 | +|---|---|---|---| +| 编译/运行是否可能分裂 | **不可能** | 可能,但会被拦下 | 会,且静默 | +| 需要什么输入 | gcc 烙定的 glibc 版本 | 同左 | — | +| 多 glibc 场景 | 天然正确(跟工具链走) | 需要人工介入 | 坏 | +| 结果是否确定 | ✅ | ✅ | ❌ **随目录序变** | + +**推荐 A,B 作为过渡期的护栏。** + +「gcc 烙定的是哪份」不需要猜:`post_install.cppm` 已经有 `extract baked loader from clang cfg` 的能力(扫 `/ld-linux-`),而 gcc 侧的 specs 里也写着。**更好的做法是让 xlings 落盘**(见 §5),但即使不落盘,从 gcc 自己的产物反推也比「取最高」正确 —— 因为它反推的正是运行期会用的那一份。 + +**必须同时做的一件事**:`mcpp doctor` 增加一条 —— *home 里有多份 glibc 且编译期选中的不是 gcc 烙定的那份*。这是 D1 唯一的可观测入口;没有它,下一次触发同样只会表现为一个与 glibc 无关的包炸掉。 + +--- + +## 2. D4:沙箱 xlings 永不升级,而 mcpp 知道却不说 + +### 2.1 实测 + +``` +$ mcpp self env +xlings binary = /home/speak/.mcpp/registry/bin/xlings ← 2026.8.2.1 +xlings pinned = 2026.8.6.3 +``` + +**mcpp 把两个数字并排打印出来,不作任何评价。** + +`fallback/xlings_binary.cppm:22`: + +```cpp +if (std::filesystem::exists(destBin)) return destBin; // 无任何版本检查 +``` + +### 2.2 两种布局,只有一种会更新 + +| 布局 | xlings 从哪来 | 升级 mcpp 时 | +|---|---|---| +| **tarball 相对**(`MCPP_HOME=<解包目录>/registry`) | release 自带的 `registry/bin/xlings` | ✅ 随之更新 | +| **默认 home**(`MCPP_HOME=~/.mcpp`) | 首次 `self init` 时 acquire 的那份 | ❌ **永不更新** | + +CI 走第一种(所以 CI 一直是新的),开发机与普通用户走第二种。**「我记得已经修过」和「实测是旧的」都对,只是说的不是同一种布局。** + +### 2.3 后果:整条修复链断在这里 + +`#352` 需要四环:mcpp 读 `subos_info` → index 拉 `xim:graphics` → **graphics 把声明写进 subos** → subos 有 `subos_info`。第三环需要沙箱 xlings ≥ 2026.8.5.1;`graphics.lua` 显式探测 `type(subos.env) ~= "function"`,**老客户端上声明被静默丢弃**。 + +实测本机沙箱 xlings 2026.8.2.1 的二进制里,`subos_info` 这个字符串出现 **0 次**。 + +### 2.4 方案 + +| | A. 版本低于 pin 就自动替换 | B. doctor 报告 + 一条显式命令 | C. 每次启动检查 | +|---|---|---|---| +| 无感 | ✅ | ❌ 用户要动手 | ✅ | +| 风险 | 换掉用户正在用的 xlings | 低 | 每次 exec 多一次版本读 | +| 与「mcpp 不接管 subos 状态」的边界 | ⚠️ 擦边 | 一致 | ⚠️ | + +**推荐 A + B**:acquire 时比对版本,低于 pin 则替换(这是 mcpp 自己 vendored 的那份,不是用户的系统 xlings,替换它不越界);同时 doctor 把偏差列为一条 finding。 + +**A 的判据必须是「低于 pin」而不是「不等于 pin」** —— 用户手动放了一份更新的进去,不该被降级。 + +> **⚠️ 这条要小心**:替换沙箱 xlings 会改变依赖解析的行为。参考 `index-floor-must-degrade` 的教训——**发布数据不得让程序失效**。所以替换后必须能回退,且 doctor 要能说出「现在用的是哪一份、为什么」。 + +--- + +## 3. D2/D3:测试证明的是「我和我自己一致」 + +### 3.1 D2 —— wire format 读错,功能完全不工作 + +盘上真实的 `envs` 是**以 binding 为键的对象**;我的 reader 期望**数组**。`is_array()` 为假 ⇒ 循环一次不执行 ⇒ 对着真实 subos **一个变量都不应用**。 + +**10 个单测 + 1 条 e2e 一路全绿**,因为每一份 fixture 都是我按同一个臆想手写的。 + +### 3.2 根因不是笔误,是方法 + +> **用与实现同源的理解去造 fixture,抓不出对 wire format 的误解。只有取自写入方的 fixture 能抓。** + +已加 `RealXlingsCapture`(真实输出逐字副本),parser 改为从 xlings 的 reader **誊写**而非建模 —— 这顺带抓到第二处分歧:xlings 丢弃 `op` 不是 `set`/`prepend` 的声明,我原来全收。 + +### 3.3 D3 —— 自证断言,以及注释撒的谎 + +我原本的 e2e 只跑一次 `mcpp run`,PASS —— 而它存在的目的是覆盖**缓存快路径**,那次根本没走。加上「这次必须走快路径,否则报错」之后,立刻暴露快路径丢环境变量,**同一条断言接着又抓到第二个**(缓存行读写顺序不匹配)。 + +更难看的是:reader 的注释**声称**它把旧缓存当 miss —— 那个检查从来没写过。**注释描述了代码没有的行为,比漏掉更糟:它让下一个读代码的人不去查。** + +### 3.4 提议:三条可执行规则 + +- **R-A 跨仓 wire format 的 fixture 必须取自写入方**,不得手写。形式:golden 文件 + 注明来源与采集时间。 +- **R-B 一条测试如果存在的目的是覆盖某条路径,必须先断言「这次确实走了那条路径」。** 否则它可能一直 PASS 而零覆盖。 +- **R-C 注释若描述一条行为,该行为必须有对应断言。** 「reader 把旧缓存当 miss」这句话本身就该是一个测试名。 + +--- + +## 4. 四个缺陷的公共上游 + +``` +D1 载荷版本 ── 猜(取最高) ─┐ +D4 xlings ── 快照(装机时那份) ─┼─→ 把可变的外部世界当成不变的 +D2 wire fmt ── 猜(臆想格式) ─┤ +D3 缓存契约 ── 猜(注释当实现) ─┘ +``` + +**D1/D4 是「外部世界会变,而我记的是装机那一刻」;D2/D3 是「外部契约有确定形状,而我记的是我以为的形状」。** + +两者的解药是同一句话:**凡是别人拥有的事实,读它,并且用它自己的产物来验证你读对了。** + +--- + +## 5. 与上游设计的关系:这轮验证了什么、推翻了什么 + +`2026-08-07-xlings-as-runtime-substrate-design.md` 的核心论点是「mcpp 把 xlings 当目录约定而不是可查询的运行时」。**这一轮四个缺陷全部是那个论点的实例**,而且其中两个是我一边写着那份设计一边犯的 —— 这本身是最强的证据。 + +它也**修正**了那份设计的一处轻描淡写:§7 把「xlings 落盘 exports」排在 P1,理由是「A 降级路径足以工作」。**D1 证明那不成立** —— 没有权威落盘,mcpp 只能靠目录扫描猜一个,而猜出来的那个与 gcc 烙定的那个是两个独立答案。**`exports` 落盘应升到 P0**,且内容要包含「gcc 这份载荷烙定的 glibc 是哪一份」。 + +--- + +## 6. 分期 + +**P0(main 已红过,先止血)** +1. ✅ revert mcpp-index#179(PR#180) +2. ✅ PR#377:wire format + 旧缓存(D2/D3) +3. **D1 护栏**:`mcpp doctor` 报告「多份 glibc 且编译期选中的不是 gcc 烙定的那份」 + +**P1(让 #352 真正到达用户)** +4. **D4**:acquire 时版本比对 + 替换,doctor 报告偏差 +5. **D1 正解**:载荷探测改为跟随 gcc 烙定的版本 +6. 之后才重新落 mcpp-index 的图形栈迁移 + +**P2** +7. xlings 侧落盘 `exports`(含 gcc↔glibc 的绑定),mcpp 改读它 +8. R-A/R-B/R-C 写进贡献规范 + +--- + +## 7. 明确不做 + +- **不在 recipe 里回避 D1**(比如把 mesa 的 glibc 下界改成钉死)。那是把引擎缺陷摊给包作者,而且下一个用下界的包会再犯 —— 与 `aarch64-native-gcc-payload-is-musl` 那次「归因成包缺 arch 轴是错的」同型。 +- **不让 mcpp 接管 subos 生命周期**。D4 的方案只替换 mcpp 自己 vendored 的那份二进制,不碰用户的系统 xlings、不管理 subos。 +- **不追 glibc 版本**。D1 的正解是「跟随工具链烙定的那一份」,不是「总用最新」。 + +--- + +## 8. 开放问题 + +- **Q1** D1 的「gcc 烙定的是哪份 glibc」:从 gcc 产物反推(specs / PT_INTERP),还是等 xlings 落盘?前者能立刻做,后者更正确 —— 是否两者都要,反推作为降级? +- **Q2** D4 的替换时机:`self init` 时?每次 acquire 时?还是只在 doctor --fix?自动替换与「用户手动放了一份新的」如何共存(判据「低于 pin 才换」是否足够)? +- **Q3** 多 glibc 是 xlings 明确要支持的方向(`subos new --runtime glibc@2.44`)。D1 修好之后,mcpp 对「一个 home 里多份 glibc」的正式立场是什么 —— 跟随工具链,还是跟随 subos 的 `runtime`?后者更符合上游设计的 S1,但需要 D4 先修。 +- **Q4** 本轮 revert 掉的图形栈迁移,重新落地时如何验证「装这个栈不会影响 home 的其余部分」?**这一条是这次事故真正缺失的那个测试。** 一个可执行形状:装栈前后各跑一次与图形无关的成员(`asio-module`、`core`),断言产物的 `PT_INTERP` 与 `readelf -V` 的 glibc 符号版本上界不变。 +- **Q5**(xlings 侧,§1.2d)范围解析取最高满足者,压过了 `latest` 别名承载的「这个 home 该用哪一份」。是否该让 `latest` 在满足区间时优先?这决定了「加一个新版本进索引」是不是一个安全操作。 + +--- + +## 9. 实施回写(2026-08-08,PR #378) + +方案落地过程中,有三条是设计当时没看见的,其中第一条**修正了本方案自己的一个假设**。 + +### 9.1 兼容回落读的是「我刚删掉的机制的产物」 + +§3.5 写的兼容路径是:subos 不自述时,读工具链自身烙入的值 —— gcc 的 `specs`、 +clang 的 `.cfg`。这在**已有机器**上成立,因为那些文件是**旧 fixup 写出来的**,还留在 +盘上。 + +在**全新安装**的机器上不成立:mcpp 已经不写它们了,所以它们不存在。链条是 +`无 binding → 无 payloadPaths → 无 --dynamic-linker → 产物拿宿主 loader`,hermetic +检查如实报出 `/lib64/ld-linux-x86-64.so.2 (outside the sandbox)`。 + +**每台开发机都还留着那些文件,所以本机验证永远是绿的,CI 才红。** 这条应当写进判据: + +> 删掉一个机制时,先列出谁在读它的产物。兼容回落尤其危险 —— 它读的往往正是旧机制 +> 留下的东西。 + +补的第三个来源是**编译器自身的 `PT_INTERP`**:由 patchelf 走查写入(该机制仍在运行), +指向的正是这套工具链对齐的那个 glibc payload,与 specs 曾经持有的是同一个事实,但 +**不依赖 mcpp 写过任何东西**。自己读 ELF 头而不用 `patchelf --print-interpreter`—— +本函数在 prepare 期运行,那里不保证 patchelf 已解析。 + +验证方式是把 `specs` 文件真的挪开再跑,而不是读代码。 + +### 9.2 归属谓词加在了一个到不了的函数里 + +§3.5 的 D5 把 `remap_xlings_baked_sysroot` 的判据从「存在」改为「归属」。但调用方 +`probe_sysroot` 的**第一步**是「存在且可用 ⇒ 直接 return」,只有在路径**不存在**时才 +往下走到 remap —— 而该函数存在的全部意义,正是处理「存在、可用、却属于别人」。 + +本机实测:这个仓库的 gcc 报的 sysroot 指向 `xim-pkgindex-fromsource` 下的 +`.xlings/subos/default`,每次构建都在吃另一个仓库的头文件。修法是把归属问在可用性 +**之前**;「外来但可用」保留为最后兜底,因为没有 registry subos 可映射的机器上,取 +「什么都不要」会直接坏掉。 + +### 9.3 生成的 spec 放在了缓存能删的目录里 + +`--no-cache` 会 `remove_all(outputDir)`,而 spec 是 prepare 期写进那个目录的 ⇒ **每次 +冷构建 + gcc 必挂**。修法不是调顺序,而是把「ninja 跑之前该文件必须存在」当作**不变量 +在消费点持有** —— 这样也扛得住用户手动 `rm -rf target`。 + +值得记的是它**怎么被发现的**:e2e 201 是专为这套机制写的测试,但它只跑暖构建,而暖 +构建下文件从上一次留着。**是全量本地 e2e 扫出来的,不是定向子集。** + +### 9.4 链接行去重 + +C-runtime 的那组 flags 同时经 `link_toolchain_flags` 与 `payload_ld` 发出,整组在链接行 +上出现两次。对正确性无害,但链接行有 128KiB 硬上限(`MAX_ARG_STRLEN`),真实 +workspace 已经用掉 43%。`payload_ld` 现在只服务 clang-with-cfg 的 PayloadFirst,其余组合 +本就已经有了。 + +### 9.5 一条跨平台低级错误 + +`rel.native().rfind("..", 0)` 在 Windows 上**编译不过**(`native()` 是 `wstring`),所有 +Windows job 直接挂;而且它在能编译的地方语义也错 —— 名叫 `..cache` 的目录并不是逃逸。 +包含关系是**路径分量**的问题,`path_is_under` 按分量问。 From 9840ca902c9b7d528795534f9c1bd277a9263b78 Mon Sep 17 00:00:00 2001 From: speak-agent Date: Sat, 8 Aug 2026 09:49:55 +0800 Subject: [PATCH 12/21] fix(toolchain): one payload is an answer; two are a question CI still had no runtime binding, and therefore no loader on the link line: -o bin/mcpp --sysroot=<...>/subos/default -B<...>/binutils/bin -static-libstdc++ Every compatibility source is something an earlier mechanism wrote -- gcc's specs, clang's cfg, and now the compiler's PT_INTERP -- and a machine can legitimately have none of them. Recorded loader paths can also name a subos VIEW rather than the payload; those carry no version at all, so they are canonicalised first (R6: artifacts bind the payload, never the view). The last resort is the installed payload set, and only when it is a singleton. This is not the rule the design removed: that one asked a directory for "the glibc" and took whatever readdir yielded first -- a choice, made by something unrelated to what the artifact loads, silently wrong as soon as a dependency's `>=` floor installed a second payload. Where exactly one glibc exists there is no choice to make; it is the only runtime any artifact from this toolchain could bind. Two or more and this stays silent, which is precisely the case the incident was and the case the subos has to settle. Verified under conditions this machine had to be forced into: gcc's specs file moved aside AND gcc's own PT_INTERP repointed at the host loader. The binding still resolves and the artifact still takes the payload loader. tests/unit/test_runtime_binding_fallback.cpp +5 --- src/toolchain/post_install.cppm | 41 ++++++++++++ tests/unit/test_runtime_binding_fallback.cpp | 69 ++++++++++++++++++++ 2 files changed, 110 insertions(+) create mode 100644 tests/unit/test_runtime_binding_fallback.cpp diff --git a/src/toolchain/post_install.cppm b/src/toolchain/post_install.cppm index a2d3a8f7..e996b9a0 100644 --- a/src/toolchain/post_install.cppm +++ b/src/toolchain/post_install.cppm @@ -597,6 +597,14 @@ std::string baked_runtime_binding(const std::filesystem::path& compilerBin) { auto from_text = [](const std::string& text) -> std::string { auto loader = detect_baked_loader(text); if (loader.empty()) return {}; + // The recorded path may name a subos VIEW (`/subos/default/lib/ + // ld-linux-...`), which is a symlink into the payload. R6: artifacts + // bind the payload, never the mutable view -- so resolve to the + // payload before reading a version off it. A view path has no version + // component at all, and parsing one yields nothing. + std::error_code cec; + if (auto real = std::filesystem::canonical(loader, cec); !cec) + loader = real.string(); // .../xim-x-glibc//lib64/ld-linux-... — the version is the // grandparent of the lib dir. auto dir = std::filesystem::path(loader).parent_path(); // lib64 @@ -643,6 +651,39 @@ std::string baked_runtime_binding(const std::filesystem::path& compilerBin) { // the same fact the specs held, from a mechanism that has not gone away. if (auto r = from_text(read_elf_interp(compilerBin)); !r.empty()) return r; + + // Last: the installed payload set, but ONLY when it is a singleton. + // + // This is not the rule this design removed. That rule asked a directory + // for "the glibc" and took whatever `readdir` yielded first -- a CHOICE, + // made by something with no bearing on what the artifact would load, and + // silently wrong the moment a second payload appeared. Where exactly one + // payload is installed there is no choice to make: it is the only glibc + // any artifact from this toolchain could bind, and declining would refuse + // a question that has a single answer. + // + // Two or more and this stays silent. That is the case the incident was, + // and it is the case the subos must answer. + if (auto xpkgs = mcpp::xlings::paths::xpkgs_from_compiler(compilerBin)) { + std::vector versions; + for (auto it = std::filesystem::directory_iterator(*xpkgs / "xim-x-glibc", ec); + !ec && it != std::filesystem::directory_iterator{}; it.increment(ec)) { + if (!it->is_directory(ec)) continue; + auto v = it->path().filename().string(); + if (!v.empty() && v.front() != '.') versions.push_back(v); + } + if (versions.size() == 1) { + mcpp::log::verbose("probe", std::format( + "runtime binding glibc@{} — the only glibc payload installed, " + "so there is nothing to choose between", versions[0])); + return "glibc@" + versions[0]; + } + if (versions.size() > 1) + mcpp::log::verbose("probe", std::format( + "{} glibc payloads installed and nothing declares which one " + "this build binds; declining rather than picking", + versions.size())); + } return {}; } diff --git a/tests/unit/test_runtime_binding_fallback.cpp b/tests/unit/test_runtime_binding_fallback.cpp new file mode 100644 index 00000000..c232b111 --- /dev/null +++ b/tests/unit/test_runtime_binding_fallback.cpp @@ -0,0 +1,69 @@ +// When nothing declares the runtime, how far may mcpp go on its own? +// +// The rule this replaced asked a directory for "the glibc" and took the first +// entry readdir yielded. That is a CHOICE made by something with no bearing on +// what the artifact loads, and it went wrong the moment a dependency's +// `xim:glibc@>=2.38` floor installed a second payload: the compile side took +// 2.44 while the interpreter still named 2.39. +// +// So the axis is not "may mcpp look at the payload directory" -- it is +// "is there anything to choose between". One payload is an answer; two are a +// question only the subos can settle. +// +// This mattered because the compatibility sources (gcc's specs, clang's cfg, +// the compiler's PT_INTERP) are all things some earlier mechanism wrote, and a +// machine can legitimately have none of them. + +#include + +import std; +import mcpp.toolchain.post_install; + +namespace tc = mcpp::toolchain; + +namespace { + +struct Home { + std::filesystem::path root, compiler; + explicit Home(std::initializer_list glibcVersions) { + root = std::filesystem::temp_directory_path() + / std::format("mcpp_binding_{}", std::random_device{}()); + auto xpkgs = root / "data" / "xpkgs"; + compiler = xpkgs / "xim-x-gcc" / "16.1.0" / "bin" / "g++"; + std::filesystem::create_directories(compiler.parent_path()); + std::ofstream(compiler) << "not an elf"; + for (auto v : glibcVersions) + std::filesystem::create_directories( + xpkgs / "xim-x-glibc" / v / "lib64"); + } + ~Home() { std::error_code ec; std::filesystem::remove_all(root, ec); } +}; + +TEST(RuntimeBindingFallback, OnePayloadIsAnAnswer) { + Home h{"2.39"}; + EXPECT_EQ(tc::baked_runtime_binding(h.compiler), "glibc@2.39"); +} + +// The incident, in one assertion. Two payloads and nothing declaring which: +// silence, not a coin flip. +TEST(RuntimeBindingFallback, TwoPayloadsIsSilence) { + Home h{"2.39", "2.44"}; + EXPECT_EQ(tc::baked_runtime_binding(h.compiler), ""); +} + +TEST(RuntimeBindingFallback, NoPayloadIsSilence) { + Home h{}; + EXPECT_EQ(tc::baked_runtime_binding(h.compiler), ""); +} + +// A compiler outside any xpkgs tree (a system gcc) has no payload set to read, +// and must not acquire one from somewhere else. +TEST(RuntimeBindingFallback, NonPayloadCompilerGetsNothing) { + EXPECT_EQ(tc::baked_runtime_binding("/usr/bin/g++"), ""); +} + +TEST(RuntimeBindingFallback, EmptyPathIsSilence) { + EXPECT_EQ(tc::baked_runtime_binding({}), ""); +} + +} // namespace From 7b3b540eaa00887d52a4835cabf3fe12145ccb7c Mon Sep 17 00:00:00 2001 From: speak-agent Date: Sat, 8 Aug 2026 09:50:33 +0800 Subject: [PATCH 13/21] docs: record where 'refuse without an authority' was too tight MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit §9.6. The design's rule -- no authority, no payload-first -- assumed a machine always has at least one of the compatibility sources. All three are artifacts some earlier mechanism wrote, and a machine can legitimately have none, at which point refusing hands the artifact the HOST loader: worse than guessing a version, because it leaves the sandbox entirely. The axis is whether there is anything to choose between, not whether mcpp may look. One payload is an answer; two are the incident. --- ...yload-version-and-contract-drift-design.md | 24 +++++++++++++++++++ 1 file changed, 24 insertions(+) diff --git a/.agents/docs/2026-08-08-payload-version-and-contract-drift-design.md b/.agents/docs/2026-08-08-payload-version-and-contract-drift-design.md index f6a89c45..3ed69d8b 100644 --- a/.agents/docs/2026-08-08-payload-version-and-contract-drift-design.md +++ b/.agents/docs/2026-08-08-payload-version-and-contract-drift-design.md @@ -313,3 +313,27 @@ workspace 已经用掉 43%。`payload_ld` 现在只服务 clang-with-cfg 的 Pay `rel.native().rfind("..", 0)` 在 Windows 上**编译不过**(`native()` 是 `wstring`),所有 Windows job 直接挂;而且它在能编译的地方语义也错 —— 名叫 `..cache` 的目录并不是逃逸。 包含关系是**路径分量**的问题,`path_is_under` 按分量问。 + +### 9.6 「拒绝」的边界:一个 payload 是答案,两个才是问题 + +§3.2 把判据写成「没有权威就拒绝走 payload-first」。落地后发现这条**收得过紧**。 + +所有兼容来源(gcc 的 `specs`、clang 的 `.cfg`、编译器自身的 `PT_INTERP`)都是**某个更早 +的机制写出来的东西**,而一台机器可以合法地一个都没有。这时按 §3.2 的字面判据就该拒绝, +产物于是拿宿主 loader —— 比「猜错版本」更糟,因为它连沙箱都出去了。 + +真正的轴不是「mcpp 能不能去看 payload 目录」,而是**有没有得选**: + +- **恰好一个 glibc payload** ⇒ 没有选择可言。它是这套工具链产出的任何产物唯一可能绑定的 + 运行时,拒绝等于拒答一个只有一个答案的问题。 +- **两个或以上** ⇒ 沉默。这正是事故本身的形状,必须由 subos 来定。 + +这与被移除的旧规则的区别是决定性的:旧规则在**有得选**的时候按 `readdir` 顺序选,而且 +一直到装进第二个 payload 之前都看起来是对的。 + +另外,记录下来的 loader 路径可能指向 subos **视图**(`/subos/default/lib/ld-linux-…`) +而不是 payload —— 视图路径里根本没有版本段。解析前先 canonical 化(R6:产物绑 payload, +不绑可变视图)。 + +验证方式:把 gcc 的 `specs` 挪开**并且**把 gcc 自身的 `PT_INTERP` 改指宿主 loader,即三条 +兼容来源全部失效,binding 仍解析得出,产物仍拿 payload loader。 From a6348e0bdeea6742116e740f9c1621ccd3b7803f Mon Sep 17 00:00:00 2001 From: speak-agent Date: Sat, 8 Aug 2026 10:02:01 +0800 Subject: [PATCH 14/21] fix(toolchain): a record naming a payload that is gone is not an authority CI's log gave the answer directly: probe: runtime 'glibc@2.39' is not installed in this home The machine had one glibc payload, 2.44, and a compatibility record naming 2.39. specs, cfg and PT_INTERP are all RECORDS of a past state -- written at some install, some fixup -- and nothing revisits them when the payload underneath is replaced. The probe matches exactly and never falls back, quite deliberately, so it refused; no payload paths resolved; no --dynamic-linker reached the link line; the artifact took the host loader and the hermetic check rejected it. Each record is now checked against the payloads actually installed, and a record naming an absent one is skipped rather than trusted. Where two payloads exist and a record names one of them, that still wins -- it is the value the artifact would load. Where nothing present is named, silence. Two more, both from the full local e2e rather than the targeted subset: - musl targets were getting the glibc payload's lib dir and loader. The sysroot branch pushed them unconditionally, but a musl sysroot is self-contained and the glibc payload there belongs to the HOST toolchain, merely probed alongside. ld then pulled glibc's static libc.a into a musl link: `undefined reference to _DYNAMIC` out of dl-reloc-static-pie.o (e2e 103). The musl guard the header branch already had now covers libraries and the loader too. - The new "vendored xlings is behind the pin" note went to stdout, and `mcpp test --json` promises every stdout line is NDJSON (e2e 155). It is a remark about the environment, not output of the command; stderr. tests/unit/test_runtime_binding_fallback.cpp +3 (8 total) 66 unit tests; e2e 103/155/184/201/86/28/30 pass. --- src/fallback/xlings_binary.cppm | 12 +++- src/toolchain/linkmodel.cppm | 14 ++++- src/toolchain/post_install.cppm | 63 +++++++++++++------- tests/unit/test_runtime_binding_fallback.cpp | 49 +++++++++++++++ 4 files changed, 112 insertions(+), 26 deletions(-) diff --git a/src/fallback/xlings_binary.cppm b/src/fallback/xlings_binary.cppm index ff21c07f..43f4ce56 100644 --- a/src/fallback/xlings_binary.cppm +++ b/src/fallback/xlings_binary.cppm @@ -6,6 +6,7 @@ // 3. Fail with user-facing instructions module; +#include #include #include @@ -68,15 +69,22 @@ acquire_xlings_binary(const std::filesystem::path& destBin, bool quiet = false, // restore. Look before leaping. auto candidate = candidate_source_version(); if (candidate.empty() || !version_is_older(have, candidate)) { + // stderr, not stdout. This is a remark about the environment, + // not output of the command that happens to be running -- and + // `mcpp test --json` promises every stdout line is NDJSON, a + // promise this line broke the moment a machine fell behind the + // pin (e2e 155). if (!quiet) - std::println("{:>12} vendored xlings {} is older than the " + std::println(stderr, + "{:>12} vendored xlings {} is older than the " "pinned {}, but no newer source is available " "(keeping it; run `xlings self update`)", "Note", have, pinnedVersion); return destBin; } if (!quiet) - std::println("{:>12} vendored xlings {} -> {} (pinned {})", + std::println(stderr, + "{:>12} vendored xlings {} -> {} (pinned {})", "Updating", have, candidate, pinnedVersion); std::error_code rec; std::filesystem::remove(destBin, rec); diff --git a/src/toolchain/linkmodel.cppm b/src/toolchain/linkmodel.cppm index fde7442f..f5891447 100644 --- a/src/toolchain/linkmodel.cppm +++ b/src/toolchain/linkmodel.cppm @@ -344,11 +344,21 @@ ToolchainLinkModel resolve_link_model(const Toolchain& tc) { // point — the baked copy was a per-toolchain-install decision that no // longer matched the per-build one, and it accumulated one dead entry // per home that ever touched the shared payload. - if (tc.payloadPaths && !tc.payloadPaths->glibcLib.empty()) { + // + // Never for a musl target. Its sysroot is self-contained -- it brings + // its own libc, CRT and loader -- and the glibc payload here belongs + // to the HOST toolchain, which merely happens to be probed alongside. + // Putting it on the link path let ld pull glibc's static libc.a into a + // musl link: `undefined reference to _DYNAMIC`, `hidden symbol + // _DYNAMIC isn't defined`, from dl-reloc-static-pie.o. The musl branch + // below already says this about headers; it holds for libraries and + // the loader too. + if (!is_musl_target(tc) + && tc.payloadPaths && !tc.payloadPaths->glibcLib.empty()) { lm.loader = resolve_loader(tc.payloadPaths->glibcLib, tc.targetTriple); lm.libDirs.push_back(tc.payloadPaths->glibcLib); } - if (!tc.binaryPath.empty()) { + if (!is_musl_target(tc) && !tc.binaryPath.empty()) { std::error_code lec; auto gccLib = tc.binaryPath.parent_path().parent_path() / "lib64"; if (std::filesystem::exists(gccLib, lec)) diff --git a/src/toolchain/post_install.cppm b/src/toolchain/post_install.cppm index e996b9a0..3230273d 100644 --- a/src/toolchain/post_install.cppm +++ b/src/toolchain/post_install.cppm @@ -614,6 +614,34 @@ std::string baked_runtime_binding(const std::filesystem::path& compilerBin) { return "glibc@" + ver; }; + // Which glibc payloads are actually installed. Every source below is + // checked against this, because all of them are RECORDS of a past state: + // specs and cfg were written at some install, PT_INTERP at some fixup. A + // payload can be replaced afterwards -- upgraded, garbage-collected -- and + // the record keeps naming what used to be there. + // + // That is not hypothetical. CI resolved `glibc@2.39` from a record while + // the only payload on disk was 2.44, so the exact-match probe correctly + // refused, no payload paths resolved, and the artifact fell through to the + // host loader. A fossil that names something no longer present is not an + // authority; it is just old. + std::vector installed; + if (auto xpkgs = mcpp::xlings::paths::xpkgs_from_compiler(compilerBin)) { + for (auto it = std::filesystem::directory_iterator(*xpkgs / "xim-x-glibc", ec); + !ec && it != std::filesystem::directory_iterator{}; it.increment(ec)) { + if (!it->is_directory(ec)) continue; + auto v = it->path().filename().string(); + if (!v.empty() && v.front() != '.') installed.push_back(v); + } + } + auto still_there = [&](const std::string& binding) { + if (binding.empty()) return false; + const auto at = binding.find('@'); + if (at == std::string::npos) return false; + auto ver = binding.substr(at + 1); + return std::ranges::find(installed, ver) != installed.end(); + }; + auto read_file = [&](const std::filesystem::path& p) -> std::string { if (!std::filesystem::exists(p, ec)) return {}; std::ifstream is(p); @@ -624,7 +652,7 @@ std::string baked_runtime_binding(const std::filesystem::path& compilerBin) { // clang: the sibling .cfg. auto cfg = compilerBin.parent_path() / (compilerBin.stem().string() + ".cfg"); - if (auto r = from_text(read_file(cfg)); !r.empty()) return r; + if (auto r = from_text(read_file(cfg)); still_there(r)) return r; // gcc: lib/gcc///specs, one level of globbing each. auto gccRoot = compilerBin.parent_path().parent_path(); @@ -632,7 +660,7 @@ std::string baked_runtime_binding(const std::filesystem::path& compilerBin) { !ec && t != std::filesystem::directory_iterator{}; t.increment(ec)) { for (auto v = std::filesystem::directory_iterator(t->path(), ec); !ec && v != std::filesystem::directory_iterator{}; v.increment(ec)) { - if (auto r = from_text(read_file(v->path() / "specs")); !r.empty()) + if (auto r = from_text(read_file(v->path() / "specs")); still_there(r)) return r; } } @@ -649,7 +677,7 @@ std::string baked_runtime_binding(const std::filesystem::path& compilerBin) { // PT_INTERP is produced by the patchelf walk, which still runs on every // install, and it names the glibc payload this toolchain was aligned to -- // the same fact the specs held, from a mechanism that has not gone away. - if (auto r = from_text(read_elf_interp(compilerBin)); !r.empty()) + if (auto r = from_text(read_elf_interp(compilerBin)); still_there(r)) return r; // Last: the installed payload set, but ONLY when it is a singleton. @@ -664,26 +692,17 @@ std::string baked_runtime_binding(const std::filesystem::path& compilerBin) { // // Two or more and this stays silent. That is the case the incident was, // and it is the case the subos must answer. - if (auto xpkgs = mcpp::xlings::paths::xpkgs_from_compiler(compilerBin)) { - std::vector versions; - for (auto it = std::filesystem::directory_iterator(*xpkgs / "xim-x-glibc", ec); - !ec && it != std::filesystem::directory_iterator{}; it.increment(ec)) { - if (!it->is_directory(ec)) continue; - auto v = it->path().filename().string(); - if (!v.empty() && v.front() != '.') versions.push_back(v); - } - if (versions.size() == 1) { - mcpp::log::verbose("probe", std::format( - "runtime binding glibc@{} — the only glibc payload installed, " - "so there is nothing to choose between", versions[0])); - return "glibc@" + versions[0]; - } - if (versions.size() > 1) - mcpp::log::verbose("probe", std::format( - "{} glibc payloads installed and nothing declares which one " - "this build binds; declining rather than picking", - versions.size())); + if (installed.size() == 1) { + mcpp::log::verbose("probe", std::format( + "runtime binding glibc@{} — the only glibc payload installed, so " + "there is nothing to choose between", installed[0])); + return "glibc@" + installed[0]; } + if (installed.size() > 1) + mcpp::log::verbose("probe", std::format( + "{} glibc payloads installed and nothing still present declares " + "which one this build binds; declining rather than picking", + installed.size())); return {}; } diff --git a/tests/unit/test_runtime_binding_fallback.cpp b/tests/unit/test_runtime_binding_fallback.cpp index c232b111..b32147c5 100644 --- a/tests/unit/test_runtime_binding_fallback.cpp +++ b/tests/unit/test_runtime_binding_fallback.cpp @@ -66,4 +66,53 @@ TEST(RuntimeBindingFallback, EmptyPathIsSilence) { EXPECT_EQ(tc::baked_runtime_binding({}), ""); } +// A record that names a payload which is no longer installed. +// +// specs, cfg and PT_INTERP are all RECORDS of a past state: written at some +// install or some fixup, and never revisited when the payload underneath is +// replaced. CI hit exactly this -- a record said glibc@2.39 while the only +// payload on disk was 2.44 -- and because the probe matches exactly and never +// falls back, nothing resolved, no loader reached the link line, and the +// artifact took the host's. A fossil naming something absent is not an +// authority; it is just old, and the resolution must carry on past it. +struct HomeWithSpecs { + std::filesystem::path root, compiler; + HomeWithSpecs(std::initializer_list glibcVersions, + std::string_view specsNames) { + root = std::filesystem::temp_directory_path() + / std::format("mcpp_fossil_{}", std::random_device{}()); + auto xpkgs = root / "data" / "xpkgs"; + auto gccRoot = xpkgs / "xim-x-gcc" / "16.1.0"; + compiler = gccRoot / "bin" / "g++"; + std::filesystem::create_directories(compiler.parent_path()); + std::ofstream(compiler) << "not an elf"; + for (auto v : glibcVersions) + std::filesystem::create_directories(xpkgs / "xim-x-glibc" / v / "lib64"); + auto sd = gccRoot / "lib" / "gcc" / "x86_64-linux-gnu" / "16.1.0"; + std::filesystem::create_directories(sd); + std::ofstream(sd / "specs") + << "*link:\n%{!static:--dynamic-linker " << xpkgs.string() + << "/xim-x-glibc/" << specsNames << "/lib64/ld-linux-x86-64.so.2}\n"; + } + ~HomeWithSpecs() { std::error_code ec; std::filesystem::remove_all(root, ec); } +}; + +TEST(RuntimeBindingFallback, RecordNamingAnAbsentPayloadIsSkipped) { + HomeWithSpecs h{{"2.44"}, "2.39"}; + // Not "glibc@2.39": that payload is gone. The singleton rule answers. + EXPECT_EQ(tc::baked_runtime_binding(h.compiler), "glibc@2.44"); +} + +TEST(RuntimeBindingFallback, RecordNamingAPresentPayloadWins) { + HomeWithSpecs h{{"2.39", "2.44"}, "2.39"}; + // Two installed, so the singleton rule cannot answer -- but the record + // names one that IS there, and that is what the artifact would load. + EXPECT_EQ(tc::baked_runtime_binding(h.compiler), "glibc@2.39"); +} + +TEST(RuntimeBindingFallback, StaleRecordWithNothingToFallBackOnIsSilence) { + HomeWithSpecs h{{"2.39", "2.44"}, "2.28"}; + EXPECT_EQ(tc::baked_runtime_binding(h.compiler), ""); +} + } // namespace From d99d1e3fc525fc31881f09ffc8ac5c8631276c28 Mon Sep 17 00:00:00 2001 From: speak-agent Date: Sat, 8 Aug 2026 10:02:46 +0800 Subject: [PATCH 15/21] docs: a record is only an authority while what it names still exists MIME-Version: 1.0 Content-Type: text/plain; charset=UTF-8 Content-Transfer-Encoding: 8bit §9.7-9.8. Two more corrections to the design's refusal rule, both from CI. The compatibility sources are records of a past state, and nothing revisits them when the payload underneath is replaced -- so 'exact match, never fall back' correctly refused a record naming a glibc that had been upgraded away, and the artifact took the host loader. §9.6 said 'no choice is not a guess'; §9.7 says 'a fossil is not an authority'. Both narrow where refusing applies: it belongs to genuine ambiguity, nothing else. §9.8 records the musl guard the sysroot branch was missing. --- ...yload-version-and-contract-drift-design.md | 40 +++++++++++++++++++ 1 file changed, 40 insertions(+) diff --git a/.agents/docs/2026-08-08-payload-version-and-contract-drift-design.md b/.agents/docs/2026-08-08-payload-version-and-contract-drift-design.md index 3ed69d8b..0ebe4dcd 100644 --- a/.agents/docs/2026-08-08-payload-version-and-contract-drift-design.md +++ b/.agents/docs/2026-08-08-payload-version-and-contract-drift-design.md @@ -337,3 +337,43 @@ Windows job 直接挂;而且它在能编译的地方语义也错 —— 名叫 ` 验证方式:把 gcc 的 `specs` 挪开**并且**把 gcc 自身的 `PT_INTERP` 改指宿主 loader,即三条 兼容来源全部失效,binding 仍解析得出,产物仍拿 payload loader。 + +### 9.7 兼容来源是**记录**,不是权威 + +§9.6 补上单例回落后 CI 依然红,日志直说了原因: + +``` +probe: runtime 'glibc@2.39' is not installed in this home +``` + +那台机器上只有一个 glibc payload:**2.44**;而兼容记录烙的是 **2.39**。 + +三条兼容来源(`specs`、`.cfg`、`PT_INTERP`)都是**过去某个时刻的记录** —— 装机时写的、 +fixup 时写的 —— 而 payload 被替换(升级、回收)时**没有任何东西回头去改它们**。§3.2 的 +「精确匹配、绝不回落」于是如实地拒绝了 2.39,结果是没有 payloadPaths、没有 +`--dynamic-linker`、产物拿宿主 loader。 + +**判据要补一句:一条记录只有在它指的东西还在,才算数。** + +现在每条记录都对「实际安装了哪些 payload」核对一遍: + +- 记录指的 payload **还在** ⇒ 采信(即使装了多个,它就是产物会加载的那个) +- 记录指的 payload **不在了** ⇒ 跳过,继续往下(单例回落往往能答) +- 什么都答不上来 ⇒ 沉默 + +这条与 §9.6 是同一件事的两面:**§9.6 说「没得选就不算猜」,§9.7 说「化石不算权威」。** +两条都是在收窄「拒绝」的适用范围 —— 拒绝该留给**真正有歧义**的情况。 + +### 9.8 musl 目标不碰 glibc payload + +`sysroot_mode` 里补 loader / lib 目录时漏了 musl 护栏。musl 的 sysroot 是自包含的,而 +同时被探测到的 glibc payload 属于**宿主**工具链 —— 把它放进链接路径后,ld 会把 glibc 的 +静态 `libc.a` 拉进一次 musl 链接: + +``` +undefined reference to `_DYNAMIC' (dl-reloc-static-pie.o) +hidden symbol `_DYNAMIC' isn't defined +``` + +同一函数里紧接着的 header 分支**本来就有** `is_musl_target` 判断 —— 那句话对库和 loader +一样成立,只是当时没写上。由 e2e 103 抓到,同样是全量跑扫出来的。 From 0987b1658f8285cca69797af324d84c26e13b3f5 Mon Sep 17 00:00:00 2001 From: speak-agent Date: Sat, 8 Aug 2026 10:10:06 +0800 Subject: [PATCH 16/21] fix(build): a subos declaring a payload that is gone cannot be honoured The stale value was not a fossil after all -- it was the subos's own `subos_info`, saying `glibc@2.39` on a machine whose only payload is 2.44. The subos description is written once; the payloads beneath it are replaced independently. Same staleness, a layer up, and the previous commit only guarded the layer below. Two things kept it alive: 1. The declaration was returned unchecked. Honouring it is not the conservative choice -- the exact-match probe finds nothing, no loader reaches the link line, and the artifact lands on the HOST loader, outside the sandbox entirely. Worse than any substitution. 2. The second resolution pass was gated on `runtimeBinding.empty()`. A stale declaration is not empty, so the pass that has the compiler in hand -- the only one that can check whether the payload exists -- never ran. The gate is now `!tc->payloadPaths`, which is what both failures (undiscoverable binding, uninstallable binding) actually look like. Reproduced locally by injecting `runtime: glibc@2.99` into this machine's subos: the declaration is passed over, resolution carries on to the 2.39 that is installed, and the artifact takes the payload loader and runs. 66 unit tests; e2e 201/86/200/103/155. --- src/build/prepare.cppm | 42 +++++++++++++++++++++++++++++++++++------- 1 file changed, 35 insertions(+), 7 deletions(-) diff --git a/src/build/prepare.cppm b/src/build/prepare.cppm index 442a8e39..0d61b018 100644 --- a/src/build/prepare.cppm +++ b/src/build/prepare.cppm @@ -929,14 +929,36 @@ std::string resolve_runtime_binding(const mcpp::manifest::Manifest& m, auto runtime_of = [](const std::filesystem::path& dir) -> std::string { return mcpp::xlings::subos::read(dir).runtime; }; + // A declaration can only be honoured while the payload it names is still + // installed. Subos descriptions are written once and the payloads beneath + // them are replaced independently -- upgraded, garbage-collected -- so a + // subos can go on saying `glibc@2.39` on a machine that now has only 2.44. + // + // Taking that at face value is not conservative, it is worse than any + // substitution: the exact-match probe finds nothing, no loader reaches the + // link line, and the artifact ends up on the HOST loader -- outside the + // sandbox entirely. Measured on CI, three rounds running. + // + // So a declaration naming an absent payload is passed over rather than + // returned, and resolution carries on to something that can be honoured. + auto usable = [&](const std::string& binding) { + if (binding.empty()) return false; + if (compilerBin.empty()) return true; // cannot check yet; caller re-asks + if (mcpp::toolchain::probe_payload_paths(compilerBin, binding)) + return true; + mcpp::log::verbose("probe", std::format( + "subos declares runtime {}, but that payload is not installed here " + "— looking for one that is", binding)); + return false; + }; if (auto active = mcpp::xlings::paths::subos_dir_of(compilerBin)) { // 1 — the project names a subos; it is a sibling of the active one. if (!m.xlings.subos.empty()) { auto named = active->parent_path() / m.xlings.subos; - if (auto r = runtime_of(named); !r.empty()) return r; + if (auto r = runtime_of(named); usable(r)) return r; } // 2 — whatever is active. - if (auto r = runtime_of(*active); !r.empty()) return r; + if (auto r = runtime_of(*active); usable(r)) return r; } // 3 — COMPATIBILITY: the value this toolchain already has baked in. @@ -1705,12 +1727,18 @@ prepare_build(bool print_fingerprint, auto runtimeBinding = resolve_runtime_binding(*m, {}); auto tc = mcpp::toolchain::detect(explicit_compiler, runtimeBinding); if (!tc) return std::unexpected(tc.error().message); - // The binding may only be discoverable from the compiler's own home - // (an inherited toolchain resolves into its owner). Re-resolve once we - // know where the compiler is, then re-probe if that changed the answer. - if (runtimeBinding.empty()) { + // The first pass has no compiler, so it can neither discover a binding + // that only the compiler's own home knows (an inherited toolchain resolves + // into its owner) nor check that a declared one is still installed. Both + // failures look identical from here: no payload paths. Re-ask with the + // compiler in hand whenever that is what happened. + // + // The old condition was `runtimeBinding.empty()`, which meant a subos + // naming a payload that had since been upgraded away won on the first pass + // and the second pass never ran. + if (!tc->payloadPaths) { auto fromCompiler = resolve_runtime_binding(*m, tc->binaryPath); - if (!fromCompiler.empty()) { + if (!fromCompiler.empty() && fromCompiler != runtimeBinding) { tc = mcpp::toolchain::detect(explicit_compiler, fromCompiler); if (!tc) return std::unexpected(tc.error().message); } From af8d4fd29de210912627e7c1b6c34ef32789137f Mon Sep 17 00:00:00 2001 From: speak-agent Date: Sat, 8 Aug 2026 10:23:02 +0800 Subject: [PATCH 17/21] fix(build): -specs= is one argv word, and its path lives under the project `cannot read spec file '/tmp/tmp.XXX/my'` for a project in `my project` (e2e 179, on CI). escape_path adds ninja's `$` escapes and nothing else, but ninja hands the command to `sh -c`, so a space still splits the word. Quoted the way include_token quotes: join the prefix, then escape and quote the whole token -- quoting only the path would put the opening quote in the wrong place and re-split exactly what the join was for. Every other payload flag survives a spaced path by luck rather than design: they name directories under MCPP_HOME, and this is the only one that lives under the PROJECT. (A spaced MCPP_HOME would break `--sysroot=` and `-B` the same way, uniformly, and did before this branch.) Locally 179 passes either way -- this machine resolves llvm for it, which emits no specs at all. Verified by building gcc explicitly in a directory with a space: artifact takes the payload loader and runs, and the ninja token comes out `'-specs=/tmp/.../my$ project/...'`. Also gates RuntimeBindingFallback.RecordNamingAPresentPayloadWins to POSIX. detect_baked_loader parses GCC specs, whose loader paths are absolute and POSIX, and its character whitelist deliberately excludes `\` and `:` so a `%{...}` spec body is never swallowed -- so a `C:\Users\...` fixture cannot be spelled at all. The other cases exercise the payload set, not the parser, and keep running everywhere. --- src/build/flags.cppm | 14 +++++++++++++- tests/unit/test_runtime_binding_fallback.cpp | 14 ++++++++++++++ 2 files changed, 27 insertions(+), 1 deletion(-) diff --git a/src/build/flags.cppm b/src/build/flags.cppm index 82589f58..40fa3fbe 100644 --- a/src/build/flags.cppm +++ b/src/build/flags.cppm @@ -729,8 +729,20 @@ CompileFlags compute_flags(const BuildPlan& plan) { // BEFORE our own -Wl flags is not required (specs are processed by the // driver, not positionally against -Wl), but keeping it adjacent to the // payload flags keeps the C-runtime decisions in one place on the line. + // + // Quoted the way include_token quotes: `-specs=` and the path are ONE argv + // word, so the prefix is joined first and the whole token quoted after -- + // quoting only the path would put the opening quote in the wrong place. + // `escape_path` alone is not enough here: it adds ninja's `$` escapes and + // nothing else, while ninja hands the command to `sh -c`. This path is the + // only payload flag that lives under the PROJECT rather than the payload, + // so it is the only one a spaced project directory splits: + // `cannot read spec file '/tmp/tmp.XXX/my'` for a project in `my project` + // (e2e 179, on CI -- no local path here has a space). if (!plan.gccCleanSpecs.empty()) - payload_ld = " -specs=" + escape_path(plan.gccCleanSpecs) + payload_ld; + payload_ld = " " + shell_quote_arg(escape_ninja_chars( + "-specs=" + plan.gccCleanSpecs.string())) + + payload_ld; std::string link_extra; if (prof.lto) link_extra += " -flto"; diff --git a/tests/unit/test_runtime_binding_fallback.cpp b/tests/unit/test_runtime_binding_fallback.cpp index b32147c5..b61e0d95 100644 --- a/tests/unit/test_runtime_binding_fallback.cpp +++ b/tests/unit/test_runtime_binding_fallback.cpp @@ -103,7 +103,21 @@ TEST(RuntimeBindingFallback, RecordNamingAnAbsentPayloadIsSkipped) { EXPECT_EQ(tc::baked_runtime_binding(h.compiler), "glibc@2.44"); } +// Linux only, and not as an exemption -- the mechanism does not exist +// elsewhere. detect_baked_loader parses GCC specs, whose loader paths are +// POSIX and absolute, and its character whitelist deliberately excludes `\` +// and `:` so that a `%{...}` spec body is never swallowed. A Windows temp +// directory is `C:\Users\...`, so this fixture cannot be spelled there at +// all: the scan stops at the first backslash and the result does not start +// with `/`. Rightly so -- there are no glibc payloads or ld-linux loaders on +// Windows for it to find. +// +// The other cases in this file are platform-neutral (they exercise the +// payload set, not the parser) and keep running everywhere. TEST(RuntimeBindingFallback, RecordNamingAPresentPayloadWins) { + if constexpr (!std::filesystem::path::preferred_separator || + std::filesystem::path::preferred_separator != '/') + GTEST_SKIP() << "GCC specs loader paths are POSIX; no such record here"; HomeWithSpecs h{{"2.39", "2.44"}, "2.39"}; // Two installed, so the singleton rule cannot answer -- but the record // names one that IS there, and that is what the artifact would load. From 938b7d8bb8c18995d1c0fde1848a412665db3406 Mon Sep 17 00:00:00 2001 From: speak-agent Date: Sat, 8 Aug 2026 10:36:57 +0800 Subject: [PATCH 18/21] test(e2e): say why the first-run binary did not run `echo "first-run binary did not run"; exit 1` reports the one thing already known and none of the things that would explain it. An ELF fails to start for a short, enumerable set of reasons -- a missing interpreter, an unresolvable RUNPATH entry, a symbol the runtime does not have -- and which one it is IS the answer. 29 fails on CI and passes on every local run here, and after three rounds of reading code I still cannot say why, because the assertion discards the process's own account of itself. It now prints the exit status and output, the PT_INTERP and whether it exists, each RUNPATH entry and whether it exists, and ldd. No behaviour change; the test passes locally exactly as before. --- tests/e2e/29_toolchain_partial_versions.sh | 23 +++++++++++++++++++++- 1 file changed, 22 insertions(+), 1 deletion(-) diff --git a/tests/e2e/29_toolchain_partial_versions.sh b/tests/e2e/29_toolchain_partial_versions.sh index cf87ab1e..43823b77 100755 --- a/tests/e2e/29_toolchain_partial_versions.sh +++ b/tests/e2e/29_toolchain_partial_versions.sh @@ -89,7 +89,28 @@ file "$binary" | grep -q 'statically linked' && { echo "first-run default unexpectedly produced a static binary (glibc default expected)" exit 1 } -"$binary" >/dev/null 2>&1 || { echo "first-run binary did not run"; exit 1; } +# Say WHY when it does not. "did not run" alone is unactionable: an ELF fails +# to start for exactly a few reasons -- a missing interpreter, an unresolvable +# RUNPATH entry, a symbol the runtime does not have -- and which one it is is +# the whole answer. This assertion sat here reporting none of them while CI +# failed on it and every local run passed. +if ! out=$("$binary" 2>&1); then + rc=$? + echo "first-run binary did not run (exit $rc): $out" + interp=$(file "$binary" | sed -n 's/.*interpreter \([^,]*\).*/\1/p') + echo " PT_INTERP: ${interp:-}" + [[ -n "$interp" ]] && { [[ -e "$interp" ]] && echo " exists" \ + || echo " MISSING"; } + rp=$(readelf -d "$binary" 2>/dev/null \ + | sed -n 's/.*R\(UN\)\?PATH.*\[\(.*\)\]/\2/p') + echo " RUNPATH: ${rp:-}" + while IFS= read -r d; do + [[ -z "$d" ]] && continue + [[ -d "$d" ]] && echo " ok $d" || echo " MISSING $d" + done <<< "$(echo "$rp" | tr ':' '\n')" + ldd "$binary" 2>&1 | sed 's/^/ ldd: /' | head -12 + exit 1 +fi # Second build should be silent on toolchain — no re-install banner. "$MCPP" build > "$TMP/secondrun.log" 2>&1 || { From b87a438d59fbdfaf44d6aa0e01733392149ae8f0 Mon Sep 17 00:00:00 2001 From: speak-agent Date: Sat, 8 Aug 2026 10:52:13 +0800 Subject: [PATCH 19/21] fix(toolchain): PayloadFirst artifacts had nowhere to find libgcc_s The diagnostic added in the previous commit answered it on the first run: error while loading shared libraries: libgcc_s.so.1 RUNPATH: /data/xpkgs/xim-x-glibc/2.44/lib64 <- one entry ldd: libgcc_s.so.1 => /lib/x86_64-linux-gnu/libgcc_s.so.1 The compiler's own runtime lives beside the compiler, not in the C library, and a produced binary links it whether or not the build mentions it. gcc's patched specs used to put that directory on the artifact; removing the specs rewrite has to put it back, and only the Sysroot path did. One helper now serves both modes. Invisible where it was written: on any machine with a host toolchain the artifact resolves libgcc_s.so.1 from /lib and runs. It failed only in a throw-away home on CI. The guard is a unit test, not an e2e. Which link mode an e2e reaches depends on whether the machine happens to have a usable sysroot -- this one does, so a PayloadFirst leg written for exactly this defect quietly ran Sysroot twice and passed with the bug in place. Verified by removing the fix: the unit test goes red, the e2e leg does not. The link model takes its mode from its input, so a test can name the mode instead of hoping for it. e2e 201 also now asserts that no artifact loads a library from /lib or /usr/lib at all. "Does it run" was the wrong question -- it is answerable yes on every developer machine while the artifact is quietly reaching outside the sandbox. tests/unit/test_link_model_runtime_dirs.cpp +4 (67 unit tests total) --- mcpp.toml | 2 +- src/toolchain/linkmodel.cppm | 31 ++++- tests/e2e/201_gcc_no_specs_pollution.sh | 83 ++++++++++++- tests/unit/test_link_model_runtime_dirs.cpp | 122 ++++++++++++++++++++ 4 files changed, 230 insertions(+), 8 deletions(-) create mode 100644 tests/unit/test_link_model_runtime_dirs.cpp diff --git a/mcpp.toml b/mcpp.toml index 60309c3d..6e44600d 100644 --- a/mcpp.toml +++ b/mcpp.toml @@ -17,7 +17,7 @@ default-profile = "release" include_dirs = ["src/libs/json"] [toolchain] -default = "gcc@16.1.0" +default = "llvm@22.1.8" macos = "llvm@22.1.8" windows = "llvm@20.1.7" diff --git a/src/toolchain/linkmodel.cppm b/src/toolchain/linkmodel.cppm index f5891447..1bf69f8e 100644 --- a/src/toolchain/linkmodel.cppm +++ b/src/toolchain/linkmodel.cppm @@ -320,11 +320,35 @@ ToolchainLinkModel resolve_link_model(const Toolchain& tc) { // cross-compile resolves by what it builds FOR. if (is_msvc_target(tc) || is_mingw_target(tc)) return lm; + // The compiler's OWN runtime lives beside it, not in the C library: + // libgcc_s.so.1 for GCC. A produced binary links it whether or not the + // build ever mentions it, so its directory has to be on the artifact's + // RUNPATH -- and gcc's patched specs used to put it there, which is + // exactly why removing that rewrite has to put it back. + // + // Both modes need this, and only one got it at first. PayloadFirst + // artifacts came out with a single RUNPATH entry (the glibc payload) and + // resolved libgcc_s.so.1 from the HOST -- or, on a machine without one, + // not at all: `error while loading shared libraries: libgcc_s.so.1`. This + // developer machine has a usable sysroot, so every local build took the + // other branch and the gap only surfaced on CI (e2e 29). + // + // Not for musl (self-contained sysroot, static world) and not for clang, + // which brings compiler-rt and libunwind instead. + auto add_compiler_runtime_dir = [&] { + if (is_musl_target(tc) || lm.clangDriver || tc.binaryPath.empty()) return; + std::error_code lec; + auto gccLib = tc.binaryPath.parent_path().parent_path() / "lib64"; + if (std::filesystem::exists(gccLib, lec)) + lm.libDirs.push_back(gccLib); + }; + auto payload_first = [&] { auto& pp = *tc.payloadPaths; lm.mode = CLibMode::PayloadFirst; lm.crtDir = pp.glibcLib; lm.libDirs.push_back(pp.glibcLib); + add_compiler_runtime_dir(); lm.systemIncludes.push_back(pp.glibcInclude); if (!pp.linuxInclude.empty()) lm.systemIncludes.push_back(pp.linuxInclude); @@ -358,12 +382,7 @@ ToolchainLinkModel resolve_link_model(const Toolchain& tc) { lm.loader = resolve_loader(tc.payloadPaths->glibcLib, tc.targetTriple); lm.libDirs.push_back(tc.payloadPaths->glibcLib); } - if (!is_musl_target(tc) && !tc.binaryPath.empty()) { - std::error_code lec; - auto gccLib = tc.binaryPath.parent_path().parent_path() / "lib64"; - if (std::filesystem::exists(gccLib, lec)) - lm.libDirs.push_back(gccLib); - } + add_compiler_runtime_dir(); // Supplement kernel headers when the sysroot lacks them (glibc's // local_lim.h needs ). Self-contained musl sysroots // ship their own; a cross target must not see host-arch headers. diff --git a/tests/e2e/201_gcc_no_specs_pollution.sh b/tests/e2e/201_gcc_no_specs_pollution.sh index d17cf818..22a3d394 100755 --- a/tests/e2e/201_gcc_no_specs_pollution.sh +++ b/tests/e2e/201_gcc_no_specs_pollution.sh @@ -20,6 +20,7 @@ # never something a build asked for. set -euo pipefail +SELF_DIR=$(cd "$(dirname "$0")" && pwd) # captured before any cd TMP=$(mktemp -d) trap 'rm -rf "$TMP"' EXIT export MCPP_HOME=$HOME/.mcpp @@ -80,4 +81,84 @@ echo "$file_out" | grep -q 'interpreter .*xim-x-glibc' || { out=$("$bin" 2>&1) || { echo "binary did not run: $out"; exit 1; } echo "$out" | grep -q 'Hello' || { echo "unexpected output: $out"; exit 1; } -echo "PASS: gcc artifact carries only the RUNPATH this build asked for" +# 4. Nothing loads from the host. +# +# "Does it run" is not this question, and answering the wrong one is how a +# real gap survived: with the compiler's own lib dir missing from RUNPATH, +# artifacts resolved libgcc_s.so.1 from /lib and ran perfectly on any machine +# that has a host toolchain -- which is every developer machine. It failed +# only on CI's throw-away home, and only as `error while loading shared +# libraries`, with no indication of what had gone missing or why. +# +# Asking where each library actually came from makes the same defect visible +# everywhere, host toolchain or not. +if command -v ldd > /dev/null 2>&1; then + host_libs=$(ldd "$bin" 2>/dev/null \ + | grep -E '=> +(/lib|/usr/lib|/lib64|/usr/lib64)/' || true) + if [[ -n "$host_libs" ]]; then + echo "the artifact loads libraries from the host:" + echo "$host_libs" | sed 's/^/ /' + echo "Every shared library a payload build needs is in the payload;" + echo "reaching outside it means a directory is missing from RUNPATH." + exit 1 + fi +fi + +# ── The same assertions again, in the OTHER link mode ──────────────────── +# +# RUNPATH is built by two separate paths -- CLibMode::Sysroot and +# CLibMode::PayloadFirst -- and a machine only ever takes one of them. This +# one has a usable sysroot, so everything above ran through Sysroot mode and +# PayloadFirst went untested here for as long as this file has existed. +# +# It was not academic: PayloadFirst omitted the compiler's own lib dir, so +# artifacts resolved libgcc_s.so.1 from the host. Invisible on any machine +# with a host toolchain; on CI's throw-away home it was `error while loading +# shared libraries` with no further explanation. +# +# A home with payloads but no subos usually has no sysroot to find, which is +# what selects PayloadFirst -- but "usually" is the honest word: where gcc +# reports a baked sysroot that happens to exist (another checkout on the same +# machine), mcpp accepts it as a last resort and this leg runs Sysroot mode a +# second time instead. Which is what it does on the machine these words were +# written on. +# +# So the deterministic guard for this invariant is the unit test +# (tests/unit/test_link_model_runtime_dirs.cpp), which names the mode rather +# than arranging for it. This leg is still worth running: it covers a second +# home shape end to end, and on CI -- where the baked path does not exist -- +# it is the real thing. +export MCPP_HOME="$TMP/payload-first-home" +MCPP_INHERIT_CONFIG=0 MCPP_INHERIT_SUBOS=0 \ + source "$SELF_DIR/_inherit_toolchain.sh" + +cd "$TMP" +"$MCPP" new pf > /dev/null +cd pf +cat >> mcpp.toml <<'EOF' + +[toolchain] +linux = "gcc@16.1.0" +EOF + +"$MCPP" build > "$TMP/pf.log" 2>&1 || { + cat "$TMP/pf.log"; echo "payload-first build failed"; exit 1; } + +pfbin=$(find target -type f -name pf -path '*/bin/*' | head -1) +[[ -n "$pfbin" ]] || { echo "no payload-first binary produced"; exit 1; } + +pf_host=$(ldd "$pfbin" 2>/dev/null \ + | grep -E '=> +(/lib|/usr/lib|/lib64|/usr/lib64)/' || true) +if [[ -n "$pf_host" ]]; then + echo "the payload-first artifact loads libraries from the host:" + echo "$pf_host" | sed 's/^/ /' + echo "RUNPATH was:" + "$readelf" -d "$pfbin" 2>/dev/null | grep -iE 'RUNPATH|RPATH' | sed 's/^/ /' + exit 1 +fi + +"$pfbin" > /dev/null 2>&1 || { + echo "payload-first binary did not run: $("$pfbin" 2>&1 | head -2)"; exit 1; } + +echo "PASS: gcc artifacts carry only the RUNPATH the build asked for and load" +echo " nothing from the host — in both sysroot and payload-first modes" diff --git a/tests/unit/test_link_model_runtime_dirs.cpp b/tests/unit/test_link_model_runtime_dirs.cpp new file mode 100644 index 00000000..c38b46b8 --- /dev/null +++ b/tests/unit/test_link_model_runtime_dirs.cpp @@ -0,0 +1,122 @@ +// An artifact must find every library it needs inside the sandbox — in BOTH +// link modes. +// +// RUNPATH is assembled by two separate paths, CLibMode::Sysroot and +// CLibMode::PayloadFirst, and any given machine only ever takes one of them. +// The compiler's own runtime (libgcc_s.so.1) lives beside the compiler, not in +// the C library, and a produced binary links it whether or not the build ever +// mentions it. gcc's patched specs used to supply that directory; removing the +// specs rewrite therefore has to supply it explicitly — and at first only the +// Sysroot path did. +// +// The consequence was invisible where it was written: on a machine with a host +// toolchain the artifact simply resolved libgcc_s.so.1 from /lib and ran. It +// failed only in a throw-away home on CI, as `error while loading shared +// libraries` with nothing said about what was missing. +// +// Asserted here rather than in e2e on purpose. Which mode an e2e reaches +// depends on whether the machine happens to have a usable sysroot — this one +// does, so an e2e leg written for PayloadFirst quietly ran Sysroot twice and +// passed with the defect in place. The link model takes its mode from its +// input, so a test can name the mode instead of hoping for it. + +#include + +import std; +import mcpp.toolchain.linkmodel; +import mcpp.toolchain.model; + +namespace tc = mcpp::toolchain; + +namespace { + +void touch(const std::filesystem::path& p) { + std::filesystem::create_directories(p.parent_path()); + std::ofstream(p) << "x"; +} + +// A payload tree with a glibc beside a gcc, as xlings lays them out. +struct Payload { + std::filesystem::path root, compiler, glibcLib, gccLib, sysroot; + Payload() { + root = std::filesystem::temp_directory_path() + / std::format("mcpp_linkmodel_{}", std::random_device{}()); + auto xpkgs = root / "data" / "xpkgs"; + compiler = xpkgs / "xim-x-gcc" / "16.1.0" / "bin" / "g++"; + touch(compiler); + gccLib = xpkgs / "xim-x-gcc" / "16.1.0" / "lib64"; + touch(gccLib / "libgcc_s.so.1"); + glibcLib = xpkgs / "xim-x-glibc" / "2.39" / "lib64"; + touch(glibcLib / "libc.so.6"); + touch(glibcLib / "ld-linux-x86-64.so.2"); + touch(xpkgs / "xim-x-glibc" / "2.39" / "include" / "features.h"); + sysroot = root / "subos" / "default"; + touch(sysroot / "usr" / "include" / "stdlib.h"); + } + ~Payload() { std::error_code ec; std::filesystem::remove_all(root, ec); } + + tc::Toolchain toolchain(bool withSysroot) const { + tc::Toolchain t; + t.compiler = tc::CompilerId::GCC; + t.version = "16.1.0"; + t.binaryPath = compiler; + t.targetTriple = "x86_64-linux-gnu"; + t.runtimeBinding = "glibc@2.39"; + tc::PayloadPaths pp; + pp.glibcLib = glibcLib; + pp.glibcInclude = glibcLib.parent_path() / "include"; + t.payloadPaths = pp; + if (withSysroot) t.sysroot = sysroot; + return t; + } +}; + +std::string joined(const tc::ToolchainLinkModel& lm) { + auto id = [](const std::filesystem::path& p) { return p.string(); }; + std::string s; + for (auto& tok : lm.link_tokens(id)) { s += tok; s += ' '; } + return s; +} + +TEST(LinkModelRuntimeDirs, PayloadFirstCarriesTheCompilerRuntime) { + Payload p; + auto lm = tc::resolve_link_model(p.toolchain(/*withSysroot=*/false)); + ASSERT_EQ(lm.mode, tc::CLibMode::PayloadFirst); + auto line = joined(lm); + EXPECT_NE(line.find("-Wl,-rpath," + p.gccLib.string()), std::string::npos) + << "libgcc_s.so.1 lives here and nothing else will find it:\n" << line; +} + +TEST(LinkModelRuntimeDirs, SysrootCarriesTheCompilerRuntime) { + Payload p; + auto lm = tc::resolve_link_model(p.toolchain(/*withSysroot=*/true)); + ASSERT_EQ(lm.mode, tc::CLibMode::Sysroot); + auto line = joined(lm); + EXPECT_NE(line.find("-Wl,-rpath," + p.gccLib.string()), std::string::npos) + << line; +} + +// The C library's own directory, in both modes, for the same reason. +TEST(LinkModelRuntimeDirs, BothModesCarryTheCLibrary) { + Payload p; + for (bool withSysroot : {false, true}) { + auto line = joined(tc::resolve_link_model(p.toolchain(withSysroot))); + EXPECT_NE(line.find("-Wl,-rpath," + p.glibcLib.string()), + std::string::npos) + << "withSysroot=" << withSysroot << "\n" << line; + } +} + +// And the interpreter. A sysroot says where headers live; it says nothing +// about which loader runs the result, and gcc's `*link:` no longer answers. +TEST(LinkModelRuntimeDirs, BothModesNameTheInterpreter) { + Payload p; + for (bool withSysroot : {false, true}) { + auto line = joined(tc::resolve_link_model(p.toolchain(withSysroot))); + EXPECT_NE(line.find("--dynamic-linker=" + p.glibcLib.string()), + std::string::npos) + << "withSysroot=" << withSysroot << "\n" << line; + } +} + +} // namespace From ec34195f1e0bdf10f0774e884c154b83e6087285 Mon Sep 17 00:00:00 2001 From: speak-agent Date: Sat, 8 Aug 2026 11:07:23 +0800 Subject: [PATCH 20/21] fix: restore the project's toolchain default, and stop assuming it is installed mcpp.toml's `[toolchain] default` went from gcc@16.1.0 to llvm@22.1.8 in the previous commit. Not deliberate -- a local experiment wrote it and `git add -A` carried it along. The whole repository then built with llvm, and five CI jobs failed on assertions that name gcc: toolchain: gcc (cold self-host) grep -q "Resolved gcc@16.1.0" toolchain: musl + llvm sed expects `default = "gcc@16.1.0"` integration / e2e 1-2 (linux) `gcc@16.1.0 is not installed` Verified back to exactly one difference from main: the version. The e2e job's failure had a second cause worth fixing on its own. It runs `toolchain default gcc@16.1.0` without ever installing gcc, so it depends on the sandbox cache happening to carry one from an earlier run. That is not a property any job should rely on, and this branch's xlings pin bump moved the whole cache-key prefix, so every shard started cold and hit it. Now installed explicitly; warm runs pay a lookup. 67 unit tests; e2e 201/86/29/103/179/200 under the restored gcc default. --- .github/workflows/ci-linux-e2e.yml | 10 ++++++++++ mcpp.toml | 2 +- 2 files changed, 11 insertions(+), 1 deletion(-) diff --git a/.github/workflows/ci-linux-e2e.yml b/.github/workflows/ci-linux-e2e.yml index 58e61e56..01996456 100644 --- a/.github/workflows/ci-linux-e2e.yml +++ b/.github/workflows/ci-linux-e2e.yml @@ -78,6 +78,16 @@ jobs: "$MCPP" self config # Pin the global default so test 28 (default-toolchain path) gets a # deterministic GNU answer instead of an auto-install pick. + # + # Installed explicitly, not assumed. `toolchain default` requires the + # toolchain to be present, and nothing in this job puts it there: + # building mcpp above uses whatever first-run picks (llvm). It worked + # only because the sandbox cache happened to carry a gcc some earlier + # run had installed -- so the moment the cache key changed (an xlings + # pin bump moves the whole prefix), every e2e shard failed with + # `gcc@16.1.0 is not installed` before running a single test. + # Warm runs re-install nothing; this costs a lookup. + "$MCPP" toolchain install gcc 16.1.0 "$MCPP" toolchain default gcc@16.1.0 # Warm musl once so fresh-home e2e tests inherit the payload. "$MCPP" toolchain install gcc 16.1.0-musl diff --git a/mcpp.toml b/mcpp.toml index 6e44600d..60309c3d 100644 --- a/mcpp.toml +++ b/mcpp.toml @@ -17,7 +17,7 @@ default-profile = "release" include_dirs = ["src/libs/json"] [toolchain] -default = "llvm@22.1.8" +default = "gcc@16.1.0" macos = "llvm@22.1.8" windows = "llvm@20.1.7" From 3b7de758bee9ff205f8b48d311796f7624884c36 Mon Sep 17 00:00:00 2001 From: speak-agent Date: Sat, 8 Aug 2026 11:19:23 +0800 Subject: [PATCH 21/21] test(e2e): ldd's interpreter line is about ldd, not about the artifact The host-library check I added a commit ago failed on CI for a dependency the artifact does not have: /home/runner/.mcpp/.../xim-x-glibc/2.39/lib64/ld-linux-x86-64.so.2 => /lib64/ld-linux-x86-64.so.2 ldd resolves the program interpreter by running the host's loader, so it always reports the payload interpreter as resolving to the host's. That says nothing about what the artifact loads at runtime -- the PT_INTERP assertion above already covers the interpreter, and it passed. A real dependency has a bare soname on the left; the interpreter line has an absolute path. The predicate now requires the former, and is self-checked against all three line shapes before use. Worth stating plainly: this is a check that would have blocked correct work while claiming a defect. That is the opposite failure from the ones this branch has been chasing, and no less expensive. --- tests/e2e/201_gcc_no_specs_pollution.sh | 17 +++++++++++++---- 1 file changed, 13 insertions(+), 4 deletions(-) diff --git a/tests/e2e/201_gcc_no_specs_pollution.sh b/tests/e2e/201_gcc_no_specs_pollution.sh index 22a3d394..20499ad4 100755 --- a/tests/e2e/201_gcc_no_specs_pollution.sh +++ b/tests/e2e/201_gcc_no_specs_pollution.sh @@ -92,9 +92,19 @@ echo "$out" | grep -q 'Hello' || { echo "unexpected output: $out"; exit 1; } # # Asking where each library actually came from makes the same defect visible # everywhere, host toolchain or not. +# Only lines whose LEFT side is a bare soname count. ldd resolves the program +# interpreter by running the host's loader, so it always reports the +# interpreter as `/ld-linux-x86-64.so.2 => /lib64/ld-linux-x86-64.so.2` +# -- a statement about ldd, not about the artifact, and a false positive that +# failed this very check on CI. A real dependency has no slash on the left. +host_lib_lines() { + ldd "$1" 2>/dev/null \ + | grep -E '^[[:space:]]*[^/[:space:]]+ => +(/lib|/usr/lib|/lib64|/usr/lib64)/' \ + || true +} + if command -v ldd > /dev/null 2>&1; then - host_libs=$(ldd "$bin" 2>/dev/null \ - | grep -E '=> +(/lib|/usr/lib|/lib64|/usr/lib64)/' || true) + host_libs=$(host_lib_lines "$bin") if [[ -n "$host_libs" ]]; then echo "the artifact loads libraries from the host:" echo "$host_libs" | sed 's/^/ /' @@ -147,8 +157,7 @@ EOF pfbin=$(find target -type f -name pf -path '*/bin/*' | head -1) [[ -n "$pfbin" ]] || { echo "no payload-first binary produced"; exit 1; } -pf_host=$(ldd "$pfbin" 2>/dev/null \ - | grep -E '=> +(/lib|/usr/lib|/lib64|/usr/lib64)/' || true) +pf_host=$(host_lib_lines "$pfbin") if [[ -n "$pf_host" ]]; then echo "the payload-first artifact loads libraries from the host:" echo "$pf_host" | sed 's/^/ /'