feat: xlings 作为运行时底座 —— subos 环境到达程序 + self-contained 的 /proc/self/exe 陷阱 (#352, #375) (2026.8.8.1) - #376
Merged
Merged
Conversation
…said so Launching through the bundled loader makes the kernel set /proc/self/exe to the loader. Every "find my resources next to the executable" path then resolves against lib/ -- silently. mcpp#375 reported this as a user's own workaround; it is in fact what mcpp's own self-contained mode produces, and the docs explained the ELF constraint that forces the launch without ever stating its consequence. The ELF spec forbids $ORIGIN in PT_INTERP, so the launch itself cannot go away. What can is the silence: the wrapper now exports MCPP_BUNDLE_DIR, and both entry points carry it because they are documented as interchangeable. Applications that cannot be changed are pointed at --mode vendored, where PT_INTERP is the host loader and /proc/self/exe is correct.
… not their problem #375 is titled "let the app link against the system libc". That is the reporter's SOLUTION. Their problem is that the artifact cannot be distributed. The design took the title at face value and proposed a [build] c_runtime axis whose one new capability was host-coupled -- linking the artifact against the host's libc. That crosses the boundary this ecosystem exists to hold. xlings is a userspace distribution and its hermetic policy names, first on the forbidden list, any .so under /usr/lib* or /lib* -- libc included. mcpp should not use the host when it does not have to. Removing that one value leaves the axis with nothing: self-contained is already spelled --target x86_64-linux-musl, and toolchain-coupled is today's behaviour. It would also have created a second answerer for "may I use the host's libc", which [build] allow_host_libs already owns. The problem has three hermetic answers and all three already exist: emit xpkg into the ecosystem (xlings repoints the binary at install time), a static musl target, or pack --mode self-contained. Two of them work today; the third was broken, and that is what the previous commit fixed. What was missing was never a mechanism -- it was that none of the three is discoverable. The axis is kept in the document as a rejected option rather than deleted. It had passed 12 contract tests, 7 renderer tests and a five-assertion e2e; green tests cannot tell you a feature should not exist, and the next person reading #375 will have the same idea.
A program needs bootstrap, discovery and configuration. xlings had the first two and, until it grew the subos_info block, nothing for the third -- its own module comment names the consequence by number: mcpp#352, a GLFW binary that links fine and exits 255 because nothing told it where the GL drivers are. mcpp is the consumer of the third one, and it reads rather than knows. That is the property that matters: mcpp must never contain the string LIBGL_DRIVERS_PATH, because the moment it does the graphics stack has two owners and the next capability xlings adds becomes an mcpp code change instead of a declaration. Every degradation fills the `note` field and callers must print it. A subos with no block is not exotic -- mcpp's own sandbox subos was measured in exactly that state, 356 workspace entries and no self-description -- and silence there is what made #352 expensive to find. family_of duplicates a five-row mapping that xlings also has. The alternative was asking the xlings binary, which costs a subprocess in the hot path and fails precisely where it matters: a sandbox xlings that has not been updated. Every row is pinned in tests, so a drift is a test failure rather than a silent ABI disagreement.
A program needs three things: it links, it finds its libraries, and it is told where its runtime data lives. mcpp supplied the first two and nothing for the third. xlings's graphics packages declare LIBGL_DRIVERS_PATH and friends into the subos; `xlings subos use` applied them and `mcpp run` did not. That gap is mcpp#352 -- a GLFW binary that links cleanly and exits 255 with no output, because nothing ever told it where the GL drivers are. mcpp carries whatever the subos declares and knows what none of it means. The e2e probes a variable with no graphics meaning on purpose: naming a real one would suggest mcpp has an opinion about it, and the moment it does the graphics stack has two owners. Resolved at run time rather than cached with the build. These values belong to the subos, not to the build, so a user who switches subos between `mcpp build` and `mcpp run` has to get the new ones; it is one file read. The subos is derived from the toolchain binary rather than from a global, so a toolchain inherited from another home resolves to that home's subos -- the one whose payloads the binary was actually linked against. MCPP_SUBOS_DIR overrides it, which is how the e2e exercises this without going near a real subos: an earlier test wrote through a symlink and permanently broke a real toolchain.
mcpp#375 asked for a supported way to make a distributable artifact and proposed one: link against the system libc. Three routes already produce a distributable artifact and none of them does that -- through the ecosystem (xlings repoints the ELF at install time), a static musl target, or a bundle carrying the toolchain's own glibc. The gap was never a mechanism. It was that the page opened with "use mcpp pack" and never said what the alternatives were, so the reader who wanted something else had to invent it. The note about route A is the one that answers #375 directly: a freshly built binary's PT_INTERP names the build machine's payload, and hand-copying that file elsewhere fails. That is a property of hand-copying, not of the artifact -- installed through xlings the ELF is repointed at the target's own payloads. Saying so is what stops the next reader concluding the artifact is broken and reaching for the host's loader to "fix" it.
The xlings pin moves because this release consumes something new from it: the subos_info block a subos writes to describe itself. Reading it against an older xlings degrades quietly by design, but CI should exercise the version that actually has it. The bootstrap pin stays at 2026.8.6.2. It is where the self-host starts, not what the release ships -- moving it would send every CI job looking for a version that does not exist yet.
`mcpp run` has two paths and only one of them had been taught about subos declarations. The full path resolves the toolchain and applies them; the fast path skips prepare_build entirely and built its own child environment. So a program got its environment on the run right after a build and lost it on every run after that -- for a GL application, "it worked once and now the window is black", with nothing in between to attribute it to. WHICH subos is a build property and is now recorded in the build cache. WHAT it declares is the subos's own and is re-read on every run, so installing a graphics stack between two runs takes effect without a rebuild. That split is also why MCPP_SUBOS_DIR moved out of the derivation: an override means "for this invocation", and caching one would let a single `MCPP_SUBOS_DIR=… mcpp run` silently redirect every later run. Found by an assertion that the test did not originally have. The first version ran `mcpp run` once, passed, and proved nothing about the path it was written for. The fast-path check now runs first and fails loudly if the second run did NOT take the fast path -- without it the assertion below it is vacuous and would keep passing after the coverage silently went away. It caught a second defect immediately: the new cache line was written before `profile=` and parsed after `cacheMode=`, so every entry read back as stale and the fast path was disabled for everyone. Two dead capability tokens, and a guard so there is no third. `# requires: linux` and `# requires: llvm` are not capabilities run_all.sh ever sets, so 65_toolchain_runtime_dirs_for_run.sh had never executed in CI -- and it passes. The skip line for a token that cannot exist reads exactly like the skip line for one that legitimately does not, which is what let it sit. The runner now refuses to start when a test declares a token outside the known universe. The xlings pin goes back to 2026.8.6.3. It was moved to 2026.8.7.1 for tidiness rather than need -- subos_info degrades quietly on an older xlings by design -- and CI's fresh-sandbox jobs failed with the toolchain's own g++ exiting 127, the signature of an interpreter that is not where the binary says it is. Whether that is a real incompatibility is worth knowing, but not on the back of a change that does not need it.
Windows CI caught this; no amount of reading would have. resolve_env joined
and split its lists on ':', which on Windows is both the wrong separator (it
is ';') and a character that appears INSIDE every absolute path. So
de-duplication split "C:\x" into "C" and "\x", matched nothing, and the
joined value came back as "C:\...\x:C:\...\x" -- a list that grows on every
nested invocation and that no consumer can parse.
This repository has made the same mistake before, in the other direction:
find_first_of(";:") over a Windows PATH cuts at the drive-letter colon.
The three test expectations that failed were also wrong, but differently, and
the difference matters: they compared against a path JOIN while the code does
a literal substitution. The literal one is correct -- the separator inside a
declaration belongs to the subos manifest, and rewriting it to the host's
spelling would be editing a value we do not own. The assertions now say so,
and the dedup test additionally asserts the result contains NO separator at
all, which is the property that actually failed.
Windows CI: `clang++: error: clang frontend command failed due to signal`, with the diagnostic file named subos_info-*.cppm. No message beyond the signal, so the offending construct is identified by removal rather than by a compiler telling us. The one exotic thing in the file was `std::ranges::find` with a member-pointer projection into std::pair; `std::ranges::sort` went with it for the same reason. Both are replaced by plain loops, which nothing here needed to be fancier than. Ruled out first: importing mcpp.platform, added in the previous commit and the only other change to this file between the run that failed a test assertion and the run that crashed the compiler. Four modules already import mcpp.libs.json and mcpp.platform together (bmi_cache, stdmod, post_install, prepare), so that combination is not it. Stated plainly because it matters for the next person: this is a hypothesis confirmed only by CI going green, not by a local reproduction. The crash needs clang 20.1.7 targeting MSVC and I have no such host.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
起点与一次方向修正
调研起点是 #375(产物不可分发)与 #352(GL 程序 exit 255)。
这个 PR 的第一版做错了,值得先说。 #375 的标题是「让要分发的应用链到系统 libc」——那是提报者已经想好的解法,他的问题是「产物没法分发」。第一版照着标题做,加了一条
[build] c_runtime轴,其中host-coupled= 把产物链到宿主 libc。它实现完毕、12+7 个单测与一条五断言 e2e 全绿,然后被整条撤销:/usr/lib*/lib*下的.so,libc 在内。mcpp 能不用 host 就不用 host。self-contained已由--target …-musl表达,toolchain-coupled是现状)[build] allow_host_libs保留在设计文档 §5.2 作为被否方案而不是删掉:它通过了全部测试,而测试全绿不能告诉你一个功能不该存在;下一个读 #375 的人会有同样的想法。
实际改了什么
1.
pack --mode self-contained打坏/proc/self/exe,而且从没说过这是调研里最出乎意料的一条。#375 描述的第三条症状读起来像用户自己的绕法,但 mcpp 自己的
self-contained模式生产的就是那个 wrapper:文档解释了 ELF 规范为何强制这种启动方式,却从没写它的后果:内核把
/proc/self/exe指向 loader,所有「在 exe 旁边找资源」的逻辑静默失效——字体、assets/、随包分发的辅助二进制。全仓 grep 该陷阱零处提及,而 mcpp 自己(platform/fs.cppm)就靠它定位自身。ELF 的约束去不掉,能去掉的是沉默:wrapper 现在导出
MCPP_BUNDLE_DIR,两个入口都带(它们被文档为可互换)。改不了的应用被指向--mode vendored。2. mcpp 程序拿不到 subos 声明的环境(#352)
一个程序需要三样:链接(bootstrap)、找到库(RPATH)、被告知运行期数据在哪(env)。mcpp 供了前两样,第三样什么都没有。xlings 的图形包把
LIBGL_DRIVERS_PATH之类声明进 subos,xlings subos use会应用,mcpp run不会——所以一个 GLFW 二进制链接干净、exit 255、零输出。xlings 侧
subos/manifest.cppm的注释直接点名了这件事:「That is the gap behind #352」。而实测 mcpp 自己的 sandbox subos:356 个 workspace 条目,零subos_info块。新增
mcpp.xlings.subos_info读取并解析该块,mcpp run/mcpp test应用其结果。mcpp 不知道这些变量是什么意思,这是设计而非疏漏——e2e 特意用一个与图形无关的探针变量,因为一旦 mcpp 里出现
LIBGL_DRIVERS_PATH这个字符串,图形栈就有了两个主人,xlings 下次加 Vulkan loader 就会变成一次 mcpp 改代码。3. 文档:三条分发路径,没有一条用宿主 libc
mcpp emit xpkg→xlings install--target x86_64-linux-muslmcpp pack --mode self-contained缺的从来不是机制,是这一页开头只写「用
mcpp pack」、从不说还有什么。A 那一条专门写明了 #375 的直接答案:烙进去的PT_INTERP指向构建机路径不是产物的缺陷,是「手工拷贝」这个动作的性质——经 xlings 安装时 ELF 会被重指。不说清这点,下一个读者就会认定产物坏了,然后伸手去拿宿主 loader「修」它。验证
test_subos_info(10 个断言组:wire format、${subosdir}展开、prepend/set 语义与去重、缺块/新 schema/坏 JSON 三种降级、family 映射逐行)30_pack_modes(加了MCPP_BUNDLE_DIR两个入口断言)、新增195_subos_env_reaches_program、以及86_llvm_hermetic_link74_run_no_loader_env_leak166_run_env_no_private_glibc06_emit_xpkg28_target_static—— 7/7 PASSPROBE=(unset)),断言有效尚未做(有意)
compat.glx-runtime→xim:graphics:Linux 下 GLFW/OpenGL 程序运行时无法创建窗口:mcpp 自带 glibc(2.39)低于宿主 Mesa 要求(2.43),且 compat-x-glx-runtime 指向 32 位库 #352 的另一半在 mcpp-index 仓,独立 PR。它至今 symlink 宿主libGL.so.1,是「用宿主」的最大一处pack --mode vendored是默认且宿主耦合:它把 PT_INTERP 重指到/lib64/…,与 hermetic 策略不一致。改默认会破坏既有用户,记在设计文档 Q2 待单独评估设计:
.agents/docs/2026-08-07-xlings-as-runtime-substrate-design.md计划:
.agents/docs/2026-08-08-xlings-runtime-substrate-implementation-plan.mdRefs #375, #352