diff --git a/.agents/docs/2026-08-09-mcpp-template-runtime-graphics-aur-focused-design.md b/.agents/docs/2026-08-09-mcpp-template-runtime-graphics-aur-focused-design.md new file mode 100644 index 00000000..fc41f7ee --- /dev/null +++ b/.agents/docs/2026-08-09-mcpp-template-runtime-graphics-aur-focused-design.md @@ -0,0 +1,1034 @@ +# mcpp 模板、运行时、图形栈与 AUR 聚焦设计 + +> 状态:Review Draft +> +> 日期:2026-08-09 +> +> 基线:mcpp main@80291ca01a98 +> +> 上位分析:.agents/docs/2026-08-09-xlings-mcpp-ecosystem-convergence-design.md +> +> 本文只冻结四项产品设计,不代表实现、发布或 AUR 状态已经改变。 + +## 1. 已确认的四项产品决策 + +本文将以下内容视为已确认方向,不再把旧方案并列为推荐项: + +1. **模板 selector 与 mcpp add 保持同一风格** + - 稳定形式:ns.name@version:tname。 + - ns. 可省略;省略时使用 mcpp/mcpp.toml 的默认 namespace mcpplibs。 + - @version 可省略。 + - :tname 可省略。 + - 不引入 --variant。 +2. **xlings 拥有图形栈与运行时,mcpp 构建在该生态之上** + - xlings 负责图形 runtime/provider 的解析、安装、激活和生命周期。 + - xim-pkgindex 提供 Mesa、NVIDIA、WSL、Vulkan ICD/driver 等 package/provider recipe。 + - mcpp-index 声明 C++ 图形包到 xlings/xim 图形能力的依赖关系。 + - mcpp 只通过 mcpp-index 的包描述、模板和通用构建契约完成构建。 + - mcpp 不探测 GPU,不按厂商写构建分支。 +3. **运行时分为 mcpp 默认与项目显式 SubOS** + - 不配置时使用 mcpp 管理、发布验证过的默认运行时。 + - 项目可在 mcpp.toml[xlings] 中选择命名 SubOS。 + - SubOS 是根项目本地 build/run 环境,类似选择“在哪个 OS 开发”;它不是库的传递依赖,也不要求消费者使用同一 SubOS。 + - 多个项目可以并存于不同 SubOS/不同 glibc,缓存和产物按实际 RuntimeBinding 隔离。 +4. **AUR 自动同步聚焦 mcpp-bin** + - mcpp-bin 是唯一自动对账并纳入漂移告警的 AUR 包。 + - AUR 是 GitHub Release 的最终一致投影,AUR 故障不阻塞 GitHub Release 本身。 + - mcpp-m 本阶段完全不动:不修改 package 文件、不推送、不退役、不改变其现有状态。 + - mcpp-git 不进入主发布关键路径。 + +## 2. 目标、边界与成功定义 + +### 2.1 目标 + +- 用户只学习一种包 selector 风格;namespace 可省略,省略时默认 mcpplibs。 +- 默认用户无需理解 xlings/SubOS 即可获得稳定运行时;需要固定 ABI/环境的项目能声明 SubOS。 +- 图形包的新平台、驱动和 backend 由 xlings/xim 生态演进;mcpp-index 表达 C++ 依赖关系,不要求修改 mcpp 核心。 +- SubOS 只影响当前根项目本地构建/运行,不从库依赖向消费者传递。 +- AUR 临时故障恢复后,mcpp-bin 能自动收敛到最新稳定 GitHub Release。 +- 设计保持 CLI 简洁、内部身份明确、构建热路径无网络和 GPU 探测。 + +### 2.2 非目标 + +- 不在本轮增加 --variant#template 或新的顶层 template 命令。 +- 不让项目 mcpp.toml 写本机绝对 xlings binary/home。 +- 不在 mcpp 中实现 Mesa/NVIDIA/Vulkan/WSL 专用逻辑。 +- 不要求 mcpp-m 与每次 release 同步。 +- 不修改或重新定义 mcpp-m 的维护策略。 +- 不在本文中解决 C1–C9 的全部实现;只保留与这四项直接相关的契约。 +- 本轮不实现代码、不重跑 workflow、不推送 AUR。 + +### 2.3 成功定义 + +~~~text +mcpp new app --template ocornut.imgui@1.92.8:glfw-opengl3 + -> dotted selector 显式解析为 namespace=ocornut, name=imgui + -> 解析出完整 PackageId 与 exact version + -> 从 mcpp-index 获取模板/构建描述 + -> mcpp-index 声明 xlings/xim 图形依赖 + -> xlings 解析并物化图形运行时 + -> 使用 mcpp 默认 runtime 或 [xlings].subos + -> mcpp 只执行通用 build/link/run plan +~~~ + +发布侧: + +~~~text +latest stable GitHub Release + -> immutable release manifest + -> mcpp-bin generator and validation + -> idempotent AUR reconcile + -> remote git + AUR RPC + clean install verification +~~~ + +## 3. 方案比较 + +| 主题 | 方案 A | 方案 B | 方案 C | 选择 | +|---|---|---|---|---| +| 模板名 | --variant 独立参数 | #tname 新分隔符 | [ns.]name[@version][:tname] | **C**:与现有 CLI/TOML 风格一致 | +| runtime 配置 | 跟随全局 active SubOS | 新增 [runtime].provider | 默认 mcpp runtime;已有 [xlings].subos 覆盖 | **C**:字段最少、可复现 | +| graphics | mcpp 内置 GL/Vulkan/provider 逻辑 | mcpp 直接调用 xim 图形接口 | xlings 拥有图形 runtime;mcpp-index 声明 C++ 依赖 | **C**:职责单一 | +| AUR | 双包串行同步 | 三包 matrix 同步 | 只自动对账 mcpp-bin | **C**:主用户路径优先 | + +方案 C 的共同原则是:**用户表面复用已有概念,内部不复用含糊字符串;每层只拥有自己能证明的事实。** + +## 4. 总体架构 + +~~~mermaid +flowchart LR + U[mcpp.toml / CLI] --> M[mcpp generic resolver and builder] + M --> I[mcpp-index
package, template, build contract] + I --> X[xlings
graphics stack owner, install, SubOS] + X -->|reads provider recipes| XI[xim-pkgindex
provider recipes and sentinels] + X --> RC[resolved graphics and runtime contract] + RC --> M + M --> A[artifact] + + R[GitHub Release Manifest] --> AR[AUR mcpp-bin reconciler] + AR --> AB[AUR mcpp-bin] +~~~ + +核心依赖方向: + +- mcpp 可以理解 PackageSelector、PackageId、LinkIntent、RuntimeBinding 等通用结构。 +- mcpp 不理解 mesanvidiawslvulkan icd 的选择规则。 +- mcpp-index 声明图形包如何编译、链接,以及它依赖哪些 xlings 生态 package/capability。 +- xlings 负责解析、安装、激活和导出图形 runtime contract。 +- xim-pkgindex 是 xlings 消费的 provider/recipe 数据源,实现 capability 并记录 artifact/provenance。 +- AUR reconciler 不读取工作区临时状态,只读取不可变 release manifest。 + +## 5. 模板 selector 统一设计 + +### 5.1 用户语法 + +稳定文法: + +~~~text +TemplateSpec := PackageSelector [ "@" ExactVersion ] [ ":" TemplateName ] +PackageSelector := Name | NamespacePath "." Name +NamespacePath := Segment { "." Segment } +TemplateName := NameAtom +~~~ + +NamespacePath 可省略。省略时: + +~~~text +imgui -> namespace = mcpplibs, name = imgui +~~~ + +它不是全索引 short-name 搜索,也不是“跟随当前 index 的 namespace”;默认值固定为 mcpplibs,与 mcpp.toml 默认 dependency table 对齐。 + +分隔符唯一职责: + +| 分隔符 | 含义 | +|---|---| +| . | 与 mcpp add 相同的 dotted package selector | +| @ | exact package version | +| : | package 内的 template name | + +合法示例: + +| 输入 | package selector | version | template | +|---|---|---|---| +| imgui | mcpplibs.imgui(ns 省略) | 默认稳定版 | descriptor default | +| ocornut.imgui | ocornut.imgui | 默认稳定版 | descriptor default | +| ocornut.imgui@1.92.8 | ocornut.imgui | 1.92.8 | descriptor default | +| ocornut.imgui:glfw-opengl3 | ocornut.imgui | 默认稳定版 | glfw-opengl3 | +| ocornut.imgui@1.92.8:vulkan | ocornut.imgui | 1.92.8 | vulkan | +| mcpplibs.gui.templates@2.0.0:window | mcpplibs.gui.templates | 2.0.0 | window | + +非法示例: + +| 输入 | 错误 | +|---|---| +| ocornut.imgui@ | empty version | +| ocornut.imgui: | empty template name | +| .imgui@1.0 | empty selector segment | +| ocornut..imgui | empty namespace segment | +| ocornut.imgui@1.0:vulkan:extra | more than one template delimiter | + +pkg: 不再兼任列表语法。列举模板继续使用现有显式表面: + +~~~text +mcpp new --list-templates ocornut.imgui@1.92.8 +~~~ + +这样“省略 tname”永远表示选择 default,不会同时表示 list。 + +### 5.2 与 mcpp add / mcpp.toml 的一致性 + +模板不能只复制 mcpp add 的视觉风格,两个命令最终必须复用同一条规范化链: + +~~~text +raw selector + -> shared PackageSelector parser + -> fill default namespace mcpplibs when omitted + -> one normalized PackageId + -> IndexRoute::lookup_descriptor by exact PackageId + -> resolved PackageId + descriptor provenance +~~~ + +目标规则: + +- imgui(mcpplibs, imgui)。 +- acme.widget(acme, widget)。 +- mcpplibs.capi.lua(mcpplibs.capi, lua)。 +- 多级 selector 总是以最后一个 segment 为 name,其余为 namespace。 + +对应的 mcpp.toml 语义: + +~~~toml +[dependencies] +imgui = "1.92.8" # default namespace mcpplibs + +[dependencies.acme] +widget = "1.0.0" # explicit namespace acme +~~~ + +CLI dotted form acme.widget@1.0.0 是第二种 TOML 形式的紧凑输入,不再表示“先猜 mcpplibs.acme,再猜 acme”。 + +当前 mcpp add 的 ordered dotted candidates 属于兼容实现。为了真正一致,迁移后 add/template 都使用上述单一规范化规则;旧项目清单已解析并锁定的 dependency 不被自动重写。 + +selector 是用户输入,最终身份仍是结构化: + +~~~text +PackageSelector { + namespace?: NamespacePath, + name: NameAtom +} + +PackageId { + namespace: NamespacePath, + name: NameAtom +} + +ResolvedTemplatePackage { + id: PackageId, + version: ExactVersion, + indexRoute, + descriptorDigest, + payloadDigest, + root +} + +TemplateSelection { + package: ResolvedTemplatePackage, + templateName: NameAtom +} +~~~ + +PackageSelector 规范化时填入默认 namespace,所以进入 resolver 后不再存在“namespace 未知”状态。xlings 安装 wire address 仍由 resolved PackageId 派生为 namespace:name@version。点号只属于 mcpp 用户 selector;冒号 wire address 不反向塞回 --template。 + +### 5.3 解析顺序 + +为了避免 namespace、version 与 tname 相互抢分隔符,解析固定为: + +1. 验证最多一个 :,分离右侧 TemplateName。 +2. 在左侧验证最多一个 @,分离 ExactVersion。 +3. 将剩余部分完整交给共享 PackageSelector parser。 +4. resolver 返回 PackageId 后再进入版本和模板选择。 + +禁止当前 scaffold 的“先按短名试空 namespace/compat,命中后再反推 namespace”旁路。 + +### 5.4 version 省略规则 + +- 写出 @version 时只接受 exact version;第一阶段不引入 range。 +- 省略时选择目标平台可用的最新 stable version。 +- prerelease 不自动成为默认;只有用户显式写出 exact prerelease 才选择。 +- 解析成功后后续流程只携带 exact version。 +- 成功输出、wire result 与生成的自依赖都展示 exact version。 + +版本选择必须复用包管理 resolver,不能由模板代码排序字符串。 + +### 5.5 tname 省略规则 + +省略 :tname 时: + +1. 正好一个 template 声明 default = true:选择它。 +2. 没有显式 default,但 package 只有一个 template:该单模板自动成为 default。 +3. 没有显式 default,且存在多个 templates:失败并列出所有 template,要求用户写 :tname 或 provider 选出 default。 +4. 多于一个显式 default:descriptor/index validation 失败。 +5. package 没有 templates/:明确报告它不是 template provider。 + +单模板自动默认是当次已解析 provider/version 的确定事实;同一已锁版本不会因未来 release 新增模板而改变。新版本若增加第二个模板却未声明 default,省略 tname 会明确失败,不进行目录排序猜测。 + +### 5.6 生成与依赖注入 + +模板解析后必须保留完整 PackageId,解决当前 namespace 在 fetch 后丢失的问题。 + +RenderVars: + +~~~text +project.name +project.namespace +project.qualifiedName +template.package.namespace +template.package.name +template.package.selector +template.package.version +template.name +~~~ + +自依赖注入复用 mcpp add 的 manifest editor: + +- 不做字符串搜索。 +- 使用用户风格的 dotted selector key。 +- 写 exact resolved version。 +- 保留 namespace 与 index provenance 到 lock/build resolution。 +- 模板已经声明相同 PackageId 时不重复写入。 +- 同 short name、不同 namespace 不视为同一个 dependency。 + +### 5.7 输出与诊断 + +成功的人类输出: + +~~~text +Created app +Template ocornut.imgui@1.92.8:glfw-opengl3 +Runtime mcpp-default +~~~ + +机器输出至少包含: + +~~~json +{ + "packageSelector": "ocornut.imgui", + "resolvedPackage": { + "namespace": "ocornut", + "name": "imgui", + "version": "1.92.8" + }, + "template": "glfw-opengl3", + "runtimeSelection": "mcpp-default" +} +~~~ + +诊断规则: + +- package 不存在:展示规范化后的 exact PackageId;namespace 省略时明确注明使用了默认 mcpplibs。 +- version 不存在:列出该平台可用 stable versions。 +- template 不存在:列出 provider 内 template names 和 default。 +- 用户写 ocornut:imgui 时,按文法它表示 package ocornut 的 template imgui;若解析失败,诊断额外建议 namespace 风格 ocornut.imgui。 +- 所有失败发生在创建目标目录前,或由临时目录事务回滚。 + +### 5.8 兼容迁移 + +旧的无 namespace 形式天然兼容: + +~~~text +pkg +pkg@version +pkg:tname +pkg@version:tname +~~~ + +变化只有: + +- 新增 dotted namespace selector。 +- bare selector 明确填入默认 namespace mcpplibs。 +- mcpp add 与 template 最终收敛为“dot 表示显式 namespace、无 dot 表示默认 mcpplibs”,不再各自猜候选。 +- pkg: 的 legacy list 含义先 warning 一个 release train,之后错误;用户改用 --list-templates pkg。 +- 不再新增 --variant,上一份综合设计中的该建议由本文覆盖。 +- builtin bin/gui 可暂时保留 alias,但 package template 输出统一用 TemplateSpec。 + +当前 add 的 dotted selector 具有默认 namespace 前缀候选,例如 capi.lua 会先尝试 mcpplibs.capi:lua。新规则下: + +~~~text +capi.lua -> capi:lua +mcpplibs.capi.lua -> mcpplibs.capi:lua +lua -> mcpplibs:lua +~~~ + +迁移要求: + +1. 已有 mcpp.toml 与 lockfile 不自动改写,仍按其已记录身份工作。 +2. 一个 release train 检测“旧候选结果与新 exact 结果不同”,warning 同时给出两种完整 selector。 +3. 新 mcpp addmcpp new --template 在迁移窗口后统一采用 exact dotted 规则。 +4. mcpp-index 中所有 nested mcpplibs.* 文档示例改成完整 namespace,不能依赖隐式前缀。 + +## 6. mcpp 默认 runtime 与 mcpp.toml SubOS + +### 6.1 两种模式 + +只定义两种项目运行时选择: + +~~~text +McppDefault +NamedSubos(name) +~~~ + +不增加第三种“跟随当前全局 active subos”模式。 + +| mcpp.toml | 选择 | +|---|---| +| 没有 [xlings].subos | McppDefault | +| [xlings] subos = "dev" | NamedSubos("dev") | +| [xlings] subos = "default" | 显式选择 xlings home 中的 default SubOS | + +### 6.2 默认模式 + +McppDefault 的定义: + +- 使用 mcpp 所选择的 xlings provider。 +- 使用 mcpp home 内已初始化、release 验证过的 default SubOS/RuntimeBinding。 +- 不受用户另一个 shell 中执行 xlings subos use 的 active 状态影响。 +- 缺失时走 mcpp bootstrap,不从任意目录挑第一个 glibc。 +- xlings/runtime contract schema 不兼容时明确失败或升级,不静默回退宿主。 + +默认路径的用户体验是“安装 mcpp 后直接 build/run”,不要求用户了解 SubOS。 + +### 6.3 项目显式 SubOS + +复用当前已支持的配置: + +~~~toml +[xlings] +subos = "dev" +deps = ["cmake@3.28", "python@3.13"] + +[xlings.workspace] +clang = "20.1.7" + +[xlings.envs] +OPENBLAS_NUM_THREADS = "1" +~~~ + +语义: + +- subos 只选择当前根项目 build/run/test 共用的本地命名环境。 +- deps、workspace pins 与 envs 在该环境中物化。 +- Linux 上 RuntimeBinding 同时决定 loader/libc。 +- macOS/Windows 上仍选择一致的工具与环境契约,但不伪造 Linux glibc 语义。 +- 指定 SubOS 不存在或无法回答 runtime contract 时 hard error,不退回 default。 +- 同一台机器可同时保留 el8trixiedefault 等多个环境;每个项目按自己的 mcpp.toml 选择,互不切换全局 active 状态。 + +### 6.4 xlings binary/home 属于机器配置 + +项目清单必须可移植,所以不在 mcpp.toml 接受绝对 binary/home: + +~~~toml +# ~/.mcpp/config.toml +[xlings] +binary = "bundled" # or "system" / absolute administrator path +home = "" +~~~ + +职责分离: + +- 全局 config 决定“使用哪一个 xlings 实例和 home”。 +- 项目 mcpp.toml 决定“在该实例中使用 default 还是哪个 named SubOS”。 + +这样团队可以提交 subos = "el8",而不提交某位开发者的本地 XLINGS_HOME 路径。 + +### 6.5 选择优先级 + +稳定顺序: + +~~~text +project [xlings].subos exists + -> NamedSubos +otherwise + -> McppDefault +~~~ + +本设计不增加临时 --subos CLI override。runtime 是 build identity,命令行临时覆盖会让同一份 mcpp.toml 产生不同 ABI,并污染缓存解释。 + +global xlings binary/home 只决定 provider,不插入第三个 runtime selection rung。 + +### 6.6 一个 snapshot 贯穿生命周期 + +~~~text +RuntimeSelection + -> resolve exact SubOS + -> read RuntimeBinding contract + -> include contract hash in toolchain/build fingerprint + -> configure/link + -> post-link validation + -> run/test environment +~~~ + +build、run、test、post-install fixup 不得分别重新猜 runtime。 + +建议内部模型: + +~~~text +RuntimeSelection { + mode: McppDefault | NamedSubos, + subosName, + source: DefaultPolicy | Manifest +} + +RuntimeBinding { + schema, + providerId, + platform, + arch, + contractHash, + loader?, + libc?, + libraryDirs[], + environment[], + capabilities[], + provenance +} +~~~ + +### 6.7 [runtime] 与 [xlings] 不混用 + +现有 [runtime] 表示程序/包启动时需要的 library dirs、dlopen libs 和 capabilities;它不是“选择哪一个 SubOS”的配置。 + +~~~toml +[xlings] +subos = "dev" # 选择 build/run 环境 + +[runtime] +capabilities = ["opengl"] # 程序需要的通用 capability +~~~ + +选择环境属于 [xlings],声明程序需求属于 [runtime]。禁止再增加 [runtime] provider = "subos" 形成第二入口。 + +### 6.8 SubOS 是本地、根项目级、非传递环境 + +SubOS 的心智模型是: + +> mcpp 在一台机器上选择一个本地开发 OS 环境来 configure/build/run 当前项目。 + +它不是库依赖约束。规则如下: + +1. 只有本次构建的 root manifest/workspace root 能选择 SubOS。 +2. dependency manifest 中的 [xlings].subos 不合并、不继承、也不要求消费者创建同名 SubOS。 +3. 一个库作为独立根项目开发时,它自己的 [xlings].subos 生效;同一源码作为另一个项目的 dependency 时,使用消费者 root 选择的环境构建。 +4. workspace 整体构建时由 workspace root 选择一个环境;member 的 SubOS 不覆盖 root。member 独立构建时才成为自己的 root。 +5. SubOS name 不写入 dependency requirement,不从 lockfile 向下游传播,也不成为 mcpp-index package identity 的一部分。 + +典型源码分发: + +~~~text +application root selects subos=el8 + -> source dependency A builds inside el8 + -> source dependency B builds inside el8 + -> application runs inside el8 + +another application selects subos=trixie + -> the same A/B sources rebuild under trixie +~~~ + +因此可以在同一机器上用不同 glibc 环境开发同一组源码依赖,而不是要求所有库声明或传递 glibc=2.x。 + +间接关系只来自实际产物: + +- RuntimeBinding 必须进入当前项目 build fingerprint,防止跨 SubOS 复用 object/BMI。 +- 如果发布的是预构建 binary,发布流程应从最终 artifact 派生 ABI、loader、GLIBC symbol floor 等兼容元数据。 +- 这些是 artifact 的客观兼容属性,不是把开发时的 SubOS name 传播给消费者。 +- package 确实需要某项运行时能力时,通过 mcpp-index/xim package requirement 表达,不通过 [xlings].subos 表达。 + +### 6.9 当前 active SubOS 的迁移 + +当前未显式配置时可能读取 active SubOS。切换到稳定 default 的迁移: + +1. 一个 release train 输出 warning,显示当前 active 与未来 default 是否不同。 +2. 提供可复制配置:[xlings] subos = "current-name"。 +3. 下一 release 将 absence 固定为 McppDefault。 +4. 不自动修改用户 mcpp.toml。 + +冷 HOME、新安装可以直接采用新规则,不需要 legacy 过渡。 + +## 7. OpenGL/Vulkan 图形栈职责 + +### 7.1 强制边界 + +| 层 | 拥有 | 不拥有 | +|---|---|---| +| mcpp | 通用 PackageId、source/build graph、LinkIntent、RuntimeBinding 消费、产物验证 | OpenGL/Vulkan 包名、GPU vendor、ICD 选择、WSL 探测 | +| mcpp-index | ImGui/GLFW/OpenGL/Vulkan C++ 包、features、templates、平台 build/link 声明,以及这些包对 xlings 图形能力的依赖关系 | 安装宿主驱动、判断 NVIDIA/WSL 当前状态 | +| xlings | 图形栈 orchestration、依赖解析/安装/激活、SubOS、runtime contract、provider/sentinel 生命周期 | C++ GUI template 与项目源码 | +| xim-pkgindex | xlings 消费的 Mesa、Vulkan loader/ICD、NVIDIA/WSL sentinel recipe 与 provenance schema | 运行时自行做全局选择、mcpp 工程生成和 source graph | + +mcpp 源码中不新增以 openglvulkanmesanvidiawsl 为条件的 planner 分支。 + +### 7.2 数据流 + +~~~text +mcpp.toml dependency + -> mcpp resolves descriptor through mcpp-index + -> mcpp-index descriptor contributes sources, features, LinkIntent + and xlings graphics package/capability dependencies + -> xlings resolves and materializes the dependency closure + -> xlings activates providers/sentinels described by xim-pkgindex + -> xlings exports RuntimeArtifacts and provenance + -> mcpp consumes only generic resolved contract + -> link, validate and run +~~~ + +mcpp 不直接选择 xim:graphics。mcpp-index 的平台 package contract 声明 xlings 生态依赖;xlings 才是图形栈运行时 owner,负责从 xim-pkgindex recipe 中选择和物化具体 provider。 + +### 7.3 通用构建接口 + +mcpp-index 输出: + +~~~text +LinkIntent { + libraries[], + linkLibraryDirs[], + transitiveNeededDirs[], + runtimeSearchDirs[], + frameworks[], + deployFiles[] +} + +RuntimeRequirement { + kind: soname | capability | icd_manifest | display | host_service, + value, + phase: link | run, + requester: PackageId, + required +} +~~~ + +xlings 解析 xim-pkgindex provider recipe 后提供: + +~~~text +RuntimeArtifact { + role: loader | library | driver | manifest | host_bridge, + provider: PackageId, + path, + provenance: payload | subos_view | host_link | system_sdk, + abi, + digest?, + hostFingerprint? +} +~~~ + +mcpp 只检查这些通用结构是否完整、目标平台是否匹配、最终 artifact 是否满足 loader/ABI 物理约束。 + +### 7.4 OpenGL 收口 + +mcpp-index: + +- ocornut.imgui 声明 core 与 backend features。 +- glfw-opengl3 template/feature 引入 GLFW、OpenGL headers 与通用 runtime requirement。 +- Linux package contract 依赖由 xim 提供的 graphics capability。 +- macOS contract 使用 native frameworks。 +- Windows contract 使用 Win32/system SDK 与声明的 runtime DLL。 + +xlings(基于 xim-pkgindex recipes): + +- Mesa/GLVND 与软件/硬件 driver closure。 +- NVIDIA/WSL host-link sentinel。 +- runtime dirs、实际 libraries 和 provenance 的解析/导出。 +- sentinel 的 applicable/not-applicable/inconclusive 生命周期。 + +目标状态是移除 mcpp-index 中复制 SubOS view library 的长期 symlink bridge;runtime artifact 通过 contract 传递。 + +### 7.5 Vulkan 收口 + +mcpp-index: + +- Vulkan headers、loader-facing API、backend sources 和 templates。 +- vulkan template 声明 loader、ICD manifest、display/surface requirements。 +- 不从 /usr/lib* 自行收集宿主 ICD/DSOs。 + +xlings(基于 xim-pkgindex recipes): + +- Vulkan loader、ICD manifests 与对应 driver。 +- Mesa RADV/Intel 等 payload provider。 +- NVIDIA/WSL host-link provider。 +- macOS MoltenVK、Windows system/provider 策略由平台包契约表达。 + +验证不能止于 loader symbol: + +1. create instance。 +2. enumerate physical devices。 +3. 记录实际 ICD manifest 与 driver provider。 +4. 有窗口 lane 时创建 surface。 +5. 无硬件环境明确报告 NOT_EXERCISED,不能汇总为 GPU pass。 + +### 7.6 ImGui feature/template 形态 + +建议 mcpp-index 将图形入口拆成: + +~~~text +core +headless +backend-glfw-opengl3 +backend-vulkan +app +docking +viewports +~~~ + +示例: + +~~~text +mcpp new app --template ocornut.imgui:glfw-opengl3 +mcpp new app --template ocornut.imgui@1.92.8:vulkan +mcpp new app --template ocornut.imgui@1.92.8:docking +~~~ + +- core/headless 不应无条件拉入 GLFW/OpenGL。 +- app 可以组合默认 backend,但 resolved dependency/features 必须写入生成清单。 +- docking/viewports 与 renderer backend 正交。 + +### 7.7 可观察性 + +mcpp why runtime 可以展示 xlings 已解析完成的结果: + +~~~text +requirement + -> selected canonical provider + -> runtime artifact + -> provenance + -> ABI/loader verdict +~~~ + +它是通用 contract 的解释器,不进行 GPU 探测。GPU/driver 诊断与重探测由 xlings 统一入口及其 xim provider/sentinel 暴露,mcpp 只给出跳转提示。 + +图形依赖和 SubOS 选择是正交的: + +- mcpp-index/xim 依赖描述“当前项目需要哪些图形能力”。 +- root [xlings].subos 描述“当前项目在哪个本地环境 build/run”。 +- library dependency 不通过自己的 SubOS 要求消费者环境;源码会在消费者已选环境中构建。 +- 某个图形 provider 对实际 ABI/glibc 的要求,由 xlings 在当前环境解析并验证,而不是把 provider 的 SubOS name 传播出去。 + +## 8. AUR 只自动维护 mcpp-bin + +### 8.1 包策略 + +| AUR 包 | 策略 | +|---|---| +| mcpp-bin | 自动生成、验证、对账和漂移告警;不阻塞 GitHub Release | +| mcpp-m | 本阶段不更新;不修改 package 文件或 AUR remote,只从 mcpp-bin 自动对账路径隔离 | +| mcpp-git | 不进入 release workflow;如要发布,单独认领与设计 | + +用户主路径是安装 release 预构建产物,因此自动化可靠性集中投入 mcpp-bin。 + +### 8.2 当前事实 + +2026-08-09 的审计快照: + +- GitHub latest stable:v2026.8.8.4。 +- AUR mcpp-bin2026.8.1.1-1。 +- 连续 push 失败的直接响应是 AUR maintenance。 +- workflow 缺少 retry、schedule 与状态对账,所以临时故障变成长时间漂移。 +- mcpp-m 存在已知独立问题,但本文不处理、不推送,也不让它成为 mcpp-bin reconciler 的前置条件或后置条件。 + +旧失败 run 不应直接 rerun,因为它从旧 release commit 执行旧生成逻辑。 + +### 8.3 单一 desired state + +release 产出不可变 manifest: + +~~~json +{ + "schema": 1, + "version": "2026.8.8.4", + "tag": "v2026.8.8.4", + "commit": "", + "assets": [ + { + "platform": "linux", + "arch": "x86_64", + "name": "mcpp-2026.8.8.4-linux-x86_64.tar.gz", + "sha256": "" + }, + { + "platform": "linux", + "arch": "aarch64", + "name": "mcpp-2026.8.8.4-linux-aarch64.tar.gz", + "sha256": "" + } + ] +} +~~~ + +reconciler 只消费 latest complete、非 draft、非 prerelease manifest。 + +### 8.4 mcpp-bin reconciler + +触发: + +- successful release workflow_run:低延迟。 +- schedule:建议每 6 小时。 +- workflow_dispatch:人工恢复,默认仍指向 latest stable。 + +流程: + +~~~text +read latest stable release manifest + -> query AUR mcpp-bin version and remote git head + -> compare with Arch vercmp + -> download both Linux assets + -> recompute sha256 and compare manifest/sidecar + -> generate PKGBUILD + -> generate .SRCINFO from PKGBUILD in Arch container + -> makepkg --verifysource + -> dry-run diff + -> fast-forward push with bounded retry + -> verify AUR git head + -> bounded poll AUR RPC + -> clean Arch install and mcpp --version smoke +~~~ + +### 8.5 幂等和失败分类 + +| 状态 | 行为 | +|---|---| +| desired == current 且内容一致 | success no-op | +| desired > current | 生成、验证、push | +| desired < current | 默认拒绝降级 | +| asset/hash 缺失 | push 前 permanent failure | +| AUR maintenance/timeout | transient;指数退避,后续 schedule 补偿 | +| SSH auth/metadata invalid | permanent;立即告警 | +| git 已更新但 RPC 延迟 | poll,不重复 commit | + +已知 package clone 失败不能自动当作首次发布。禁止 force-push AUR 历史。 + +### 8.6 mcpp-m 完全不动的边界 + +本文对 mcpp-m 的要求只有隔离,不包含任何维护动作: + +- 不修改 scripts/aur/mcpp-m/**。 +- 不生成或推送 mcpp-m AUR commit。 +- 不改变其版本、checksum、maintainer、deprecated 状态或现有远端历史。 +- 不宣称它与 GitHub Release 同步。 +- mcpp-bin reconciler 的成功/失败只由 mcpp-bin desired/observed state 决定。 + +实现 mcpp-bin reconciler 时应建立独立 job/workflow;现有 combined workflow 唯一允许涉及 mcpp-m 的变化是停止自动调用其 publish leg,以保证“不推送 mcpp-m”。不得读取、生成或改写 mcpp-m 内容。 + +### 8.7 AUR SLO + +建议初始目标: + +- AUR 可用时,release 后 30 分钟内 mcpp-bin 收敛。 +- event path 失败后,6 小时 schedule 再尝试。 +- 超过 24 小时仍漂移,自动更新固定告警 issue 或发高优先级通知。 + +## 9. 四项设计如何协同 + +### 9.1 默认用户 + +~~~text +install mcpp-bin + -> bundled xlings/bootstrap default runtime + -> mcpp new app --template ocornut.imgui:glfw-opengl3 + -> mcpp-index supplies template/build contract + -> xlings/xim supplies graphics runtime + -> mcpp build + -> mcpp run using the same default RuntimeBinding +~~~ + +用户不需要手工选择 GPU provider,也不需要配置 SubOS。 + +### 9.2 固定运行时用户 + +~~~toml +[xlings] +subos = "el8" +~~~ + +~~~text +mcpp new/build/run + -> same template/package selector + -> same mcpp-index graph + -> named SubOS RuntimeBinding + -> cache fingerprint changes + -> artifact targets that runtime +~~~ + +### 9.3 平台差异 + +| 平台 | mcpp default runtime | graphics provider | +|---|---|---| +| Linux x86_64 | mcpp-managed default SubOS/runtime binding | xim Mesa or explicit host sentinel | +| Linux aarch64 | mcpp-managed native runtime | capability must fail clearly until graphics recipe supports it | +| macOS arm64/x86_64 | mcpp-managed tool/SDK environment | native frameworks/MoltenVK package contract | +| Windows x64 | mcpp-managed tool/runtime environment | Win32/system DLL or declared Vulkan provider | + +跨平台只共享抽象契约,不假装共享底层文件布局。 + +## 10. 迁移阶段 + +### Stage 0 — 文法与边界冻结 + +- 冻结 TemplateSpec grammar:namespace/version/tname 均可省略,namespace 默认 mcpplibs。 +- 冻结“单模板在没有显式 default 时自动成为 default”。 +- 冻结 absence → McppDefault、[xlings].subos → NamedSubos。 +- 冻结 SubOS 是 root-local、非传递 build/run 环境。 +- 冻结 xlings 是 graphics runtime owner,mcpp 只有 generic contract knowledge。 +- 冻结 AUR managed set = {mcpp-bin}。 +- 冻结 mcpp-m package/AUR remote 不动。 + +### Stage 1 — 模板统一 + +- 提取共享 PackageSelector parser/resolver。 +- 将 add/template 统一为 bare → mcpplibs、dotted → exact namespace。 +- scaffold 保存完整 PackageId/provenance。 +- manifest 注入复用 add editor。 +- dotted namespace/template E2E。 +- pkg: list deprecation。 + +### Stage 2 — runtime 选择统一 + +- 引入 RuntimeSelection。 +- default 与 named SubOS 生成同一 RuntimeBinding snapshot。 +- build/run/test 共用 snapshot 和 contract hash。 +- 只从 root/workspace root 读取 SubOS;dependency/member 不向消费者传播。 +- active SubOS legacy warning。 + +### Stage 3 — graphics 收口 + +- mcpp-index 调整 ImGui/OpenGL/Vulkan features/templates 及 xlings 生态依赖。 +- xlings 基于 xim-pkgindex recipes 收口 GL/Vulkan runtime artifacts 和 sentinels。 +- 移除 host ICD farm 与 SubOS view symlink bridge。 +- mcpp 保持 generic planner/validator。 + +### Stage 4 — mcpp-bin 对账 + +- release manifest。 +- 单包 generator、dry-run 与 Arch validation。 +- event + schedule reconciler。 +- 从旧状态恢复到 latest stable,并验证真实安装。 + +这些 stage 是设计级依赖顺序,不是 implementation plan;每个 stage 仍需在设计获批后拆成独立可评审 PR。 + +## 11. 验收矩阵 + +### 11.1 模板 + +| 场景 | 期望 | +|---|---| +| pkg | mcpplibs:pkg + latest stable + default/single template | +| pkg@1.2.0 | default namespace + exact version + default/single template | +| ns.pkg:t | exact namespace ns + named template | +| ns.pkg@1.2.0:t | 完整解析,输出 exact PackageId/version/template | +| capi.lua / mcpplibs.capi.lua | 分别解析为 capi:lua / mcpplibs.capi:lua | +| add/template 相同 selector | 规范化为同一个 PackageId | +| 同 short name 不同 namespace | 不按 short name 注入/缓存碰撞 | +| 单 template、无显式 default | 自动选择该 template | +| 多 templates、无 default | 失败并列出 templates | +| 多 default | provider validation 失败 | +| prerelease only 且省略版本 | 失败并要求显式 version | +| 失败渲染/I/O | 目标目录不存在或事务回滚 | + +### 11.2 runtime + +| 场景 | 期望 | +|---|---| +| fresh HOME,无 [xlings] | bootstrap mcpp default,build/run 同 binding | +| named SubOS 存在 | build/run/test 使用该 contract | +| named SubOS 不存在 | hard error,不退回 default | +| active SubOS 与 default 不同 | 迁移期 warning;最终仍选择 default | +| 切换 subos | build fingerprint 改变,不复用旧 ABI objects | +| dependency 声明自己的 SubOS | 作为 dependency 时不传递、不覆盖 root 环境 | +| 同一源码在 el8/trixie 两个 root 中构建 | 各自在自己的 RuntimeBinding 下生成独立缓存/产物 | +| 预构建库 | 传递实际 artifact ABI requirement,不传递 SubOS name | +| Linux loader/libc 混源 | post-link 失败或严格迁移阶段明确 warning | +| macOS/Windows | 不执行 Linux glibc 规则 | + +### 11.3 graphics + +| Gate | 必须证明 | +|---|---| +| mcpp source ownership | 无 GPU vendor/ICD selection branch,只消费 xlings generic contract | +| xlings ownership | 根据 mcpp-index 依赖解析 xim recipes、provider/sentinel 与 provenance | +| mcpp-index static | OpenGL/Vulkan features、templates、platform dependencies 可解析 | +| Linux software GL | Xvfb/Wayland headless + llvmpipe 创建窗口/帧 | +| Linux native GL | AMD/Intel/NVIDIA/WSL 分别记录实际 provider | +| Vulkan | instance、physical device、ICD manifest/driver provenance | +| macOS | native framework/MoltenVK 路径 | +| Windows | Win32 backend 与 Vulkan provider 路径 | +| unsupported arch | 明确 capability error,不以 SKIP 计 pass | + +### 11.4 AUR + +| 场景 | 期望 | +|---|---| +| 新 stable release | mcpp-bin 在 SLO 内收敛 | +| 重复事件 | no-op,无新 commit | +| 迟到旧事件 | 不降级 | +| 两架构任一资产/hash 错 | push 前失败 | +| AUR maintenance | transient retry + schedule 补偿 | +| mcpp-m | package 文件和 AUR remote 均不变,不参与 mcpp-bin verdict | +| push 后 | git head、RPC version、clean install 全部验证 | + +## 12. 性能与简洁性约束 + +- TemplateSpec parse 为 O(length),不访问网络。 +- resolver 与 mcpp add 共用缓存与 IndexRoute,不增加第二轮全索引扫描。 +- RuntimeBinding 每次 configure 解析一次,contract hash 进入 snapshot;hot no-op 不启动 xlings。 +- mcpp build 热路径不探测 GPU、不执行 Vulkan/OpenGL 工具。 +- graphics provider 探测发生在 xim install/doctor 生命周期,并缓存 host fingerprint。 +- AUR schedule 在 desired == current 时只做轻量查询与一致性检查。 +- 不因这四项新增顶层 CLI 命令。 + +## 13. 错误与安全边界 + +- selector、version、template name 任一非法,在网络/install/目录创建前失败。 +- package descriptor 命中后必须校验声明的完整身份。 +- 模板为纯数据,不执行 provider hooks/scripts。 +- 生成在 sibling temp dir 完成,验证后原子 rename。 +- manifest 不接收项目级 absolute xlings binary/home。 +- named SubOS 不存在时不 fallback。 +- mcpp 不读取 host GPU library 目录来补齐索引缺口。 +- AUR 先验证全部 mcpp-bin assets,再加载 SSH secret/push。 +- AUR 只 fast-forward,不改写历史。 + +## 14. 第二轮 review 后已冻结的细节 + +1. namespace 可以省略;省略固定为 mcpplibs,写出 dotted namespace 时按 exact namespace 解释。 +2. 只有一个 template 且没有 default = true 时,该单模板自动成为 default。 +3. SubOS 暂时只允许通过 root/workspace-root mcpp.toml 选择,不增加 CLI override。 +4. SubOS 是本地 build/run 开发环境,不作为库的传递要求;不同项目可选择不同 SubOS/glibc。 +5. xlings 是 graphics stack/runtime owner;mcpp-index 声明 C++ 图形包到 xlings 生态的依赖,mcpp 只做 generic build。 +6. AUR 自动对账只面向 mcpp-bin;mcpp-m package 与 AUR remote 本阶段不动。 + +## 15. 与上一份综合设计的关系 + +本文仅在以下四处覆盖上位文档: + +| 上位文档建议 | 本文最终方向 | +|---|---| +| --template namespace:name@version --variant name | --template [ns.]name[@version][:tname];默认 ns=mcpplibs | +| mcpp 参与更宽的 graphics runtime 规划 | xlings 负责 graphics runtime;mcpp-index 声明依赖;mcpp 只消费通用契约 | +| runtime contract 为大范围跨仓主线 | McppDefault + existing [xlings].subos,且 SubOS root-local/non-transitive | +| mcpp-bin/mcpp-m 都进入自动 reconciler | 自动 reconciler 只管理 mcpp-bin;mcpp-m 不动 | + +未被本文覆盖的 C1–C9、artifact physics、identity 类型化与机器输出分析继续保留在上位文档中。 + +## 16. 当前实现证据锚点 + +| 主题 | 证据 | +|---|---| +| mcpp add dotted/colon selector | tests/e2e/12_add_command.sh:82-106src/pm/commands.cppm:75-143 | +| shared dotted candidate rules | src/pm/dependency_selector.cppm:78-120 | +| package identity spec | docs/spec/package-identity.md:150-169 | +| current TemplateSpec | src/scaffold/template.cppm:22-47 | +| scaffold short-name loss | src/scaffold/create.cppm:30-129 | +| template E2E | tests/e2e/69_package_templates.sh:94-194 | +| existing [xlings].subos | src/manifest/types.cppm:469-485src/manifest/toml.cppm:988-996 | +| root project environment materialization | src/build/prepare.cppm:1997-2009 | +| current runtime resolution | src/build/prepare.cppm:928-974 | +| user documentation | docs/05-mcpp-toml.md:946-978 | +| AUR workflow | .github/workflows/aur-publish.yml:14-102 | + +本文是待 review 设计,不应据此宣称上述行为已落地或关闭相关 issue。 diff --git a/.agents/docs/2026-08-09-mcpp-template-runtime-graphics-aur-implementation-plan.md b/.agents/docs/2026-08-09-mcpp-template-runtime-graphics-aur-implementation-plan.md new file mode 100644 index 00000000..cbeb4007 --- /dev/null +++ b/.agents/docs/2026-08-09-mcpp-template-runtime-graphics-aur-implementation-plan.md @@ -0,0 +1,408 @@ +# mcpp Template, Runtime, Graphics, and AUR Convergence Implementation Plan + +> **For agentic workers:** REQUIRED SUB-SKILL: Use superpowers:subagent-driven-development (recommended) or superpowers:executing-plans to implement this plan task-by-task. Steps use checkbox (`- [ ]`) syntax for tracking. + +**Goal:** 在一张 mcpp Draft PR 中落地已冻结的模板 selector、项目级 RuntimeSelection/RuntimeBinding、provider-neutral 图形运行时契约与仅管理 mcpp-bin 的 AUR reconciler;随后完成 review-gated 普通合入、release、mcpp-index/GitCode 对接和隔离环境下的 xlings 全生态验证。 + +**Architecture:** CLI 和 mcpp.toml 共用一个 exact dotted PackageSelector;scaffold 在任何落盘前解析出完整 PackageId/version/template,并在 sibling 临时目录事务生成。运行时只允许 McppDefault 或 root/workspace-root NamedSubos,两者解析成一次性 RuntimeBinding snapshot,进入构建指纹并由 build/run/test 复用,dependency/member 的 SubOS 不传递。mcpp-index 声明图形 RuntimeRequirement,xlings/xim 解析 provider,mcpp 仅记录 canonical requester/provider/artifact provenance 和执行平台通用链接/运行验证。GitHub Release manifest 是 AUR desired state,mcpp-bin reconciler 单调、幂等、校验实物并只做 fast-forward push。 + +**Tech Stack:** C++23 modules, mcpplibs.cmdline, nlohmann/json, TOML manifest parser, Bash/Python 3 release tooling, GitHub Actions, Arch makepkg, gh/git, xlings/xim/mcpp-index. + +## Global Constraints + +- [ ] 所有行为改动遵循 RED → GREEN → refactor;每个 RED 命令和预期失败原因写入提交/验证台账。 +- [ ] mcpp 实现只使用一张 PR,基于最新 `origin/main`;不直接 push main,不 force-push,不 amend/rebase 历史。 +- [ ] 所有改动先形成可审阅的独立 commit,立即 push 到 Draft PR,并追加验证/checkpoint 评论;不在提交、PR 或日志摘录中记录本地用户名、绝对工作区路径或私有信息。 +- [ ] Draft PR 经用户 review 后才可转 ready;不使用 admin/bypass,只有最新 PR HEAD 的 required checks 全部终态成功后才进行普通 merge。 +- [ ] `mcpp-m` 边界为字节级不变:不修改 `scripts/aur/mcpp-m/**`,不读取/生成/发布其内容,不访问其 AUR remote。 +- [ ] 不增加 `--variant`、`--subos` 或其他顶层 CLI;不在 mcpp 中出现 GPU vendor、Mesa、NVIDIA、WSL 或 ICD 选择分支。 +- [ ] SubOS 只来自本次构建 root/workspace root 的 `mcpp.toml`,不写入 dependency requirement/lock identity,不从 dependency/member 继承。 +- [ ] 平台物理规则放入 `src/platform/` 或已有平台模块;Linux ELF/glibc 校验在 macOS/Windows 明确 no-op,不用 Linux 推断替代原生 CI。 +- [ ] 所有 stateful 本地/生态验证使用隔离 `HOME`、`MCPP_HOME`、`XLINGS_HOME` 和 SubOS root;验证前后确认宿主状态未变化。 +- [ ] 只显式 stage 本计划列出的文件;保留主 checkout 中用户的未跟踪 issue triage 文档。 +- [ ] GitHub release 是主发布真源;AUR 暂时不可用时 GitHub release 不回滚,reconciler 留下可重试的精确失败分类。 + +--- + +## Task 1: Freeze Baseline, Issue, Branch, and Test Evidence + +**Files:** + +- Modify: `.agents/docs/2026-08-09-mcpp-template-runtime-graphics-aur-implementation-plan.md` +- Add: `.agents/docs/2026-08-09-mcpp-template-runtime-graphics-aur-validation.md` +- Preserve: `.agents/docs/2026-08-09-mcpp-template-runtime-graphics-aur-focused-design.md` +- Preserve: `.agents/docs/2026-08-09-xlings-mcpp-ecosystem-convergence-design.md` + +- [x] Create focused issue #398 referencing #397, #380, #392, and #396. +- [x] Create isolated worktree `feat/template-runtime-graphics-aur` from `origin/main@80291ca`. +- [x] Record baseline versions, latest releases, open PRs, and exact hashes for mcpp, xlings, mcpp-index, and xim-pkgindex in the validation ledger. +- [x] Run baseline `mcpp build`, unit suite, focused scaffold E2E, runtime E2E, AUR script tests (if any), and `git diff --check`; record any pre-existing failure without weakening later gates. +- [x] Snapshot hashes of `scripts/aur/mcpp-m/**` and the host xlings configuration for final boundary comparison. +- [x] Commit the approved design documents, implementation plan, and baseline validation ledger as the first explicit-files commit. + +## Task 2: Replace Candidate Guessing with One Exact PackageSelector + +**Files:** + +- Modify: `src/pm/dependency_selector.cppm` +- Modify: `src/pm/commands.cppm` +- Modify: `src/manifest/toml.cppm` +- Modify: `src/manifest/xpkg.cppm` +- Modify: `src/pm/dep_spec.cppm` +- Modify: `src/pm/index_route.cppm` +- Modify: `src/build/prepare.cppm` +- Modify: `src/cli.cppm` +- Modify: `tests/unit/test_pm_compat.cpp` +- Modify: `tests/unit/test_pm_index_route.cpp` +- Modify: `tests/unit/test_manifest.cpp` +- Modify: `tests/unit/test_pm_package_fetcher.cpp` +- Modify: `tests/e2e/12_add_command.sh` +- Modify: `tests/e2e/27_namespace_dependencies.sh` +- Modify: `tests/e2e/62_dotted_dependency_selector_priority.sh` +- Modify: `tests/e2e/63_bare_dependency_peer_root_priority.sh` +- Modify: `tests/e2e/78_test_main_combinations.sh` +- Modify: `tests/e2e/79_gtest_regular_dep_feature_main.sh` +- Modify: `tests/e2e/162_bare_name_namespace_scope.sh` +- Modify: `tests/e2e/165_bare_name_cross_namespace_wire_address.sh` +- Add: `tests/e2e/203_exact_selector_lock_migration.sh` +- Modify: `docs/05-mcpp-toml.md` +- Modify: `docs/zh/05-mcpp-toml.md` +- Modify: `docs/06-workspace.md` +- Modify: `docs/zh/06-workspace.md` +- Modify: `docs/spec/package-identity.md` + +**Interfaces:** + +- `PackageSelector { optional namespace; NameAtom name; string spelling; }` +- `parse_package_selector(string_view) -> expected` +- `normalize_package_selector(PackageSelector, defaultNs="mcpplibs") -> DependencyCoordinate` +- `format_package_selector(DependencyCoordinate) -> dotted selector` + +- [x] RED: add unit rows proving `lua -> (mcpplibs,lua)`, `capi.lua -> (capi,lua)`, `mcpplibs.capi.lua -> (mcpplibs.capi,lua)`, and rejection of empty/double-dot/control segments. +- [x] RED: update add E2E so `mcpp add capi.lua@5.4.7` probes/writes exact `capi:lua`, not `mcpplibs.capi:lua`; prove a same-short-name sibling cannot win. +- [x] Implement the O(length), no-I/O shared parser/normalizer and return one exact coordinate after default namespace filling. +- [x] Route `mcpp add`, dependency TOML parsing, feature dependency parsing, and xpkg dependency parsing through the same normalized coordinate. +- [x] Preserve already-parsed/locked identities; during the migration release, emit a warning only when an old dotted candidate exists and differs from the new exact coordinate, with both copyable selectors. +- [x] Update existence-gate and not-found diagnostics to show the normalized PackageId and explicitly mention default `mcpplibs` when namespace was omitted. +- [x] GREEN: run the focused unit and namespace/add E2E tests, then `git diff --check`. +- [x] Commit exact selector normalization and identity documentation. + +## Task 3: Make TemplateSpec Typed, Namespace-Aware, and Deterministic + +**Files:** + +- Modify: `src/scaffold/template.cppm` +- Modify: `src/scaffold/create.cppm` +- Modify: `src/cli/cmd_new.cppm` +- Modify: `src/cli.cppm` +- Add: `tests/unit/test_scaffold.cpp` +- Modify: `tests/e2e/69_package_templates.sh` +- Modify: `.github/workflows/ci-fresh-install.yml` + +**Interfaces:** + +- `TemplateSpec { PackageSelector package; optional version; optional templateName; bool legacyList; }` +- `parse_template_spec(string_view) -> expected` +- `ResolvedTemplatePackage { DependencyCoordinate id; string version; string indexRoute; string descriptorDigest; string payloadDigest; path root; }` +- `TemplateSelection { ResolvedTemplatePackage package; string templateName; }` + +- [x] RED: unit-test every valid and invalid grammar row from focused design §5.1, including multiple `:`/`@`, empty components, exact dotted namespaces, and `pkg:` legacy list recognition. +- [x] RED: add E2E packages with `mcpplibs:widget`, `acme:widget`, and `mcpplibs.capi:lua`; prove template resolution uses the exact PackageId through `IndexRoute::lookup_descriptor`. +- [x] Implement TemplateSpec parsing in the fixed order template delimiter → version delimiter → shared PackageSelector; reject before config/network access. +- [x] Replace scaffold short-name/compat probing with `IndexRoute` exact lookup and keep canonical namespace, version, route, descriptor digest, payload digest, and root through fetch/render/output. +- [x] Reuse package-manager semver resolution for latest stable; omitted versions must not select prereleases, while explicit exact prerelease remains allowed. +- [x] RED: test default selection rules: one explicit default wins; one sole non-default auto-wins; multiple without default list choices and fail; multiple defaults fail validation; no templates directory reports provider error. +- [x] Implement the default rules and `pkg:` one-release warning pointing to `--list-templates`. +- [ ] Update human and machine output to include canonical selector, resolved namespace/name/version, template, and runtime selection. +- [ ] GREEN: run new unit tests, template E2E, and three-platform fresh-install template lanes. +- [x] Commit typed TemplateSpec and exact template-package resolution. + +## Task 4: Make Scaffolding Safe and Transactional (#380) + +**Files:** + +- Add: `src/scaffold/project_name.cppm` +- Add: `src/platform/project_name.cppm` +- Add: `src/platform/scaffold_fs.cppm` +- Modify: `src/scaffold/template.cppm` +- Modify: `src/scaffold/create.cppm` +- Modify: `src/manifest/toml.cppm` +- Modify: `src/pm/commands.cppm` +- Modify: `tests/unit/test_scaffold.cpp` +- Modify: `tests/unit/test_manifest.cpp` +- Modify: `tests/e2e/69_package_templates.sh` +- Add: `tests/e2e/204_new_transactional_scaffold.sh` + +**Interfaces:** + +- `validate_project_name(string_view) -> expected` +- `render_tokens(string_view, RenderVars) -> expected` +- `ScaffoldTransaction::begin(parent,name)`, `commit()`, destructor rollback. + +- [x] RED: reject empty, absolute, separators, `.`, `..`, C0/DEL, Windows reserved device names, trailing dot/space, quote/tab and names containing the legacy `PROJECT` marker before target creation. +- [x] RED: prove inserted values containing placeholder-like text are not rescanned and all RenderVars render canonical project/template identities. +- [x] RED: inject read/write/copy/rename failures and assert neither final target nor sibling temporary directory remains. +- [x] Implement shared portable project-name validation with platform-specific reserved-name rules isolated under `src/platform/project_name.cppm`. +- [x] Replace repeated string substitution with a single-pass token renderer that never rescans inserted values. +- [x] Extend RenderVars with project namespace/qualified name and template package namespace/name/selector/version/template. +- [x] Replace substring dependency detection with structured manifest editing keyed by canonical PackageId; exact resolved version and features must be idempotent across same short names in different namespaces. +- [x] Generate builtin and package templates in a same-parent temporary directory, check every filesystem/stream operation, fsync/close as supported, and atomically rename only after validation. +- [x] GREEN: run scaffold unit/E2E tests, reproduce #380 cases with timeouts, and confirm no path escape or partial output. +- [ ] Commit transactional scaffold and close #380 from the final PR only after CI. + +## Task 5: Introduce Root-Local RuntimeSelection and One RuntimeBinding Snapshot + +**Files:** + +- Add: `src/xlings/runtime_selection.cppm` +- Add: `src/platform/runtime_binding.cppm` +- Modify: `src/build/prepare.cppm` +- Modify: `src/build/plan.cppm` +- Modify: `src/build/execute.cppm` +- Modify: `src/xlings/subos_info.cppm` +- Modify: `src/toolchain/model.cppm` +- Modify: `src/toolchain/detect.cppm` +- Modify: `src/toolchain/fingerprint.cppm` +- Inspect (no change required; policy is owned by `runtime_selection`): `src/project.cppm` +- Add: `tests/unit/test_runtime_selection.cpp` +- Modify: `tests/unit/test_subos_info.cpp` +- Modify: `tests/unit/test_fingerprint.cpp` +- Add: `tests/e2e/205_root_local_subos.sh` + +**Interfaces:** + +- `RuntimeSelection { enum Mode { McppDefault, NamedSubos }; string subosName; Source source; path ownerRoot; }` +- `RuntimeBinding { int schema; string providerId; string platform; string arch; string contractHash; optional loader; optional libc; vector libraryDirs; vector environment; vector capabilities; string provenance; }` +- `select_runtime(rootManifest, optional workspaceManifest, rootPath) -> expected` +- `resolve_runtime_binding(selection, compiler, GlobalConfig) -> expected` + +- [x] RED: no `[xlings].subos` selects `McppDefault` even if active SubOS differs; an explicit `subos = "default"` is a NamedSubos selection. +- [x] RED: missing named SubOS is a hard error and cannot fall back to default/active/compiler-baked runtime. +- [x] RED: workspace root SubOS overrides member declaration during workspace build; a member declaration applies only when that member is built independently. +- [x] RED: dependency manifest SubOS never merges into the consumer, lockfile, or cache identity; the same sources under el8/trixie produce distinct RuntimeBinding contract hashes. +- [x] Implement selection before workspace member substitution and preserve its owner root; materialize project xlings config from the root/workspace-root selection only. +- [x] Define McppDefault as the mcpp-managed default SubOS/runtime in configured xlings home, independent of xlings active/current symlink; bootstrap it when absent. +- [x] Read one RuntimeBinding snapshot, canonicalize/sort its fields, compute contractHash, store it in BuildContext/BuildPlan/cache metadata, and feed it to toolchain detection/fingerprint. +- [x] Reuse that snapshot for run/test environment; remove build/run fast-path re-reads of active SubOS. +- [x] Match current xlings `op=set` presence semantics: preserve any ambient value, including an explicitly empty value, while `prepend` remains ordered and de-duplicated. +- [x] On macOS/Windows return platform-native bindings without invented glibc/ELF fields; platform-specific derivation stays in `src/platform/runtime_binding.cppm`. +- [x] GREEN: run unit tests, named/default/workspace/dependency SubOS E2E, and compare binding/fingerprint output. +- [x] Commit root-local runtime selection and shared binding snapshot. + +## Task 6: Fix Runtime Payload Selection and Add Linux Artifact Physics (#392/#396) + +**Files:** + +- Modify: `src/toolchain/post_install.cppm` +- Modify: `src/toolchain/lifecycle.cppm` +- Add: `src/platform/elf_runtime.cppm` +- Modify: `src/platform/runtime_binding.cppm` +- Add: `src/build/runtime_validation.cppm` +- Modify: `src/build/ninja_backend.cppm` +- Modify: `src/build/execute.cppm` +- Modify: `src/doctor.cppm` +- Modify: `src/xlings/subos_info.cppm` +- Add: `tests/unit/test_elf_runtime.cpp` +- Add: `tests/e2e/206_runtime_binding_physics.sh` + +**Interfaces:** + +- `ElfRuntimeFacts { artifact; interp; runpaths; needed; requiredGlibcVersions; definedGlibcVersions; resolvedLibc; resolvedObjects; }` +- `validate_runtime_artifact(path, RuntimeBinding, RuntimeResolution) -> RuntimeVerdict` +- `RuntimeVerdict { Pass | ProvenMismatch | Inconclusive; diagnostics[]; }` + +- [x] RED: select the glibc payload named by RuntimeBinding, never the first directory entry; stale/absent payload is an explicit error. +- [x] RED: fixture ELFs prove Rule B rejects interpreter/libc from different payloads and accepts same-payload paths. +- [x] RED: fixture version tables prove a required GLIBC symbol floor above the selected libc exports is a hard proven mismatch; a lower/equal floor passes; unavailable closure data is inconclusive, not falsely green. +- [x] Implement semantic exact payload lookup in post-install fixup and bump the fixup revision so existing toolchains repair against the selected binding. +- [x] Implement internal ELF64 little-endian parsing for PT_INTERP, DT_RPATH/RUNPATH, DT_NEEDED and GNU version need/definition sections under `src/platform/elf_runtime.cppm`; no shell parsing on the build hot path. +- [x] Validate only newly linked Linux ELF outputs; cache the verdict by artifact stat/link fingerprint so hot no-op performs zero parses. +- [x] Emit canonical requester/provider/artifact paths and a copyable SubOS remediation; hard-fail proven Rule B/A mismatches, classify unresolvable host/hardware closure as inconclusive with an explicit diagnostic. +- [x] Ensure macOS/Windows validators compile to a typed no-op and never apply Linux glibc rules. +- [x] Extend doctor/runtime explanation to reuse stored verdict rather than re-probe or guess. +- [x] GREEN: run ELF unit fixtures, form-X E2E, current #392 reproduction shape, and a safe host-DSO control. +- [x] Commit runtime physics validation; only close #392/#396 if the final released E2E proves their exact acceptance cases. + +## Task 7: Carry Provider-Neutral Graphics Runtime Provenance + +**Files:** + +- Modify: `src/manifest/types.cppm` +- Modify: `src/manifest/toml.cppm` +- Modify: `src/manifest/xpkg.cppm` +- Modify: `src/build/plan.cppm` +- Modify: `src/build/flags.cppm` +- Modify: `src/build/prepare.cppm` +- Modify: `src/build/runtime_validation.cppm` +- Modify: `src/doctor.cppm` +- Modify: `src/platform/runtime_binding.cppm` +- Modify: `src/xlings/subos_info.cppm` +- Add: `tests/unit/test_runtime_contract.cpp` +- Modify: `tests/unit/test_link_model_runtime_dirs.cpp` +- Modify: `tests/unit/test_runtime_selection.cpp` +- Modify: `tests/unit/test_subos_info.cpp` +- Modify: `tests/e2e/62_runtime_library_dirs.sh` +- Modify: `tests/e2e/66_runtime_provides.sh` +- Modify: `tests/e2e/200_subos_env_reaches_program.sh` +- Modify: `tests/e2e/205_root_local_subos.sh` +- Add: `tests/e2e/207_runtime_contract_provenance.sh` +- Modify: `docs/05-mcpp-toml.md` +- Modify: `docs/zh/05-mcpp-toml.md` +- Modify: `docs/08-toolchain-internals.md` +- Modify: `docs/zh/08-toolchain-internals.md` + +**Interfaces:** + +- `RuntimeRequirement { kind; value; phase; canonical requester PackageId; required; }` +- `RuntimeArtifact { role; canonical provider PackageId; path; provenance; abi; digest; hostFingerprint; }` +- `LinkIntent { libraries; linkLibraryDirs; transitiveNeededDirs; runtimeSearchDirs; frameworks; deployFiles; }` + +- [x] RED: same-short-name providers in two namespaces remain distinguishable in runtime resolution JSON and `mcpp why runtime`. +- [x] RED: required capabilities and provided capabilities are separate; a requester cannot become its own provider merely because it requires a capability. +- [x] RED: runtime search dirs do not enter `-L`; Linux transitive-needed dirs use `-Wl,-rpath-link`, macOS emits rpath/install-name semantics, and Windows uses explicit deploy files. +- [x] Introduce structured generic requirement/artifact/provenance values while keeping legacy descriptor fields readable for one compatibility train. +- [x] Populate requester/provider from the resolved package identity, including namespace/version/index provenance; never use bare `package.name` as provider identity. +- [x] Write the resolved RuntimeBinding, requirements, artifacts, search mechanism, and validation verdict into `resolution.json`. +- [x] Make `mcpp why runtime` a pure interpreter of stored generic facts; GPU/driver diagnostics point to xlings and never probe hardware. +- [x] Add a static ownership gate that rejects new mcpp source branches containing provider-specific GPU/ICD selection vocabulary outside docs/tests. +- [x] GREEN: run runtime-contract unit/E2E tests and prove the build hot path launches no GL/Vulkan probe. +- [x] Commit provider-neutral runtime contract and link-intent separation. + +## Task 8: Generate an Immutable Release Manifest + +**Files:** + +- Add: `scripts/release/generate_manifest.py` +- Add: `tests/scripts/test_release_manifest.py` +- Modify: `.github/workflows/release.yml` +- Modify: `docs/09-release.md` +- Modify: `docs/zh/09-release.md` + +**Interface:** + +- Release asset `mcpp-release.json` schema 1 with version, tag, release commit, and exact name/SHA256 for Linux x86_64/aarch64 plus every shipped platform asset. + +- [x] RED: fixtures reject duplicate platform/arch rows, missing sidecars, mismatched hashes, draft/prerelease input, wrong tag/version, and non-deterministic ordering. +- [x] Implement deterministic manifest generation from downloaded release artifacts and sidecars, recomputing every SHA256. +- [x] Wire the release workflow so the manifest is uploaded only after all required release assets exist and validation passes. +- [x] Add a release gate that downloads the uploaded manifest and compares it to the final GitHub release inventory. +- [x] GREEN: run manifest tests and a local fixture generation twice with byte-identical output. +- [x] Commit immutable release desired-state manifest. + +## Task 9: Replace Combined AUR Publishing with an mcpp-bin Reconciler + +**Files:** + +- Add: `scripts/aur/reconcile_mcpp_bin.py` +- Add: `scripts/aur/render_mcpp_bin.py` +- Add: `tests/scripts/test_aur_reconcile.py` +- Modify: `scripts/aur/update.sh` +- Modify: `scripts/aur/README.md` +- Modify: `.github/workflows/aur-publish.yml` +- Preserve byte-for-byte: `scripts/aur/mcpp-m/**` + +**Interface:** + +- `inspect -> DesiredState/ObservedState/ReconcilePlan` +- exit classification `noop | updated | transient | permanent | refused-downgrade` +- manual inputs `publish=false|true`, optional exact latest stable tag only; no downgrade override in this phase. + +- [x] RED: fixture tests cover desired==observed no-op, upgrade, late old event refusal, missing/hash-mismatched asset, AUR maintenance retry, auth/permanent failure, RPC lag after git update, and known clone failure not becoming first publish. +- [x] RED: assert the reconciler never opens, hashes, copies, stages, or addresses `scripts/aur/mcpp-m/**`; compare pre/post tree hashes. +- [x] Split `update.sh` into an mcpp-bin-only compatibility wrapper around the renderer; remove all mcpp-m reads/writes without editing mcpp-m files. +- [x] Render PKGBUILD from `mcpp-release.json`; regenerate `.SRCINFO` with non-root Arch `makepkg --printsrcinfo`, then `makepkg --verifysource`. +- [x] Query AUR RPC and HTTPS git, compare versions with Arch `vercmp`, validate both Linux assets/sidecars, and produce a dry-run diff before secrets are loaded. +- [x] Publish only by normal fast-forward SSH push with pinned AUR host key and bounded exponential retry for maintenance/timeouts; never force or initialize a missing known package. +- [x] Verify remote git head, bounded-poll RPC, then install in a clean Arch container and assert `mcpp --version`. +- [x] Change workflow triggers to successful release workflow_run + six-hour schedule + workflow_dispatch; all call the same latest-stable reconciler. Remove the mcpp-m publish leg. +- [x] Emit Actions summary with trigger, desired/observed versions, hashes, remote commit, retry count, classification, and drift age; AUR failure must not alter GitHub release conclusion. +- [ ] GREEN: run Python tests, shell lint, dry-run against current latest release, and an Arch container source verification. +- [x] Commit mcpp-bin-only AUR reconciliation and verify mcpp-m byte hashes are unchanged. + +## Task 10: Pin Latest xlings, Version mcpp, and Update User Documentation + +**Files:** + +- Modify: `src/xlings.cppm` +- Modify: `.github/actions/bootstrap-mcpp/action.yml` +- Modify: `.github/workflows/bootstrap-macos.yml` +- Modify: `.github/workflows/ci-linux-e2e.yml` +- Modify: all other authoritative xlings pin sites found by `tests/unit/test_xlings_version_pin.cpp` +- Modify: `mcpp.toml` +- Modify: `README.md` +- Modify: `docs/00-getting-started.md` +- Modify: `docs/zh/00-getting-started.md` +- Modify: `docs/05-mcpp-toml.md` +- Modify: `docs/zh/05-mcpp-toml.md` +- Modify: `docs/08-toolchain-internals.md` +- Modify: `docs/zh/08-toolchain-internals.md` +- Modify: `docs/spec/package-identity.md` +- Modify: `scripts/aur/README.md` + +- [x] Re-query latest non-draft/non-prerelease xlings immediately before pinning; pin the exact version and verify every authoritative pin site matches. +- [x] Choose the next unused calendar version (expected `2026.8.9.1` after live tag check), update `mcpp.toml`, and leave AUR snapshots to release-time generation. +- [x] Document `[ns.]name[@version][:tname]`, default `mcpplibs`, exact dotted namespaces, sole-template default, and legacy list migration. +- [x] Document McppDefault vs root/workspace-root `[xlings].subos`, no CLI override, non-transitive dependency semantics, coexistence across glibc bindings, and prebuilt ABI metadata boundary. +- [x] Document xlings/xim ownership of OpenGL/Vulkan providers and that mcpp never probes GPU/driver/ICD. +- [x] Document mcpp-bin-only eventual AUR reconciliation and explicitly state mcpp-m/mcpp-git are outside automation. +- [x] Update English and Chinese examples together; regenerate command reference if CLI help changed. +- [ ] GREEN: run pin tests, docs example tests, generated command reference tests, `git diff --check`, and forbidden-vocabulary/boundary scans. +- [ ] Commit version, xlings pin, and documentation. + +## Task 11: Full Local Validation and PR Publication + +**Files:** + +- Modify: `.agents/docs/2026-08-09-mcpp-template-runtime-graphics-aur-validation.md` + +- [ ] Build a fresh mcpp binary with the pinned xlings in isolated homes; do not validate with a stale installed mcpp. +- [ ] Run the complete unit suite and every applicable Linux E2E, then focused no-cache workspace/self-host/release builds. +- [ ] Run scaffold/template, default/named/workspace SubOS, runtime physics, runtime provenance, release-manifest, and AUR reconcile gates independently and record commands/results. +- [ ] Run sanitizers/static checks available in repository CI, `git diff --check origin/main...HEAD`, forbidden mcpp-m diff, and explicit scope inventory. +- [ ] Measure hot no-op build and template parse against baseline; require no material regression and zero xlings/GPU subprocess on hot no-op. +- [ ] Rebase is forbidden; if origin/main advanced, merge origin/main normally, rerun all affected gates, and preserve visible history. +- [x] Push the branch and open one Draft PR with `Closes #398`, related issue notes, architecture decisions, RED/GREEN evidence, boundary proof, version/pin, and cross-repo follow-up plan. +- [ ] After every logical commit, push immediately and append a checkpoint with commit id, focused evidence, remaining failures, and cross-repository boundary. +- [ ] Request review using the repository review workflow and address feedback with fresh evidence. + +## Task 12: Native GitHub Actions, Review-Gated Merge, and Release + +**Skills required at this task:** `mcpp-release`, `superpowers:verification-before-completion`, and `superpowers:finishing-a-development-branch`. + +- [ ] Wait for all latest-HEAD checks to reach terminal success across Linux, macOS, Windows, cross/QEMU aarch64, native aarch64, fresh install, release manifest, and AUR dry-run lanes. +- [ ] Treat skipped native/runtime hardware rows as NOT_EXERCISED, not pass; create an isolated temporary CI lane if a required platform gate is absent. +- [ ] Verify remote PR HEAD equals local HEAD, branch diff is clean, no unresolved reviews, no conflict, and `scripts/aur/mcpp-m/**` is unchanged. +- [ ] After explicit user review, mark the Draft ready and use the repository's normal merge path without admin/bypass; verify the merge commit is on main and PR/issue states are correct. +- [ ] Follow `mcpp-release`: verify version strings, create/push the release tag, monitor every release job, asset, checksum, `mcpp-release.json`, GitCode mirror, and GitHub release until terminal success. +- [ ] Do not block/rollback the GitHub release for an AUR transient; dispatch the fixed latest-stable mcpp-bin reconciler and record its exact terminal state. +- [ ] Close #380/#392/#396 only when their released acceptance paths are proven; otherwise comment with delivered subset and keep the residual open. + +## Task 13: mcpp-index, GitCode Resources, and xlings Ecosystem Follow-Through + +**Repositories:** + +- `mcpplibs/mcpp-index` +- `openxlings/xim-pkgindex` +- `openxlings/xlings` +- GitCode helper/config already used by the release workflow + +- [ ] In isolated repo-specific worktrees, update mcpp-index minimum/current mcpp pins and template descriptors to canonical selectors and sole/default template lint. +- [ ] Refactor ImGui features/templates to core/headless, backend-glfw-opengl3, backend-vulkan, app, docking, and viewports without making core pull graphics unconditionally. +- [ ] Express OpenGL/Vulkan runtime requirements through generic xlings/xim package/capability dependencies and canonical provider identities; remove mcpp-index host ICD/DSO collection paths. +- [ ] In xim-pkgindex, add/fix Vulkan loader/ICD and host-link provider recipes/provenance only where current recipes cannot satisfy the new generic contract; keep provider detection/lifecycle outside mcpp. +- [ ] Use the established index CI workflow, one repo PR per external repository as required by repository boundaries, and wait for every latest-head validation job before merge. +- [ ] Confirm the mcpp release job opens/merges the mcpp index version bump and local GitCode resources contain the exact new release assets plus SHA256 sidecars; verify GitCode with ranged GET and full SHA256. +- [ ] Publish/rebuild index artifacts and verify their public hashes/refs. +- [ ] From fresh isolated HOME/MCPP_HOME/XLINGS_HOME, install the released mcpp and pinned/released xlings, run `xlings update`, canonical template new/build/run/test, default runtime, two named SubOS/glibc builds, and package lifecycle smoke. +- [ ] Run software OpenGL and Vulkan instance/device lanes where hardware exists; report AMD/Intel/NVIDIA/WSL/macOS/Windows hardware rows as PASS/FAIL/NOT_EXERCISED with actual provider/ICD provenance. +- [ ] Verify mcpp does not probe GPU and the same graphics dependencies resolve through xlings/xim under each exercised environment. +- [ ] Record all PRs, commits, CI run/job IDs, release tags, asset hashes, GitCode URLs, AUR state, and public smoke results in the validation ledger. + +## Task 14: Completion Audit and Final Report + +- [ ] Verify mcpp main/release/index heads, tags, PR merge states, issue comments/states, AUR mcpp-bin desired/observed state, and public artifact hashes from live sources. +- [ ] Verify local worktrees are clean and main checkout user files remain untouched. +- [ ] Re-run `git diff --check`, version/pin consistency, mcpp-m boundary hash, and one final cold-home released-binary smoke. +- [ ] Mark every checkbox with evidence; do not convert NOT_EXERCISED hardware rows into green. +- [ ] Update the active goal to complete only when no required item remains. +- [ ] Report concise core outcomes plus links to the implementation plan, validation ledger, issue, PR, CI, release, index PRs/artifacts, GitCode, AUR and ecosystem evidence. diff --git a/.agents/docs/2026-08-09-mcpp-template-runtime-graphics-aur-validation.md b/.agents/docs/2026-08-09-mcpp-template-runtime-graphics-aur-validation.md new file mode 100644 index 00000000..14e95f37 --- /dev/null +++ b/.agents/docs/2026-08-09-mcpp-template-runtime-graphics-aur-validation.md @@ -0,0 +1,408 @@ +# mcpp Template, Runtime, Graphics, and AUR Validation Ledger + +> Started: 2026-08-09 Asia/Shanghai +> +> Implementation issue: https://github.com/mcpp-community/mcpp/issues/398 +> +> This ledger separates baseline, RED, GREEN, CI, release, and public-ecosystem evidence. A running, skipped, cancelled, or superseded job is never recorded as pass. + +## 1. Scope and repository boundaries + +The mcpp implementation is developed in one branch and one pull request: + +- worktree: isolated feature worktree; local path intentionally omitted +- branch: `feat/template-runtime-graphics-aur` +- base: `mcpp-community/mcpp main@80291ca01a982c1e8c00e43bfa97ffe68516e6d7` +- focused issue: `mcpp-community/mcpp#398` +- umbrella issue: `mcpp-community/mcpp#397` + +External repositories retain their own history and review boundaries. Their implementation, if required after the mcpp release, uses separate repository PRs; it does not create a second mcpp implementation PR. + +The implementation worktree was created from `origin/main`; pre-existing checkout state was left untouched. + +## 2. Live baseline + +Captured after fresh `git fetch origin main --prune` on 2026-08-09: + +| Repository | local HEAD | `origin/main` | local state | latest release | +|---|---|---|---|---| +| `mcpp-community/mcpp` | `80291ca01a982c1e8c00e43bfa97ffe68516e6d7` | same | isolated feature worktree clean before docs | `v2026.8.8.4`, published 2026-08-08 10:51:31Z | +| `openxlings/xlings` | `2913a0949af3b26192a6c1d8f12b78f976bea6b8` on `feat/version-grammar` | `f203b6b9a5e3e0d3f707468ed56cdb8b50cc7acc` | dirty user checkout; must use a new worktree | `v2026.8.9.2`, published 2026-08-08 20:38:57Z | +| `mcpplibs/mcpp-index` | `b86fc7c0c80a93f4ccdf797c13ec0cca2557d6eb` | same | clean main | no GitHub release list | +| `openxlings/xim-pkgindex` | `576ef09b69becefca00de8c94c6ba17d5cdb1ee4` | `e8029381beb2e0c83c4ec4318f01bd18c523890a` | local main behind; must use a new worktree | no GitHub release list | + +Open PR inventory at capture time: + +- mcpp: #387, #372 (Draft), #353, #351 (Draft), #272. +- xlings: none. +- mcpp-index: #150 (Draft), #61. +- xim-pkgindex: none. + +Local executables visible before the isolated cold build: + +- PATH mcpp: `mcpp 2026.8.6.2`. +- latest existing source-built binary: `mcpp 2026.8.8.4` at `target/x86_64-linux-gnu/09887d532ce30543/bin/mcpp` in the main checkout. +- PATH xlings: `xlings 2026.8.9.2`. + +The release version proposed for this implementation is `2026.8.9.1`; the tag is re-queried immediately before changing `mcpp.toml`. + +## 3. Immutable boundary snapshots + +Approved design hashes: + +- focused design: `e06821b73102049e1f2184a2967e5dd81dfb6f05389a763dad326d139d0e3f25` +- parent convergence design: `ad93db80a3222f6536629a15d9b5ba430b222e179b91de01131c619ecbd3bd65` +- initial implementation plan: `aea2963f593ef00e71b2876f52a773014a28098f513f82260063dcd336acb09e` + +`scripts/aur/mcpp-m/**` baseline: + +| File | SHA256 | +|---|---| +| `.SRCINFO` | `9d8b8279aadfa6b1915850fbbfe33c2ef237456f94dd706e726e3169b60be0f8` | +| `PKGBUILD` | `545fe0de51f0cf7e5979871ba8f8d6cd3233c485338a9768f57e53c0a6f8fa7f` | +| `mcpp.sh` | `cbab68984b02c415f8ae42bf9417647b842e900fbf7bf292067e4a384b924f8f` | +| sorted aggregate | `afb8a647e04483a86985119e07086016f49d55f177ee6257094c336d226113c6` | + +Host xlings configuration snapshot: + +- scope: sorted SHA256 rows for `$XLINGS_HOME/.xlings.json` and `$XLINGS_HOME/subos/*/.xlings.json`; contents were not copied. +- aggregate: `f218aadf3792ee815c8535ce0ca0bb53f634fecbdbc5d0d47db442052d786d1b`. + +All stateful verification uses a separate temporary root and must reproduce both aggregate hashes before completion. + +## 4. Baseline verification + +Isolated root: disposable task root; exact local path intentionally omitted. + +| Gate | Command shape | Status | Evidence | +|---|---|---|---| +| cold source build | isolated `MCPP_HOME`, `XLINGS_HOME`, vendored xlings 2026.8.9.2; `mcpp build --no-cache` | PASS | cold bootstrap installed `glibc@2.44` and resolved `gcc@16.1.0`; release build finished in 64.80s; binary `target/x86_64-linux-gnu/72abd390cce53924/bin/mcpp` reports `2026.8.8.4` | +| unit suite | fresh baseline binary, isolated homes; `mcpp test` | PASS | `68 passed; 0 failed`, 89.06s total | +| template E2E | isolated homes; `MCPP= bash tests/e2e/69_package_templates.sh` | PASS | exit 0, `OK` | +| runtime E2E | isolated homes; scripts 74, 166 and 201 | PASS with one fixture rerun | #74 proved payload glibc 2.44 PT_INTERP; #166 proved private glibc env inclusion/exclusion; #201 passed both sysroot/payload-first modes after using a non-`/tmp` isolated HOME | +| AUR tests | inventory + `bash -n scripts/aur/update.sh` | PASS for syntax; coverage absent | baseline has no reconciler tests; Task 9 adds fixture coverage | +| diff check | `git diff --check origin/main...HEAD` before edits | PASS | no tracked diff before documentation was added | + +### Baseline fixture diagnosis + +The first #201 run exited 1 because the isolated `MCPP_HOME` itself was below the system temporary directory. The test intentionally rejects every RUNPATH beginning with that directory as evidence of another temporary home, so it classified the current, valid isolated home as pollution. The observed two paths were the current glibc and gcc payload directories, not a stale foreign home. + +The hypothesis was tested by hard-link cloning the disposable state into a non-temporary isolated root, letting the normal fixup rebind paths, and running the unmodified script again. It exited 0 and proved both link modes load nothing from the host. No product or test source was changed for this baseline result. + +## 5. RED/GREEN ledger + +Each behavior entry is appended with: + +1. exact test command; +2. RED exit code and why the old code failed; +3. production change; +4. GREEN exit code and relevant assertions; +5. refactor/full-gate result. + +### 5.1 Exact PackageSelector and dependency identity (Task 2) + +| Gate | RED evidence | Production change | GREEN evidence | +|---|---|---|---| +| shared parser | Selector tests initially could not compile because `PackageSelector` / `parse_package_selector` did not exist; unsafe-segment rows then exposed missing character validation | Added one O(length), no-I/O parser, default `mcpplibs` normalization, dotted formatter, and diagnostic-only legacy coordinate helper | `test_pm_compat`: 17 tests passed, including bare/dotted/nested/invalid rows | +| manifest/xpkg parsing | Updated manifest expectations produced 3 failures under ordered-candidate behavior; invalid TOML/xpkg selectors were previously not hard parse failures | Routed direct, nested, feature, target and xpkg dependency inputs through the shared exact parser | `test_manifest`: 148 passed; `test_pm_index_route`: 7 passed; `test_pm_package_fetcher`: 10 passed | +| add/exact sibling | `12_add_command.sh` first showed `capi.lua` written/resolved with the old default-prefix candidate and no migration diagnostic; a later RED left both legacy `"acme.util"` and canonical `[dependencies.acme] util` rows | `mcpp add` now accepts canonical `[ns.]name@version`, retains `ns:name` as a warned one-release alias, probes only one PackageId, removes an equivalent legacy flat row, and writes/upserts the canonical table | `12_add_command.sh`: `capi.lua` cannot be stolen by `(mcpplibs.capi,lua)`; warning names both exact selectors; legacy source shape becomes one canonical row; malformed input leaves TOML byte-stable | +| scoped remove | Added same-name `[dev-dependencies]` before `[dependencies]`; RED removed the dev row and left the regular row | Limited flat removal to the exact `[dependencies]` body and kept nested namespace removal exact | Rebuilt source binary; the same E2E preserves the dev row and removes only the regular dependency | +| exact build miss | `162_bare_name_namespace_scope.sh` initially passed an exact miss into install-time compatibility retries | Every version dependency is identity-validated before install; miss reports the exact coordinate and did-you-mean remains diagnostic-only | `162`: bare gtest stays `(mcpplibs,gtest)` while explicit `(compat,gtest)` resolves; `165` proves the descriptor supplies the exact wire namespace | +| lock migration | New `203_exact_selector_lock_migration.sh` first failed because Form-B synthesis received the ambiguous manifest map key instead of the resolved short name | Existing v2 lock identity anchors an old dotted selection for one release train; unlocked selection never falls back; Form-B synthesis uses canonical short name | `203`: warning names old/new selectors, build/run use the locked old identity, rewritten lock retains its namespace | + +Task 2 refactor/full gates on the isolated source binary +`target/x86_64-linux-gnu/aee81584bf66d3f8/bin/mcpp`: + +- `mcpp build --no-cache`: PASS, release build completed in 64.12s. +- `mcpp test --no-cache`: PASS, **68 passed; 0 failed**, 86.37s. +- focused E2E set `12, 27, 62, 63, 78, 79, 162, 165, 203`: all PASS/OK. +- changed shell scripts `bash -n`: PASS. +- `git diff --check`: PASS. + +### 5.2 Typed exact TemplateSpec and deterministic provider selection (Task 3) + +| Gate | RED evidence | Production change | GREEN evidence | +|---|---|---|---| +| grammar | New `test_scaffold` could not build because typed `parse_template_spec` / `select_template` APIs did not exist | Added fixed-order `:` then `@` parsing over the shared PackageSelector; exact safe version keys; atomic tname validation; one-release `pkg:` list marker | 9 scaffold tests pass for every valid design row and empty/double-delimiter/range/unsafe failures | +| exact provider identity | Updated E2E is red on the baseline binary at the first new exact-identity diagnostic; baseline implementation probes empty/compat short names and cannot retain an explicit foreign/nested PackageId | Registry template fetch now uses one normalized coordinate through `IndexRoute::lookup_descriptor`; no short-name or compat probe remains | E2E same-short `(mcpplibs,tpl-demo)` / `(acme,tpl-demo)` selects acme exactly; nested `(mcpplibs.capi,lua)` survives output and injected selector | +| deterministic version | Baseline accepted only its ad-hoc version flow and exposed no payload provenance | Shared SemVer resolver chooses latest stable, explicit prerelease is validated, indirect aliases are refused, and xpkg version entries retain declared SHA256 | E2E omits `2.0.0-rc.1` for default selection, accepts its exact pin, rejects `latest`, rejects unpublished `9.9.9`, and prints descriptor/payload provenance | +| default template | Old code failed whenever no `default=true` existed, including a package with exactly one template | Shared selection implements sole-template auto-default, unique explicit default, explicit tname, and deterministic provider errors | Unit/E2E cover sole auto-default, multiple ambiguous choices, duplicate defaults, missing templates directory, and explicit disambiguation | +| migration/CI surfaces | Legacy `pkg:` silently meant listing and native fresh-install only exercised a bare short name | `pkg:` warns with copyable `--list-templates`; fresh-install Linux/macOS/Windows lanes now use explicit `mcpplibs.imgui` and assert resolved identity | Local legacy-list E2E passes; native lanes are configured and remain pending PR/latest-head CI evidence | + +Task 3 local gates on source binary +`target/x86_64-linux-gnu/94da92f90aedbe7f/bin/mcpp`: + +- source build: PASS, full release rebuild completed in 72.92s. +- `mcpp test --no-cache`: PASS, **69 passed; 0 failed**, 33.07s. +- `test_scaffold`: 9/9 behavior rows PASS; `test_manifest`: 148/148 PASS. +- `69_package_templates.sh`: PASS with exact/default/nested/alias/provider/wire assertions. +- `bash -n tests/e2e/69_package_templates.sh` and `git diff --check`: PASS. + +### 5.3 Portable, durable, transactional scaffolding (Task 4 / #380) + +| Gate | RED evidence | Production change | GREEN evidence | +|---|---|---|---| +| portable project name | New scaffold tests failed to compile because `mcpp.scaffold.project_name` did not exist | Added an ASCII package/directory validator and isolated Windows device-basename policy under `mcpp.platform.project_name`; `cmd_new` validates before config/index/network/staging work | Unit rows accept bare/qualified names and reject empty, absolute, separators, dot components, C0/DEL, Windows-invalid/device names, trailing dot/space, and the legacy `PROJECT` marker | +| single-pass rendering | Baseline only repeatedly replaced the literal `PROJECT`/two legacy variables | Added the complete eight-field `RenderVars` vocabulary, strict unknown/unterminated-token diagnostics, and append-only single-pass rendering | Unit test proves every canonical project/provider identity renders and an inserted `{{template.name}}` sequence is not rescanned; E2E renders qualified/nested identities | +| exact dependency edit | New manifest-editor tests failed to compile because `upsert_dependency_text` did not exist; baseline scaffold used substring presence/insertion | Added one parsed, source-preserving dependency editor shared by `mcpp add` and scaffold injection; it writes default tables or `[dependencies.]` plus short key and reparses/verifies the exact PackageId/version/features | `test_manifest` 150/150; same-short `compat.widget` survives while exact `acme.widget` is injected idempotently; add E2E remains green | +| transaction and I/O | Baseline created the final directory before rendering and ignored several stream/filesystem results; exclusive platform rename API was absent | Both builtin/package paths write a same-parent `.mcpp-new-*` tree, close+sync files, validate the complete manifest, then commit with Linux `renameat2(RENAME_NOREPLACE)`, macOS exclusive rename, or Windows write-through no-replace move | 18 scaffold tests cover rollback plus deterministic read/write/copy/rename failures; E2E 204 covers path escape, unknown token, type collision, symlink rejection, no final tree, and no staging residue | + +Task 4 focused local evidence on source binary +`target/x86_64-linux-gnu/94da92f90aedbe7f/bin/mcpp`: + +- source build: PASS, release rebuild completed in 47.89s after the shared manifest editor change. +- full unit suite: PASS, **69 test binaries passed; 0 failed**, 16.77s cached refactor gate. +- `test_scaffold`: **18/18** behavior tests PASS; `test_manifest`: **150/150** PASS. +- `02_new_build_run.sh`: PASS, generated builtin project builds and runs. +- `12_add_command.sh`, `69_package_templates.sh`, and `204_new_transactional_scaffold.sh`: all PASS/`OK`. +- `mcpp-m` aggregate remains `afb8a647e04483a86985119e07086016f49d55f177ee6257094c336d226113c6`. +- host xlings config aggregate remains `f218aadf3792ee815c8535ce0ca0bb53f634fecbdbc5d0d47db442052d786d1b`. +- macOS/Windows native compilation and filesystem semantics remain pending latest-HEAD PR CI; they are not inferred from this Linux run. + +### 5.4 Root-local RuntimeSelection and immutable RuntimeBinding (Task 5) + +| Gate | RED evidence | Production change | GREEN evidence | +|---|---|---|---| +| two-mode selection | `test_runtime_selection` first failed to compile because `mcpp.xlings.runtime_selection` did not exist | Added presence-preserving manifest parsing and one pure `McppDefault`/`NamedSubos` selector with portable name validation and an explicit owner root | 10 runtime selection/binding tests pass; absence, explicit `default`, workspace override, standalone member and invalid names are distinct | +| exact binding | Binding tests then failed to compile because `mcpp.platform.runtime_binding` did not exist | Resolves only the configured default or exact root-owned named SubOS; missing directory, missing block, empty runtime and incompatible schema are hard errors; canonical provider/runtime/env snapshot gets a stable contract hash | Unit tests prove missing named cannot fall back, same-content `el8`/`dev` hashes differ, and serialized cache round-trip verifies its hash | +| lifecycle/cache | Baseline execute path derived a compiler-owned SubOS, honored `MCPP_SUBOS_DIR`, and re-read `.xlings.json` independently on full/fast run | BuildContext/BuildPlan/toolchain/fingerprint/cache carry one snapshot; fast paths require it and invalidate on contract mtime; full and cached run/test resolve environment only from that snapshot | E2E 200 proves first/cached run equality, ignored legacy override, mutation-triggered rebuild, old-cache miss and missing-contract error | +| root-local scope | A member/dependency declaration could become the manifest visible after workspace substitution or nested prepare | Selection happens before member substitution; workspace root materializes its own xlings config; dependency/tool sub-builds inherit the consumer snapshot and never consult their manifest SubOS | E2E 205 proves active-state independence, explicit-default identity, workspace-root ownership, standalone-member ownership and non-transitive path dependency behavior | +| xlings env semantics | New tests showed `set` overwrote both a caller value and an explicitly empty variable | `set` now follows xlings presence semantics; `prepend` retains ordered element-wise de-duplication | `test_subos_info`: 17/17 pass, including caller/empty preservation | + +Task 5 local gates on source binary +`target/x86_64-linux-gnu/e49880a389812d1b/bin/mcpp`: + +- source build: PASS, incremental release rebuild completed in 47.59s. +- full unit suite: PASS, **70 test binaries passed; 0 failed**, 87.78s. +- `test_manifest`: **151/151**; `test_runtime_selection`: **10/10**; `test_subos_info`: **17/17**; `test_fingerprint`: **8/8**. +- E2E `02`, `88`, `200`, `205`, `12`, `69`, and `204`: all PASS/`OK`. +- changed shell scripts `bash -n` and `git diff --check`: PASS. +- `mcpp-m` aggregate remains `afb8a647e04483a86985119e07086016f49d55f177ee6257094c336d226113c6`. +- host xlings config aggregate remains `f218aadf3792ee815c8535ce0ca0bb53f634fecbdbc5d0d47db442052d786d1b`. +- Linux behavior is locally exercised. The platform-native branch stores `runtimeId` while leaving `libc`/ELF fields empty off Linux; native macOS/Windows compilation remains a latest-HEAD PR CI gate, not a local claim. + +### 5.5 Exact runtime payload and Linux ELF physics (Task 6 / #392 / #396) + +| Gate | RED evidence | Production change | GREEN evidence | +|---|---|---|---| +| exact payload | `test_elf_runtime` initially failed to compile because `mcpp.platform.elf_runtime` did not exist; the old post-install helper returned the first loader-bearing glibc directory | Added exact `glibc@` parsing/lookup, explicit malformed/missing/stale errors and fixup revision `hermetic-4-exact-runtime`; RuntimeBinding now carries canonical loader/lib dir and optional `host_glibc` | RuntimePayload unit rows prove 2.44 wins with 2.39 also installed and every malformed/absent identity refuses instead of falling back | +| internal ELF facts | The same RED had no typed artifact facts or GNU version-table reader | Added bounded ELF64-LE parsing of PT_INTERP, PT_DYNAMIC, DT_RPATH/RUNPATH/NEEDED and GNU verneed/verdef, preserving loader search order and invoking no shell tool | Synthetic stripped-shape fixture returns exact interpreter, ordered runpaths, needed soname, `GLIBC_2.40` need and `GLIBC_2.44` definition; text/truncated inputs refuse | +| Rule B | No post-link seam compared the emitted interpreter with the libc actually resolved through the closure | Closure resolution canonicalizes artifacts/providers and rejects host/private or multi-private libc mixtures against the selected RuntimeBinding | Unit rows reject interpreter/libc and transitive two-libc mixtures and accept one selected payload; E2E 206 forces host PT_INTERP + private libc and gets a canonical Rule-B hard failure | +| Rule A | No code compared a host DSO's GNU symbol floor with the selected libc exports | Validator compares every requester `GLIBC_*` need to the selected libc definitions; proven higher floors fail, missing closure data is typed inconclusive | Unit rows reject 2.42 > 2.39, accept equal/lower, and keep missing GPU closure inconclusive; E2E 206 links real host `libtinfo` under glibc 2.44 as the safe #392-shaped control | +| cache/doctor | A hot fast path could not know whether its artifact had ever been checked; doctor reparsing would observe a different current host | Stored verdict is keyed by artifact stat plus RuntimeBinding contract; hot paths require current PASS and compare pre/post-Ninja stats; mismatch/inconclusive records remain sticky without reparsing; doctor reads the stored record | Self-host artifact verdict is `pass`; bare build finishes in 0.01s with artifact/verdict stat unchanged; E2E 206 proves PASS no-op, cached mismatch remains red with unchanged verdict, and doctor prints the stored PASS | +| platform boundary | Linux-specific rules had no typed cross-platform seam | Validator compiles as a typed no-op off Linux; RuntimeBinding leaves ELF/libc-only data absent on macOS/Windows | Linux unit/E2E pass; native macOS/Windows compile/behavior remains an explicit PR CI gate, not inferred locally | + +Task 6 focused local evidence on source binary +`target/x86_64-linux-gnu/665e89fc782f5d0f/bin/mcpp`: + +- two-stage self-host: first new implementation build PASS; RuntimeBinding-aware rebuild PASS in 65.06s with canonical glibc 2.44 loader/lib directory and host floor 2.39. +- `test_elf_runtime`: **10/10** tests PASS (exact payload, parser, Rule A/B, inconclusive). +- `206_runtime_binding_physics.sh`: PASS for real host-DSO control, post-link mismatch, sticky cached mismatch, doctor reuse, and hot zero-rewrite cache. +- regressions `200_subos_env_reaches_program.sh`, `205_root_local_subos.sh`, and `02_new_build_run.sh`: PASS/PASS/OK. +- full unit/refactor gate: PASS, **71 test binaries passed; 0 failed**, 50.99s (49.30s build + 1.54s run). +- `bash -n` for changed E2E and `git diff --check`: PASS. +- #392/#396 remain open until the merged release artifact repeats the exact acceptance E2E; local source evidence alone is not closure evidence. + +### 5.6 Provider-neutral runtime provenance and LinkIntent (Task 7) + +| Gate | RED evidence | Production change | GREEN evidence | +|---|---|---|---| +| structured descriptor contract | `test_runtime_contract` initially failed to compile because `RuntimeRequirement`, `RuntimeArtifact`, `LinkIntent`, canonical `PackageId`, and the resolver API did not exist; the first xpkg GREEN attempt then exposed a dangling `string_view` over a temporary parser body; review RED proved an explicit relative library file remained relative to the consumer instead of its declaring package | TOML and xpkg descriptors now read the same typed requirement/artifact/link-intent grammar; explicit library paths resolve at the declaring package; legacy `library_dirs`, `dlopen_libs`, and `capabilities` remain readable for one train and normalize only to runtime-search/requirement facts | `test_runtime_contract` TOML/xpkg rows pass and assert every structured channel, package-root path anchoring, plus retained legacy fields | +| exact ownership | The old runtime model retained only short provider strings and treated a requirement as a weak provider candidate | Build-plan resolution stamps every requirement/artifact/provider with namespace, name, version, and source provenance; only `provides` creates a descriptor provider; an explicit override is exact or uniquely compatible and missing/ambiguous cases hard-fail | Unit rows keep `alpha.backend@2.0.0` and `beta.backend@3.0.0` distinct and prove the requester is never self-promoted; E2E 66 rejects the legacy weak-requester override | +| xlings-selected facts | RuntimeBinding had no generic provider/artifact payload and its hash/cache could not preserve a host selection; review RED then proved equal-prefix facts were not ordered by source/provenance/digest and could make the contract hash input-order-dependent | Optional schema-1 `subos_info.runtime_contract` facts resolve `${subosdir}`/relative paths once, sort by their complete identity into RuntimeBinding, participate in its hash/serialization, and precede descriptor fallback facts in BuildPlan | `test_subos_info` and `test_runtime_selection` round-trip provider identity, artifact provenance, digest and host fingerprint; the full-fact ordering row and merge row prove deterministic selected-before-fallback behavior | +| link/search separation | Legacy `library_dirs` entered both `-L` and rpath; there was no typed seam to assert all output formats | Added a pure four-flavor LinkIntent renderer: runtime dirs never enter link lookup, ELF alone gets `-rpath-link`, Mach-O gets rpath/frameworks, PE gets link dirs while deploy files remain copy edges | `test_link_model_runtime_dirs` passes all flavor assertions; E2E 62 inspects a real Linux link and finds RUNPATH without the forbidden `-L` token | +| durable explanation | Runtime explanation could resolve again and provider provenance was not present in the stored schema; an added RED also showed `required=false` ABI facts were incorrectly promoted into hard compatibility constraints | `resolution.json` schema 2 stores the exact binding, canonical requirements/providers/artifacts, link intent, search mechanism, and synchronized post-link verdict; optional requirements remain recorded but stay out of hard ABI/doctor projections; `mcpp why runtime` reads only the newest stored file | E2E 207 builds successfully with an intentionally mismatched optional `abi:musl`, preserves same-short identities/artifact provenance, corrupts `mcpp.toml`, installs fake probe commands, then proves `why runtime` still succeeds without launching them | +| ownership boundary | No automated fence prevented mcpp from acquiring graphics-provider or hardware-probe policy | Added a source gate over executable code branches and process-launch neighborhoods; diagnostics redirect provider/host re-diagnosis to `xlings doctor` | Static gate passes over the complete `src/` tree; E2E 207 proves the live why path is probe-free | +| runtime-physics compatibility | Synthetic named SubOS fixtures contained only metadata, which became physically false after Task 6 began enforcing loader/libc closure | E2E 200/205 now reuse the managed default's exact libc/loader view while varying only environment and root ownership; no product fallback was introduced | E2E 200, 205, and 206 pass with truthful bindings; Task 6 Rule A/B enforcement remains intact | + +Task 7 focused local evidence on the latest self-hosted source binary +`target/x86_64-linux-gnu/4091fed9f558ec8a/bin/mcpp`: + +- self-host source builds: PASS; initial RuntimeBinding implementation completed in 68.66s and the post-review deterministic/optional/path refinement rebuild completed in 78.83s. +- focused units: `test_runtime_contract` **5/5**, `test_link_model_runtime_dirs` **5/5**, `test_runtime_selection` **10/10**, and `test_subos_info` **18/18** PASS. +- focused E2E `62_runtime_library_dirs.sh`, `66_runtime_provides.sh`, and `207_runtime_contract_provenance.sh`: PASS/`OK`. +- RuntimeBinding regressions `200_subos_env_reaches_program.sh`, `205_root_local_subos.sh`, and `206_runtime_binding_physics.sh`: PASS. +- full no-cache unit gate: PASS, **72 test binaries passed; 0 failed**, 109.40s (107.48s build + 1.78s run); after the review refinements, the complete latest-source refactor gate repeated **72/72** in 14.58s (12.98s build + 1.48s run). +- changed shell scripts `bash -n`, executable mode for E2E 207, and `git diff --check`: PASS. +- `mcpp-m` aggregate remains `afb8a647e04483a86985119e07086016f49d55f177ee6257094c336d226113c6`. +- host xlings config aggregate remains `f218aadf3792ee815c8535ce0ca0bb53f634fecbdbc5d0d47db442052d786d1b`. +- Linux semantics are locally exercised. Mach-O and PE flag spelling has pure unit coverage on Linux; native macOS/Windows compilation and behavior remain latest-HEAD PR CI gates, not inferred passes. + +### 5.7 Immutable release desired state (Task 8) + +| Gate | RED evidence | Production change | GREEN evidence | +|---|---|---|---| +| manifest contract | The initial 11-case suite produced 12 failing assertions because no generator existed; fixtures cover duplicate normalized platform/arch, missing sidecars, payload/sidecar hash disagreement, draft/prerelease releases, wrong tag/version, missing required assets, duplicate release names, and invalid commit identity | Added schema-1 generation from the GitHub release JSON plus downloaded payloads; four current platforms are mandatory, every additional versioned platform payload is included, every digest is recomputed, and output is sorted deterministically | `test_release_manifest.py` passes **12/12**; randomized release-API asset order produces byte-identical output and every manifest hash equals the independently recomputed fixture payload hash | +| future platform closure | Review fixture added a valid FreeBSD/riscv64 versioned payload; the first implementation stayed green while silently omitting that row | Primary-platform discovery is provider-neutral and version-exact rather than a fixed four-platform allowlist; the four present release targets remain a required floor | The focused future-platform test changed RED-to-GREEN and asserts the complete row and SHA256 occur in the manifest | +| immutable publication | Previously each platform uploaded independently and downstream jobs inferred completeness from timing and filenames | New `release-manifest` job waits for all four uploaders, validates non-draft/non-prerelease identity, uploads only when absent, refuses a byte-different existing manifest, polls public API visibility, then refetches all public assets and regenerates/compares the manifest | Ruby/Psych parses `release.yml` and confirms six jobs, `publish-ecosystem` depends on `release-manifest`, Python bytecode compilation passes, and `git diff --check` is clean; live GitHub upload/refetch remains a PR/release CI gate | +| maintainer contract | Release docs had no machine-readable completeness boundary or reproducible local audit command | English and Chinese release docs define schema, row scope, immutability/rerun behavior, public refetch command, and checklist gates | Documentation examples invoke the same checked-in generator and explicitly recompute payload hashes rather than trusting manifest or sidecar values | + +Task 8 focused local evidence: + +- RED: missing generator yielded 12 assertion failures across 11 initial tests; the later additional-platform review row independently failed because FreeBSD/riscv64 was omitted. +- GREEN: `python3 tests/scripts/test_release_manifest.py` reports **12 tests OK** in 0.309s; `python3 -m py_compile` succeeds for generator and tests. +- Determinism: the complete fixture shuffles API inventory with seed 398, invokes the CLI twice to separate output files, and compares exact bytes. +- Public-inventory replay: downloaded the eight real versioned payload/sidecar assets from stable `v2026.8.8.4`, resolved tag commit `55a39d90fe98b5475fc394ac8487fe6804b2b84f`, and generated twice byte-identically. The four-row manifest SHA256 is `6614ba2db65c8c28cb5a9d3466bd6cf3007634221da76d25216785061c647edb`; every real sidecar and recomputed payload digest agreed. +- Workflow structure: Ruby/Psych parses `.github/workflows/release.yml`; all six jobs are present and ecosystem publication is gated by the immutable manifest job. +- Public GitHub upload/refetch and eventual-consistency polling cannot be truthfully claimed from local fixtures; they remain explicit latest-HEAD release CI evidence. + +### 5.8 mcpp-bin-only AUR reconciliation (Task 9) + +- State-machine/contract suite: `python3 -m unittest tests/scripts/test_aur_reconcile.py` + passes **12/12**. It covers no-op/upgrade/repair/RPC-lag/refused-downgrade, + missing or mismatched release material, bounded maintenance retry, permanent + auth failure, failed known-package clone without repo initialization, exact + latest-complete release selection, workflow recovery triggers, and the pinned + AUR ED25519 fingerprint. It also requires both the rendered `mcpp-bin` + maintainer and reconciler commits to use the public `speak-agent` GitHub + noreply identity rather than a personal address. +- Release fixture: the real public `v2026.8.8.4` payloads/sidecars and locally + generated immutable manifest validate as desired `2026.8.8.4-1`; both Linux + digests match manifest and sidecar. The first Arch render attempt classified + the Docker Hub registry-header timeout as **transient**, not success. A later + real Arch run completed `makepkg --printsrcinfo` and non-root + `makepkg --verifysource`; the rendered `.SRCINFO` and both source payloads + verified successfully. +- `bash -n scripts/aur/update.sh`, Python compilation/tests, Ruby/Psych workflow + parse, and `git diff --check`: PASS. +- Protected `scripts/aur/mcpp-m/**` aggregate before/after remains + `afb8a647e04483a86985119e07086016f49d55f177ee6257094c336d226113c6`. +- No AUR publish was performed. The public package remains an observed + `2026.8.1.1-1` while the verified desired fixture is `2026.8.8.4-1`; release + publication must later exercise the reconciler from a fixed merged + main/release state. + +### 5.9 latest pin, version, and user contract docs (Task 10) + +- Live GitHub release API on 2026-08-09: newest stable xlings is + `v2026.8.9.2`; newest stable mcpp is `v2026.8.8.4`, so the next unused mcpp + version is `2026.8.9.1`. +- Live `openxlings/xim-pkgindex` main identifies `2026.8.8.4` as the current + mcpp latest for Linux/macOS/Windows; `.xlings.json` now bootstraps that + published version rather than the previous stale `2026.8.6.2`. +- `.github/tools/check_version_pins.sh` passes with every xlings pin at + `2026.8.9.2`, build version `2026.8.9.1`, and bootstrap mcpp `2026.8.8.4`. +- English/Chinese README, getting-started, manifest, toolchain, identity, AUR, + and changelog text now state the same template/SubOS/graphics/AUR boundaries. + +## 6. Pull request and CI + +Draft PR [#400](https://github.com/mcpp-community/mcpp/pull/400) was opened from +`feat/template-runtime-graphics-aur`. Every logical change is a separate pushed +commit followed by a PR checkpoint comment. Branch history is not rewritten; +the PR remains Draft until explicit user review. + +Initial head `234a4df` exposed two real boundaries: stale declared glibc +identity versus the resolved SubOS view on Linux, and Linux-only +runtime-physics assertions executing on macOS/Windows. Both received focused +regressions at `dae4384`. Subsequent native and local review exposed and fixed: + +- exact namespace inheritance and dependency-owner retention; +- ELF SONAME reuse and exact-miss diagnostic cause retention; +- truthful runtime contracts in fake xlings fixtures; +- macOS BMI-settling scope; +- Windows open-stream cleanup; +- isolated `MCPP_HOME` use in the libc poison fixture; +- lexical workspace-root anchoring for inherited relative indices on Windows. + +At implementation head `ed4cf64`, the complete latest-source local C++ gate is +**72/72**. One isolated Linux full-E2E audit produced **186 pass, 1 fixture +failure, 19 platform/capability skips**; the only failure was the wrong state +root in E2E 156, and its exact rerun passes after `0cc6a2a`. Exact E2E 12 also +passes after `ed4cf64`. This is deliberately not restated as a full latest-head +206-case rerun. + +The native `ed4cf64` matrix is terminal with 12 successes, five failures and one +concurrency cancellation: + +- PASS: hermetic Linux E2E, macOS ARM64 E2E, musl + LLVM, MinGW + Linux-to-Windows + Wine, Windows-to-Linux cross-build and Linux artifact run, + Linux unit/cold-GCC/both E2E shards, Windows build/unit/package and Windows + E2E 1/2; +- FAIL at one known cross-repository boundary: Linux xlings integration, macOS + xlings LLVM E2E, aarch64 mcpp + xlings, and Windows xlings regressions all + encounter xlings' bare `ftxui` declaration under exact selection; +- FAIL independently in Windows E2E 2/2 (run `31321961040`, job + `93266267511`): workspace-member `mcpp add acme.util@2.0.0` cannot read its + root-owned local index even though the corresponding native unit passes; +- CANCELLED: the bare-Windows/no-Visual-Studio job was superseded by a later + push and is not counted as pass. + +The exact Windows artifact succeeds for both a hand-authored member and `mcpp +new m1` under an isolated Wine reproduction. That narrows the remaining failure +to native Git Bash/process/workspace state, but does not prove it fixed. +Commit `9a47ccf` therefore adds a permanent, privacy-safe `route:` error line +that reports only local-index root/`pkgs` presence. Its TDD cycle is RED on the +missing API, then GREEN for `PmIndexRoute` 12/12 and real E2E 12. The next native +Windows run remains the authority for the root cause; no latest-head full-suite +claim is made yet. + +The xlings boundary is tracked by +[openxlings/xlings#521](https://github.com/openxlings/xlings/pull/521), whose +eight native/cross checks pass. It remains Draft and REVIEW_REQUIRED. After the +handoff (`9f6161a`) and validation-ledger (`6e42d6c`) commits, privacy review +produced `cf39cb2`: the AUR reconciler and rendered `mcpp-bin` now use the public +`speak-agent` noreply identity, with a RED/GREEN contract regression. `189c6d1` +records that 12/12 contract result. The current diagnostic head `9a47ccf` must +reach terminal state before any latest-head claim. + +The Chinese operator handoff is +`.agents/docs/2026-08-09-pr400-handoff-zh.md`. It records implementation scope, +all pushed commits, diagnosed failures, honest validation boundaries, remaining +release/ecosystem work, and the mandatory Draft/commit/checkpoint/privacy +workflow. + +## 7. Merge and release + +Not started. After explicit user review, this section will record the normal merge commit, tag, GitHub release, release workflow/job IDs, asset inventory, checksums, `mcpp-release.json`, and GitCode mirror evidence. Admin/bypass merge is outside the approved workflow. + +## 8. AUR mcpp-bin + +Current live snapshot from the preceding audit: + +- GitHub desired release: `v2026.8.8.4`. +- AUR observed `mcpp-bin`: `2026.8.1.1-1`. +- latest failed AUR run: `31254088758`, after version and both checksums resolved, then AUR SSH returned maintenance. +- public replay manifest SHA256: + `6614ba2db65c8c28cb5a9d3466bd6cf3007634221da76d25216785061c647edb`. +- real Arch dry-run: `.SRCINFO`, non-root `makepkg --verifysource`, and both + Linux source checksums PASS for desired `2026.8.8.4-1`. + +No old failed run will be rerun. Recovery uses the reconciler from a fixed main/release state. `mcpp-m` is outside every verdict and publish path. + +## 9. Cross-repository and public ecosystem verification + +### 9.1 mcpp-index exact identity bridge + +- Issue [mcpplibs/mcpp-index#196](https://github.com/mcpplibs/mcpp-index/issues/196) + is closed by merged PR + [mcpplibs/mcpp-index#197](https://github.com/mcpplibs/mcpp-index/pull/197). +- Merge commit: `b974cbba5a5ab7da7908422e44ff8e4b63243dc3`. +- The immutable `mcpplibs.capi.lua@0.0.3` payload remains unchanged; an + index-side Form-B bridge declares the actual `compat.lua@5.4.7` dependency. +- All ten PR checks passed, including descriptor lint/select, Linux/macOS/ + Windows workspace consumers, mirror reachability, graphics side-effect + isolation, and timings. + +### 9.2 xlings exact compatibility identities + +- Issue [openxlings/xlings#520](https://github.com/openxlings/xlings/issues/520) + and Draft PR + [openxlings/xlings#521](https://github.com/openxlings/xlings/pull/521) + carry the xlings repository boundary. +- Head `0ff9e502f6857b5d1a48657dfeeb94244e322cb9` declares + `compat.ftxui@6.1.9` and test-only `compat.gtest@1.15.2`, then refreshes the + lock so the transitive Lua C library retains exact identity. +- All eight Linux/macOS/Windows/aarch64 checks pass. The PR remains Draft, + MERGEABLE but REVIEW_REQUIRED; no admin/bypass merge is authorized. +- Current xlings release `2026.8.9.2` predates this fix. After review and normal + merge, a separate xlings version bump/release is required before updating the + mcpp pin. + +### 9.3 Still pending + +GitCode ranged/full resource publication, any required xim-pkgindex update, +fresh-home package lifecycle, the canonical template build/run, multiple +SubOS/glibc bindings, and OpenGL/Vulkan provider +PASS/FAIL/NOT_EXERCISED provenance remain release-follow-through work. None is +claimed from source-only or local Linux evidence. diff --git a/.agents/docs/2026-08-09-pr400-handoff-zh.md b/.agents/docs/2026-08-09-pr400-handoff-zh.md new file mode 100644 index 00000000..33e0f620 --- /dev/null +++ b/.agents/docs/2026-08-09-pr400-handoff-zh.md @@ -0,0 +1,422 @@ +# PR #400 中文交接文档 + +> 更新时间:2026-08-10 00:06 CST +> +> 维护身份:`speak-agent` +> +> 状态:Draft,尚未批准合并、尚未发布 +> 本文只记录可公开的仓库事实;本地用户名、绝对工作区路径、临时目录、凭据和个人邮箱均有意省略。 + +## 1. 交接结论 + +本轮已经把用户确认的四组设计决策实现到 +[mcpp-community/mcpp#400](https://github.com/mcpp-community/mcpp/pull/400): + +1. 包和模板统一使用 `[ns.]name[@version][:tname]`,省略 `ns` 时严格等于 + `mcpplibs`;`version` 和 `tname` 可省略;不引入 `--variant`;只有一个模板时, + 即使没有声明 `default = true`,它也自动成为默认模板。 +2. mcpp 不探测 GPU、Mesa、NVIDIA、WSL、ICD 或驱动来源。xlings/xim 负责图形栈和 + 运行时事实,mcpp-index 负责 C++ 包依赖关系,mcpp 只消费规范化后的 provider、 + artifact、provenance 和 link intent。 +3. mcpp 默认使用自己的运行时;根项目或 workspace 根可以在 `mcpp.toml` 中选择 + xlings SubOS。暂不提供 CLI override。SubOS 只是本次根构建/运行的本地 OS 环境, + 不传递为依赖约束,成员和源码库自己的 SubOS 声明不会污染消费者。 +4. AUR 自动化只收口 `mcpp-bin`。`mcpp-m` 的文件、远端和发布路径均未修改。 + +实现本身已经完成主要代码、单元测试、E2E、英文/中文文档、release manifest 和 +AUR reconciler。当前不能合并的主要原因不是上述核心实现,而是新“精确命名空间” +契约暴露了 xlings 仓库中仍然存在的裸 `ftxui`/`gtest` 声明。修复已经放在独立 +Draft PR [openxlings/xlings#521](https://github.com/openxlings/xlings/pull/521), +8/8 CI 通过,等待用户 review 和普通合并。 + +## 2. 关键入口和当前边界 + +| 项目 | 当前状态 | 说明 | +|---|---|---| +| mcpp 实施 issue | [#398](https://github.com/mcpp-community/mcpp/issues/398),OPEN | 冻结范围和验收条件;已更新为 Draft、逐 commit/checkpoint、隐私安全、普通合并 | +| mcpp 汇总 issue | [#397](https://github.com/mcpp-community/mcpp/issues/397),OPEN | C1-C9、特殊保留项和最终残留工作的统一入口 | +| mcpp Draft PR | [#400](https://github.com/mcpp-community/mcpp/pull/400),Draft | 本轮唯一 mcpp 实施 PR | +| scaffold 缺陷 | [#380](https://github.com/mcpp-community/mcpp/issues/380),OPEN | 代码已实现,仍需发布后二次验收才可关闭 | +| glibc 冲突 | [#392](https://github.com/mcpp-community/mcpp/issues/392),OPEN | 代码已实现,仍需发布后二次验收才可关闭 | +| Rule A/B | [#396](https://github.com/mcpp-community/mcpp/issues/396),OPEN | 代码已实现,仍需发布后二次验收才可关闭 | +| mcpp-index issue | [#196](https://github.com/mcpplibs/mcpp-index/issues/196),CLOSED | 修复不可变 `mcpplibs.capi.lua@0.0.3` 的裸 Lua 依赖 | +| mcpp-index PR | [#197](https://github.com/mcpplibs/mcpp-index/pull/197),MERGED | Form-B bridge 精确声明 `compat.lua@5.4.7`,10/10 CI 通过 | +| xlings issue | [#520](https://github.com/openxlings/xlings/issues/520),OPEN | 跟踪 xlings 裸兼容包声明 | +| xlings Draft PR | [#521](https://github.com/openxlings/xlings/pull/521),Draft | 精确声明 `compat.ftxui`、`compat.gtest` 并刷新 lock,8/8 CI 通过 | + +mcpp 分支为 `feat/template-runtime-graphics-aur`,基线为 +`80291ca01a982c1e8c00e43bfa97ffe68516e6d7`。核心实现和原生诊断 checkpoint 是 +`ed4cf64279a5da94c736742338e5b01d90e297b8`;随后增加中文交接、验证账本、公共发布身份, +以及不泄露本地路径的 exact-index 路由诊断。本文更新时远端 HEAD 为 +`9a47ccf5727a2756451812da2c98e18e1f034409`。相对基线共有 31 个可追溯 commit,改动 +111 个文件,约 15,049 行新增、1,692 行删除。接手时仍须重新比较本地与远端 HEAD, +不要把本文快照当成可变分支的永久 HEAD。 + +设计和验证文档按以下顺序阅读: + +1. `.agents/docs/2026-08-09-mcpp-template-runtime-graphics-aur-focused-design.md` +2. `.agents/docs/2026-08-09-mcpp-template-runtime-graphics-aur-implementation-plan.md` +3. `.agents/docs/2026-08-09-mcpp-template-runtime-graphics-aur-validation.md` +4. `.agents/docs/2026-08-09-xlings-mcpp-ecosystem-convergence-design.md` +5. 本文 + +## 3. 已完成的实现 + +### 3.1 包身份和模板语法 + +- 新增共享 `PackageSelector`,统一解析包依赖、`mcpp add`、xpkg 和模板 provider。 +- 裸名称只规范化为 `(mcpplibs, name)`,不再按短名称跨命名空间猜测。 +- 显式点分命名空间保持精确,例如 `mcpplibs.capi.lua`、`compat.ftxui`。 +- `mcpp add` 与 `mcpp new` 使用同一身份风格。 +- 模板语法固定为 `[ns.]name[@version][:tname]`;先解析 `:`,再解析 `@`。 +- `version` 省略时选择最新稳定版本;不会把 prerelease 当默认,也不接受模糊 + `latest` 别名。 +- `tname` 省略时:一个模板自动默认;多个模板必须有且仅有一个显式 default, + 否则给出确定性错误。 +- 旧 `pkg:` 列表写法仅保留一个迁移周期并打印可复制的新命令提示。 +- exact miss 会打印实际尝试的 canonical selector,并只把其它命名空间作为 + “did you mean” 诊断,不自动回退。 + +### 3.2 `mcpp new` 事务和跨平台安全 + +- 在访问配置、索引或网络前验证项目名称。 +- 拒绝绝对路径、路径分隔符、`.`/`..`、控制字符、Windows device basename、 + 尾随点/空格以及旧 `PROJECT` 占位名等不安全输入。 +- 模板变量改为单次渲染,插入值中的 `{{...}}` 不会被二次扫描。 +- 未知变量、未闭合 token、目录逃逸、symlink、类型冲突、读取/写入/复制失败都会 + 明确失败。 +- 在目标同级 staging 目录完整生成、关闭并同步文件、重新解析 manifest 后,才用 + no-replace rename 提交;失败不留下半成品目标或 staging 残留。 +- Linux、macOS、Windows 的独占提交分别放在平台实现中,不把平台 API 混入通用层。 + +### 3.3 根本地运行时和 SubOS + +- 引入纯 `RuntimeSelection`:只有 `McppDefault` 与 `NamedSubos` 两种状态。 +- 只有根项目/workspace 根的 `[xlings].subos` 生效;没有 CLI override。 +- workspace 成员独立运行时可以有自己的根选择;作为 workspace 成员或依赖被消费时, + 不传播自己的 SubOS。 +- 构建、运行、测试和快速路径共享一次解析出的不可变 `RuntimeBinding`。 +- binding 保存 owner、provider、runtime id、环境、路径和 contract hash;cache key + 包含该 hash,运行时配置变化会正确失效。 +- 旧 `MCPP_SUBOS_DIR` 不再改变选择结果。 +- xlings 环境的 `set`/`prepend` 语义保留调用方已存在变量并去重,不把显式空值误当缺失。 + +### 3.4 Linux 运行时物理约束(#392/#396) + +- 精确解析并查找 `glibc@`,不会从多个 payload 中取“第一个看起来能用的”。 +- 内部 ELF64-LE 读取器解析 `PT_INTERP`、`DT_RPATH/RUNPATH`、`DT_NEEDED`、 + GNU verneed/verdef,不调用外部 shell probe。 +- Rule A:host DSO 的 `GLIBC_*` 需求不得高于选中 libc 的 export floor。 +- Rule B:最终解释器、直接/传递 libc 必须与同一个 RuntimeBinding 一致,拒绝 host/private + 或两个 private libc 混用。 +- post-link verdict 与 artifact stat、RuntimeBinding contract 一起缓存;缓存 mismatch + 仍然保持失败,不能靠下一次运行重新探测变绿。 +- `mcpp doctor` 读取构建时已经保存的 verdict,不重新观察另一个当前 host。 +- 非 Linux 平台只保留 typed no-op 边界,不执行 ELF/glibc 规则。 + +### 3.5 图形栈和 provider provenance + +- mcpp 只接受结构化 `RuntimeRequirement`、`RuntimeProvider`、`RuntimeArtifact` 和 + `LinkIntent`。 +- 每条事实携带 canonical namespace/name/version/source/provenance;同名 provider + 不会相互覆盖。 +- xlings 选中的 runtime facts 优先于 descriptor fallback,并以完整身份排序,保证 + contract hash 与输入顺序无关。 +- link-time search 与 runtime search 分离:runtime dirs 不再错误进入 `-L`;ELF、Mach-O、 + PE 分别渲染自己的 link intent。 +- `resolution.json` schema 2 保存 binding、requirements、providers、artifacts、link intent + 和 post-link verdict。 +- `mcpp why runtime` 只读取保存结果,不启动探测命令。 +- 增加源码边界测试,防止 mcpp 后续引入 GPU、vendor、driver、ICD 探测分支;需要重新诊断 + host/provider 时,提示用户使用 `xlings doctor`。 + +### 3.6 release desired state + +- release workflow 新增不可变 `mcpp-release.json` schema 1。 +- manifest 由最终公开 GitHub release inventory 和实际下载 payload 重新计算,不信任上传 + 顺序或 sidecar 文本。 +- 四个现有平台是最低完整集合,未来符合命名契约的平台也必须进入 manifest,不能静默遗漏。 +- manifest 只允许首次创建;同名且字节不同会拒绝,rerun 只能验证相同内容。 +- `publish-ecosystem` 必须等待 manifest job 完成。 +- 版本已经准备为 `2026.8.9.1`;当前 mcpp 内部 xlings pin 为已发布的 + `2026.8.9.2`,待 #521 合并并发布后还需要更新到新 xlings 版本。 + +### 3.7 `mcpp-bin` AUR reconciler + +- 只管理 `mcpp-bin`;`mcpp-m` 和 `mcpp-git` 不在 reconcile verdict 或 push 范围。 +- 支持 release event、定时恢复和手动触发。 +- 选择最新完整稳定 release,验证 `mcpp-release.json`、两个 Linux payload 和 sidecar, + 再生成 `.SRCINFO`。 +- 使用 Arch `vercmp`,拒绝倒退;相同版本可检测并修复 drift。 +- clone、认证、AUR maintenance、RPC lag 都有明确分类和有界重试。 +- 只允许 fast-forward push,不提供 force、空仓库首次发布或旧 run 重跑旁路。 +- 已对公开 `v2026.8.8.4` 做真实 Arch `makepkg --printsrcinfo`/ + `--verifysource` dry-run;目标应为 `2026.8.8.4-1`,公开 AUR 观测仍是 + `2026.8.1.1-1`,本轮没有发布。 + +### 3.8 文档和用户契约 + +- 同步 README、中文 README、getting-started、manifest、toolchain、identity、AUR、release + 和 changelog。 +- 明确无 `--variant`、无 CLI SubOS override、SubOS 非传递、mcpp 不探测 GPU、AUR 只管理 + `mcpp-bin`。 +- `.agents/docs` 中保留详细设计、拆分计划和 RED/GREEN/CI/release 验证账本。 +- 已清理文档中的本地绝对路径和旧 squash/admin-bypass 描述;当前流程是 Draft、逐 commit、 + 用户明确 review 后普通合并。 + +## 4. Commit 追踪 + +以下顺序就是实现历史;不要 rebase、amend、squash 或 force-push: + +| Commit | 内容 | +|---|---| +| `52706c9` | 写入聚焦设计、实施计划和初始验证账本 | +| `527b26e` | 共享精确 PackageSelector,收口包身份 | +| `93db300` | 精确 TemplateSpec、版本和默认模板选择 | +| `dce8619` | 事务式、跨平台安全的项目脚手架 | +| `ac0670f` | 根本地 RuntimeSelection/RuntimeBinding | +| `7850702` | Linux ELF/runtime closure Rule A/B | +| `6904673` | provider-neutral runtime provenance 与 LinkIntent | +| `234a4df` | 不可变 release manifest | +| `08857c7` | `mcpp-bin` AUR desired-state reconciler | +| `dae4384` | 真实 SubOS view 与 RuntimeBinding 对齐 | +| `76fee4e` | 准备 `2026.8.9.1` 版本和 pin/docs | +| `04a348e` | E2E capability 正确识别 Python | +| `0108ee1` | ELF parser fixture 只在 Linux 执行 | +| `e6050b7` | 依赖精确继承 index 声明命名空间 | +| `d008a21` | 依赖选择保留 index owner identity | +| `c569152` | runtime closure 正确处理 ELF SONAME 复用 | +| `679c7ab` | E2E fixture 与精确身份契约对齐 | +| `65fc77b` | fake xlings fixture 提供真实 runtime contract | +| `e3b93ac` | exact miss 保留 version-floor 根因 | +| `1ef3112` | malformed exact descriptor 变为明确错误 | +| `4d8d080` | module mangling 使用实际 authored modules | +| `12b0b95` | macOS BMI E2E 只检查自身稳定 edge | +| `92caaf9` | Windows scaffold 测试清理前关闭文件流 | +| `4e39a8e` | Draft PR 流程和文档隐私清理 | +| `0cc6a2a` | libc poison fixture 尊重隔离 `MCPP_HOME` | +| `ed4cf64` | workspace 继承相对 index 时使用 lexical root anchor | +| `9f6161a` | 新增完整中文 PR 交接文档 | +| `6e42d6c` | 同步 Arch、CI 和跨仓库验证账本 | +| `cf39cb2` | AUR 使用 `speak-agent` 公共 noreply 身份,并增加隐私回归 | +| `189c6d1` | 记录隐私安全 AUR checkpoint 和 12/12 契约结果 | +| `9a47ccf` | exact miss 输出隐私安全的 index route 状态,供 Windows 原生定位 | + +每个逻辑 commit push 后都在 PR #400 留有 checkpoint。issue #380、#392、#396 也已经 +收到“实现已进入 Draft、但发布前不关闭”的状态评论。 + +## 5. 遇到的问题、根因和处理 + +### 5.1 精确身份暴露 xlings 的裸依赖 + +症状:Linux、macOS、Windows 和 aarch64 中“使用新 mcpp 构建 xlings”的任务报错: +`ftxui` 被严格解释为 `(mcpplibs, ftxui)`,而索引中的真实包是 `compat.ftxui`。 + +这不是应该在 mcpp 中恢复 fallback 的理由;恢复 fallback 会破坏用户确认的命名空间 +语义。处理方式分两层: + +1. mcpp-index #197 已把不可变 `mcpplibs.capi.lua@0.0.3` 的裸 Lua 依赖用 + index-side Form-B bridge 精确改为 `compat.lua@5.4.7`,已经正常合并。 +2. xlings #521 把根依赖改为 `compat.ftxui`、测试依赖改为 `compat.gtest`,并刷新 lock。 + 该 PR 仍是 Draft,8/8 CI 通过,不能 admin/bypass 合并。 + +### 5.2 RuntimeBinding 最初与真实 SubOS view 不一致 + +初版 binding 只信任声明 identity;真实安装中 loader/libc view 可能经过 xlings 重定位或 +版本视图转换,导致“声明 glibc”与实际闭包不一致。修复后,binding 从选中的 SubOS view +获取 canonical payload,并保留声明/provider provenance;缺失、过期或混合闭包都明确失败。 + +### 5.3 原有测试含 Linux 特有假设 + +- ELF parser synthetic fixture 在非 Linux 也执行:改为 typed platform skip。 +- Python E2E capability 通过错误方式推断:改为真实可执行能力判断。 +- fake xlings 只有元数据、没有 loader/libc 物理事实:补足真实 runtime contract,未增加产品 + fallback。 + +### 5.4 macOS 增量构建断言过宽 + +E2E #170 原先要求第三次构建“整个图没有任何工作”。macOS 合法重建生成的 iOS init/link +edge,但 stage/main BMI 已经稳定。测试现在只约束它负责验证的 stage/main edge,避免把 +平台合法工作误判成 BMI 回归。最新 macOS ARM64 E2E 已通过。 + +### 5.5 Windows 清理和 workspace path identity + +- 单元 fixture 在 `remove_all` 前仍持有两个 `ifstream`;Linux 允许删除打开文件,Windows + 拒绝。修复为在清理前结束 stream lifetime。 +- workspace 根 `[indices] acme = { path = "index" }` 在 Windows 上经 + `weakly_canonical` 后可能因为 short-name/case alias 与成员视图失去字符串身份。现在使用 + `(workspace_root / relative_path).lexically_normal()`,既把相对 index 锚定到根,又不改变 + 用户声明的 lexical identity。新增 focused unit,相关本地 E2E #12 通过;但 + `ed4cf64` 的原生 Windows E2E 2/2 仍在 workspace member 的 `mcpp add + acme.util@2.0.0` 失败,因此该提交不是完整修复。 +- 失败 job 为 run `31321961040` / job `93266267511`;错误只有 `package 'acme.util' + not found in any configured index` 和 `tried: (acme, util)`。同一原生 Windows 矩阵的 + `PmIndexRoute.WorkspaceMemberReadsRootAnchoredRelativeIndex` 已执行并通过,说明纯单元 fixture + 没有覆盖真实进程/工作目录视图。 +- 已下载该 job 构建的 Windows artifact,并在隔离 Wine 环境中同时用手工 member 和 + `mcpp new m1` 重放;两条 `mcpp add acme.util@2.0.0` 都通过,所以不能把 Wine 结果外推为 + GitHub Windows 已修复。 +- `9a47ccf` 增加永久、隐私安全的 `route:` 诊断,只报告 local index 的 root/`pkgs` + present/absent,不输出盘符、用户目录或绝对路径。下一轮原生日志将区分 workspace + inheritance、路径锚定和 descriptor read 三层;拿到证据前不猜测修复。 + +### 5.6 隔离 HOME fixture 使用错误根 + +E2E #156 用 `$HOME/.mcpp` 注入旧 libc,但测试实际选择了 `MCPP_HOME`。因此它先污染了当前 +mcpp,而没有进入目标 nested-tool regression。fixture 现在使用 +`${MCPP_HOME:-$HOME/.mcpp}` 对应的状态根,精确重跑通过。 + +### 5.7 本地无法替代原生 Windows 验证 + +曾尝试用隔离环境的 MinGW cross toolchain 构建当前 mcpp,以更快复现 Windows 行为;构建 +停在现有 GCC module 与 `windows.h` language-linkage 冲突,未修改产品代码,也未污染 host。 +因此 Windows 结论只接受 GitHub 原生 runner 终态,不从 Linux source review 或 cross-build +推断。 + +### 5.8 AUR 外部状态不稳定 + +第一次真实 Arch 容器拉取受到 Docker Hub registry header timeout,按 transient 记录,未 +伪装成通过。随后真实 Arch dry-run 已完成并验证两个 Linux 源。公开 AUR 仍落后,之前的 +发布任务遇到 AUR SSH maintenance;本轮没有绕过、force-push 或手工覆盖远端。 + +## 6. 已有验证证据 + +### 6.1 本地代码和 E2E + +- 最新源码完整 C++ unit/integration:`72/72`,0 fail。 +- 一轮隔离 Linux 全量 E2E:186 pass、1 fail、19 个平台/能力 skip。唯一失败是 #156 的 + fixture 根错误,修复后精确重跑通过。 +- `ed4cf64` 后精确重跑 #12 和 #156 通过,完整 72/72 unit 再次通过。 +- `9a47ccf` 按 TDD 先得到缺少 `IndexRoute::describe` 的 RED;实现后 focused + `PmIndexRoute` 12/12、真实 E2E #12 通过,且诊断不包含本机路径。该提交尚未重新宣称 + 最新源码全量 unit/E2E 均通过。 +- 重点 E2E 已覆盖 package/template、事务 scaffold、RuntimeBinding、root-local SubOS、 + Rule A/B、provider provenance、stored `why runtime`、AUR/release 脚本。 +- release manifest contract:12/12。 +- AUR state-machine/contract:12/12,包括公共 `speak-agent` 发布身份契约。 +- 所有变更 shell 通过 `bash -n`,Python generator/tests 通过 bytecode compile,workflow YAML + 可解析,`git diff --check` 通过。 +- 公开 `v2026.8.8.4` 四平台 payload/sidecar replay 生成的 manifest 字节稳定;manifest + SHA256 为 `6614ba2db65c8c28cb5a9d3466bd6cf3007634221da76d25216785061c647edb`。 + +不要把上述“一轮全量后精确修复”写成“最新 HEAD 已重新全量 206 个 E2E 全绿”。最终完整 +结论仍需要 latest-head CI 和发布后二次验证。 + +### 6.2 边界未被污染 + +- `scripts/aur/mcpp-m/**` 聚合 SHA256 在实现前后都为 + `afb8a647e04483a86985119e07086016f49d55f177ee6257094c336d226113c6`。 +- host xlings 配置聚合 SHA256 在状态化测试前后都为 + `f218aadf3792ee815c8535ce0ca0bb53f634fecbdbc5d0d47db442052d786d1b`。 +- 原始 checkout 的用户改动未被暂存、覆盖或清理;实施在隔离 worktree 中进行。 +- PR diff 已扫描常见 Linux/macOS/Windows home path、runner 用户名和临时目录模式,未发现 + 本地路径泄露。 + +### 6.3 GitHub CI 快照 + +以下是 `ed4cf64` 的终态快照;共 12 success、5 failure、1 cancelled。cancelled 不是 pass: + +| 状态 | Job | +|---|---| +| PASS | hermetic Linux E2E | +| PASS | macOS ARM64 E2E | +| PASS | musl + LLVM toolchain | +| PASS | MinGW Linux→Windows build + Wine run | +| PASS | Windows→Linux cross-build及 Linux artifact run | +| PASS | Linux unit、GCC cold toolchain、Linux E2E 1/2 和 2/2 | +| PASS | Windows build/unit/package、Windows E2E 1/2 | +| FAIL(已知外部边界) | Linux 构建/运行 xlings | +| FAIL(已知外部边界) | macOS xlings LLVM E2E | +| FAIL(已知外部边界) | aarch64 mcpp + xlings | +| FAIL(已知外部边界) | Windows toolchains + regressions 中的 xlings 构建 | +| FAIL(mcpp 内部,仍在定位) | Windows E2E 2/2 的 workspace root local index | +| CANCELLED | bare Windows/no-Visual-Studio;被后续 push 的 concurrency 取消,不作结论 | + +四个已知失败都发生在构建当前 xlings source 时的裸 `ftxui` exact miss;对应 #521, +不能在 mcpp 中用 fallback 掩盖。Windows E2E 2/2 是独立的 mcpp 问题,不能归入 #521; +`9a47ccf` 的 latest-head 原生矩阵需先给出 route 状态,再做下一笔最小修复。 + +## 7. 尚未完成 + +以下项目必须明确交接,不能因为“代码已经很多”而省略: + +1. **等待 `9a47ccf` 的 mcpp latest-head CI 证据。** 特别读取 Windows E2E 2/2 的 + `route:` 行;`ed4cf64` 已经由原生 runner 证明没有完整解决根相对 index。 +2. **用户 review xlings #521。** 8/8 CI 已通过,但仍为 Draft/REVIEW_REQUIRED。 +3. **普通合并 xlings #521。** 不使用 admin、bypass、squash、force 或历史改写。 +4. **发布新的 xlings 稳定版本。** 当前 `2026.8.9.2` 不含 #521;需要正常 version bump、 + tag、全平台 release CI 和资产核验。 +5. **更新 mcpp #400 的 xlings pin。** pin 到包含 #521 的新稳定版本,作为独立 commit、push + 和 checkpoint;然后重新跑 latest-head mcpp 全矩阵。 +6. **补最终交付账本。** 更新 `.agents/docs/...validation.md` 中过时的 CI、Arch 和跨仓库 + “not started” 段落,记录终态 job/run、PR/merge commit、release asset/checksum。 +7. **用户 review mcpp #400。** 只有用户明确同意后才能从 Draft 转 ready,并走普通合并。 +8. **发布 mcpp `v2026.8.9.1`。** 验证 tag、版本字符串、所有平台资产、sidecar、最终公开 + inventory 和不可变 `mcpp-release.json`;running/cancelled/superseded 不算通过。 +9. **GitCode/生态镜像。** 按发布边界补 GitCode resource,验证 ranged/full download 和 hash; + 必要时更新 xim-pkgindex,但不把 provider policy 搬进 mcpp。 +10. **AUR `mcpp-bin` 收敛。** 从合并后的固定 main/release 状态运行 reconciler,等待 AUR + maintenance 恢复,验证 RPC 可见、`.SRCINFO`、两个架构 checksum 和 fresh install。 +11. **全生态 fresh-home 验证。** 在隔离 HOME/XLINGS_HOME/SubOS root 中验证:安装、 + `[ns.]name[@version][:tname]` 模板、唯一默认模板、多个 SubOS/glibc、build/run/test、 + OpenGL/Vulkan provider provenance 的 PASS/FAIL/NOT_EXERCISED 表达。 +12. **issue 收口。** #398 随 PR 合并关闭;#380/#392/#396 只在发布后二次验收通过后关闭, + 并留言精确 release/测试证据。残留工作统一写回 #397。 +13. **原始 issue/PR 全量整理未完成。** C1-C9 和其它开放 PR/issue 的最终关闭/合并仍需按 + #397 清单继续;特殊保留 #43、#260 以及标记为保留/Draft/do-not-merge 的项不要动。 + +## 8. 推荐接手顺序 + +1. 确认 `gh api user --jq .login` 返回 `speak-agent`。 +2. 检查 PR #400 本地 HEAD、远端 HEAD 和 clean worktree;不要在原始用户 checkout 上工作。 +3. 等 `9a47ccf` 的 Windows E2E 2/2 输出隐私安全的 `route:` 状态;只按原生证据修复。 + 四个裸 `ftxui` 失败归入 #521,其余失败必须单独 RED/根因/修复/commit/checkpoint。 +4. 把最新终态快照评论到 #400;不要重复 rerun 已被新 HEAD 取代的旧 workflow。 +5. 请用户 review #521;得到明确同意后,转 ready 并普通合并,随后按 xlings release skill + 正常发版。 +6. mcpp 独立 commit 更新新 xlings pin,push 后评论 checkpoint,再等待新的 latest-head + 全矩阵。 +7. 所有代码和依赖 gate 通过后,独立 commit 更新验证账本;做 diff/privacy/mcpp-m/host-config + 边界复核并 push/comment。 +8. 把本文“尚未完成”逐项清零,再请求用户 review #400;没有 review 不转 ready。 +9. 普通合并、release、GitCode、AUR、fresh-home 生态验证完成后,再按证据关闭 issue。 + +## 9. 每个后续改动的强制流程 + +每个逻辑变化都执行同一套流程: + +1. 先写或选中能复现问题的 focused test,记录 RED; +2. 做最小实现,记录 GREEN; +3. 跑相关回归和 `git diff --check`; +4. 扫描 diff,确保没有本地用户名、绝对路径、临时目录、个人邮箱或凭据; +5. 只暂存明确文件,不使用宽泛 `git add -A`; +6. 使用 `speak-agent` 的 GitHub noreply commit identity; +7. 一个逻辑变更一个 commit,立即 push 到 Draft PR; +8. 立即在 PR 留中文 checkpoint:问题、根因、改动、测试、当前阻塞; +9. 不 amend/rebase/squash/force-push,不使用 admin/bypass; +10. pending、skipped、cancelled、superseded 任务绝不记为 PASS。 + +## 10. 不应改变的产品边界 + +- 不恢复跨命名空间短名称 fallback。 +- 不增加 `--variant`。 +- 暂不增加 CLI SubOS override。 +- 不把 SubOS 作为库的传递依赖要求。 +- 不让 mcpp 探测 GPU、Mesa、NVIDIA、WSL、ICD 或驱动来源。 +- 不把 xlings/xim provider policy 复制进 mcpp-index 或 mcpp。 +- 不修改或发布 `mcpp-m`。 +- 不碰明确特殊保留的 issue/PR。 +- 不在用户未 review 时把 Draft 转 ready 或合并。 + +## 11. 当前可供 review 的核心点 + +用户 review 时建议优先确认以下五点: + +1. `ns` 省略严格等于 `mcpplibs` 是否符合预期,包括 exact miss 不 fallback。 +2. 唯一模板自动默认、多模板显式 default/tname 的错误边界是否足够直观。 +3. RuntimeBinding 是否真正只由根/workspace 根选择,成员和依赖完全不传播 SubOS。 +4. 图形栈是否保持“xlings/xim 诊断和选择,mcpp-index 描述关系,mcpp 只构建”的责任分层。 +5. `mcpp-bin` reconciler 是否满足幂等、单调、checksum、fast-forward-only,同时完全隔离 + `mcpp-m`。 + +在这五点确认前,PR #400 和 #521 都应保持 Draft。 diff --git a/.agents/docs/2026-08-09-xlings-mcpp-ecosystem-convergence-design.md b/.agents/docs/2026-08-09-xlings-mcpp-ecosystem-convergence-design.md new file mode 100644 index 00000000..01cd8c34 --- /dev/null +++ b/.agents/docs/2026-08-09-xlings-mcpp-ecosystem-convergence-design.md @@ -0,0 +1,985 @@ +# xlings × mcpp 生态契约收敛与优化设计 + +> 状态:Review Draft +> +> 日期:2026-08-09 +> +> 基线:mcpp main@80291ca01a98;xlings 最新发布 2026.8.9.2;mcpp 发布包当前内置 xlings 2026.8.8.1。#397 的跨仓证据快照为 mcpp-index b86fc7c、xim-pkgindex c0aded29;2026-08-09 复核的远端 main 分别为 b86fc7ce802938,设计不依赖本地旧 checkout。 +> +> 范围:设计与迁移方案,不代表本文所列实现已经完成。 +> +> 关联问题:[#397(C1–C9 汇总)](https://github.com/mcpp-community/mcpp/issues/397)、[#396](https://github.com/mcpp-community/mcpp/issues/396)、[#392](https://github.com/mcpp-community/mcpp/issues/392)、[#380](https://github.com/mcpp-community/mcpp/issues/380)。 + +## 1. 结论先行 + +这些问题不是九个独立 bug,也不是“再补一轮 if”能稳定解决的问题。它们共同暴露了五类边界不清: + +1. **身份边界不清**:namespace:name 在 CLI、模板、索引、渲染器之间被拆掉或退化为裸 name。 +2. **状态边界不清**:构建缓存只记“输入看起来没变”,却没有声明一次成功必须留下哪些输出与证明。 +3. **运行时边界不清**:xlings、SubOS、索引和 mcpp 都能推断或改写运行时环境,却没有统一、可持久化、可验证的运行时契约。 +4. **策略边界不清**:用户构建时的物理可运行性、索引仓库的生态闭包策略、GPU/宿主探测混在同一层。 +5. **发布边界不清**:GitHub Release 是事件源,AUR/Homebrew 等下游却按“一次事件必成功”实现,缺少最终一致性对账。 + +建议采用 **分层、类型化、可验证契约**,而不是把 xlings 扩成一个全知守护进程: + +- **xlings** 是环境与运行时 substrate:安装、版本选择、payload provenance、SubOS、RuntimeBinding、环境操作语义。 +- **xim-pkgindex** 是系统级 payload 与宿主桥接策略:glibc、Mesa、NVIDIA/WSL sentinel、图形运行时闭包。 +- **mcpp-index** 是 C/C++ 包、模板和链接元数据:模块、头文件、link plan、GUI 模板与平台依赖声明。 +- **mcpp** 是项目规划器与产物验证器:构建图、缓存后置条件、模板事务、实际二进制物理检查、可解释诊断。 +- **发布流水线** 是投影对账器:以不可变 Release Manifest 为源,将 AUR 等渠道收敛到期望状态。 + +推荐方案的核心不是“增加更多配置”,而是减少重复推断:每个事实只有一个所有者,其他组件消费版本化契约。 + +## 2. 目标与非目标 + +### 2.1 目标 + +- 将 C1–C9、#380、#392、#396 收敛到少量可复用的根契约。 +- 让 mcpp newmcpp add、索引和生成后的清单使用同一套包身份。 +- 让构建“成功”同时意味着必要产物、元数据和运行时证明完整。 +- 让私有 glibc、宿主库、图形运行时的来源和兼容性可计算、可解释、可缓存。 +- 支持 Linux、macOS、Windows 的真实平台差异,不制造虚假对称。 +- 让发布渠道从一次性推送转成幂等对账,外部服务恢复后可自动收敛。 +- 保持无变化构建快速,不在构建热路径启动 xlings 或访问网络。 + +### 2.2 非目标 + +- 不在 mcpp 中实现 GPU 驱动管理。 +- 不要求每个 GUI 测试都在普通共享 runner 上打开真实窗口。 +- 不用 xlings daemon 替代 mcpp 的项目图规划。 +- 不把索引闭包策略变成用户本地每次构建的强制联网检查。 +- 不以一个跨四仓库的巨型 PR 交付。 +- 本文不直接关闭 issues、重跑 AUR 工作流或发布新版本。 + +## 3. 当前事实与根因 + +### 3.1 C1–C9 不是同一优先级,但可以归并 + +| 编号 | 当前现象 | 直接根因 | 应归入的长期契约 | 优先级 | +|---|---|---|---|---| +| C1 | fast path 在 compile_commands.json 等配置产物缺失时仍可直接进入 Ninja,编辑器反复请求;普通 executable 缺失通常仍由 Ninja 重建 | 缓存没有声明配置成功的后置条件 | BuildSnapshot + OutputManifest | P1 | +| C2 | 注释中的 R"(、普通字符串中的注释记号会污染模块扫描状态 | 先做脆弱的行级 strip,再解析 token | 线性词法 masker | P1 | +| C3 | TTY 下 --no-color 无效 | color 状态延迟初始化覆盖显式参数 | 一次初始化的 UiPolicy | P2 | +| C4 | namespaced 包 exports 非 strict 时漏检,strict 时会把合法 exports 误报 missing | qualified owner 与短 manifest name 做字符串比较 | PackageId 类型贯穿 | P1 | +| C5 | Windows 版本探测含 POSIX 2>/dev/null | 命令以 shell 字符串表达 | CommandSpec + direct exec | P1 | +| C6 | 部分安装形态下 release 内置 xlings 不进入候选更新链 | 候选只按位置推断,没有 provenance/capability | XlingsCandidate 选择策略 | P1/P2 | +| C7 | xlings 端已按 presence 语义修复,但 mcpp 仍把 op=set 无条件覆盖 | reader 与当前 wire 语义漂移,且缺少显式 replace | EnvOp schema v2 | P1 | +| C8 | 两段无版本安装循环在进入 fetch 前即失败并吞掉错误;当前用户可达影响尚未证明 | 多个安装 owner、错误被忽略 | 单一 solver/installer owner | P2 | +| C9 | 路径、源输出、glob、脚手架、链接、版本、canary、CI、机器输出等 11 个长期项 | 同一事实多处生产、字符串协议与 fail-open | 分别映射到下表所列基础契约 | P1–P3 | + +C9 的 11 项应明确归属,而不是留作“杂项”: + +| C9 项 | 归属 | +|---|---| +| Windows 路径与 action 规范化(#393) | typed Path + CommandSpec | +| generated source output 被静默排除(#393) | OutputManifest + action postcondition | +| 默认 source glob 漂移(#386) | 单一 SourceSet producer | +| 脚手架安全与卡死(#380) | NameAtom + 单次渲染 + 事务目录 | +| runtime link order / macOS runtime dir(#304) | LinkPlan token + 平台 capability | +| prerelease 依赖解析(#370) | 单一 VersionReq parser | +| cfg version 被忽略(#290) | 封闭条件词汇;未知条件 hard error | +| operator-template canary 只 precompile、不 import(#256) | 真实 importer + crash/unsupported/fail 三态 | +| cppfly resolver 后续候选不可达、canary 可自我跳过(#215) | 遍历全部候选 + fail-closed capability gate | +| fresh install 被缓存掩盖(#259) | 冷 HOME 独立 gate | +| machine output 不一致(#379) | UiPolicy + WireEnvelope | + +其中 #386 的“四项 fallback glob 与七项 canonical 默认不一致”是已证实的代码漂移,但正常 xpkg 装载路径已补默认并要求 sources 非空;当前用户可达性尚未建立,应作为 P3 清理而不是阻塞前两阶段。 + +### 3.2 #380:输入验证只是第一层,真正缺的是脚手架事务 + +当前 mcpp new 可接受路径穿越、控制字符和会重新引入占位符的名称;builtin renderer 对含 PROJECT 的替换值可静态证明无限循环,package-template renderer 虽推进游标,不会以同一种方式无限循环,但仍会发生占位符相互消费。两条路径都有 I/O 错误被忽略和半成品风险。修复不能只加一个正则: + +- 项目逻辑名必须是一个 NameAtom,与目标目录 --dir 分离。 +- 新项目默认执行 portable policy:拒绝绝对路径、./..、任一平台路径分隔符、NUL/控制字符、Windows 保留设备名及尾随点/空格;已有 legacy manifest 继续可读,不借此批量改写用户身份。 +- namespace 是结构化字段,不允许塞回项目名或模板字符串。 +- 渲染必须单次完成;未知占位符、重复键、非法 TOML/C++ 标识均 hard error。 +- 先写同父目录临时目录,验证生成清单与必需文件,再原子 rename。 +- 任一步失败清理临时目录;目标已存在默认不覆盖。 + +### 3.3 #392 与 #396:问题是“实际装载物理”,不是版本号猜测 + +#392 展示了私有 glibc 与宿主 libtinfo/Mesa 混装时的两个失败方向: + +- 私有 loader/libc 较旧,宿主库要求更高 GLIBC symbol version。 +- 通过全局 LD_LIBRARY_PATH 暴露私有 glibc,又让外部宿主命令加载到错误 libc/私有符号。 + +当前 main 已有隔离私有 glibc 环境的局部缓解,因此不能表述成“完全未修”。但仅靠目录过滤仍不能证明最终 ELF 的实际闭包可运行。 + +另一个独立的不确定性是 post-install 的 sandbox glibc 查找:注释声称选择 newest,代码实际返回 directory_iterator 的第一个命中;该顺序未规定,不能描述为“字典序第一”。目标设计必须按完整 PackageId/version/RuntimeBinding 选择,不从目录枚举顺序推断。 + +#396 中应保留两条物理不变量,但修正 Rule A 的计算方式: + +- **Rule B — 单一 RuntimeBinding**:实际 PT_INTERP 与最终解析到的 libc 必须来自同一个 payload/runtime binding。 +- **Rule A — symbol ceiling**:对每一个实际借用的宿主 ELF,对其 .gnu.version_r 所需的最高 GLIBC 版本逐对象检查,不用粗糙的“私有 glibc 版本 ≥ host glibc 版本”替代。 + +生态策略另行定义: + +- **Policy D — 生态闭包**:索引包的运行时依赖应由生态包闭包满足;宿主对象只允许出现在显式、类型化的 host-link sentinel 后。 +- D 在索引 CI/安装物化时强制。用户构建的最终目标是对**已证明**的 A/B 冲突 hard fail,并清晰展示 provenance;在精确闭包解析器积累真机证据前,按 observe → warning → strict opt-in → 默认 strict 分阶段,不能拿粗略的 host glibc 代理在第一天全局 hard fail。 + +这一区分很重要:A/B 是物理事实,D 是仓库治理策略。 + +### 3.4 模板 namespace:现有语法先天歧义 + +当前 --template 解析 pkg@ver:template,第一个冒号被当作 template 分隔符。因此 ocornut:imgui 会被理解为 package=ocornut、template=imgui,无法表达规范身份。scaffold resolver 又只尝试空 namespace 与 compat,没有复用依赖路径的 IndexRoute;裸 imgui 会先命中冻结的 mcpplibs:imgui,而不是新的 ocornut:imgui,也不会报告跨 namespace 候选歧义。 + +同时,模板获取阶段即使从 descriptor 得到 namespace,返回值和渲染变量仍只保留裸 name;依赖注入也用字符串搜索和短名。这会重新引入此前包索引已经修复的同名碰撞。 + +正确方向是复用 mcpp add 已有身份语法,而不是再发明第三套压缩分隔符: + +~~~text +mcpp new app --namespace acme \ + --template ocornut:imgui@1.92.8 \ + --variant glfw-opengl3 \ + --index official + +mcpp new app --template mcpplibs:templates@0.0.1 +mcpp new --list-templates ocornut:imgui@1.92.8 +~~~ + +- --template:规范 PackageRef,即 namespace:name@version。 +- --variant:该 provider 内的模板变体。 +- --namespace:新项目自身 namespace。 +- --dir:文件系统目的地。 +- --index:路由来源;index alias/path/URL 不属于 PackageId。 +- 旧 pkg@ver:template 只作为 deprecated ingress,解析后立即规范化并告警。 +- 对 pkg:word,新语义“namespace:name”与旧语义“pkg 的 variant”可能同时成立:只在一侧候选唯一时兼容;两侧都存在或无法排除时返回稳定的 ambiguity error,并给出可复制的显式 --template/--variant 写法。 +- builtin bin 可保留兼容 alias,但内部也使用明确的 builtin provider identity。 + +可选的紧凑糖 namespace:name@version#variant 没有冒号歧义,但建议在显式 flags 与 wire model 稳定后再决定;内部始终是 TemplateSelection,不让紧凑语法成为第三种身份。 + +### 3.5 图形程序:当前已有正确积木,但责任仍有重复 + +当前生态已经包含: + +- mcpp-index 的 ocornut:imguicompat.glfwcompat.glx-runtime。 +- xim-pkgindex 的 xim:graphics、Mesa,以及 NVIDIA/WSL host-link sentinels。 +- Linux Mesa 的软件和多类硬件 backend;macOS native frameworks;Windows Win32/system SDK 路径。 + +问题在于 compat.glx-runtime 仍需要从 SubOS view 选择并 symlink GL 库到自己的 runtime 目录。这是过渡桥,而不是最终模型。最终应让 xlings 持久化可传递 runtime exports,mcpp 直接消费 resolved runtime contract;mcpp-index 只声明依赖,不复制 xlings 的视图布局。 + +Vulkan 还存在更明显的旁路:compat.vulkan-runtime 仍从 host 的 /lib*//usr/lib* 收集 ICD 和 transitive DSOs,测试主要覆盖 loader API,没有证明真实 ICD/device、来源或 GLIBC 闭包。GL 的收敛方案必须同时覆盖 Vulkan loader、ICD manifest 与 driver;否则只是把同类风险移到另一个 API。 + +### 3.6 AUR:外部维护是触发原因,缺少对账才是长期漂移原因 + +2026-08-09 的只读审计显示: + +- GitHub 最新 release 是 [v2026.8.8.4](https://github.com/mcpp-community/mcpp/releases/tag/v2026.8.8.4),AUR RPC 中 mcpp-binmcpp-m 均停在 2026.8.1.1-1,相差 20 个正式 release。 +- 最后成功的 aur-publish run 是 [30649165652](https://github.com/mcpp-community/mcpp/actions/runs/30649165652);从 [30718043364](https://github.com/mcpp-community/mcpp/actions/runs/30718043364) 起到 2026-08-08,共核对到 18 个同类失败、4 个因 release gate skipped;最新失败是 [31254088758](https://github.com/mcpp-community/mcpp/actions/runs/31254088758)。 +- 失败日志均已完成版本与 sha 刷新,随后在 AUR SSH 阶段收到 The AUR is down due to maintenance 并退出 128。 +- 当前 workflow 只有 release 完成事件与手工触发;没有 retry、schedule/reconcile、推送后验证。 + +还存在一个必须先修的独立 P0: + +- scripts/aur/update.sh:65 只匹配单行 sha256sums=('...')。 +- scripts/aur/mcpp-m/PKGBUILD:34-35 是两行数组,所以 source checksum 没有被更新。 +- workflow 生成的 .SRCINFO 使用新 checksum,而 PKGBUILD 仍保留旧 checksum,形成同一提交内部不一致。 +- 当前顺序先推 mcpp-bin 再推 mcpp-m;AUR 恢复后直接重跑可能先产生部分成功,再尝试推送坏的 mcpp-m 元数据。 + +因此不能把“重跑工作流”作为修复。应先增加本地结构化更新与一致性 gate,再恢复发布。 + +### 3.7 多视角评估 + +| 视角 | 当前风险 | 评价 | 目标变化 | +|---|---|---|---| +| 架构 | 高 | xlings、probe、post-install、build plan、run env 多次推导 runtime truth;scaffold 又绕开 canonical resolver | 单一 owner + versioned contract + typed snapshot | +| 稳定性 | 高 | fail-open、忽略 I/O/error_code、配置副作用未入 manifest;外部发布一次失败即永久漂移 | transaction/postcondition/reconciler | +| 兼容性 | 高 | GLIBC 风险取决于实际 DSO;namespace、EnvOp、runtime dirs 在读写端语义漂移 | exact artifact proof + schema negotiation | +| 优雅简洁 | 中高 | CLI 表面不算大,但内部靠短名、目录顺序、字符串命令和目录复制造成隐性复杂度 | 保持少量 CLI,内部用 PackageRef/CommandSpec/LinkIntent | +| 用户体验 | 高 | 成功可能留下半状态;错误只报 GLIBC/包不存在,无法说明来源;机器输出不稳定 | 原子操作、canonical ref、why provenance、稳定 envelope | +| 性能 | 中 | 现有 fast path 很快,但把完整性排除在“命中”之外;runtime/版本探测可能重复 | 本地 hash snapshot、按产物增量验证、无热路径 xlings/network | +| 多平台 | 高 | POSIX shell 片段泄漏到 Windows;Mach-O runtime dirs 与 Linux ELF 语义混用;硬件结论常由 headless/cross build 代替 | LinkIntent 平台 lowering + native cold-home/hardware evidence | + +这里的目标不是用“更严格”换“更慢”。类型化契约让昂贵解析只发生在安装、configure 或 changed artifact 上,hot no-op 只校验 fingerprint 与 required outputs。 + +## 4. 三种架构选择 + +| 方案 | 架构 | 稳定性 | 兼容性 | 简洁性 | 性能 | 多平台 | 结论 | +|---|---|---|---|---|---|---|---| +| A. 按 issue 打补丁 | 局部修改快,但同一事实继续多处推断 | 短期变绿,回归概率高 | 表面影响小,长期漂移大 | 初看简单,维护复杂 | 可维持当前热路径 | 平台分支继续散落 | 只用于 P0 止血 | +| B. xlings 全知 daemon | 所有解析/探测集中 | daemon 生命周期与状态成为新故障域 | 强耦合 xlings 版本 | 接口表面少,系统更重 | 构建热路径多进程/IPC | Windows/macOS 服务语义复杂 | 不采用 | +| C. 分层 typed contracts | 每个事实单一 owner,边界清楚 | 可校验、可缓存、可回滚 | 支持 schema/legacy ingress | 数据模型略增,重复逻辑显著减少 | 构建只读本地契约 | 原生 provider 表达差异 | **推荐** | + +方案 C 可以允许 P0 局部修复先落地,但所有 P0 修复都应朝目标契约收敛,不能制造第二套临时协议。 + +## 5. 目标架构 + +~~~mermaid +flowchart LR + XI[xlings
install / SubOS / RuntimeBinding] --> RC[Runtime & Install Contract] + XP[xim-pkgindex
system payload / host sentinel] --> XI + MP[mcpp-index
C++ package / template / LinkPlan] --> MC[mcpp
planner / builder / inspector] + RC --> MC + MC --> OM[BuildSnapshot / OutputManifest] + MC --> ELF[Artifact Physics Verdict] + GH[Immutable GitHub Release Manifest] --> REC[Channel Reconciler] + REC --> AUR[AUR mcpp-bin / mcpp-m] + REC --> OTHER[other package channels] +~~~ + +### 5.1 单一事实所有者 + +| 事实 | 唯一 owner | 消费者 | +|---|---|---| +| 包 canonical identity、版本与安装 payload | xlings/libxpkg | mcpp、索引工具、模板 | +| runtime binding、loader/libc、exports、env op | xlings + xim package descriptor | mcpp | +| C/C++ source/module/link/template metadata | mcpp-index | mcpp | +| 项目 build graph 与 required outputs | mcpp | IDE、CI、用户 | +| 实际 ELF 闭包与兼容 verdict | mcpp post-link inspector | 用户、CI、缓存 | +| GitHub release 资产与 checksum | Release Manifest | AUR/Homebrew/其他渠道 | +| AUR 当前版本 | AUR | reconciler,仅作为 observed state | + +### 5.2 不可违反的不变量 + +1. **Identity**:内部永远用结构体,不用拼接字符串承担身份。 +2. **Success**:返回成功前,声明的 required outputs 必须存在并通过验证。 +3. **Runtime**:loader、libc 与解析到的共享库必须能追溯到 provider。 +4. **No hot-path orchestration**:构建热路径不启动 xlings、不访问索引网络。 +5. **Atomicity**:cache snapshot、模板目录、投影元数据和渠道更新必须原子提交或明确部分失败。 +6. **Fail closed on ambiguity**:同名候选、多 payload、未知 cfg/placeholder 不按目录顺序或短名猜。 +7. **Native evidence**:Windows/macOS/aarch64 行为只由对应 runner 证明。 + +## 6. 核心数据契约 + +### 6.1 Identity types + +建议共享概念定义,语言实现可独立: + +~~~text +NameAtom = one validated identifier atom +NamespacePath = one or more NameAtom segments +PackageId = { namespace: NamespacePath, name: NameAtom } +PackageRef = { id: PackageId, versionReq?: VersionReq } +ResolvedPackageId= { id: PackageId, version, indexRoute, descriptorDigest, payloadDigest } +ProjectIdentity = { namespace: NamespacePath, name: NameAtom, version } +TemplateSelection= { provider: PackageRef, variant?: NameAtom } +~~~ + +规则: + +- wire/display 规范形式是 namespace:name。 +- bare name 只能在候选唯一时作为便捷输入;歧义必须列出候选并失败。 +- legacy FQN 只在 ingress 解析一次;之后不再拆字符串。 +- map/cache key 使用完整 PackageId。 +- qualified display 由类型派生,不在各层自行加前缀。 +- index route 和 transport provenance 随 ResolvedPackageId 保留用于诊断/lock,但不混进 PackageId 本身。 + +### 6.2 xlings Runtime & Install Contract v2 + +xlings 在安装/物化时写入 versioned、只含相对路径的契约。可以演进现有安装描述文件,也可以使用专用文件;名称不是本文的关键决策,schema 是。 + +建议字段: + +~~~json +{ + "schema": 2, + "package": {"namespace": "xim", "name": "glibc", "version": "2.44"}, + "payload": { + "id": "content-addressed-id", + "digest": "sha256:...", + "root": "." + }, + "runtimeBinding": { + "id": "linux-glibc-2.44-x86_64", + "platform": "linux", + "arch": "x86_64", + "loader": "lib/ld-linux-x86-64.so.2", + "libc": "lib/libc.so.6", + "glibcProvidedCeiling": "2.44" + }, + "exports": { + "includeDirs": [], + "libraryDirs": ["lib"], + "runtimeDirs": ["lib"], + "crtObjects": [] + }, + "runtimeDependencies": [ + {"namespace": "xim", "name": "ncurses", "version": "..."} + ], + "environment": [ + {"name": "PATH", "op": "prepend", "value": "bin"} + ], + "capabilities": ["runtime.opengl", "runtime.egl"], + "provenance": "ecosystem", + "hostObservation": { + "glibc": "2.43", + "fingerprint": "optional-host-fingerprint" + } +} +~~~ + +约束: + +- 所有路径相对 payload root;移动安装根不使契约失效。 +- 契约文件有 content hash;mcpp cache key 使用 hash,不使用 mtime 猜测。 +- SubOS 只引用 active provider contract id,不复制或重新解释 payload 目录。 +- provenance 是封闭枚举:ecosystemsystem-sdkhost-link。 +- 复用 xlings 现有 host_glibc 观测并带入诊断/host-link fingerprint;mcpp 不再忽略该字段,但它不能替代逐 DSO 的 symbol requirement 检查。 +- 旧 schema 可读时告警并转成内存 v2;多 payload/歧义时 fail closed,不再按字典序挑目录。 + +### 6.3 EnvOp schema v2 + +废弃歧义的单一 set: + +事实边界:xlings 自 2026.8.8.2 起,POSIX/fish/PowerShell/进程内路径已经统一为“变量存在即不覆盖”,包括显式空字符串;剩余缺陷是 mcpp reader 仍把 Set 解释成 replace。schema v2 是为了把这一区别永久写入协议,而不是声称 xlings writer 仍未修。 + +| op | 精确定义 | +|---|---| +| default | 变量不存在时设置;变量存在但为空也视为已存在 | +| prepend | 按平台路径分隔符前置,去重并保序 | +| replace | 无条件覆盖,必须显式使用 | + +兼容规则: + +- schema 1 的 set 按 xlings 历史行为映射为 default。 +- xlings writer 与 mcpp reader 同一发布窗口支持 v2。 +- POSIX shell、fish、PowerShell、xlings 进程内应用和 mcpp 读取共享同一组 golden fixtures。 + +### 6.4 BuildSnapshot 与 OutputManifest + +一次配置成功写入原子、版本化 snapshot: + +~~~text +BuildSnapshot + schema + inputFingerprint + graphFile + sourceSetDigest + toolchainContractHash + runtimeContractHashes[] + outputManifest + postLinkVerdicts[] + +OutputManifest + ninjaOwnedArtifacts[] + metadataProjections[] + generatedSources[] + requiredPostconditions[] +~~~ + +fast path 决策: + +- Ninja-owned artifact 缺失:交给 Ninja 依据图重建。 +- compile_commands.json 等 metadata projection 缺失:从 output-dir 中 fingerprint 对应的 canonical copy 原子重投影,不必完整 prepare。 +- graph/snapshot/runtime contract 缺失或 schema 不支持:完整 prepare。 +- 任一 required postcondition 缺失:不得输出 “Finished”。 +- compile_commands.json 加入新项目默认 ignore;已有项目即使未 ignore 也应正确修复。 + +### 6.5 UiPolicy 与 WireEnvelope + +进程启动时仅初始化一次: + +~~~text +UiPolicy { + color: Auto | Always | Never, + quiet: bool, + output: Human | Wire +} +~~~ + +- --no-color 直接得到 Never,后续 TTY 探测不能覆盖。 +- human/status/progress 写 stderr。 +- machine stdout 只允许 versioned envelope;无彩色、无 spinner、无说明文字。 +- pseudo-TTY、Windows console、管道和重定向分别测试。 + +### 6.6 CommandSpec + +所有子进程用结构化 argv,不拼 shell: + +~~~text +CommandSpec { + executable, + argv[], + cwd, + envDelta, + stdinPolicy, + stdoutPolicy, + stderrPolicy +} +~~~ + +2>/dev/null、引号、重定向不再进入参数字符串。POSIX 与 Windows 使用各自 native spawn backend,共享上层语义测试。 + +## 7. 详细设计 + +### 7.1 xlings 候选发现与升级 + +当前 release-bundled xlings 被排除在候选更新链之外。改为显式候选: + +1. CLI/config 显式覆盖。 +2. mcpp release-bundled sibling。 +3. distro/AUR 提供的环境候选。 +4. mcpp writable sandbox 已安装候选。 +5. system PATH。 + +每个候选携带: + +~~~text +XlingsCandidate { + path, + version, + provenance, + writable, + contractCapabilities[], + digest? +} +~~~ + +选择原则: + +- 先满足 required contract capability/schema。 +- 默认不降级。 +- 在可信候选中选择最高兼容版本,而不是只按目录位置。 +- 需要复制到 sandbox 时原子替换并验证 digest/version。 +- mcpp self env 显示候选、被选原因、版本和 provenance。 + +release pin 同时表达两件事: + +- 本次 release 原生 CI 验证过的 exact bundled version。 +- 运行时允许的 minimum contract capability/schema。 + +这样可接受未来兼容版本,又不丢失发布可复现性。 + +### 7.2 构建完整性、C1 和 generated source + +将构建分成三个明确阶段: + +1. **Plan**:解析 identity、source set、toolchain/runtime contract,生成 graph 与 snapshot。 +2. **Execute**:Ninja/runner 生产 artifacts 和 generated sources。 +3. **Verify/Project**:检查 OutputManifest、运行 post-link inspector、原子投影 CDB/机器元数据。 + +generated action 必须声明 outputs;其输出并入下一轮 SourceSet 或明确标为最终产物。没有声明或声明后缺失均失败,不能静默排除。 + +默认 source glob 只由一个 SourceSet producer 展开;清单、scanner、Ninja graph 不再各自 glob。 + +性能要求: + +- snapshot 校验按 digest/stat 快速路径完成。 +- CDB 重投影是文件复制/rename,不重新求解全部依赖。 +- no-change build 目标维持 < 0.5 s,或相对当前基线不回退超过 10%;两者取更严格、但先在 CI 固定硬件建立基线。 + +### 7.3 C++ scanner + +用 O(bytes) streaming lexical masker 代替启发式 strip,状态至少包括: + +- Normal +- line comment +- block comment +- string / char / escape +- raw string delimiter / raw body +- line splice 与 CRLF 处理 + +masker 保留换行与列宽,将注释/字符串内容替换为空白;module/import parser 只读取 masked code。注释只可在 Normal 状态开始。 + +回归 corpus 必含: + +- 行注释和块注释中的 R"(。 +- 普通字符串中的 /*//。 +- raw string 中的假 import。 +- char literal、encoding prefix、自定义 raw delimiter。 +- CRLF、行拼接、文件末尾未闭合状态。 + +scanner 保持热路径无编译器子进程;若未来接 P1689,编译器扫描作为权威慢路径/不确定输入 fallback,不与 masker 产生第三套模块身份。 + +### 7.4 exports 与 namespace + +SourceUnit.owner、manifest owner、dependency owner 全部改为 PackageId。校验使用结构体相等;诊断需要字符串时调用一个 formatter。 + +这个改动同时解决: + +- C4 namespaced exports 被跳过。 +- namespace 被重复前缀化。 +- 模板依赖注入回退短名。 +- cache/graph 在同短名包之间碰撞。 + +### 7.5 脚手架与模板 + +#### 7.5.1 CLI + +推荐稳定表面: + +~~~text +mcpp new + [--namespace ] + [--dir ] + [--template ] + [--variant ] +~~~ + +交互输出必须同时展示 provider 的 qualified identity 和 variant;--list-templates 的 wire 模式返回结构化数组。 + +#### 7.5.2 renderer + +RenderVars 至少包含: + +~~~text +project.name +project.namespace +project.qualifiedName +project.version +template.provider.namespace +template.provider.name +template.provider.qualifiedName +template.variant +~~~ + +实现要求: + +- parse template 成 token stream,一次替换,不扫描插入值。 +- 未知 token hard error;需要 literal token 使用明确 escape。 +- 文件路径渲染也走同一验证器,禁止绝对路径、.. 和目录逃逸。 +- 依赖注入操作 TOML AST,不用字符串查找;写出完整 namespace 分组。 +- 模板 descriptor 声明兼容的 mcpp contract/version、必需 capability 与平台。 + +#### 7.5.3 transaction + +流程: + +~~~text +parse and validate input + -> resolve exact template provider + -> materialize to sibling temp dir + -> render once + -> parse generated mcpp.toml + -> validate required files and no path escape + -> optional offline configure smoke + -> atomic rename to destination +~~~ + +任何错误返回非零并删除 temp;诊断给出字段、非法值和允许形式,不留下半项目。 + +#### 7.5.4 compatibility + +- 旧 pkg@ver:variant 支持两个 release train,仅限能唯一解析的无 namespace provider。 +- 第一个 release train warning;第二个可通过 compatibility flag 使用;之后删除。 +- 模板索引新增 canonical identity,不原地改变旧模板含义。 + +### 7.6 Artifact Physics Inspector + +当前 pre-link driver -### 检查只看到“编译器计划”,不能证明最终 ELF。新增 post-link inspector: + +1. 读取实际 PT_INTERPDT_NEEDED、RPATH/RUNPATH。 +2. 根据 artifact 路径、Runtime Contract 和平台规则递归解析动态闭包。 +3. 给每个对象标注 payload/provider/provenance。 +4. 从 .gnu.version_r 计算每个 host-borrowed object 的 GLIBC requirements。 +5. 验证 Rule B 与 Rule A。 +6. 生成 versioned verdict,按 artifact digest + Runtime Contract hash 缓存。 + +可复用 vendored patchelf 获取 interp/rpath/needed;GLIBC version requirement 建议实现最小只读 ELF parser,避免假设用户安装 readelf,也避免仅用系统 glibc 版本近似。 + +诊断示例信息应包含: + +~~~text +artifact +selected loader and provider +selected libc and provider +offending object +required GLIBC symbol ceiling +provided runtime ceiling +resolution path and remediation candidates +~~~ + +build fast path 必须把 verdict 当作 required output;artifact 或 Runtime Contract hash 变化后失效。 + +allow_host_libs 可以调整 Policy D 的告警/允许范围,但不能关闭物理 A/B:配置项不能让不可装载的 ELF 变得可运行。 + +上线分四步: + +1. **observe**:只记录闭包与 provider,对照 linker trace/map、readelf/patchelf fixtures。 +2. **warning**:对 proven mismatch 报高质量诊断,但保留显式 strict opt-in。 +3. **strict opt-in**:在 mcpp/xim-index native matrix 中积累低误报数据。 +4. **default strict**:只对精确解析得到的 mismatch/unresolved required object hard fail;解析器自身无法证明的情况返回“inconclusive”,不得假装兼容,也不得用 our_glibc ≥ host_glibc 代理误拒绝。 + +### 7.7 GUI / graphics 分层 + +当前 [runtime] 的 soname、capability、provider 和目录大多是扁平字符串:provider 只记 short name,library_dirs 又同时影响 -L 与 RPATH,capability 还可能同时表示 requires 与 provides。目标模型先拆开“需求”和“已解析实物”: + +~~~text +RuntimeRequirement { + kind: soname | capability | icd_manifest | display | host_service, + value, + phase: link | run, + required, + target, + requester: PackageId +} + +RuntimeArtifact { + role: loader | library | driver | manifest | host_bridge, + relativePath, + provider: ResolvedPackageId, + provenance: payload | subos_view | host_link | system_sdk, + abi, + requiredGlibcCeiling?, + digest?, + hostFingerprint? +} +~~~ + +链接和运行目录也必须分义: + +~~~text +link.libraryDirs +link.transitiveNeededDirs +runtime.rpathDirs +runtime.dlopenDirs +runtime.environment +deploy.files +~~~ + +runtime 目录不得隐式进入 -L。ELF lowering 分别生成 link search、-rpath-link 和 RPATH/RUNPATH;Mach-O 使用 @rpath/install name;PE 使用 app-local DLL 与 system DLL contract。resolution.json 输出 requirement → canonical provider → artifact → search mechanism → provenance → ABI verdict 的完整链。 + +#### 7.7.1 层次 + +| 层 | 职责 | 禁止事项 | +|---|---|---| +| xlings | payload、SubOS view、RuntimeBinding、runtime exports、host sentinel 激活 | 不理解 ImGui/GLFW 项目语义 | +| xim-pkgindex | Mesa/系统运行时闭包、NVIDIA/WSL host-link sentinel、平台系统包 | 不生成 mcpp 工程 | +| mcpp-index | ImGui/GLFW 等 C++ 包、LinkPlan、模板、平台依赖选择 | 不复制/symlink xlings 内部 view 作为长期 ABI | +| mcpp | 根据 capability 规划、链接、artifact physics、why 诊断 | 不探测 GPU 型号或选择驱动 | + +#### 7.7.2 平台策略 + +- **Linux**:xim:graphics 作为图形运行时入口;Mesa 默认闭包;NVIDIA/WSL 通过 sentinel 表达显式 host-link。 +- **macOS**:使用系统 SDK/framework capability;不伪装成 Linux Mesa 布局。 +- **Windows**:使用 Win32/system SDK 与明确 DLL runtime contract;命令执行和路径由 native backend。 + +模板声明平台依赖,不在源码中散落“如果 Linux 就手工找 libGL”。 + +ImGui 包本身也应按 feature 解耦: + +- core/headless 不拉窗口系统。 +- backend-glfw-opengl3 显式引入 GLFW、OpenGL headers 与 runtime。 +- backend-vulkan 显式引入 Vulkan loader + ICD requirement。 +- app 可组合默认 backend;docking/viewports 是正交 feature。 + +当前 ocornut:imgui 的 app facade/后端依赖仍容易让只用 core 的项目拉入 graphics 栈,这一拆分应由 mcpp-index 完成,不放进 xlings。 + +#### 7.7.3 过渡 + +compat.glx-runtime 的 symlink bridge 暂时保留: + +1. xlings contract v2 能表达 resolved transitive runtime exports。 +2. mcpp 能直接消费并生成正确 LinkPlan/runtime dirs。 +3. mcpp-index native tests 证明不再需要桥。 + +然后在一个有 warning 的 release train 中弃用,避免双份 runtime dir 漂移。 + +#### 7.7.4 测试矩阵 + +| 层级 | Linux | macOS | Windows | +|---|---|---|---| +| 常规 PR | headless llvmpipe configure/build/run;artifact A/B | native framework build + minimal smoke | Win32 backend build + minimal smoke | +| 原生/计划任务 | X11/Wayland;AMD/Intel/NVIDIA;WSL2 | arm64 + x86_64 native window | x64/arm64 native window | +| side-effect | 安装 graphics 前后无关 ELF 的 interpreter 与 GLIBC ceiling 不变 | SDK selection 不污染全局 env | DLL/path contract 不污染宿主 shell | + +Vulkan native gate 至少创建 instance、枚举 physical device,并验证实际加载的 ICD manifest/driver provenance;只测试 loader symbol 不算通过。真实 GPU 行为只能由对应 native runner 证明;Linux headless 通过不外推成所有硬件通过。当前 xim:graphics recipe 明确仅支持 Linux x86_64,所以 aarch64 在支持落地前应返回清晰的 capability unavailable,而不是把 cross-build 或 skipped runtime test 汇总成绿色。 + +### 7.8 安装 owner 与错误传播 + +删除 C8 的两段 versionless 预安装循环。唯一流程: + +~~~text +parse PackageRef + -> solve exact graph + -> install/materialize exact nodes through xlings + -> verify install contracts + -> expose graph to planner +~~~ + +- 任何节点失败立即带完整 PackageId/version/provenance 返回。 +- 不允许 catch (...) {} 或忽略 error_code 后继续。 +- 离线模式只使用已验证 materialization;缺失时明确报错。 + +### 7.9 Release Manifest 与 AUR reconciler + +#### 7.9.1 Release Manifest + +release 成功后发布不可变 manifest: + +~~~json +{ + "schema": 1, + "version": "2026.8.8.4", + "tag": "v2026.8.8.4", + "commit": "...", + "assets": [ + {"name": "...", "sha256": "...", "platform": "...", "arch": "..."} + ], + "bundled": {"xlings": "2026.8.8.1"} +} +~~~ + +AUR 生成器只消费 manifest,不重新从日志/文件名猜版本和资产。 + +mcpp-m 应消费 release 自己发布且带 sidecar 的版本化 source asset,不再依赖可能重生成的 tag archive;生成器下载实物重算 hash,并与 manifest/sidecar 交叉验证。 + +#### 7.9.2 先修 P0 生成器 + +- 不再用行级 sed 更新 Bash array;使用小型、确定性生成器从 model 完整生成 PKGBUILD 与 .SRCINFO。 +- 在推送前下载/校验 source archive 与二进制资产。 +- 比较 PKGBUILD 解析结果与 .SRCINFO:version、source URL、每个 checksum 必须一致。 +- 在 Arch container 中运行 makepkg --printsrcinfo 并与提交文件 diff。 +- 在 Arch container 中运行 makepkg --verifysource;PKGBUILD 是源,.SRCINFO 只由 makepkg 生成。 +- 分别构建/安装 mcpp-binmcpp-m 的最小 smoke。 +- 所有 package 预检全部通过后才允许任何 push,避免 mcpp-bin 先成功、mcpp-m 后失败。 +- dry-run 默认不加载 SSH secret、不 push,只输出 desired/current、生成文件、diff 和验证报告。 + +#### 7.9.3 对账工作流 + +触发: + +- release workflow 成功事件:低延迟路径。 +- schedule:建议每 6 小时。 +- workflow_dispatch:恢复/指定版本,但默认仍以 latest stable manifest 为准。 + +算法: + +~~~text +read latest immutable Release Manifest + -> query AUR RPC and git heads for every managed package + -> compare desired/current with Arch vercmp; reject implicit downgrade + -> generate all expected repositories in temp dirs + -> validate all package postconditions + -> for each drifting package, push idempotently with bounded retry/backoff + -> query AUR RPC/git again + -> succeed only if observed state matches expected state +~~~ + +行为: + +- AUR maintenance、连接超时等 retryable failure 使用指数退避与抖动。 +- auth、invalid metadata、checksum mismatch 属于 permanent failure,立即失败并告警。 +- 每个 package 的 push 可独立重试,但总体状态明确显示 partial convergence。 +- GitHub Release 是 desired state,AUR 是 observed projection;无需再维护一份可漂移的 mutable ledger。 +- 错过中间 release 时允许直接收敛 latest stable,不要求重放全部历史版本。 +- workflow_run 只负责唤醒;每次都重新读取最新完整、非 draft、非 prerelease manifest,迟到事件不得降级 AUR。 +- 已知 AUR package clone 失败时不得自动当作“首次发布”初始化空仓;首次认领必须是独立、显式流程。 +- 读取可走 HTTPS,SSH 只用于 push;固定官方 host key,不在发布时盲信动态 ssh-keyscan。 +- 使用专用发布身份/Ed25519 key;普通 PR 与不受信任代码永远拿不到 secret。 +- AUR git head 是立即验证源,RPC 允许 bounded poll;RPC 延迟不得触发重复提交。 +- 使用普通 fast-forward push,禁止 force-push/历史改写。 + +建议 SLO(待 review): + +- AUR 可用时,release 后 30 分钟内收敛。 +- 事件失败后,scheduled backstop 最迟 6 小时再次尝试。 +- 连续 24 小时未收敛触发高优先级告警。 + +mcpp-git 是否发布是独立产品决策;若保留,必须加入 managed package matrix 和相同 postcondition,不能只在仓库里放模板。 + +## 8. 兼容与迁移策略 + +### 8.1 双读单写 + +- xlings/mcpp 在迁移窗内读取 schema 1 和 2,只写 schema 2。 +- build cache schema 改变视为 cache miss,不尝试就地猜测迁移。 +- 唯一可转换的 legacy identity 在 ingress 转换;歧义立即失败。 +- 旧模板语法有明确两个 release train 的弃用窗口。 + +### 8.2 Fail-open 与 fail-closed 边界 + +| 情况 | 行为 | +|---|---| +| 缺少可再生成的 CDB projection | 本地重投影 | +| 旧但唯一可转换的 contract | 转换 + warning | +| 多 payload、同短名多候选 | fail closed | +| Rule A/B 已证明的物理冲突 | 经过 observe/warning 迁移后 hard fail;解析不确定单独报告 inconclusive | +| Policy D 仅在本地用户项目违反且 A/B 安全 | 默认清晰 warning;索引 CI hard fail | +| 未知 cfg、placeholder、output | hard fail | +| AUR 外部维护 | retry + scheduled reconcile,不回滚 GitHub release | + +### 8.3 可回滚性 + +- contract/snapshot 都有 schema 与原子文件;回滚二进制时旧 reader 忽略不支持的新文件并重新 materialize。 +- symlink bridge 只在新 runtime export 经过至少一个 release train 后移除。 +- 渠道 reconciler 生成提交前保存 expected diff;不 force-push AUR 历史。 +- 不通过修改全局 host env 回滚 runtime;切换 active RuntimeBinding。 + +## 9. 分阶段交付 + +### Phase 0 — 止血,不等待完整架构 + +1. mcpp:C1 输出完整性检查/CDB repair;C2 masker;C3 color;C4 PackageId 比较;C5 CommandSpec;删除 C8 循环。 +2. scaffold:#380 NameAtom、路径约束、单次渲染、临时目录事务。 +3. AUR:修复 mcpp-m checksum 生成;增加 PKGBUILD/.SRCINFO 一致性 gate;全部预检后再 push;再人工恢复一次对账。 +4. release:先完成 C5/C6 的候选/版本探测闭环,再将 bundled xlings 升到已验证的 2026.8.9.x,并跑冷 HOME 原生 gate;不能只改 pin。 + +每项独立小 PR,避免 P0 被 contract v2 设计阻塞。 + +### Phase 1 — Identity 与本地契约 + +1. PackageId/PackageRef/ProjectIdentity 全链路。 +2. 新 --template PackageRef --variant 表面和 legacy ingress。 +3. BuildSnapshot/OutputManifest。 +4. EnvOp schema v2 golden fixtures。 +5. xlings Runtime & Install Contract v2 writer;mcpp 双 reader。 +6. XlingsCandidate capability/provenance 选择。 + +### Phase 2 — Runtime physics 与 graphics + +1. post-link ELF parser/verdict 与 mcpp why runtime。 +2. xim-pkgindex Policy D CI。 +3. mcpp 读取 transitive runtime exports。 +4. Linux llvmpipe/native 图形矩阵。 +5. 弃用 compat.glx-runtime symlink bridge。 + +### Phase 3 — 脚手架 UX 与原生 GUI vertical slice + +1. TOML AST 依赖注入与模板 contract。 +2. ImGui/GLFW canonical template。 +3. Linux、macOS、Windows 各自一条从 mcpp new 到真实运行的冷 HOME vertical slice。 +4. 机器输出与 IDE 消费统一。 + +### Phase 4 — 发布渠道最终一致性 + +1. Release Manifest。 +2. AUR event + schedule reconciler、retry 分类、推后验证。 +3. 将 Homebrew/其他渠道逐步迁到同一 manifest 模型。 +4. 发布 dashboard 展示 GitHub release、索引、AUR 和 bundled xlings 的期望/实际版本。 + +## 10. 跨仓库 PR 切分建议 + +| 顺序 | 仓库 | 小 PR 主题 | 依赖 | +|---|---|---|---| +| 0A | mcpp | AUR generator/checksum consistency + dry-run validation | 无 | +| 0B | mcpp | C1/C3/C5 快速修复与回归测试 | 无 | +| 0C | mcpp | scanner masker(C2) | 无 | +| 0D | mcpp | scaffold transaction(#380) | 无 | +| 1A | xlings | install/runtime contract v2 + EnvOp v2 writer | 设计字段冻结 | +| 1B | mcpp | contract v1/v2 reader + candidate selection | 1A fixtures | +| 1C | mcpp/libxpkg consumer | PackageId 全链路与模板 CLI | identity spec | +| 1D | mcpp | BuildSnapshot/OutputManifest | 0B | +| 2A | mcpp | ELF actual-artifact inspector | 1B | +| 2B | xim-pkgindex | Policy D closure CI + sentinel schema | 1A | +| 2C | mcpp-index | graphics runtime exports 消费,移除桥的准备 | 1A/1B/2B | +| 3A | 三平台仓库矩阵 | GUI cold-home vertical slices | 1C/2A/2C | +| 4A | mcpp release | Release Manifest + AUR reconciler | 0A | + +协调者负责 schema fixtures、跨仓集成顺序和最终 native gate;各 PR 保持可独立回滚,不改写历史。 + +## 11. 验收标准 + +### 11.1 功能与回归 + +- #380 的 hang、路径逃逸、控制字符、部分目录均有回归测试。 +- 同短名不同 namespace 的 package/template 可同时存在,bare ambiguity 明确失败。 +- 删除 artifact、CDB、graph、verdict 任一项后,下一次 build 能正确重建/重投影或完整 prepare。 +- C2 最小 corpus 及现有真实项目 corpus 零 pass-to-fail。 +- --no-color 在 TTY、pipe、Windows console 均无 ANSI。 +- EnvOp golden fixtures 在 xlings 与 mcpp 结果字节一致。 +- A/B 测试含:同 binding 成功、loader/libc 混源失败、host object ceiling 高于 runtime 失败、低于等于成功。 +- 安装 graphics 不改变无关 ELF 的 interpreter/provider/GLIBC ceiling。 + +### 11.2 用户路径 + +每个平台都从隔离 HOME/MCPP_HOME/XLINGS_HOME 开始: + +~~~text +install released mcpp + -> verify selected bundled/upgraded xlings + -> mcpp new namespaced GUI project + -> resolve/install dependencies + -> configure/build + -> run platform-appropriate smoke + -> delete CDB/artifact and verify repair + -> inspect mcpp why runtime / wire output +~~~ + +不得用已缓存开发机状态替代。 + +### 11.3 性能 + +- scanner O(bytes),以大 translation-unit corpus 防止超线性回退。 +- no-change build 不启动 xlings、不访问网络。 +- artifact physics 仅在新 artifact 或 contract hash 变化时运行。 +- 固定硬件建立 median/p95 基线后,hot no-op build 回退不超过 max(5%, 10 ms);缺 CDB 只允许一次轻量 reconfigure/project。 +- 增量链接的 artifact inspector 额外成本目标不超过 max(5%, 50 ms/产物);hot no-op parse 次数必须为 0。 +- contract 解析与 snapshot 校验有单独 benchmark,避免 JSON/磁盘布局成为热路径瓶颈。 + +### 11.4 多平台与发布 + +- Linux x86_64/aarch64、macOS arm64/x86_64、Windows x64 至少有原生 cold-home gate。 +- 不从 Linux source review 推断 Windows spawn/console 正确。 +- release 完成后验证 remote tag/commit、assets、checksums、bundled xlings、索引版本。 +- AUR push 后同时验证 AUR git head 与 RPC version;只看到 workflow green 不算渠道收敛。 + +## 12. 可观测性与用户体验 + +新增统一解释命令,优先扩展现有 mcpp self/mcpp why,不创建大量顶层命令: + +~~~text +mcpp self env + selected xlings, all candidates, provenance, contract schema + +mcpp why package namespace:name + normalized PackageRef, selected version, source index, dependency path + +mcpp why runtime + loader/libc provider, dynamic closure, host-link leaves, A/B verdict + +mcpp build --output wire + versioned event/result envelope only +~~~ + +human 模式先给解决动作,再给细节。例如 loader/libc 混源时直接指出是哪个对象、来自哪里、需要哪个 runtime contract,而不是只输出 “GLIBC not found”。 + +## 13. 风险与缓解 + +| 风险 | 缓解 | +|---|---| +| schema v2 同时改 xlings/mcpp,发布错位 | 双读单写、共享 fixtures、capability negotiation | +| typed identity 改动面大 | 先在 ingress/graph 边界引入,禁止新增裸字符串 key | +| ELF parser 容易遗漏格式 | 最小只读范围、fixture 与 readelf/patchelf 对照、fuzz | +| Policy D 过严伤害用户自定义宿主库 | 只在 index CI hard fail;用户项目区分 A/B 与 D | +| GUI native CI 不稳定 | headless PR gate + 有标签的 native scheduled gate,分别报告 | +| AUR 部分发布 | 全包预检、独立状态、post-push reconcile;不 force-push | +| contract 使 fast path 变慢 | content hash 缓存、只读本地文件、按 artifact digest 复用 verdict | + +## 14. Review 需要确认的决策 + +1. 是否接受 **方案 C:分层 typed contracts**,并明确拒绝 xlings daemon 化? +2. 是否接受模板 CLI 将 provider 与 variant 分离:--template namespace:name@version --variant name?是否还需要后续提供 #variant 紧凑糖? +3. 是否接受项目逻辑 namespace、项目名和输出目录分别由 --namespace、位置参数、--dir 表达? +4. 是否接受 Rule A 按实际 host object 的 GLIBC symbol requirement 计算,而不是比较两端 glibc 发行版本? +5. 是否接受 A/B 在用户构建 hard fail、Policy D 只在索引 CI hard fail的分层? +6. 是否接受 mcpp 不做 GPU 探测,GPU/宿主驱动选择只由 xim sentinel/provider 负责? +7. 是否接受 Runtime & Install Contract v2 使用相对路径、content hash、双读单写迁移? +8. 是否接受 AUR 先修 generator consistency,再启用 6 小时 scheduled reconcile;30 分钟/6 小时/24 小时作为初始 SLO? +9. mcpp-git 是正式维护的第三个 AUR package,还是从自动发布范围明确移除? +10. Phase 0 的顺序是否同意:AUR 元数据安全、C1–C5/C8、#380 事务、bundled xlings 升级并行止血? + +## 15. 证据锚点与验证边界 + +下列行号对应本文基线,后续代码移动时以符号为准: + +| 主题 | 当前实现证据 | +|---|---| +| C1 fast path / CDB | src/build/execute.cppm:705-774src/build/ninja_backend.cppm:1547-1549src/build/compile_commands.cppm:282-307 | +| C2 scanner | src/modgraph/scanner.cppm:137-189,579-590 | +| C3 color | src/cli.cppm:97-108src/ui.cppm:233-240,266-331 | +| C4 exports | src/modgraph/scanner.cppm:786-792src/modgraph/validate.cppm:90-118 | +| C5 process | src/fallback/xlings_binary.cppm:156-183src/platform/process.cppm:191-226,335-350 | +| C7 EnvOp | src/xlings/subos_info.cppm:250-300tests/unit/test_subos_info.cpp:290-310 | +| C8 dead loops | src/toolchain/lifecycle.cppm:551-565src/build/prepare.cppm:1664-1670src/pm/package_fetcher.cppm:923-936 | +| #380 | src/scaffold/create.cppm:183-298src/scaffold/template.cppm:134-182 | +| template identity | src/scaffold/template.cppm:22-47,125-228src/scaffold/create.cppm:30-141 | +| #392 selection | src/toolchain/post_install.cppm:378-388src/xlings.cppm:35-48 | +| #396 pre-link only | src/build/hermetic.cppm:103-211src/build/ninja_backend.cppm:1575-1582 | +| runtime flattening | src/manifest/types.cppm:455-467src/build/plan.cppm:631-712src/build/flags.cppm:709-724,860-885 | +| AUR trigger/push | .github/workflows/aur-publish.yml:14-102 | +| AUR checksum defect | scripts/aur/update.sh:61-66scripts/aur/mcpp-m/PKGBUILD:32-35 | + +本次只读交叉审计没有执行会安装 graphics 或创建工程的命令,也没有在 WSL2、AMD/Intel/NVIDIA、macOS MoltenVK、Windows Vulkan 或 Linux aarch64 graphics 上实跑。mcpp 基线已有的 7 个被触发 Actions workflow 全绿,只能证明现有覆盖集;它们没有针对性覆盖上述大部分缺陷,native aarch64 在该提交也未触发。因此本文不建议仅凭现有 CI 关闭 #397/#396/#392/#380。 + +## 16. 相关既有设计 + +本文是跨问题的收敛层,不替代以下文档的细节: + +- .agents/docs/2026-08-07-xlings-as-runtime-substrate-design.md +- .agents/docs/2026-08-08-payload-version-and-contract-drift-design.md +- .agents/docs/2026-08-08-machine-readable-output-protocol-design.md +- .agents/docs/2026-08-05-build-mcpp-extensibility-architecture.md +- xlings 的 .agents/docs/2026-08-09-ecosystem-closure-design.md + +如果本文获批,下一步不是直接开启一个大实现,而是把第 14 节决策写成冻结的 ADR/schema fixtures,再按第 10 节拆分小 PR。 diff --git a/.agents/docs/2026-08-10-pr400-completion-design.md b/.agents/docs/2026-08-10-pr400-completion-design.md new file mode 100644 index 00000000..c1942d62 --- /dev/null +++ b/.agents/docs/2026-08-10-pr400-completion-design.md @@ -0,0 +1,613 @@ +# PR #400 收尾设计方案 —— 重新判定阻塞点,并把串行收口改成并行 + +> 日期:2026-08-10 +> 对象:[mcpp-community/mcpp#400](https://github.com/mcpp-community/mcpp/pull/400)(Draft,HEAD `64803fc`) +> 关联:#398(实施)、#397(汇总)、#380/#392/#396、 +> [mcpplibs/mcpp-index#197](https://github.com/mcpplibs/mcpp-index/pull/197)(已合)、 +> [openxlings/xlings#521](https://github.com/openxlings/xlings/pull/521)(Draft) +> 本文只记录可公开的仓库事实,不含本机路径、用户名、凭据。 + +--- + +## 0. 这份文档做什么 + +PR #400 的实现主体已经完成(31 commit、111 文件、+15085/−1692)。现有交接文档 +`.agents/docs/2026-08-09-pr400-handoff-zh.md` 把"还没做完的事"列得很完整,但它对**唯一一个 +mcpp 内部失败**的根因判定是错的,而整条收口路线又建立在"等 xlings 发版"这个跨仓库串行依赖上。 + +所以本文不重述已完成的实现,只做三件事: + +1. **§1** 用 CI 原始日志重新判定 Windows E2E 2/2 的根因(结论与 PR 正文不同); +2. **§2** 给出 review 结论,标出**必须在合并前处理**的项; +3. **§3–§5** 给出把串行等待改成并行推进的收口设计与分阶段计划。 + +### 0.1 基线事实(已核验) + +| 项 | 值 | 证据 | +|---|---|---| +| PR #400 HEAD | `64803fc` | `gh pr view 400` | +| main | `80291ca` | `git log` | +| main 的 Windows E2E | **success** | run `31275102045` | +| `64803fc` 矩阵终态 | **13 pass / 5 fail** | `gh pr checks 400` | +| 4 个 fail | 全部是裸 `ftxui` exact miss | job `93271797138` / `93271798469` / `93271802557` / `93271790099` | +| 第 5 个 fail | `12_add_command.sh` (exit 2) | job `93271793304` | +| `bare Windows: no Visual Studio` | 已 **pass**(`ed4cf64` 时是 cancelled) | job `93272767480` | + +`64803fc` 比交接文档记录的 `ed4cf64` 快照好一格:cancelled 已消失,仍是 5 个 fail。 + +--- + +## 1. 重新判定:Windows E2E 2/2 的根因不是 workspace 索引继承 + +### 1.1 PR 现在的说法 + +PR 正文与交接文档 §5.5 都写:失败发生在 step (14),workspace member 读不到根 +`[indices] acme`;`ed4cf64` 用 lexical anchor 修但没修好;`9a47ccf` 加 `route:` 诊断, +"等原生 Windows 输出 root/pkgs 状态后再做最小修复"。 + +**证据已经到了**(run `31324173520`),但它指向另一处。 + +### 1.2 原始日志 + +``` +2026-08-09T16:39:28.5388080Z error: package 'acme.util' not found in any configured index +2026-08-09T16:39:28.5388719Z tried: (acme, util) +2026-08-09T16:39:28.5389134Z route: local index 'acme': root absent, pkgs absent +2026-08-09T16:39:28.5946779Z FAIL: 12_add_command.sh (exit 2, 2.06s) +``` + +### 1.3 四条独立证据都排除 step (14) + +**(a) 退出码。** step (14) 被包在 `|| { echo …; exit 1; }` 里: + +```bash +workspace_add=$("$MCPP" add acme.util@2.0.0 2>&1) || { + echo "$workspace_add" + echo "workspace member could not read its root-owned local index" + exit 1 +} +``` + +它失败只可能是 **exit 1**。日志是 **exit 2** —— 那是 mcpp 自己的退出码经 `set -e` 直传, +只有 `"$MCPP" add … > /dev/null` 这种裸调用才会这样。 + +**(b) 缺失的横幅。** step (14) 失败必然打印 +`workspace member could not read its root-owned local index`,日志里没有。 +同一 harness 会显示脚本自身的 stdout —— 同一个 job 里 +`118_purview_include_rebuild.sh` 的 `windows: depfile degradation reported as expected` +就是脚本 `echo` 出来的。 + +**(c) 顺序。** 失败前最后两条未被捕获的 mcpp 输出是 step (5) 的 `capi.lua` 迁移 warning +(16:39:27.10)和 step (12) 的 `unknownidx.thing` warning(16:39:27.98)。 +step (13)/(14) 的输出全部被 `$( )` 捕获,所以不出现。下一条落到日志的 stderr +就是 step (15)。 + +**(d) `route:` 行本身。** step (15) 的 fixture 在 +`tests/e2e/12_add_command.sh:250` 写的是: + +```bash +[indices] +acme = { path = "$TMP/myapp/index" } +``` + +`$TMP` 来自 `mktemp -d`。在 Git Bash 下它是 **MSYS 路径** `/tmp/tmp.XXXX`, +而被写进 `mcpp.toml` 的是**文件内容**,MSYS 的 argv/env 路径转换对它不生效。 +原生 `mcpp.exe` 拿到 `/tmp/tmp.XXXX/myapp/index`,按 Windows 语义那是"当前盘根相对", +即 `C:\tmp\tmp.XXXX\myapp\index` —— 不存在。 + +**这正好就是 `root absent, pkgs absent`。** + +### 1.4 为什么 main 是绿的 + +step (15) 是本 PR **新增**的(`git diff origin/main...HEAD` 中该段全为 `+`)。 +step (14) 在 main 上就存在且一直通过。 + +全仓 e2e 里,只有这一处把绝对 `$TMP` 路径写进 manifest **并且在 Windows 上真正通过它解析包**: + +- `43_indices_lockfile.sh:45` 也写绝对 `$INDEX_DIR`,但整个测试只做文本断言,从不解析包 → Windows PASS; +- `163_identity_first_resolution.sh` 用相对路径 `../idx1` → Windows PASS; +- `121_default_ns_redirect.sh` / `203_exact_selector_lock_migration.sh` / `44_*` 等 + 被 `# requires: gcc` 挡掉 → Windows SKIP。 + +### 1.5 为什么 Wine 复现是假绿 + +交接文档 §5.5 记录:"下载该 job 的 Windows artifact,在隔离 Wine 中重放,两条 +`mcpp add acme.util@2.0.0` 都通过。" + +Wine 默认把 **`Z:` 映射到 `/`**。于是 `\tmp\tmp.XXXX\myapp\index` 落回真实的 +`/tmp/tmp.XXXX/myapp/index`,fixture 可读,测试自然过。 + +> **方法论条目(值得写进 CLAUDE.md / 记忆):Wine 能验证 PE 产物"能跑", +> 不能验证"路径语义"。** 凡是结论依赖盘符、驱动器映射、`/` 开头路径解释、 +> 8.3 短名、UNC 的问题,Wine 通过 = 无信息,不是证据。 + +### 1.6 推论:`ed4cf64` 是一次无证据的产品改动 + +`ed4cf64` 把 `src/project.cppm` 的 `inherit_workspace_indices` 里的 + +```cpp +idx.path = std::filesystem::weakly_canonical(wsRoot / idx.path); // main,绿 +``` + +改成 + +```cpp +idx.path = (wsRoot / idx.path).lexically_normal(); // PR +``` + +理由写的是"weakly_canonical 可能把 Windows 短名/大小写路径改写成另一种拼写"。 + +对照事实: + +- **同样的失败在 `ed4cf64` 之前之后完全一致** —— run `31318089536`(`0108ee19`,改动之前) + 的错误文本与 `31321961040`(`ed4cf64`,改动之后)逐字相同; +- main 带着 `weakly_canonical` 是绿的; +- 该改动配的单测 `PmIndexRoute.WorkspaceMemberReadsRootAnchoredRelativeIndex` + (`tests/unit/test_pm_index_route.cpp:184`)里有一行 + `EXPECT_EQ(indices.at("acme").path, (root / "index").lexically_normal());` + —— 它断言的是**实现细节的路径拼写**,而不是能力。这正是它抓不到真 bug 的原因。 + +所以:**这条改动目前是"为了一个不存在的原因、改掉一个当时是绿的行为",且没有任何 +测试能证明它有必要。** 它不一定错(`weakly_canonical` 会解符号链接, +参见记忆 `issue344-cache-object-address`:`fs::relative` 解符号链接曾让包静默退出缓存), +但**它必须要么被证明,要么被还原**,不能就这么留在 diff 里。 + +--- + +## 2. Review 结论 + +按"合并前必须处理 / 合并前应处理 / 记录即可"三档。 + +### R1 · P0 · 修在了控制流到不了的地方 + +见 §1。当前 PR 的下一步动作("等 Windows route 证据 → 修 workspace 继承") +如果照做,会在一个本来就没坏的路径上继续改代码。 + +**处理**:按 §3 主线 A。 + +> 这是记忆 `repair-placed-where-flow-never-reaches` 的同一形状第 12 次出现: +> **修补被放在控制流永远到不了的地方。** 这次的变体是"断言的失败位置被错认"。 + +### R2 · P0 · 裸名精确化 = 让**已经发布**的数据失效 + +这是本 PR 最大的产品风险,而且它已经在用四个红 job 报警。 + +现状(`tests/e2e/162_bare_name_namespace_scope.sh` 的 diff 就是判据): + +| 写法 | main | PR #400 | +|---|---|---| +| `gtest = "1.15.2"` | 解析到 `compat.gtest` | **硬失败** | +| `ftxui = "6.1.9"` | 解析到 `compat.ftxui` | **硬失败** | +| `compat.gtest` | 解析 | 解析 | + +四个 CI 失败(Linux integration / macOS LLVM / aarch64 / Windows toolchains)全部是 +同一句: + +``` +error: dependency 'ftxui': no package found for exact selector + tried: (mcpplibs, ftxui) + compat.ftxui + ftxui = "6.1.9" +``` + +PR 把它们归类为"已知跨仓库边界,由 xlings #521 收口"。**这个归类不成立**: + +1. **mcpp 自己也被它打中**。本 PR 不得不把仓库根 `mcpp.toml` 的 + `[dev-dependencies] gtest` 改成 `[dev-dependencies.compat] gtest`。 + 一个变更需要改自己的 manifest 才能自举,就是破坏性变更的定义。 +2. **xlings 只是一个消费者**。索引里有 34 个 `compat.*` 包 + (`165_bare_name_cross_namespace_wire_address.sh` 在 main 上的注释记的数; + 本 PR 把那段注释删掉了)。 + 任何用户 `mcpp.toml` 里写着裸 `gtest`/`ftxui`/… 的项目, + 在升级到 2026.8.9.1 的那一刻全部构建失败,而**修 xlings 帮不到他们**。 +3. **它是记忆 `index-floor-must-degrade` 的镜像**。那条的教训是 + "索引是数据、mcpp 是程序,发布数据不得让程序失效"。 + 这次反过来:**发布程序不得让已发布的数据与用户 manifest 失效。** + 两个方向的判据是同一条:**必须降级,不得变砖。** +4. **PR 自己已经给出了正确的先例**。同一个 PR 对 `ns:name → ns.name` + 给了一个 release 的迁移别名 + 可复制的替换提示。裸名迁移比冒号拼写影响面**大得多**, + 却只给了硬失败。这是内部不一致。 + +**处理**:按 §3 主线 B(加一个 release 的迁移窗口)。这同时会把四个红变绿, +**并把 xlings #521 从发布关键路径上摘下来**。 + +### R3 · P1 · 合并 #400 会立刻武装 AUR 自动发布 + +`.github/workflows/aur-publish.yml`: + +```yaml +on: + workflow_run: { workflows: [release], types: [completed] } + schedule: + - cron: '17 */6 * * *' + workflow_dispatch: { inputs: { publish: … default: false } } +``` + +而发布判定是: + +```bash +if [[ "$TRIGGER" == workflow_run || "$TRIGGER" == schedule ]]; then + publish=true +else + publish=${MANUAL_PUBLISH:-false} +fi +``` + +**`schedule` 直接 `publish=true`。** 也就是说:#400 一旦合入 main, +最多 6 小时后就会有一次**无人值守的、对第三方服务(AUR)的写操作**, +它会把当时"最新完整稳定 release"(合并瞬间是 `v2026.8.8.4`)推到 `mcpp-bin`, +而公开 AUR 现在还停在 `2026.8.1.1-1`。 + +这不一定是错的结果,但**它是一个由"合并"而不是由"明确放行"触发的对外副作用**, +而且这条路径**从未真跑过**(交接文档 §3.7:只做了非 root `makepkg --verifysource` dry-run, +"本轮没有发布")。 + +**处理**:见 §3 主线 C 的 C3 —— 首次落地把 `schedule` 分支降级为 dry-run +(用一个仓库变量或直接先不带 `schedule` 触发器合入), +手工 `workflow_dispatch --publish` 验证一次成功后,再用一个独立小 PR 打开定时收敛。 + +### R4 · P1 · 身份/索引路由的 e2e 在 Windows 上几乎不跑 + +本 PR 的核心是"精确身份 + 索引路由",而这类 e2e 的 `# requires:` 把它们挡在 Windows 之外: + +| 测试 | requires | Windows | +|---|---|---| +| `203_exact_selector_lock_migration.sh` | `gcc fresh-sandbox` | SKIP | +| `121_default_ns_redirect.sh` | `gcc fresh-sandbox` | SKIP | +| `205_root_local_subos.sh` | `elf gcc` | SKIP(合理,Linux 语义) | +| `206_runtime_binding_physics.sh` | `elf gcc` | SKIP(合理) | +| `207_runtime_contract_provenance.sh` | `elf python3 unix-shell` | SKIP(合理) | + +`205/206/207` 跳过是对的 —— 它们本来就是 Linux 运行时物理。 +但 `203`(**exact selector 的 lock 迁移**)和 `121`(**索引 default 重定向**) +是纯路径 + 纯身份逻辑,跟编译器无关,却因为 `gcc` 这个 token 从来没在 Windows 上跑过。 + +**这正是本次 bug 能溜过去的结构性原因:身份与路径的 bug 只在 Windows 上出现, +而验证身份与路径的测试只在 Linux 上跑。** + +**处理**:见 §3 主线 A 的 A3。 + +> 参见记忆 `issue375-retracted-c-runtime-and-subos-env`: +> `# requires:` 的死 token 让 `65_*` 从未在 CI 跑过。同一类问题,这次是"活 token 但选错了"。 + +### R5 · P2 · PR 体量与耦合 + +111 文件 / +15085,横跨四个互不依赖的产品域(包/模板身份、事务脚手架、 +运行时绑定与 ELF 闭包、release manifest + AUR reconciler),**外加一次版本 bump**。 + +现在拆已经不划算(31 个 commit + 不得改写历史的约束),所以**不建议拆 #400**。 +但两件事要做: + +- **发布分级**:AUR(R3)与 release manifest 是第一次真跑,按 §3 C3 单独放行, + 不要和身份语义同时"上线"; +- **记录规则**:下次同类工作按域拆 PR。一个 PR 只应该有一条可以独立回滚的主线。 + +### R6 · P2 · 版本号日期已过期 + +`mcpp.toml` / `src/version.cppm` 是 `2026.8.9.1`,今天已是 2026-08-10。 +按 `YYYY.M.D.N` 约定(记忆 `mcpp-date-version-convention`), +实际发版当天必须重新对齐(如 `2026.8.10.1`)。 +**这不是收尾时顺手改的东西** —— 它牵动 CHANGELOG、docs、release manifest、 +AUR 目标版本、bootstrap pin 的判据,必须作为独立 commit 在发版前执行。 + +### R7 · 待独立复核区(本文不下结论) + +以下三块是本 PR 新增的高风险实现,本轮只做了结构性抽查,**没有**做逐行审计, +因此**不给通过或不通过的结论**,只登记为"合并前需要一次独立 review pass": + +- `src/platform/elf_runtime.cppm`(+722):手写 ELF64 读取器。 + 抽查到 `off <= bytes.size() && size <= bytes.size() - off` 这类正确的溢出安全写法 + 和 `kMaxClosureObjects` 上界,形态是对的;但 verneed/verdef 遍历、 + SONAME 复用、`$ORIGIN` 展开需要针对畸形/截断输入的定向用例。 +- `src/build/runtime_validation.cppm`(+433):verdict 缓存。 + 键包含 artifact stat + `runtimeBinding.contractHash`, + 且 `Inconclusive` 会把 summary 从 `Pass` 拉下来 —— 方向正确。 + 需要专门确认的不变量:**缓存 miss/mismatch 必须保持失败,不得靠下一次运行变绿** + (PR 声称已实现,需要一个明确的 RED 用例锁住)。 +- `scripts/aur/reconcile_mcpp_bin.py`(+1132):12/12 契约测试 + 真实 Arch dry-run, + 但 fast-forward-only、vercmp 单调性、AUR RPC 滞后分类这三条只在**真实推送**时才被检验。 + 这就是 R3 要分级放行的原因。 + +--- + +## 3. 设计:三条主线 + +### 主线 A —— 让 latest-head Windows 变绿(修测试,不是修产品) + +#### A1. 修 fixture:`tests/e2e/12_add_command.sh:250` + +写进 manifest 的绝对路径必须是**宿主原生**路径。仓库已有先例: +`171_bmi_staging_locked_dest.sh:75` 用 `cygpath -w`。 + +TOML 里反斜杠要转义,所以用 **`cygpath -m`**(混合模式,`C:/Users/...`): +Windows API 接受正斜杠,TOML basic string 也不需要转义。 + +在 `tests/e2e/run_all.sh` 里导出一个共享 helper(新建 `_paths.sh` 或直接放进 run_all): + +```bash +# 把一个 shell 侧路径转成"写进 mcpp.toml 后原生 mcpp 能解析"的形式。 +host_path() { + case "$(uname -s)" in + MINGW*|MSYS*|CYGWIN*) cygpath -m "$1" ;; + *) printf '%s' "$1" ;; + esac +} +``` + +step (15) 改为: + +```bash +IDX="$(host_path "$TMP/myapp/index")" +... +acme = { path = "$IDX" } +``` + +**不要**改成相对路径。改成相对路径能让测试变绿,但会**顺手删掉"绝对路径索引" +这条唯一在 Windows 上被真正解析的覆盖** —— 那正是这次暴露出的盲区。 + +#### A2. `ed4cf64` 的处置:证明它,或者还原它 + +两条路,选一条,不允许"留着不动": + +- **A2-a(推荐)**:写一个 RED —— *workspace root 通过符号链接访问时, + 继承的相对 index 必须仍锚在声明的 root 上*。 + 如果 `weakly_canonical` 让它失败而 `lexically_normal` 让它通过, + 这条改动就被证明了,保留,并把该测试作为它的锁。 +- **A2-b**:还原为 `weakly_canonical`(回到 main 的绿行为)。 + +无论哪条,都要修 `tests/unit/test_pm_index_route.cpp:221`: +把 `EXPECT_EQ(path, …lexically_normal())` 换成能力断言 +(`route.describe(...)` 与 `lookup_descriptor` 命中), +**不要断言路径的字符串拼写**。 + +#### A3. 补 Windows 侧的身份/路径覆盖 + +- 把 `203_exact_selector_lock_migration.sh` 与 `121_default_ns_redirect.sh` 的 + `# requires:` 里的 `gcc` 去掉(若它们确实只做解析断言;若确有编译步骤, + 就把解析断言拆成一个不需要 gcc 的新用例)。目标:**exact selector 与索引路由 + 在三个平台都跑**。 +- 新增一条专门的 e2e:*绝对路径 `[indices]` 在三个平台都能解析*, + 内部使用 `host_path`,并**显式断言 `route:` 输出为 `root present, pkgs present`**。 + 这条测试的存在本身就是防止"MSYS 路径写进 manifest"再次发生的护栏。 +- 给 `run_all.sh` 加一条极简 lint:扫描 `tests/e2e/*.sh`, + 凡出现 `path = "$...` 形式且变量不是经 `host_path` 得来的,报错。 + (宁可稍微粗糙也要有 —— 这类错误人眼在 review 里看不出来。) + +#### A4. 保留 `route:` 诊断 + +`9a47ccf` 加的 `IndexRoute::describe` 是本轮**唯一让根因可判定**的东西: +没有它,日志只会说 "not found",谁都无法区分"索引没配"和"索引配了但路径不存在"。 +它应该保留,并按 A3 被测试固定下来。 + +--- + +### 主线 B —— 裸名迁移窗口(把 R2 从"生态阻塞"降级为"一次弃用警告") + +#### B1. 语义 + +对**省略命名空间**的选择器: + +1. 精确解析 `(mcpplibs, name)`; +2. **仅在 miss 时**,按上一版本的老顺序再试 `(compat, name)` 和 `(∅, name)`; +3. 若老 rung 命中:**正常解析**,同时打印一条弃用警告,内容包含 + - 实际选中的完整身份(`compat.ftxui`), + - 可直接粘贴的 manifest 片段(复用 `src/build/prepare.cppm` 里 miss 分支 + 已有的 "did you mean" 渲染,就在 `:2447` 那条 error 之前), + - 该 rung 将在下一个版本移除; +4. 老 rung 也 miss → 保持现在的硬失败与提示(不变); +5. **`mcpp.lock` 只写规范身份**(`compat.gtest`),绝不写裸拼写; +6. `mcpp add <裸名>` 命中老 rung 时,**把规范点分形式写进 `mcpp.toml`** + —— 用户第一次碰它就自动迁移。 + +关键点:**原缺陷是"静默",不是"回退"。** +`162_bare_name_namespace_scope.sh` 的注释原文写得很清楚: +"a bare name that matched nothing fell through to the FIRST candidate **SILENTLY**, +so mcpp carried on with a namespace it had invented"。 +一条带完整身份的警告已经把"静默"消灭了,同时保住了已发布的数据。 + +#### B2. 落点 + +| 文件 | 改什么 | +|---|---| +| `src/pm/dependency_selector.cppm` | 新增 `legacy_bare_candidates(name)`,与已有的 `legacy_prefixed_coordinate` 对称:只服务于"exact miss 之后",绝不进 exact 候选表 | +| `src/pm/index_route.cppm` | `Lookup` 增加"命中来自哪一 rung",让上层能区分 exact 命中与 legacy 命中 | +| `src/build/prepare.cppm:2447` 附近 | miss 后先试 legacy rung;命中则 warning + 继续;仍 miss 则维持现有 error | +| `src/pm/commands.cppm` | `mcpp add` 命中 legacy rung 时写规范点分形式 + warning | +| `tests/e2e/162_bare_name_namespace_scope.sh` | 两个分支都锁:裸 `gtest` **命中 compat 且必须出现弃用警告**;裸 `widget`(无处可寻)**必须硬失败** | +| `mcpp.toml` | 保持 `[dev-dependencies.compat] gtest`(这是规范写法,不回退) | +| `docs/spec/package-identity.md` + zh | 明确写出窗口的起止版本 | + +#### B3. 效果 + +- 四个红 job 立刻具备变绿条件,**不需要等 xlings #521 合并 + xlings 发版**; +- xlings #521 仍然是**正确**的修复,继续正常 review/合并/发版, + 但它从 mcpp 发布的关键路径上被摘下来; +- 用户升级不炸; +- 下一个版本移除窗口时,生态已经有一个完整 release 的警告期。 + +#### B4. 备选(不推荐,但记录) + +- **B-alt-1 生态优先**:不加窗口,先把索引里 34 个 `compat.*` 全部加 + `mcpplibs.` 桥接描述符(就是 mcpp-index #197 对 `capi.lua` 做的 Form-B bridge)。 + 缺点:把裸拼写永久固化进索引,与本 PR 的精确身份目标直接冲突。 +- **B-alt-2 硬切**:照现在合并,靠文档和 release note 通知。 + 缺点:所有存量用户 manifest 在升级瞬间失败,而错误信息虽然给了替换建议, + 但**无法自动修复**,也无法回退(旧 mcpp 已被 upgrade 提示引导升级)。 + +--- + +### 主线 C —— 收口顺序:把串行改并行 + +#### C1. 当前(交接文档 §8)是一条全串行链 + +``` +xlings#521 review → 合并 → xlings 发版 → mcpp 改 pin → mcpp 全矩阵 + → mcpp review → 合并 → mcpp 发版 → GitCode → AUR → fresh-home +``` + +任何一环卡住(尤其是跨组织的 xlings 发版)整条停摆。 + +#### C2. 加入主线 B 之后可以并行 + +``` +轨道 1(mcpp #400,自足) + A1 修 fixture ─ A2 处置 ed4cf64 ─ A3 补覆盖 ─ B 迁移窗口 + → latest-head 全矩阵 → 用户 review → 普通合并 → 发版 + +轨道 2(xlings #521,独立) + 用户 review → 普通合并 → xlings 发版 + → mcpp 独立小 PR 更新 pin(不阻塞轨道 1) + +轨道 3(对外发布面,独立放行) + release manifest 首跑验证 → AUR 手工 publish 验证 → 打开定时收敛 +``` + +#### C3. AUR 分级放行(对应 R3) + +1. **合并 #400 时**:`aur-publish.yml` 的 `schedule` 分支改为 `publish=false` + (或整段先不带 `schedule` 触发器合入)。合并不产生任何对外写。 +2. **手工验证一次**:`workflow_dispatch` 带 `publish=false` 看 plan, + 再带 `publish=true` 推一次,核验 AUR RPC 可见、`.SRCINFO`、两个架构 checksum、 + 以及在真实 Arch 上 fresh install。 +3. **验证通过后**,用一个只改触发器的独立小 PR 打开定时收敛。 + +这样"第一次真实对外推送"是一次**有人盯着的、可回退到上一步的**动作, +而不是合并后 6 小时内自己发生的事。 + +#### C4. 版本对齐(对应 R6) + +发版当天,独立 commit 统一 `mcpp.toml` / `src/version.cppm` / CHANGELOG / +docs / release manifest 目标 / AUR 目标版本。 +`.xlings.json` 的 mcpp bootstrap pin 保持 `2026.8.8.4`(上一个已发布版本)—— +按记忆 `release-bootstrap-pin-two-groups`,bootstrap pin 是自举起点,不随本次发布走。 + +--- + +## 4. 分阶段计划与验收判据 + +每一阶段都遵循 PR #400 已确立的流程:先 RED、最小实现、GREEN、 +隐私扫描、单文件暂存、一个逻辑变更一个 commit、立即 push、立即中文 checkpoint、 +不 amend/rebase/squash/force-push、不 admin bypass。 + +| 阶段 | 内容 | 验收判据(**判据是原生 CI 终态,不是本机**) | +|---|---|---| +| **P0** | A1 `host_path` + step (15) fixture | Windows E2E 2/2 中 `12_add_command.sh` PASS;Linux/macOS 不回归 | +| **P1** | A2 处置 `ed4cf64`;重写 `test_pm_index_route.cpp:221` 为能力断言 | 若走 A2-a:符号链接用例先 RED 后 GREEN;若走 A2-b:还原后三平台全绿 | +| **P2** | A3 Windows 身份/路径覆盖 + `run_all.sh` lint | `203`/`121` 在 Windows 上由 SKIP 变 PASS;新绝对路径索引用例三平台 PASS;lint 对故意构造的坏 fixture 报错 | +| **P3** | B 裸名迁移窗口 | `162` 两个分支都 GREEN;**四个 xlings 相关 job 由 fail 变 pass**;`mcpp.lock` 里不出现裸拼写 | +| **P4** | R7 独立 review pass(ELF / verdict cache / AUR reconciler) | 三块各自补齐定向 RED;verdict 缓存 mismatch 必须保持失败的用例存在 | +| **P5** | C4 版本对齐 + 文档 + 验证账本 | 版本号 = 实际发版日期;`.agents/docs/...validation.md` 里所有 "not started" 段落被终态替换 | +| **P6** | 用户 review → 转 ready → 普通合并 | 全矩阵**同一个 HEAD**上 terminal 全绿;pending/skipped/cancelled/superseded 一律不算 PASS | +| **P7** | 发版 + release manifest 首跑核验 | tag、四平台资产、sidecar、不可变 `mcpp-release.json` 全部核验;索引 main 的 latest 指向它 | +| **P8** | AUR 分级放行(C3 三步) | AUR RPC 可见目标版本;真实 Arch fresh install 通过;之后才打开 `schedule` | +| **P9** | GitCode / xim-pkgindex / fresh-home 全生态 | 隔离 HOME/XLINGS_HOME/SubOS 下:安装、`[ns.]name[@version][:tname]`、唯一默认模板、多 SubOS/glibc、build/run/test、provider provenance 的 PASS/FAIL/NOT_EXERCISED 表 | +| **P10** | issue 收口 | #398 随 PR 关闭;#380/#392/#396 凭发布后二次验收证据关闭;残留写回 #397 | + +**并行关系**:P0–P3 是 #400 的关键路径;xlings #521 的 review/合并/发版 +与 P0–P6 完全并行,其 pin 更新是合并后的独立小 PR。 + +> 关于"发布完成"的判据,沿用记忆 `release-publish-pipeline`: +> **判据是索引 main 的 latest 指向它**,不是 tag 存在、也不是 PR 已合。 +> `gh pr merge` 不带 `--admin` 可能是静默空操作(退出 0),合完必须回查 `state`。 + +--- + +## 5. 需要你拍板的决策点 + +| # | 决策 | 选项 | 我的建议 | +|---|---|---|---| +| **D1** | 裸名迁移窗口 | (a) 加一个 release 的弃用窗口(主线 B)
(b) 索引侧加 34 个 `mcpplibs.*` 桥接
(c) 硬切,只靠文档通知 | **(a)** —— 与本 PR 自己对 `ns:name` 的处理一致;能保住存量用户;并把 xlings 摘出关键路径 | +| **D2** | `ed4cf64` 的 `weakly_canonical → lexically_normal` | (a) 用符号链接 RED 证明并保留
(b) 还原到 main 的行为 | **(a) 先试**,一天内证不出来就走 (b)。不允许"留着不动" | +| **D3** | AUR 首次放行 | (a) 合并时 `schedule` 降级为 dry-run,手工验证后再打开
(b) 照现状合并 | **(a)** —— 第一次对外真实推送应该有人盯着 | +| **D4** | 发版版本号 | (a) 发版当天重新对齐为 `2026.8.{当天}.1`
(b) 保持 `2026.8.9.1` | **(a)**,按日期版本约定 | +| **D5** | R7 三块的 review 深度 | (a) 合并前补定向 RED
(b) 合并后跟进 | **(a) 只对 verdict 缓存那条不变量**("mismatch 不得靠重跑变绿"),其余可 (b) | + +--- + +## 6. 明确不做 + +- **不拆 #400**(31 commit + 不改写历史的约束下,拆的成本大于收益); + 规则记录下来给下一次。 +- **不在 mcpp 里恢复"静默"的跨命名空间回退**。主线 B 的窗口是**带警告、 + 写规范身份进 lock、`add` 时自动迁移 manifest** 的,与被修掉的静默回退不是同一件事。 +- **不动 `mcpp-m`**(文件、远端、发布路径)与 `mcpp-git`。 +- **不引入 `--variant`、不引入 CLI SubOS override、不让 mcpp 探测 GPU/驱动/ICD** + —— #398 冻结的四条产品边界不变。 +- **不处理 #397 的 C1–C9**。它们是独立缺陷(fast path 不重建 CDB、 + 模块扫描器 raw-string 假阴性、`--no-color` 空操作 …), + 与本 PR 无关,按 #397 自己的节奏走。 +- **不碰特殊保留项**:#43、#260,以及标记为保留/Draft/do-not-merge 的项。 + +--- + +## 附:本文结论的可复核路径 + +| 结论 | 复核方式 | +|---|---| +| 失败在 step (15) 而非 (14) | `gh api repos/mcpp-community/mcpp/actions/jobs/93271793304/logs`,看 `exit 2` 与缺失的横幅 | +| step (15) 是本 PR 新增 | `git diff origin/main...HEAD -- tests/e2e/12_add_command.sh` | +| main 的 Windows E2E 是绿的 | run `31275102045` | +| `ed4cf64` 前后失败一致 | run `31318089536` vs `31321961040` | +| 四个 fail 同因 | job `93271797138` / `93271798469` / `93271802557` / `93271790099` | +| AUR `schedule` 即 publish | `.github/workflows/aur-publish.yml` 的 plan 步骤 | +| 身份 e2e 在 Windows 被跳过 | Windows E2E 日志中的 `SKIP: 203_… (missing capability: gcc)` | + +--- + +## 附录 A:实施结果(2026-08-10 收尾) + +本节是执行完 D1–D5 之后回填的**事实**,不是计划。凡与正文冲突的,以本节为准。 + +### A.1 D1–D5 的落地结果 + +| 决策 | 结果 | +|---|---| +| **D1** 裸名迁移窗口 | 已实施。精确未命中 + namespace 被省略 → 试 `(compat,name)` 与无 namespace rung;命中打印弃用警告 + 可粘贴片段,规范身份进 lock/install/cache;`mcpp add` 直接迁移 manifest。`2026.9` 移除,常量 `kBareNameFallbackRemovedIn`。 | +| **D2** `ed4cf64` 证明或还原 | **已证明,保留**。换回 `weakly_canonical` 让 `unit/test_pm_index_route` 失败(71 passed / 1 failed)。但理由与原提交描述不同:不是 Windows 短名,而是「anchoring 不得把 index 移出作者声明的那棵树」——经符号链接访问的 workspace 会被重定位,而 `prepare` 的报错会打印这个路径。 | +| **D3** AUR 首次放行 | 已实施。`workflow_run` 与 `schedule` 都需要仓库变量 `AUR_AUTOPUBLISH == "true"` 才推送;契约测试直接执行 workflow 自己的判定 shell。 | +| **D4** 版本对齐 | 已实施。`2026.8.9.1` → `2026.8.10.1`(版本号是日期,评审期间日期变了;`2026.8.9.1` 从未打 tag)。 | +| **D5** verdict 缓存不变量 | 已实施为 e2e `209`。**该不变量本来就是对的**(`validated_artifact_snapshot` 在任一 stored status ≠ Pass 时拒绝快速路径),只是没有测试;现在有了。 | + +### A.2 P0 的根因判定被推翻了一次,值得单独记住 + +正文 §1 的推断(step (15) 的 MSYS 路径)经原生 CI 证实。三条可复用的判据: + +1. **先看退出码,再看错误文本。** 被 `|| { …; exit 1; }` 包住的步骤失败只能是 exit 1; + 日志里的 exit 2 是 mcpp 自己的码经 `set -e` 直传,只可能来自裸调用。 +2. **失败横幅缺席本身是证据。** harness 会显示脚本 stdout,所以「该步骤失败时必然打印的 + 那行」没出现,就说明失败不在那一步。 +3. **Wine 通过 = 零信息**(对路径语义而言)。Wine 把 `Z:` 映射到 `/`, + `\tmp\…` 落回真实 `/tmp/…`。 + +修法按「一类」而不是「一行」:`_host_path.sh` + `00_fixture_path_hygiene.sh` lint + +全仓 24 文件 40 处迁移。大多数此前只是因为对应测试在 Windows 上因缺 capability 而 SKIP。 + +### A.3 计划外收进来的两项 + +- **#401(私有 glibc 泄漏进子进程环境)。** 已修,收敛为新模块 + `src/platform/runtime_env_contract.cppm` 的**作用域**决策(不是条件判断)。 + 实测对照:已发布 mcpp `LDLP=[…/runtime:…/xim-x-glibc/2.39/lib64]`,本分支 `LDLP=[…/runtime]`; + 产物 RUNPATH 改动前后逐字节相同,说明这条环境项本来就没有收益。 +- **覆盖盲区。** 身份/索引路由的 e2e 全部要 `gcc` 或 `fresh-sandbox`,Windows 两者皆无。 + 新增 `210_local_index_addressing_on_every_host.sh`:无编译器、无沙箱、无网络,三平台都跑。 + +### A.4 主线 C 的实际阻塞点变了 + +正文假设瓶颈是「等 xlings #521 合并 + 发版」。加入 D1 的迁移窗口后,#521 确实离开了关键路径。 + +**但按要求把 xlings pin 提到最新 `2026.8.10.1` 之后出现了新的、更硬的阻塞:** +冷 home 装不上 `xim:gcc@16.1.0` —— 依赖解析下载 glibc 2.44,而 gcc 的 config hook +找不到该 payload(诊断建议的是 2.39)。同一 workflow、同一 runner 镜像、两次都确认 +cache miss 的 A/B: + +| xlings | 冷缓存 | 结果 | +|---|---|---| +| `2026.8.9.2` | 已确认 cache miss | gcc 安装成功(run `31317627461`) | +| `2026.8.10.1` | 已确认 cache miss | 失败(run `31335075557`,4/4 Linux job) | + +所以不是「一直坏、被热缓存掩盖」。已带证据上报 +[openxlings/xlings#524](https://github.com/openxlings/xlings/issues/524), +mcpp 侧把 pin 停在 `2026.8.9.2`,等修复发布后用独立 commit 提升。 + +> 这条本身也是一个判据:**「pin 到最新」是一个需要被验证的动作,不是一次文本替换。** +> 它之所以在这里被抓到,只是因为换 pin 同时换掉了 CI 缓存键,把冷启动路径暴露了出来。 diff --git a/.github/actions/bootstrap-mcpp/action.yml b/.github/actions/bootstrap-mcpp/action.yml index 400311d3..68355d10 100644 --- a/.github/actions/bootstrap-mcpp/action.yml +++ b/.github/actions/bootstrap-mcpp/action.yml @@ -25,7 +25,7 @@ inputs: # `package.name`, so one of the two was simply unreachable — and which one # depended on the machine, which is why CI failed on `compat:lua` on # Windows and `mcpplibs.capi:lua` on Linux. Never pin below that. - default: '2026.8.8.1' + default: '2026.8.9.2' cache-target: description: also restore/save target/ (build artifacts + BMIs) required: false diff --git a/.github/actions/setup-macos-llvm/action.yml b/.github/actions/setup-macos-llvm/action.yml index cabba4cf..0e7fef7d 100644 --- a/.github/actions/setup-macos-llvm/action.yml +++ b/.github/actions/setup-macos-llvm/action.yml @@ -15,7 +15,7 @@ inputs: # Floor imposed by the index, not a routine bump — see # .github/actions/bootstrap-mcpp/action.yml for why 0.4.69 is required # (two packages named `lua` in one repo need openxlings/xlings#381). - default: '2026.8.8.1' + default: '2026.8.9.2' runs: using: composite diff --git a/.github/workflows/aur-publish.yml b/.github/workflows/aur-publish.yml index 661cb893..0ea09b17 100644 --- a/.github/workflows/aur-publish.yml +++ b/.github/workflows/aur-publish.yml @@ -1,102 +1,138 @@ name: aur-publish -# Publish the `mcpp-bin` and `mcpp` AUR packages after a release. -# -# Triggers on COMPLETION of the `release` workflow (not on `release: -# published`): release.yml creates the GitHub Release in its first job but -# uploads the aarch64 / macOS / Windows assets in LATER jobs, so the aarch64 -# .sha256 that mcpp-bin needs only exists once the whole workflow finishes. -# -# Requires one repository secret: -# AUR_SSH_PRIVATE_KEY — private key whose public half is registered on the -# AUR account that owns mcpp / mcpp-bin. -# See scripts/aur/README.md → "Automated publishing" for the full setup. +# Reconcile only mcpp-bin. This workflow is downstream of `release`, so its +# failure is visible without changing the already-terminal release conclusion. on: workflow_run: workflows: [release] types: [completed] + schedule: + - cron: '17 */6 * * *' workflow_dispatch: inputs: - version: - description: "Version to publish (default: [package].version in mcpp.toml)" + publish: + description: 'Publish the validated diff (false performs a dry-run only)' + type: boolean + required: true + default: false + tag: + description: 'Optional exact latest complete stable tag (no downgrade override)' + type: string required: false concurrency: - group: aur-publish + group: aur-mcpp-bin-reconcile cancel-in-progress: false +permissions: + contents: read + jobs: - publish: - runs-on: ubuntu-latest - # On the workflow_run trigger, only proceed if the release actually - # succeeded (skip failed/cancelled release runs). + reconcile: + name: reconcile mcpp-bin if: >- - github.event_name == 'workflow_dispatch' || + github.event_name != 'workflow_run' || github.event.workflow_run.conclusion == 'success' + runs-on: ubuntu-24.04 + timeout-minutes: 60 + env: + GH_TOKEN: ${{ github.token }} + PYTHONDONTWRITEBYTECODE: '1' + REQUESTED_TAG: ${{ inputs.tag }} steps: - - name: Checkout released commit + - name: Checkout reconciler source uses: actions/checkout@v4 with: - # workflow_run: the exact commit the release was built from. - # workflow_dispatch: default ref (HEAD of the branch). ref: ${{ github.event.workflow_run.head_sha || github.ref }} - - name: Refresh both PKGBUILDs to the release version - id: refresh + - name: Reconciler contract tests + run: python3 tests/scripts/test_aur_reconcile.py + + # This phase has no SSH private key in its environment. It downloads the + # immutable manifest and both Linux payload/sidecars, recomputes hashes, + # runs makepkg as non-root in Arch, inspects RPC + HTTPS git, and emits the + # exact diff before any publishing secret is loaded. + - name: Inspect and validate desired state + id: plan env: - # CI runs as root; force update.sh's template .SRCINFO path. - MCPP_AUR_NO_MAKEPKG: "1" + TRIGGER: ${{ github.event_name }} + MANUAL_PUBLISH: ${{ inputs.publish }} + # Repository variable, absent until a human has watched one publish + # succeed. See "Arming the automatic triggers" in scripts/aur/README.md. + AUTOPUBLISH: ${{ vars.AUR_AUTOPUBLISH }} run: | - VER="${{ github.event.inputs.version }}" - if [ -z "$VER" ]; then - # mcpp.toml at the released commit carries the right version. - VER=$(grep -m1 -E '^\s*version\s*=' mcpp.toml | sed -E 's/.*"([^"]+)".*/\1/') - fi - echo "version=$VER" >> "$GITHUB_OUTPUT" - ./scripts/aur/update.sh "$VER" + args=( + --trigger "$TRIGGER" + --report-json "$RUNNER_TEMP/aur-plan.json" + --summary "$GITHUB_STEP_SUMMARY" + ) + [[ -z "$REQUESTED_TAG" ]] || args+=(--tag "$REQUESTED_TAG") + python3 scripts/aur/reconcile_mcpp_bin.py "${args[@]}" - - name: Configure AUR SSH - run: | - install -dm700 ~/.ssh - printf '%s\n' "${{ secrets.AUR_SSH_PRIVATE_KEY }}" > ~/.ssh/aur - chmod 600 ~/.ssh/aur - ssh-keyscan -t rsa,ed25519 aur.archlinux.org >> ~/.ssh/known_hosts 2>/dev/null - cat > ~/.ssh/config <<'EOF' - Host aur.archlinux.org - User aur - IdentityFile ~/.ssh/aur - IdentitiesOnly yes - EOF + # An unattended push to a third-party service must be ARMED, not + # inherited from a merge. `schedule` fires every six hours off the + # default branch, so merging this workflow used to be enough to make + # mcpp start writing to the AUR on its own — before anyone had seen + # the reconciler complete a real push even once. Both automatic + # triggers therefore plan-and-report until AUR_AUTOPUBLISH is set; + # `workflow_dispatch` keeps its explicit per-run switch, which is how + # that first push is meant to happen. + case "$TRIGGER" in + workflow_run | schedule) + if [[ "${AUTOPUBLISH:-}" == "true" ]]; then + publish=true + else + publish=false + echo "::notice::AUR_AUTOPUBLISH is not set — reporting the desired state without publishing." + fi + ;; + *) + publish=${MANUAL_PUBLISH:-false} + ;; + esac + echo "needs_publish=$(jq -r '.needs_publish' "$RUNNER_TEMP/aur-plan.json")" >> "$GITHUB_OUTPUT" + echo "publish=$publish" >> "$GITHUB_OUTPUT" - - name: Push to the AUR + - name: Configure pinned AUR SSH identity + if: steps.plan.outputs.needs_publish == 'true' && steps.plan.outputs.publish == 'true' env: - VER: ${{ steps.refresh.outputs.version }} + AUR_SSH_PRIVATE_KEY: ${{ secrets.AUR_SSH_PRIVATE_KEY }} run: | - set -eu - git config --global user.name "mcpp-ci" - git config --global user.email "x.d2learn.org@gmail.com" + test -n "$AUR_SSH_PRIVATE_KEY" || { echo 'AUR_SSH_PRIVATE_KEY is empty'; exit 1; } + install -dm700 "$HOME/.ssh" + install -m600 /dev/null "$HOME/.ssh/aur" + printf '%s\n' "$AUR_SSH_PRIVATE_KEY" > "$HOME/.ssh/aur" + install -m600 scripts/aur/aur.archlinux.org.known_hosts "$HOME/.ssh/known_hosts" + ssh-keygen -lf "$HOME/.ssh/known_hosts" -E sha256 \ + | grep -F 'SHA256:RFzBCUItH9LZS0cKB5UE6ceAYhBD5C8GeOBip8Z11+4' + install -m600 /dev/null "$HOME/.ssh/config" + printf '%s\n' \ + 'Host aur.archlinux.org' \ + ' User aur' \ + ' IdentityFile ~/.ssh/aur' \ + ' IdentitiesOnly yes' \ + ' StrictHostKeyChecking yes' \ + ' UserKnownHostsFile ~/.ssh/known_hosts' \ + > "$HOME/.ssh/config" - publish() { # $1 = package name (= dir under scripts/aur/) - pkg="$1"; src="scripts/aur/${pkg}"; work="/tmp/aur-${pkg}" - # Clone the existing AUR repo; if the package doesn't exist yet - # (first publish), start an empty repo — AUR creates it on push. - if git clone "ssh://aur@aur.archlinux.org/${pkg}.git" "$work" 2>/dev/null \ - && [ -e "$work/.git" ]; then :; else - rm -rf "$work"; mkdir -p "$work" - git -C "$work" init -q - git -C "$work" remote add origin "ssh://aur@aur.archlinux.org/${pkg}.git" - fi - # AUR repos contain only PKGBUILD, .SRCINFO and local sources. - cp "$src/PKGBUILD" "$src/.SRCINFO" "$src/mcpp.sh" "$work/" - git -C "$work" add -A - if git -C "$work" diff --cached --quiet; then - echo ":: ${pkg}: no changes, skipping" - return 0 - fi - git -C "$work" commit -q -m "${pkg} ${VER}" - git -C "$work" push origin HEAD:master - echo ":: ${pkg}: published ${VER}" - } + - name: Fast-forward publish and verify convergence + if: steps.plan.outputs.needs_publish == 'true' && steps.plan.outputs.publish == 'true' + env: + TRIGGER: ${{ github.event_name }} + run: | + args=( + --publish + --trigger "$TRIGGER" + --report-json "$RUNNER_TEMP/aur-final.json" + --summary "$GITHUB_STEP_SUMMARY" + ) + [[ -z "$REQUESTED_TAG" ]] || args+=(--tag "$REQUESTED_TAG") + python3 scripts/aur/reconcile_mcpp_bin.py "${args[@]}" - publish mcpp-bin - publish mcpp-m + - name: Preserve reconciliation reports + if: always() + uses: actions/upload-artifact@v4 + with: + name: aur-mcpp-bin-reconciliation + path: ${{ runner.temp }}/aur-*.json + if-no-files-found: error diff --git a/.github/workflows/bootstrap-macos.yml b/.github/workflows/bootstrap-macos.yml index de2e817b..f69a58eb 100644 --- a/.github/workflows/bootstrap-macos.yml +++ b/.github/workflows/bootstrap-macos.yml @@ -17,7 +17,7 @@ jobs: # Dormant (workflow_dispatch only), but kept in step with the rest — # check_version_pins.sh holds it there. Floor: 0.4.69, below which the # index cannot resolve two packages that share a short name. - XLINGS_VERSION: '2026.8.8.1' + XLINGS_VERSION: '2026.8.9.2' steps: - uses: actions/checkout@v4 diff --git a/.github/workflows/ci-fresh-install.yml b/.github/workflows/ci-fresh-install.yml index d363080c..1e1c96ea 100644 --- a/.github/workflows/ci-fresh-install.yml +++ b/.github/workflows/ci-fresh-install.yml @@ -152,7 +152,7 @@ jobs: env: XLINGS_NON_INTERACTIVE: '1' run: | - curl -fsSL https://raw.githubusercontent.com/openxlings/xlings/main/tools/other/quick_install.sh | bash -s v2026.8.8.1 + curl -fsSL https://raw.githubusercontent.com/openxlings/xlings/main/tools/other/quick_install.sh | bash -s v2026.8.9.2 echo "$HOME/.xlings/subos/current/bin" >> "$GITHUB_PATH" - name: Install mcpp and config mirror @@ -179,11 +179,12 @@ jobs: # Template packages exercise the sha256-pinned mcpp-index fetch # path (user report: `mcpp new ... --template imgui` failed with # fetch 'imgui@0.0.6' exit 1 on hosts without a sha256sum binary). - - name: "Template: mcpp new --template imgui (fetch path)" + - name: "Template: exact mcpplibs.imgui selector (fetch path)" run: | cd "$(mktemp -d)" - mcpp new abc1 --template imgui + mcpp new abc1 --template mcpplibs.imgui 2>&1 | tee template.log test -f abc1/mcpp.toml + grep -F 'namespace=mcpplibs name=imgui' template.log - name: "Default: build mcpp" run: | @@ -292,7 +293,7 @@ jobs: - name: Install xlings + mcpp run: | - curl -fsSL https://raw.githubusercontent.com/openxlings/xlings/main/tools/other/quick_install.sh | bash -s v2026.8.8.1 + curl -fsSL https://raw.githubusercontent.com/openxlings/xlings/main/tools/other/quick_install.sh | bash -s v2026.8.9.2 # Deliberately NOT writing to $GITHUB_PATH here. On container # images that declare no PATH in their config (opensuse/ # tumbleweed), appending a single dir to GITHUB_PATH makes the @@ -363,7 +364,7 @@ jobs: # (older ones carry minos=15 and refuse to start). # v0.4.51+: in-process sha256 — this image has no sha256sum # binary, so pinned fetches failed before it. - curl -fsSL https://raw.githubusercontent.com/openxlings/xlings/main/tools/other/quick_install.sh | bash -s v2026.8.8.1 + curl -fsSL https://raw.githubusercontent.com/openxlings/xlings/main/tools/other/quick_install.sh | bash -s v2026.8.9.2 echo "$HOME/.xlings/subos/current/bin" >> "$GITHUB_PATH" - name: Install mcpp and config mirror @@ -391,11 +392,12 @@ jobs: # path — this is what broke on hosts without a sha256sum binary # (stock macOS / bare Windows) before xlings 0.4.51 hashed # in-process. - - name: "Template: mcpp new --template imgui (fetch path)" + - name: "Template: exact mcpplibs.imgui selector (fetch path)" run: | cd "$(mktemp -d)" - mcpp new abc1 --template imgui + mcpp new abc1 --template mcpplibs.imgui 2>&1 | tee template.log test -f abc1/mcpp.toml + grep -F 'namespace=mcpplibs name=imgui' template.log - name: "LLVM: build mcpp" run: | @@ -466,13 +468,15 @@ jobs: # path (user report: `mcpp new abc1 --template imgui` failed with # fetch 'imgui@0.0.6' exit 1 on bare Windows — no sha256sum binary # outside git-bash; fixed by xlings 0.4.51 in-process hashing). - - name: "Template: mcpp new --template imgui (fetch path)" + - name: "Template: exact mcpplibs.imgui selector (fetch path)" shell: pwsh run: | $tmp = New-TemporaryFile | ForEach-Object { Remove-Item $_; New-Item -ItemType Directory -Path $_ } Set-Location $tmp - mcpp new abc1 --template imgui + mcpp new abc1 --template mcpplibs.imgui 2>&1 | Tee-Object -Variable templateOutput + if ($LASTEXITCODE -ne 0) { exit $LASTEXITCODE } if (!(Test-Path abc1/mcpp.toml)) { exit 1 } + if (-not (($templateOutput -join "`n") -match 'namespace=mcpplibs name=imgui')) { exit 1 } - name: "LLVM: build mcpp" shell: pwsh diff --git a/.github/workflows/ci-linux-e2e.yml b/.github/workflows/ci-linux-e2e.yml index 360c135f..fe479bbc 100644 --- a/.github/workflows/ci-linux-e2e.yml +++ b/.github/workflows/ci-linux-e2e.yml @@ -133,7 +133,7 @@ jobs: - name: Bootstrap xlings + released mcpp run: | - curl -fsSL https://raw.githubusercontent.com/openxlings/xlings/main/tools/other/quick_install.sh | bash -s v2026.8.8.1 + curl -fsSL https://raw.githubusercontent.com/openxlings/xlings/main/tools/other/quick_install.sh | bash -s v2026.8.9.2 export PATH="$HOME/.xlings/subos/current/bin:$PATH" xlings update xlings install mcpp -y -g diff --git a/.github/workflows/cross-build-test.yml b/.github/workflows/cross-build-test.yml index 1ee50a9c..be57dd3a 100644 --- a/.github/workflows/cross-build-test.yml +++ b/.github/workflows/cross-build-test.yml @@ -118,7 +118,7 @@ jobs: # release assets were uploaded in a broken state (records present, # blobs missing → 404 on GET); re-uploaded clean. The stale-INDEX # half is handled by the marker-clear below. - XLINGS_VERSION: '2026.8.8.1' + XLINGS_VERSION: '2026.8.9.2' run: | tarball="xlings-${XLINGS_VERSION}-linux-x86_64.tar.gz" curl -fsSL -o "/tmp/${tarball}" \ @@ -255,7 +255,7 @@ jobs: - name: Bootstrap mcpp via xlings env: XLINGS_NON_INTERACTIVE: '1' - XLINGS_VERSION: '2026.8.8.1' + XLINGS_VERSION: '2026.8.9.2' run: | tarball="xlings-${XLINGS_VERSION}-linux-x86_64.tar.gz" curl -fsSL -o "/tmp/${tarball}" \ diff --git a/.github/workflows/release.yml b/.github/workflows/release.yml index cf4e8861..19a2f994 100644 --- a/.github/workflows/release.yml +++ b/.github/workflows/release.yml @@ -96,7 +96,7 @@ jobs: # Pin xlings to a known-good version. The upstream install # script always grabs `latest` (no version override), so we # download + self-install manually to avoid broken releases. - XLINGS_VERSION: '2026.8.8.1' + XLINGS_VERSION: '2026.8.9.2' run: | if [ ! -x "$HOME/.xlings/subos/default/bin/xlings" ]; then tarball="xlings-${XLINGS_VERSION}-linux-x86_64.tar.gz" @@ -288,7 +288,7 @@ jobs: - name: Bootstrap mcpp via xlings env: XLINGS_NON_INTERACTIVE: '1' - XLINGS_VERSION: '2026.8.8.1' + XLINGS_VERSION: '2026.8.9.2' run: | tarball="xlings-${XLINGS_VERSION}-linux-x86_64.tar.gz" curl -fsSL -o "/tmp/${tarball}" \ @@ -358,11 +358,11 @@ jobs: # below are pinned to the same version as XLINGS_VERSION; they are # NOT interpolated from it, so check_version_pins.sh scans for them # explicitly (they were absent from the old lock-step comment). - XLA="xlings-2026.8.8.1-linux-aarch64.tar.gz" + XLA="xlings-2026.8.9.2-linux-aarch64.tar.gz" if curl -fsSL -o "/tmp/$XLA" \ - "https://github.com/openxlings/xlings/releases/download/v2026.8.8.1/$XLA"; then + "https://github.com/openxlings/xlings/releases/download/v2026.8.9.2/$XLA"; then tar -xzf "/tmp/$XLA" -C /tmp - XLBIN=$(find /tmp/xlings-2026.8.8.1-linux-aarch64 -path '*/bin/xlings' -type f | head -1) + XLBIN=$(find /tmp/xlings-2026.8.9.2-linux-aarch64 -path '*/bin/xlings' -type f | head -1) if [ -n "$XLBIN" ]; then mkdir -p "$STAGING/$WRAPPER/registry/bin" cp "$XLBIN" "$STAGING/$WRAPPER/registry/bin/xlings" @@ -440,7 +440,7 @@ jobs: - name: Bootstrap mcpp via xlings env: XLINGS_NON_INTERACTIVE: '1' - XLINGS_VERSION: '2026.8.8.1' + XLINGS_VERSION: '2026.8.9.2' run: | if [ ! -x "$HOME/.xlings/subos/default/bin/xlings" ]; then WORK=$(mktemp -d) @@ -622,7 +622,7 @@ jobs: shell: bash env: XLINGS_NON_INTERACTIVE: '1' - XLINGS_VERSION: '2026.8.8.1' + XLINGS_VERSION: '2026.8.9.2' run: | # Captured before the `cd` below, in POSIX form: this step never # returns to the workspace, and GITHUB_WORKSPACE is a backslash @@ -725,16 +725,116 @@ jobs: dist/mcpp-windows-x86_64.zip dist/mcpp-windows-x86_64.zip.sha256 + # Seal the complete, non-draft release inventory only after every platform + # uploader has finished. The manifest is desired state for downstream + # reconcilers: a rerun may reproduce it byte-for-byte, but may never replace + # it with different bytes for the same tag. + release-manifest: + name: validate + seal release manifest + needs: [build-release, build-linux-aarch64, build-macos, build-windows] + runs-on: ubuntu-24.04 + permissions: + contents: write + timeout-minutes: 20 + env: + GH_TOKEN: ${{ secrets.GITHUB_TOKEN }} + steps: + - uses: actions/checkout@v4 + with: + fetch-depth: 0 + + - name: Resolve immutable release identity + id: resolve + run: | + if [ "${{ github.event_name }}" = "push" ]; then + TAG="${{ github.ref_name }}" + elif [ -n "${{ github.event.inputs.tag }}" ]; then + TAG="${{ github.event.inputs.tag }}" + else + VERSION=$(awk -F '"' '/^version[[:space:]]*=/{print $2; exit}' mcpp.toml) + test -n "$VERSION" || { echo 'failed to read [package].version from mcpp.toml'; exit 1; } + TAG="v$VERSION" + fi + git fetch --force origin "refs/tags/$TAG:refs/tags/$TAG" + COMMIT=$(git rev-list -n 1 "refs/tags/$TAG") + test -n "$COMMIT" + echo "tag=$TAG" >> "$GITHUB_OUTPUT" + echo "version=${TAG#v}" >> "$GITHUB_OUTPUT" + echo "commit=$COMMIT" >> "$GITHUB_OUTPUT" + + - name: Generate, upload once, and refetch manifest + run: | + TAG="${{ steps.resolve.outputs.tag }}" + VERSION="${{ steps.resolve.outputs.version }}" + COMMIT="${{ steps.resolve.outputs.commit }}" + AUDIT_ROOT=$(mktemp -d) + mkdir -p "$AUDIT_ROOT/assets" "$AUDIT_ROOT/publish" + + gh api "repos/${GITHUB_REPOSITORY}/releases/tags/$TAG" \ + > "$AUDIT_ROOT/release.json" + gh release download "$TAG" --repo "$GITHUB_REPOSITORY" \ + --dir "$AUDIT_ROOT/assets" + python3 scripts/release/generate_manifest.py \ + --release-json "$AUDIT_ROOT/release.json" \ + --assets-dir "$AUDIT_ROOT/assets" \ + --version "$VERSION" \ + --tag "$TAG" \ + --commit "$COMMIT" \ + --output "$AUDIT_ROOT/publish/mcpp-release.json" + + if [ -f "$AUDIT_ROOT/assets/mcpp-release.json" ]; then + cmp "$AUDIT_ROOT/assets/mcpp-release.json" \ + "$AUDIT_ROOT/publish/mcpp-release.json" + echo "Existing manifest is byte-identical; leaving it untouched." + else + gh release upload "$TAG" \ + "$AUDIT_ROOT/publish/mcpp-release.json" \ + --repo "$GITHUB_REPOSITORY" + fi + + # Do not trust the upload command alone. Fetch the final public + # inventory into a clean directory, recompute every payload digest, + # regenerate desired state, and compare the published bytes. + MANIFEST_VISIBLE=false + for attempt in {1..12}; do + if gh api "repos/${GITHUB_REPOSITORY}/releases/tags/$TAG" \ + --jq '.assets[].name' | grep -Fxq mcpp-release.json; then + MANIFEST_VISIBLE=true + break + fi + echo "Waiting for mcpp-release.json API visibility ($attempt/12)" + sleep 5 + done + test "$MANIFEST_VISIBLE" = true + mkdir -p "$AUDIT_ROOT/final-assets" + gh api "repos/${GITHUB_REPOSITORY}/releases/tags/$TAG" \ + > "$AUDIT_ROOT/final-release.json" + gh release download "$TAG" --repo "$GITHUB_REPOSITORY" \ + --dir "$AUDIT_ROOT/final-assets" + test -f "$AUDIT_ROOT/final-assets/mcpp-release.json" + python3 scripts/release/generate_manifest.py \ + --release-json "$AUDIT_ROOT/final-release.json" \ + --assets-dir "$AUDIT_ROOT/final-assets" \ + --version "$VERSION" \ + --tag "$TAG" \ + --commit "$COMMIT" \ + --output "$AUDIT_ROOT/final-expected.json" + cmp "$AUDIT_ROOT/final-assets/mcpp-release.json" \ + "$AUDIT_ROOT/final-expected.json" + cat "$AUDIT_ROOT/final-assets/mcpp-release.json" + # Publish this release into the xlings ecosystem, after ALL platform builds - # have uploaded their assets: + # have uploaded their assets and the immutable manifest gate has passed: # ① mirror binaries → xlings-res/mcpp (GitHub + GitCode) so XLINGS_RES # downloads resolve on every platform (incl. the CN/GitCode path); # ② open a PR against openxlings/xim-pkgindex bumping mcpp to this version # (a maintainer merges it — index git source is not on the critical path). - # Best-effort / non-blocking: a failure here never fails the release. + # These publication steps are required for ecosystem completeness: their + # failure leaves the GitHub Release object available but keeps this workflow + # red, so post-release verification cannot report a complete release. # Shared vendored scripts live in .github/tools/ (kept in sync with xlings). publish-ecosystem: - needs: [build-release, build-linux-aarch64, build-macos, build-windows] + needs: release-manifest runs-on: ubuntu-latest # A4 hardening: a single stuck upload once held this job >1h (6h default # ceiling). The mirror script has per-file timeouts and (post-0.0.89) diff --git a/.xlings.json b/.xlings.json index 2a35c39e..652d12f8 100644 --- a/.xlings.json +++ b/.xlings.json @@ -1,5 +1,5 @@ { "workspace": { - "mcpp": "2026.8.6.2" + "mcpp": "2026.8.8.4" } } diff --git a/CHANGELOG.md b/CHANGELOG.md index 3143d1bc..24234f5f 100644 --- a/CHANGELOG.md +++ b/CHANGELOG.md @@ -3,6 +3,72 @@ > 本文件追踪 `mcpp-community/mcpp` 公开仓的版本演进。 > 格式参考 [Keep a Changelog](https://keepachangelog.com/zh-CN/1.1.0/)。 +## [2026.8.10.1] — 2026-08-10 + +包身份、开发运行时与发布链收敛为同一组可验证事实。完整设计与验证记录见 +`.agents/docs/2026-08-09-mcpp-template-runtime-graphics-aur-focused-design.md` 与 +`.agents/docs/2026-08-09-mcpp-template-runtime-graphics-aur-validation.md`。 + +### 新增 + +- **模板 selector 与 `mcpp add` 同源。** `mcpp new --template` 统一为 + `[namespace.]name[@version][:tname]`;省略 namespace 只表示 `mcpplibs`,不再做 + 全索引短名搜索。未写 `tname` 时选择唯一显式 default;若只有一个模板且没有 + `default = true`,该单模板自动成为默认。多模板歧义明确失败并提示 + `--list-templates`,不引入 `--variant`。 +- **事务化 scaffold。** 包身份、版本、模板、变量与 hook 全部在 sibling 临时目录中 + 完成解析/渲染/校验,成功后才提交目标目录;失败不会留下半成品。 +- **根项目级 RuntimeBinding。** 未声明 `[xlings].subos` 使用 mcpp 初始化并经 release + 验证的 `McppDefault`;也可在 `mcpp.toml` 选择命名 SubOS。无 CLI/env override, + workspace root 决定整体环境,member/dependency 的 SubOS 不传递。不同 SubOS/glibc + contract 进入独立构建指纹,可在同一机器并存。 +- **provider-neutral 图形运行时契约。** mcpp-index 描述通用 requirement,xlings/xim + 选择 OpenGL/Vulkan、Mesa/NVIDIA、WSL、ICD/driver provenance;mcpp 只消费已解析的 + RuntimeBinding/LinkIntent、记录 resolution,并做平台通用链接与闭包校验,绝不探测 GPU。 +- **不可变 release manifest。** 全平台资产完成后生成 `mcpp-release.json`,逐项重算 + SHA256,并在公开 GitHub inventory 上重放比较;它是下游发布的唯一 desired state。 +- **`mcpp-bin` AUR reconciler。** 校验 manifest、双 Linux 资产/sidecar、AUR RPC/git 与 + Arch `vercmp`,先 dry-run exact diff,再以固定 host key 做普通 fast-forward push, + 有界重试并验证 RPC/公开 HEAD/干净 Arch 安装。定时恢复瞬时故障;`mcpp-m`、 + `mcpp-git` 不在自动化作用域。 + +### 修复 + +- **私有 glibc 不再泄漏进子进程环境(#401)。** `mcpp run` 曾把私有 glibc 目录放进 + `LD_LIBRARY_PATH`,而该变量会被程序派生的**每一个**进程继承;`/bin/sh` 由**宿主** + loader 加载(`PT_INTERP` 写死在可执行文件里,任何环境变量都改不了),于是在重定位 + 阶段就死于 `undefined symbol: __pointer_chk_guard, version GLIBC_PRIVATE`。 + 该目录本来只为「可执行文件 DT_NEEDED 闭包覆盖不到的 dlopen」而存在,而产物的 + RUNPATH 已经覆盖了它(link model 在 `--dynamic-linker` 旁就发了 + `-Wl,-rpath,`,改动前后产物 RUNPATH 逐字节相同),所以这条环境项没有收益、 + 只有代价。决策收敛在 `mcpp.platform.runtime_env_contract`:私有 libc 是 + **binary 作用域**,不是 environment 作用域。 +- RuntimeBinding 不再把 stale 的声明文本误当成实际 payload:有效 SubOS view 可规范化到 + 唯一受管 payload 时记录真实身份;旧 view 断链时只解析声明精确指名的 payload, + 仍绝不枚举目录挑版本。 +- Linux ELF/glibc 闭包规则的单测只在 Linux 断言相应物理语义;macOS/Windows 原生 CI + 固定验证 typed no-op 边界,不再拿 Linux 结果误判其他平台。 + +### 迁移 + +- **省略 namespace 的依赖获得一个版本的过渡期。** 省略 namespace 依然精确表示 + `mcpplibs`,但当精确坐标未命中时,`compat.` 与「不声明 namespace 的上游 + descriptor」两级会再被尝试一次;命中会打印弃用警告、给出可直接粘贴的 manifest + 片段,并把**规范身份**写入 lock/install/cache(歧义拼写只留在用户 manifest 里)。 + `mcpp add <裸名>` 直接把规范点分形式写回 `mcpp.toml`。 + 写明的身份(如 `mcpplibs.gtest`)不进入过渡期,未命中即失败;第三方 namespace 仍 + 不可被裸名触达。该过渡期在 `2026.9` 移除。 + - 理由:索引里已发布的 `compat.*` 包与既有用户 manifest 全部使用裸名写法。 + 「已发布的数据不得让程序失效」与「已发布的程序不得让既有数据失效」是同一条判据, + 两个方向都必须降级而不是变砖。#278 修掉的缺陷是**静默**回退,不是回退本身。 + +### 其他 + +- xlings pin 保持在 `2026.8.9.2`,自举 mcpp pin 提升到已发布的 `2026.8.8.4`。 + 曾提升到 `2026.8.10.1` 并回退:该版本在**冷 home** 上装不上 `xim:gcc@16.1.0` + (依赖解析到 glibc 2.44,而 gcc 的 config hook 找不到该 payload), + 同一 workflow 的冷缓存 A/B 已确认,见 openxlings/xlings#524。修复发布后再提升。 + ## [2026.8.8.4] — 2026-08-08 机器可读输出有契约了。设计与实测见 diff --git a/README.md b/README.md index 98b8e55b..2e8c76ab 100644 --- a/README.md +++ b/README.md @@ -141,6 +141,8 @@ yay -S mcpp-m # or build from source (bootstrapped with mcpp-bin) Installs the `mcpp` command system-wide; per-user data still lives in `~/.mcpp/`. On Arch the name `mcpp` is an unrelated C preprocessor, so the packages are `mcpp-bin` / `mcpp-m` (see [`scripts/aur/`](scripts/aur/)). +Stable-release automation reconciles `mcpp-bin` only; `mcpp-m` and `mcpp-git` +remain manually maintained and may intentionally lag. @@ -274,7 +276,7 @@ import mcpplibs.cmdline;
Developer experience -- `mcpp new` — create a modular project; `--template [@ver][:]` uses a **library-provided template** (e.g. `--template imgui`); `--list-templates ` lists them +- `mcpp new` — create a modular project; `--template [ns.]name[@version][:tname]` uses a **package-provided template** with the same exact identity style as `mcpp add`. A sole template is the default even without `default = true`; `--list-templates [ns.]name[@version]` lists ambiguous sets - `mcpp run [-- args]` — build and run - `mcpp test [pattern] [-- args]` — auto-discover and run tests (filter by name; `--list`, `--timeout `, `--message-format json`) - `mcpp search` — search package indices diff --git a/README.zh-CN.md b/README.zh-CN.md index 538a7b46..be4ac637 100644 --- a/README.zh-CN.md +++ b/README.zh-CN.md @@ -139,6 +139,8 @@ yay -S mcpp-m # 或源码构建(用 mcpp-bin 自举) 系统级安装 `mcpp` 命令,每个用户的数据仍在各自的 `~/.mcpp/`。 Arch 上 `mcpp` 这个名字属于一个无关的 C 预处理器,所以包名是 `mcpp-bin` / `mcpp-m`(详见 [`scripts/aur/`](scripts/aur/))。 +稳定 release 的自动对账只管理 `mcpp-bin`;`mcpp-m` 与 `mcpp-git` 仍由人工维护, +可能有意滞后。
@@ -271,7 +273,7 @@ import mcpplibs.cmdline;
开发体验 -- `mcpp new` — 创建模块化项目;`--template [@ver][:]` 使用**库自带模板**(如 `--template imgui`),`--list-templates ` 列举 +- `mcpp new` — 创建模块化项目;`--template [ns.]name[@version][:tname]` 与 `mcpp add` 使用同一精确身份风格并选择**包自带模板**。只有一个模板时即使未写 `default = true` 也自动成为默认;歧义时用 `--list-templates [ns.]name[@version]` 列举 - `mcpp run [-- args]` — 构建并运行 - `mcpp test [pattern] [-- args]` — 自动发现并运行测试(按名字过滤;`--list`、`--timeout `、`--message-format json`) - `mcpp search` — 搜索包索引 diff --git a/docs/00-getting-started.md b/docs/00-getting-started.md index 6b7b23a5..1d80e0e8 100644 --- a/docs/00-getting-started.md +++ b/docs/00-getting-started.md @@ -71,6 +71,27 @@ int main() { } ``` +### Creating from a package template + +`mcpp new --template` uses the same exact package-selector style as `mcpp add`: + +```bash +mcpp new gui-demo --template ocornut.imgui@1.92.8:docking +mcpp new --list-templates ocornut.imgui@1.92.8 +``` + +The grammar is `[namespace.]name[@version][:template]`. The namespace, version, +and template name are independently optional; omitting the namespace means the +single default namespace `mcpplibs`, not an index-wide short-name search. If the +template name is omitted, mcpp uses the sole `default = true` declaration, or +automatically uses the package's only template when none is explicitly marked. +Multiple templates without one default are an error that points to +`--list-templates`. There is no separate `--variant` vocabulary. + +The package identity, version, and template are fully resolved before the +destination is committed. A failed download, render, hook, or validation leaves +no half-created project directory. + ## Building and Running ```bash @@ -149,8 +170,9 @@ For the differences between the four modes and their artifact layouts, see [02 ## More Entry Points -- GUI quickstart: `mcpp new myapp --template imgui` (templates are distributed with the imgui library and their versions are aligned automatically; - run `mcpp new --list-templates imgui` to see all templates the library provides, or use `--template imgui:docking` to select a specific one). +- GUI quickstart: `mcpp new myapp --template ocornut.imgui@1.92.8:docking` + (templates are distributed with the package; omit `:docking` for its declared + default/sole template, or run `mcpp new --list-templates ocornut.imgui@1.92.8`). - Explaining default decisions: `mcpp why [toolchain|runtime|deps]`; host capability checkup: `mcpp self doctor`; machine-readable resolution manifest: the build artifact `target///resolution.json`. - Offline operation: `mcpp --offline` or `MCPP_OFFLINE=1` prevents index refreshes, downloads, and toolchain installation. diff --git a/docs/05-mcpp-toml.md b/docs/05-mcpp-toml.md index 999ce66c..8ed81a6d 100644 --- a/docs/05-mcpp-toml.md +++ b/docs/05-mcpp-toml.md @@ -375,18 +375,18 @@ Default convention: `src/.cppm` (e.g. package name ```toml # Packages under the default package namespace (mcpplibs) [dependencies] -gtest = "1.15.2" # Exact version -mbedtls = "3.6.1" -ftxui = "6.1.9" +cmdline = "0.0.2" # Exact version +templates = "0.0.1" -# Dotted selector: try mcpplibs. first, then fall back to the sibling peer root. -# For example, imgui.core is tried in order as mcpplibs.imgui/core, then imgui/core. -[dependencies] -capi.lua = "0.0.3" +# Dotted selector: one exact identity. Everything before the final dot is the +# namespace; the final segment is the package name. compat.gtest = "1.15.2" imgui.core = "0.0.1" imgui.backend.glfw_opengl3 = "0.0.1" +mcpplibs.capi.lua = "0.0.3" +``` +```toml # Namespace sub-table form [dependencies.mcpplibs] cmdline = "0.0.2" @@ -394,17 +394,23 @@ tinyhttps = "0.2.2" llmapi = "0.2.5" [dependencies.compat] -glfw = "3.4" # Explicit namespace, skips the mcpplibs-first candidate +glfw = "3.4" # Explicit namespace; no fallback search +``` +```toml # Path dependency (local development) [dependencies] mylib = { path = "../mylib" } +``` +```toml # Git dependency — pick exactly one of tag / branch / rev [dependencies] mylib = { git = "https://github.com/user/mylib.git", tag = "v1.0.0" } applib = { git = "https://github.com/user/applib.git", branch = "develop" } +``` +```toml # Long-form dep spec: features and backend knobs [dependencies] imgui = { version = "0.0.3", features = ["docking"] } # Request a feature of this dependency @@ -449,24 +455,20 @@ qux = ">=1.0, <2.0" # Range combination #### Namespace resolution rules -Every package has a two-part identity: a **namespace** and a **name**. How you write -a dependency key decides which namespaces mcpp will look in. - -**A bare name resolves in exactly three places**, in order: +Every package has a two-part identity: a **namespace** and a **name**. Every +selector normalizes to exactly one identity: -| # | Namespace | Example | -|---|---|---| -| 1 | `mcpplibs` — the default namespace | `cmdline = "0.0.2"` | -| 2 | `compat` — the wrapper namespace for third-party C/C++ libraries | `gtest = "1.15.2"` → `compat.gtest` | -| 3 | upstream packages that declare no namespace at all | `opencv = "4.10.0"` | +- `cmdline` → `(mcpplibs, cmdline)`; omitting the namespace means the + `mcpplibs` default, and nothing else. +- `compat.gtest` → `(compat, gtest)`. +- `mcpplibs.capi.lua` → `(mcpplibs.capi, lua)`. -**Any other namespace must be written out in full.** There is no fuzzy, index-wide -search by short name: +There is no ordered fallback or fuzzy, index-wide search by short name: ```toml # ✅ Correct — dotted selector [dependencies] -"chriskohlhoff.asio" = "1.38.1" +chriskohlhoff.asio = "1.38.1" # ✅ Correct — namespace sub-table (preferred when you have several from one org) [dependencies.chriskohlhoff] @@ -477,14 +479,41 @@ asio = "1.38.1" asio = "1.38.1" ``` -The third form fails with an error that lists the namespaces that were searched and, -when a package with that short name exists elsewhere, the exact line to write instead. +The third form fails with an error that names the exact `(mcpplibs, asio)` +identity that was tried and, when the short name exists elsewhere, gives a +copyable explicit selector. + +##### Migration window for bare names (`2026.8.10.1` → removed in `2026.9`) + +Every published `compat.*` package and every manifest written before exact +identity spells its dependency bare — `gtest = "1.15.2"`. Failing those +outright on upgrade would break builds against data that is already published +and cannot be edited retroactively, so for one release a bare name that misses +`mcpplibs` still reaches `compat.`, and a descriptor that declares no +namespace at all still answers to its bare name. -**Why not resolve bare names across every namespace?** Because dependency resolution -has to be reproducible. A global short-name search would mean that (a) two namespaces -owning the same short name are settled by index ordering, and (b) **adding an index -could silently change which package an existing dependency resolves to**. Requiring -the namespace keeps a `mcpp.toml` resolving to the same packages on every machine. +It is not quiet about it: + +``` +warning: dependency 'gtest' resolved to 'compat.gtest' through the deprecated +bare-name search; namespace omission means `mcpplibs` only. Write the exact +package: + [dependencies.compat] + gtest = "1.15.2" + (or run `mcpp add compat.gtest@1.15.2`). This fallback is removed in 2026.9. +``` + +What reaches `mcpp.lock`, the install layer and the cache is the canonical +identity, so the ambiguous spelling lives in exactly one place — your manifest +— until you change it. `mcpp add gtest@1.15.2` changes it for you. + +The window does **not** apply to a selector that states a namespace +(`mcpplibs.gtest` misses and stays missed), and a bare name still never reaches +a third-party namespace. + +**Why one identity?** Dependency resolution has to be reproducible. Candidate +search would let two namespaces with the same short name be settled by index +state, and adding an index could silently retarget an existing dependency. **For xpkg authors:** in an index descriptor, identity is the pair `(package.namespace, package.name)`. The namespace is the dotted path; **`name` is @@ -509,8 +538,17 @@ path) but not required. The older fully-qualified spelling (`name = "chriskohlhoff.asio"`) is still accepted, so already-published descriptors keep working. `mcpp xpkg parse` -enforces the rule — run it in your index CI. Requires mcpp >= 0.0.106 and -xlings >= 0.4.69; full normative text in `docs/spec/package-identity.md`. +enforces the descriptor rule — run it in your index CI. Descriptor identity +requires mcpp >= 0.0.106; exact selectors require mcpp >= 2026.8.10.1; both use +xlings >= 0.4.69. Full normative text is in `docs/spec/package-identity.md`. + +`mcpp new --template` deliberately reuses this identity model instead of +creating another package grammar: `[ns.]name[@version][:tname]`. A bare name +there also means only `mcpplibs`; version and template may be omitted +independently. Omitted `tname` selects the sole explicit default, or the only +template when no `default = true` is present. Multiple unmarked templates are +an error, never a directory-order choice. See the normative template rows in +`docs/spec/package-identity.md` §4.4. #### When mcpp refreshes the package index @@ -547,7 +585,7 @@ Run any command with `-v` to see the decision for each dependency and why. ### 2.6 `[dev-dependencies]` — Test Dependencies ```toml -[dev-dependencies] +[dev-dependencies.compat] gtest = "1.15.2" ``` @@ -905,31 +943,79 @@ exactly that: an entry's object paths are now addressed relative to the populate the entry first. `mcpp cache verify` additionally reports any entry whose recorded addresses escape it, so a recurrence is auditable offline. -### 2.11 `[runtime]` — Host Runtime Capabilities +### 2.11 `[runtime]` — Provider-neutral runtime contract ```toml [runtime] -library_dirs = ["vendor/lib"] # Directories baked into the artifact's RUNPATH (relative to the package root) -dlopen_libs = ["libGL.so.1"] # sonames dlopen'd at runtime (validated by doctor) -capabilities = ["opengl.glx.driver"] # Host capabilities required (open namespace) -provides = ["opengl.glx.driver"] # Explicitly declares capabilities this package fulfills (strong provider) - -# Explicit provider override (the "explicit" notch of the three-way knob) -[runtime."opengl.glx.driver"] -provider = "compat.glx-runtime" -``` - -- **Provider selection**: a package that declares `provides` (strong) takes precedence - over one that merely lists a capability under `capabilities` (weak, backward - compatible); `[runtime.] provider=` is an explicit override with the highest - precedence, and pointing at a provider not present in the dependency graph produces - a warning. -- The resolved result can be inspected via `mcpp why runtime`, `mcpp self doctor`, and - the build artifact `target///resolution.json` (it is not magic by - default). -- Capability naming convention: layered lowercase `domain.sub.role` (e.g. - `opengl.glx.driver`, `x11.display`) and prefix-style `abi:` (e.g. `abi:glibc`, - which participates in toolchain ABI enforcement). +requirements = [ + { kind = "capability", value = "display.present", phase = "run", required = true }, + { kind = "soname", value = "libwidget.so.1", phase = "link", required = false }, +] +provides = ["display.present"] +artifacts = [ + { role = "library", path = "runtime/libwidget.so.1", provenance = "payload", abi = "elf-x86_64", digest = "sha256:...", host_fingerprint = "host-1" }, +] + +# Platform-neutral LinkIntent. Paths are relative to this package root. +libraries = ["widget"] +link_library_dirs = ["lib"] +transitive_needed_dirs = ["runtime/closure"] +runtime_search_dirs = ["runtime"] +frameworks = ["WindowKit"] +deploy_files = ["bin/widget.dll"] + +# Use an exact canonical identity when multiple providers exist. +[runtime."display.present"] +provider = "acme.widget-runtime@2.0.0" +``` + +`requirements` records a non-empty `kind`/`value`, a `link` or `run` phase, +and whether the requirement is mandatory (`required` defaults to `true`). +Optional requirements remain visible provenance but do not become hard ABI or +doctor inputs. A `libraries` entry that is an explicit relative file path is +resolved against the declaring package root; a bare logical name remains a +platform-spelled library name. +`artifacts` requires `role`, `path`, and `provenance`; `abi`, `digest`, and +`host_fingerprint` are optional evidence. The resolver, not the descriptor, +stamps every requirement with the exact requester PackageId and every artifact +with the exact declaring provider PackageId, including namespace, version, and +source/index provenance. A descriptor therefore cannot spoof another package, +and `alpha.backend` never collapses into `beta.backend`. + +Only `provides` creates a descriptor-owned provider fact. Merely requiring a +capability never makes the requester its own provider. An explicit +`[runtime.] provider=` override accepts a canonical +`namespace.name@version` (or an unambiguous compatibility spelling); missing or +same-short-name ambiguous providers are hard errors. Provider/artifact facts +already selected by the xlings SubOS precede descriptor fallbacks. xlings/xim +owns graphics-stack, driver, ICD, WSL, and host provenance selection; mcpp +records and consumes the generic result and never probes GPU hardware. + +Link intent keeps discovery stages separate: + +| Field | ELF | Mach-O | PE/Windows | +|---|---|---|---| +| `link_library_dirs` | `-L` | `-L` | `-L` or `/LIBPATH:` | +| `transitive_needed_dirs` | `-Wl,-rpath-link` | no flag | no flag | +| `runtime_search_dirs` | RUNPATH/rpath only, never `-L` | rpath only | no flag | +| `frameworks` | no flag | `-framework` | no flag | +| `deploy_files` | copy edge | copy edge | copy beside the output; never a linker flag | + +For one compatibility train, `library_dirs` maps only to runtime search, +`dlopen_libs` maps to required run-phase soname requirements, and +`capabilities` maps to required run-phase capability requirements. None of +these legacy fields creates a provider. + +`target///resolution.json` schema 2 stores the RuntimeBinding, +canonical requirements/providers/artifacts, LinkIntent, platform search +mechanism, and post-link verdict. `mcpp why runtime` is a pure interpreter of +the latest stored file: it neither re-resolves the manifest nor launches a +graphics/hardware probe. Use `xlings doctor` when the selected host provider +itself needs re-diagnosis. + +Capability names use layered lowercase `domain.sub.role` (for example +`display.present`) and prefix-style `abi:` (for example `abi:glibc`, which +participates in toolchain ABI enforcement). ### 2.12 `[package] platforms` — Platform Declaration @@ -966,15 +1052,18 @@ build needs (`make`/`cmake`/`protoc`/…), pin tool versions per project, or set build-time env vars — without hand-editing `.xlings.json`. `[toolchain]` (§2.7) remains the ergonomic shorthand for the compiler; `[xlings.workspace]` is the general form. -`subos` carries a second meaning on Linux: it selects **which C runtime the -build binds against**. A subos describes its own runtime (xlings 2026.8.5.1+), -and mcpp takes that as the authority rather than looking around for a libc — so -`subos = "el8"` and `subos = "trixie"` in two projects on one machine produce -artifacts targeting each one's glibc, with the compile side and the run side -guaranteed to agree. The binding is part of the toolchain fingerprint, so -switching it rebuilds rather than reusing the other subos' objects. A subos -older than that (or none at all) falls back to whatever the toolchain itself -was installed against; `mcpp build -v` prints which of the two happened. See +`subos` selects the root project's **local build/run OS environment**. If the +key is absent, mcpp uses its initialized, release-verified `McppDefault` SubOS; +`subos = "default"` is an explicit `NamedSubos("default")` selection. There is +no CLI/environment override and no implicit following of xlings active/current. + +On Linux the selected environment also fixes the loader/libc contract, so +`subos = "el8"` and `subos = "trixie"` can coexist and produce separately +fingerprinted objects. A workspace root overrides member declarations during a +workspace build. Dependency/member SubOS declarations are non-transitive: a +library's declaration applies when it is an independent root, not when its +sources are consumed by another root. A missing named SubOS or missing/ +incompatible runtime contract is an error, never a fallback. See docs/08-toolchain-internals.md §2.1. ### 2.14 Host tools from a dependency (mcpp 2026.8.5.1+) @@ -1309,7 +1398,7 @@ version = "1.0.0" [targets.mymath] kind = "lib" -[dev-dependencies] +[dev-dependencies.compat] gtest = "1.15.2" ``` @@ -1417,7 +1506,7 @@ mcpp build --target x86_64-linux-musl | Static stdlib | `true` | Portable binary | | Headers | `include/` (if present) | Added to `-I` automatically | | Tests | `tests/**/*.cpp` | Discovered automatically by `mcpp test` | -| Dependency namespace | `mcpp` (default) | The flat form uses the default ns | +| Dependency namespace | `mcpplibs` (default) | A bare selector means only this exact namespace | ### 4.1 Legacy `[language]` Compatibility Layer diff --git a/docs/06-workspace.md b/docs/06-workspace.md index 7b917719..8f1fd3b4 100644 --- a/docs/06-workspace.md +++ b/docs/06-workspace.md @@ -100,7 +100,7 @@ Declare dependency versions centrally under `[workspace.dependencies]`; members # root mcpp.toml [workspace.dependencies] cmdline = "0.0.2" -capi.lua = "0.0.3" # dotted selector: mcpplibs.capi/lua, then capi/lua +mcpplibs.capi.lua = "0.0.3" # exact selector: (mcpplibs.capi, lua) [workspace.dependencies.compat] mbedtls = "3.6.1" diff --git a/docs/08-toolchain-internals.md b/docs/08-toolchain-internals.md index 6a2eb4d4..3bc59d14 100644 --- a/docs/08-toolchain-internals.md +++ b/docs/08-toolchain-internals.md @@ -8,16 +8,15 @@ ## 1. The model in one picture ``` -mcpp.toml [toolchain] / global default / `mcpp toolchain install` - │ (three entry paths — ONE shared pipeline) - ▼ -resolve payload (xim:gcc / xim:llvm / xim:musl-gcc xpkg under the sandbox) - ▼ -ensure_post_install_fixup() ← idempotent convergence (marker-gated) +mcpp.toml [xlings].subos / mcpp-managed default runtime ▼ resolve runtime binding ← which libc the artifact will load (§2.1) — an answer, not a search ▼ +resolve toolchain payload ← project/default/install paths share one pipeline + ▼ +ensure_post_install_fixup() ← exact glibc@version, marker-gated; never readdir-first + ▼ detect / probe ← triple, sysroot, payload paths (glibc, linux-headers) ▼ ToolchainLinkModel (single resolver for the C-library axis) @@ -27,6 +26,8 @@ ToolchainLinkModel (single resolver for the C-library axis) └──► cfg regeneration (the human-facing clang++.cfg) ▼ hermetic link check (`-###` dry-run) ← checks sandbox CRT/loader resolution + ▼ +link → internal ELF physics check ← validates the artifact and resolved closure (§6.1) ``` Two principles run through everything: @@ -77,17 +78,37 @@ anymore: the cfg is an output of this machinery, not an input (§5). ### 2.1 The runtime binding — which libc, decided once A payload-first build links against a specific glibc, and *which* one is a -fact about the environment, not something to be inferred. mcpp resolves it in -order: - -1. `[xlings] subos = ""` — the named subos, a sibling of the active one, - describes its own runtime in the `subos_info` block of its `.xlings.json` - (xlings 2026.8.5.1+). -2. The active subos, same block. -3. *Compatibility.* A subos created before that block existed cannot answer. - The value baked into the toolchain itself then stands in — gcc's specs, - clang's cfg. This is the value the artifact **would** load, so compile side - and run side still agree; it retires itself the moment the subos can speak. +fact about the root project's local development OS, not something to infer +from a compiler path or shell. mcpp has exactly two selection modes: + +1. No `[xlings].subos`: `McppDefault`, the initialized `subos/default` in the + xlings home selected by global mcpp configuration. +2. `[xlings] subos = ""`: `NamedSubos(name)`. Explicit `"default"` + remains a named selection; other names resolve in the root project's local + xlings scope. + +The workspace root owns the selection during a workspace build. Member and +dependency declarations do not merge or propagate; the same member's +declaration applies only when that source is an independent root. Neither +`XLINGS_ACTIVE_SUBOS`, `current`, the compiler's owner home, nor a CLI/env +override is a third selection rung. + +The selected SubOS must provide the supported `subos_info` contract. A missing +named environment or missing/incompatible contract is a hard error, never a +fallback to default/active/compiler-baked state. mcpp reads it once into a +`RuntimeBinding` snapshot, feeds its libc identity into payload probing, and +reuses the same snapshot for configure/link/run/test and the fast-path cache. +On Linux the snapshot also records the canonical selected loader/libc directory +and the optional creation-host glibc floor. These are evidence used by the +post-link validator, not another selection mechanism. + +The resolved SubOS view is authoritative when it canonically names one managed +glibc payload. Older xlings state may retain `runtime = "glibc@2.39"` after the +view has atomically moved to the managed 2.44 payload; mcpp records the physical +2.44 identity and path in that case. If an older view contains a broken link, +mcpp may resolve only the exact payload named by `runtime`; it never enumerates +installed versions or chooses a nearest/newest one. Both paths consume xlings +facts and preserve one RuntimeBinding rather than introducing an mcpp policy. No binding is a **refusal**, not a default: `CLibMode::PayloadFirst` is declined rather than picking a libc. @@ -102,9 +123,57 @@ against a runtime without them, and the failures surfaced on packages unrelated to the dependency that pulled the second glibc in. Directory order is not a decision procedure. -Because the binding decides what the artifact loads, it is part of the -toolchain fingerprint (11 fields, not 10) — two builds differing only in -runtime must not share a cache entry. +Because the binding decides what the artifact loads, the complete canonical +contract hash is field 11 of the toolchain fingerprint — two named SubOS +environments that happen to use the same libc still must not share a cache +entry when their providers or environment declarations differ. + +### 2.2 Generic runtime providers and artifacts + +`RuntimeBinding` also carries the provider-neutral facts selected by xlings for +the chosen development OS. `subos_info.runtime_contract` is an optional, +additive schema-1 block: + +```json +{ + "providers": [ + {"capability": "display.present", "provider": { + "namespace": "xim", "name": "display-runtime", "version": "1.0.0", + "source": "xim-pkgindex@"}} + ], + "artifacts": [ + {"role": "driver", "provider": { + "namespace": "xim", "name": "display-runtime", "version": "1.0.0", + "source": "xim-pkgindex@"}, + "path": "${subosdir}/lib/runtime/provider.so", + "provenance": "subos_view", "abi": "elf-x86_64", + "digest": "sha256:...", "host_fingerprint": "..."} + ] +} +``` + +The binding parser resolves `${subosdir}` and relative artifact paths once, +sorts the facts, and includes them in the contract hash and cache snapshot. In +the build plan these selected provider facts precede descriptor-declared +fallbacks. Descriptor requirements and artifacts are independently stamped +with their resolved requester's/provider's canonical PackageId, so equal short +names in different namespaces remain distinct end to end. + +The ownership boundary is deliberate: mcpp-index expresses generic runtime +requirements; xlings/xim chooses and diagnoses the host graphics/runtime stack; +mcpp consumes only the selected provider/artifact facts and generic LinkIntent. +mcpp has no hardware, driver-vendor, WSL, or ICD selection path. Its source gate +rejects introducing such provider-specific branches or coupling those terms to +a launched probe. + +`LinkIntent` keeps `linkLibraryDirs`, `transitiveNeededDirs`, and +`runtimeSearchDirs` separate. The last category is never rendered as `-L`: +ELF receives rpath plus `-rpath-link` only for the transitive category, Mach-O +receives rpath/framework flags, and PE receives link-library paths plus explicit +deploy-file copy edges. The exact RuntimeBinding, canonical identities, link +intent, search mechanism, and post-link verdict are persisted in +`resolution.json` schema 2. `mcpp why runtime` only interprets that stored file; +re-diagnosis belongs to `xlings doctor`. ## 3. The link model (`src/toolchain/linkmodel.cppm`) @@ -274,6 +343,76 @@ class that faithfully reproduces the clean-machine failure mode, plus e2e `86_llvm_hermetic_link.sh` which re-checks the `-###` resolution on every machine. +### 6.1 Post-link Linux runtime physics (`elf_runtime.cppm`) + +The hermetic check answers a pre-link question: what does the driver appear to +resolve? The runtime-physics check answers the stronger post-link question: +what did the newly produced ELF actually record and what will its closure +load? `[build] allow_host_libs = true` deliberately relaxes the first check; +it does not suppress physical impossibilities in the second. + +For each newly linked Linux executable/shared object, mcpp parses ELF64 +little-endian program/dynamic/GNU-version tables internally—no `readelf`, +`patchelf`, or `ldd` subprocess on the build path—and records `PT_INTERP`, +`DT_RPATH`/`DT_RUNPATH`, `DT_NEEDED`, and required/defined `GLIBC_*` versions. +It resolves the declared closure using the artifact search paths, selected +runtime/toolchain directories, and known host library directories, then applies: + +- **Rule B (same source):** `PT_INTERP` and every resolved `libc.so.6` must be + the canonical payload selected by `RuntimeBinding`. A host loader plus private + libc, or two private libc payloads, is a proven pre-main failure. +- **Rule A (version floor):** every closure request for `GLIBC_x.y` must be no + newer than the selected libc's exported GNU version definitions. Linking a + host DSO is allowed when this holds; mcpp is checking physics, not imposing a + no-host-library policy. + +Verdicts are typed: `Pass`, `ProvenMismatch`, or `Inconclusive`. Proven A/B +mismatches fail the build with canonical requester/provider/artifact paths and +a copyable SubOS remediation. Missing loader-cache/hardware closure data is +reported as inconclusive, never relabelled green. macOS and Windows use the +same interface as a typed no-op and never receive ELF/glibc rules. + +The verdict is stored as `.mcpp-runtime-verdicts.json` beside `build.ninja`, +keyed by artifact stat fingerprint plus the complete runtime contract hash. +Hot no-op builds require a current passing record, compare artifact stats +before/after Ninja, and perform zero ELF parses. An unexpected relink drops to +the full path before success or execution. `mcpp self doctor` reports the same +stored verdict rather than re-probing a potentially different current host. + +Post-install alignment follows the same identity rule: `glibc@2.44` resolves +only `/xim-x-glibc/2.44/{lib64,lib}`. A missing/stale exact payload is an +error; another installed version is never a fallback. + +### 6.2 Where a runtime search path is allowed to live (`runtime_env_contract.cppm`) + +There are two ways to tell a loader where to look, and they differ by blast +radius, not by convenience: + +| | reaches | | +| --- | --- | --- | +| `DT_RUNPATH` | the one object that carries it, and its `dlopen()` | per-binary | +| `LD_LIBRARY_PATH` | the process **and every process it ever spawns** | inherited, forever | + +That second row is why the private libc payload is **binary-scoped**. A glibc's +`libc.so.6` and its `ld.so` are version-locked through `GLIBC_PRIVATE`: 2.44's +libc carries an undefined `__pointer_chk_guard` that only 2.44's own loader +exports. An mcpp-built program is fine — `PT_INTERP` names the private loader. +`/bin/sh` is not: its `PT_INTERP` names the **host** loader and no environment +variable can override it, so a `popen()`/`system()` child dies during +relocation, before `main`, with no output (mcpp#401; mcpp#291 is the same shape +one hop closer in, killing mcpp's own nested host tools). + +So mcpp never publishes the private libc directory through the environment. It +does not need to: wherever a payload exists the link model already emits +`-Wl,-rpath,` beside `--dynamic-linker`, which covers the case the +directory exists for — a `dlopen()` whose own `DT_NEEDED` closure does not +consult the executable's RUNPATH. + +This is a scope, not a condition. "Only export it when a dependency might +`dlopen()`" still exports it, and the child that dies does not care why. Plain +dependency runtime directories keep their environment scope: they have no +loader coupling, so a host binary that stumbles onto them is at worst confused. + ## 7. Extending the machinery ### 7.1 Adding a new toolchain (new compiler family or distribution) @@ -350,6 +489,9 @@ everything §3–§4 does for ELF. | link model + loader resolution | `src/toolchain/linkmodel.cppm` | | unified fixup pipeline (patchelf/specs/cfg, marker) | `src/toolchain/post_install.cppm` | | install/lifecycle entry | `src/toolchain/lifecycle.cppm`; auto-install entries in `src/build/prepare.cppm` | +| root runtime selection/binding | `src/xlings/runtime_selection.cppm`, `src/platform/runtime_binding.cppm`, `src/xlings/subos_info.cppm` | +| generic runtime contract + LinkIntent | `src/manifest/types.cppm`, `src/build/plan.cppm`, `src/build/flags.cppm` | +| stored resolution explanation | `src/build/prepare.cppm`, `src/build/runtime_validation.cppm`, `src/doctor.cppm` | | flag assembly (main build) | `src/build/flags.cppm` | | `import std;` precompile | `src/toolchain/stdmod.cppm` | | build.mcpp host flags | `src/build/build_program.cppm` | diff --git a/docs/09-release.md b/docs/09-release.md index 88f9eb06..72a60bf1 100644 --- a/docs/09-release.md +++ b/docs/09-release.md @@ -41,11 +41,48 @@ tag from `mcpp.toml`) does all of this: ``` build ×4 (linux x86_64 / linux aarch64 / macOS ARM64 / Windows x64) → GitHub Release v with tarballs + .sha256 sidecars + → recompute every payload hash and publish immutable mcpp-release.json → mirror to xlings-res/mcpp on BOTH GitHub and GitCode → open the version-bump PR against openxlings/xim-pkgindex → workflow_run hook fires ci-fresh-install ``` +### 2.1 The immutable release manifest + +`release-manifest` waits for all four platform upload jobs. It downloads the +final non-draft, non-prerelease GitHub Release, recomputes each versioned +platform payload's SHA256, verifies the corresponding `.sha256` sidecar, and +publishes `mcpp-release.json` schema 1: + +```json +{ + "schema": 1, + "version": "", + "tag": "v", + "commit": "", + "assets": [ + { + "platform": "linux", + "arch": "x86_64", + "name": "mcpp--linux-x86_64.tar.gz", + "sha256": "" + } + ] +} +``` + +The array is sorted by platform, architecture, and name. It contains every +versioned platform payload, including the required Linux x86_64, Linux aarch64, +macOS ARM64, and Windows x86_64 assets. Versionless aliases and the source +archive are intentionally not desired-state rows. + +The workflow then downloads the public release again, regenerates the manifest, +and requires a byte-for-byte match. A workflow rerun accepts an existing +manifest only when it is already byte-identical; it never overwrites different +bytes for the same tag. Downstream release consumers (notably the `mcpp-bin` +AUR reconciler) must consume this manifest instead of guessing completeness +from a moving workspace or from a subset of release assets. + Two steps are **not** automated: - **merging the xim-pkgindex bump PR** — a maintainer does it. Until it lands, @@ -71,6 +108,28 @@ curl -fsSL -o /tmp/p.tgz "https://github.com/xlings-res/mcpp/releases/download/$ sha256sum /tmp/p.tgz # compare against pkgs/m/mcpp.lua in xim-pkgindex ``` +To repeat the release gate locally against the public GitHub inventory: + +```bash +V= +TAG="v$V" +AUDIT=$(mktemp -d) +mkdir -p "$AUDIT/assets" +gh api "repos/mcpp-community/mcpp/releases/tags/$TAG" > "$AUDIT/release.json" +gh release download "$TAG" -R mcpp-community/mcpp --dir "$AUDIT/assets" +python3 scripts/release/generate_manifest.py \ + --release-json "$AUDIT/release.json" \ + --assets-dir "$AUDIT/assets" \ + --version "$V" \ + --tag "$TAG" \ + --commit "$(git rev-list -n 1 "$TAG")" \ + --output "$AUDIT/expected.json" +cmp "$AUDIT/assets/mcpp-release.json" "$AUDIT/expected.json" +``` + +This command recomputes payload hashes; it does not copy hashes out of the +published manifest or blindly trust the sidecars. + Then a real install, in a **clean-room `XLINGS_HOME`** — never the machine's own `~/.xlings`, which can mask a broken index with cached state: @@ -163,7 +222,8 @@ would again let the index guard and the installed version drift apart. [ ] `bash .github/tools/check_version_pins.sh` passes (verifies `mcpp.toml` = `MCPP_VERSION`, and `.xlings.json` is not newer) [ ] merge to main, CI green [ ] gh workflow run release.yml --ref main -[ ] release.yml green (4 builds + publish-ecosystem) +[ ] release.yml green (4 builds + immutable manifest + publish-ecosystem) +[ ] downloaded mcpp-release.json regenerates byte-identically from public assets [ ] mirrors serve all four platforms on BOTH hosts, sha256 recomputed [ ] merge the xim-pkgindex bump PR [ ] clean-room XLINGS_HOME: xlings install mcpp@ succeeds diff --git a/docs/spec/package-identity.md b/docs/spec/package-identity.md index 46261995..509661cd 100644 --- a/docs/spec/package-identity.md +++ b/docs/spec/package-identity.md @@ -5,9 +5,9 @@ | **规范编号** | SPEC-001 | | **标题** | 包身份(`package.namespace` / `package.name`)、`[dependencies]` 选择器与匹配机制 | | **状态** | **评审中(Review)** —— 已实现 | -| **版本** | 1.1 | -| **最后修改** | 2026-08-03 | -| **最低实现版本** | mcpp **0.0.106**(xlings >= 0.4.69) | +| **版本** | 1.2 | +| **最后修改** | 2026-08-09 | +| **最低实现版本** | 描述符身份:mcpp **0.0.106**;精确 selector:mcpp **2026.8.10.1**(xlings >= 0.4.69) | | **作者/维护** | mcpp-community | | **相关设计文档** | `.agents/docs/2026-06-20-package-resolution-architecture.md` §4
`.agents/docs/2026-06-26-identity-first-resolution-no-filename.md`
`.agents/docs/2026-07-25-issue278-descriptor-name-form-canonicalization-design.md`
`.agents/docs/2026-07-25-name-namespace-bidirectional-verification-report.md`
`.agents/docs/2026-07-25-name-namespace-canonical-implementation-spec.md` | | **相关 issue** | [mcpp#278](https://github.com/mcpp-community/mcpp/issues/278)
[xlings#381](https://github.com/openxlings/xlings/issues/381) —— 索引键缺命名空间维度(§3.3) | @@ -26,12 +26,12 @@ | 标记 | 含义 | |---|---| -| ✅ **已实现** | 自 mcpp 0.0.106 起的行为与本规范一致 | +| ✅ **已实现** | 自对应“最低实现版本”起的行为与本规范一致 | | ⚠️ **部分实现** | 已有实现,但语义或覆盖面与本规范有差异(差异已注明) | | ❌ **未实现** | 本规范要求但尚未支持;当前行为已注明 | -> **本规范所需行为自 mcpp 0.0.106 起已全部实现,当前实现继续符合。** 索引作者按 §3 -> 书写即可。0.0.105 及更早版本要求的过渡形态(`name` 必须写成 +> 描述符身份规则自 mcpp 0.0.106 起实现;唯一精确 selector 自 2026.8.10.1 +> 起实现。0.0.105 及更早版本要求的过渡形态(`name` 必须写成 > `.`)仍被接受为**兼容写法**,见 §8。 --- @@ -42,7 +42,7 @@ - 索引描述符(`.lua`)中 `package.namespace` / `package.name` 的语义与形态 - `mcpp.toml` 中 `[dependencies]`(及 `[dev-dependencies]` / `[build-dependencies]` / `[feature-deps.*]`)的书写文法 -- 从用户书写 → 候选身份 → 描述符发现 → 身份校验 → 安装目标的完整匹配机制 +- 从用户书写 → 唯一身份 → 描述符发现 → 身份校验 → 安装目标的完整匹配机制 ### 1.2 边界:mcpp 不改动 xlings 规范 @@ -119,7 +119,7 @@ package = { namespace = "compat", name = "compat.zlib" } -- legacy,短名 = "z ✅ **规范如此**。 -✅ **已实现**。xlings **0.4.69** 起索引按 `(effectiveNamespace, name)` 建键([#381](https://github.com/openxlings/xlings/issues/381)),同一索引内两个同短名不同命名空间的包各自可寻址;裸名请求多候选时报 ambiguity 并列出候选。 +✅ **已实现**。xlings **0.4.69** 起索引按 `(effectiveNamespace, name)` 建键([#381](https://github.com/openxlings/xlings/issues/381)),同一索引内两个同短名不同命名空间的包各自可寻址;mcpp 始终传入唯一精确身份。 e2e `163_identity_first_resolution.sh` 端到端锁住:同一 path 索引内 `(alpha, widget)` 与 `(beta, widget)` 各自安装到 `alpha-x-widget` / `beta-x-widget`。 @@ -149,30 +149,30 @@ e2e `163_identity_first_resolution.sh` 锁住:身份为 `(acme, widget)` 的描 ## 4. 消费侧规范:`[dependencies]` 书写文法 -用户写的**不是身份,而是选择器(selector)** —— 它展开为一个**有序候选身份列表**,按序尝试,首个满足者胜出。 +用户写的是 selector;它必须在 O(输入长度)、无 I/O 的解析阶段规范化成**唯一一个** `(namespace, name)`。索引状态、候选顺序或已安装内容不得参与身份决定。 ### 4.1 四种书写形式 -| # | 写法 | 展开为候选(按序) | 语义 | +| # | 写法 | 规范化身份 | 语义 | |---|---|---|---| -| 1 | `[dependencies]`
`gtest = "1.15.2"` | ① `(mcpplibs, gtest)`
② `(∅, gtest)` | **裸名**。只解析默认命名空间搜索路径(§5.2) | -| 2 | `[dependencies]`
`acme.widget = "1.0"` | ① `(mcpplibs.acme, widget)`
② `(acme, widget)` | **点式选择器**。先试默认命名空间下的同名子空间,再试同级 peer root | -| 3 | `[dependencies.acme]`
`widget = "1.0"` | ① `(acme, widget)` | **命名空间子表**。权威、单候选、无猜测 —— **推荐用于第三方命名空间** | -| 4 | `[dependencies]`
`"acme.widget" = "1.0"` | ① `(acme, widget)` | **引号点式键**(legacy)。单候选,等价于 #3 | +| 1 | `[dependencies]`
`cmdline = "0.0.2"` | `(mcpplibs, cmdline)` | **裸名**。省略 namespace 只表示默认 `mcpplibs` | +| 2 | `[dependencies]`
`acme.widget = "1.0"` | `(acme, widget)` | **点式选择器**。最后一段是 name,之前所有段是 namespace | +| 3 | `[dependencies.acme]`
`widget = "1.0"` | `(acme, widget)` | **命名空间子表**。与 #2 完全等价,多个同 namespace 包时更清晰 | +| 4 | `[dependencies]`
`"acme.widget" = "1.0"` | `(acme, widget)` | **引号点式键**(legacy source shape)。身份仍与 #2/#3 相同 | -✅ **已实现**(`resolve_dependency_selector` / `make_direct_dependency_selector`)。 +✅ **已实现**(`parse_package_selector` / `normalize_package_selector` / `make_direct_dependency_selector`)。 -**#2 与 #3 的区别很重要**:`[dependencies.acme]` 是**显式 TOML 表**,mcpp 视其为命名空间根,直接产出单候选;而 `[dependencies]` 里的点式键是**有序猜测**,会先试 `mcpplibs.` 前缀。 - -判定「是否命名空间根」的条件:该路径是 TOML **显式表**(`[dependencies.acme]` 被真实书写过),**或**键名恰为默认命名空间 `mcpplibs`。 +**#2 与 #3 只有 TOML 版式差异,没有解析差异。** 新增索引或同短名 sibling 不能改变其身份。 多级命名空间同理,`ns` 逐层累积:`[dependencies.mcpplibs.capi]` + `lua = "0.0.3"` → 单候选 `(mcpplibs.capi, lua)`。 -### 4.2 裸名的解析域是封闭的 +### 4.2 裸名就是默认 mcpplibs + +> **裸名(#1)只能表示 `(mcpplibs, name)`。`compat`、第三方 namespace 与无 namespace 包都不会成为隐式候选。** -> **裸名(#1)只解析三类包:`mcpplibs`(默认命名空间)、`compat`(包装命名空间)、无 `namespace` 声明的上游包。任何声明了第三方命名空间的包,禁止用裸名请求。** +例如 gtest 必须写成 `compat.gtest` 或 `[dependencies.compat] gtest = ...`;裸 `gtest` 请求的是不同身份 `(mcpplibs, gtest)`。 -✅ **已实现**(0.0.105)。裸名请求命中一个声明了非空第三方 `namespace` 的描述符时,该候选被拒绝。 +✅ **已实现**(2026.8.10.1)。默认 namespace 的依赖身份门禁不再接纳 `compat` 或无 namespace descriptor。 **设计理由**:全域按短名搜索会让解析结果取决于「本机装了哪些索引」—— 1. 两个命名空间拥有同名包时,胜负由索引顺序决定,而用户 `[indices]` 添加的索引之间**没有全序**; @@ -181,18 +181,45 @@ e2e `163_identity_first_resolution.sh` 锁住:身份为 `(acme, widget)` 的描 依赖解析的**可复现性**优先于书写便捷性。 +#### 4.2.1 过渡期(`2026.8.10.1` 起,`2026.9` 移除) + +索引里已发布的 `compat.*` 包与既有用户 `mcpp.toml` **全部**使用裸名写法。 +让它们在一次 mcpp 升级后直接失败,等于「发布一个程序,让已经发布、且无法追溯修改的 +数据失效」——这与「索引抬高 floor 不得让旧客户端变砖」是同一条判据的两个方向, +两边都必须**降级**而不是变砖。 + +所以省略 namespace 的 selector 在一个版本内保留一条**出口坡道**: + +1. 先精确解析 `(mcpplibs, name)`; +2. **仅在未命中时**,再依次尝试 `(compat, name)` 与「descriptor 自己不声明 namespace」 + 这一级; +3. 命中即打印弃用警告,内容包含实际选中的完整身份与可直接粘贴的 manifest 片段; +4. 写入 lock、install 与 cache 的是**规范身份**(`compat.gtest`),歧义拼写只留在 + 用户 manifest 里,直到用户改它; +5. `mcpp add <裸名>` 直接把规范点分形式写回 `mcpp.toml`——碰一次就迁移一次。 + +不适用的情形(**不是**过渡期的一部分): + +- 写明了 namespace 的 selector(`mcpplibs.gtest`、`[dependencies.mcpplibs]`)—— + 那是一个身份声明,未命中就是未命中; +- 第三方 namespace——裸名从来、且仍然不可触达(§4.2 的供应链理由不变)。 + +> #278 修掉的缺陷是**静默**:mcpp 带着一个用户从未写过的 namespace 继续往下走, +> 并且不说。一条带完整身份的警告已经消灭了「静默」,同时保住了已发布的数据。 +> 同一个版本对 `ns:name → ns.name` 用的也是同一套过渡期处理。 + ### 4.3 解析失败时的诊断 -候选全部落空时,mcpp **必须**明确失败,并列出尝试过的身份;若该短名存在于其他命名空间,**应当**给出可直接抄写的正确写法。 +唯一身份落空时,mcpp **必须**明确失败并列出该身份;若同短名存在于其他 namespace,**应当**只在诊断中给出可复制的显式 selector,禁止把提示结果回灌解析。 -✅ **已实现**(0.0.105): +✅ **已实现**(2026.8.10.1): ``` -error: dependency 'asio': no package found under the namespaces mcpp searched - tried: (mcpplibs, asio), (no namespace, asio) +error: dependency 'asio': no package found + tried: (mcpplibs, asio) a package with this name exists under another namespace: chriskohlhoff.asio - bare names only resolve to the `mcpplibs` / `compat` namespaces. write it out: + namespace omission means `mcpplibs`; write the exact package: [dependencies] "chriskohlhoff.asio" = "1.38.1" or: @@ -200,7 +227,34 @@ error: dependency 'asio': no package found under the namespaces mcpp searched asio = "1.38.1" ``` -该 did-you-mean 扫描**仅**在已失败路径触发,结果**只进错误文案**,禁止回灌解析、lockfile 或安装层 —— 否则就退化成 §4.2 否决的全域模糊匹配。 +该 did-you-mean 扫描**仅**在已失败路径触发,结果**只进错误文案**,禁止回灌解析、lockfile 或安装层。 + +迁移 release 对旧的 compact dotted 搜索做两件事:已有 lock 继续固定已记录身份;无 lock 时若旧的 `mcpplibs.` primary 确实存在,warning 同时显示旧/新完整 selector,但仍不回退。 + +### 4.4 `mcpp new --template` 的同源 selector + +模板不拥有第二套包身份。`--template` 与 `--list-templates` 复用 §4 的精确 +`PackageSelector`,只在包身份/版本之后增加模板名轴: + +| 输入 | 规范结果 | +|---|---| +| `pkg` | `(mcpplibs, pkg)` + latest stable + default/单模板 | +| `pkg@1.2.0` | `(mcpplibs, pkg)` + `1.2.0` + default/单模板 | +| `acme.widget` | `(acme, widget)` + latest stable + default/单模板 | +| `acme.widget@1.2.0:docking` | `(acme, widget)` + `1.2.0` + `docking` | + +完整文法为 **`[namespace.]name[@version][:tname]`**。namespace、version、tname +分别可省略;namespace 的省略语义与依赖完全一致,只表示默认 `mcpplibs`。模板默认 +规则按以下顺序且必须唯一: + +1. 正好一个 template 声明 `default = true`,选择它; +2. 没有显式 default,但当前已解析包版本只有一个 template,选择该单模板; +3. 多个 template 且没有 default,hard error 并提示 `--list-templates`。 + +模板目录顺序、索引顺序和本机缓存都不得参与选择。`--variant` 不属于词汇表。 +scaffold 必须先解析完整 PackageId/version/template,再以 sibling 临时目录事务生成; +失败不得留下目标目录。旧 `pkg:` 的“列举模板”含义仅保留一个 release train 的迁移 +warning,规范列举命令是 `mcpp new --list-templates pkg`。 --- @@ -212,34 +266,33 @@ error: dependency 'asio': no package found under the namespaces mcpp searched 用户书写(selector) │ §4.1 展开 ▼ -有序候选身份列表 [(ns₁,n₁), (ns₂,n₂), …] - │ 逐个尝试 +唯一身份 (ns,name) + │ 精确查询 ▼ ① 发现:先探测推荐文件名,落空后按声明身份扫描描述符(§3.4) ② 校验:xpkg_lua_identity_matches 复核声明身份 ← §5.2 -③ 收敛:INV-RESOLVE 拒绝裸名命中第三方 ns ← §4.2 -④ 回填:用描述符**声明的** namespace 定身份 ← §5.3 - │ 首个通过者胜出;全部落空 → §4.3 报错 +③ 验证:descriptor 身份必须精确相等 ← §5.2 + │ 落空 → §4.3 报错 ▼ 选定身份 (ns, name) → 派生 wire key / store dir(§6) ``` ### 5.2 身份校验规则 -给定候选 `(ns, shortName)` 与一个描述符,`xpkg_lua_identity_matches` 的判定: +给定请求 `(ns, shortName)` 与一个描述符,`xpkg_lua_identity_matches` 的判定: -| 候选 `ns` | 判定 | +| 请求 `ns` | 判定 | |---|---| | 描述符无 `name` | **接受**(无从校验,宽松) | -| `ns` 为空(discovery) | 短名相等即可 | -| `ns == "mcpplibs"`(默认命名空间) | 描述符身份的 ns ∈ **{`mcpplibs`, `compat`}**,或(旧式)无 ns | +| `ns` 为空(仅内部 legacy discovery) | 短名相等即可;不得进入依赖 selector 主路径 | +| `ns == "mcpplibs"`(默认命名空间) | 依赖 selector 主路径要求描述符身份必须为 `mcpplibs`;底层 legacy API 的 `allowLegacyBareDefault` 只供非 selector 兼容调用 | | 其他具体 ns | **精确相等** | -✅ **已实现**。第三行即**默认命名空间搜索路径**:裸 `gtest` 能命中 `compat.gtest`,正是这条。 +✅ **已实现**。`compat.gtest` 与 `mcpplibs.gtest` 是两个不同身份。 -### 5.3 空命名空间的回填(P3) +### 5.3 空命名空间的兼容边界(P3) -候选 `(∅, name)` 命中后,**必须**用描述符**声明的** `namespace` 作为最终身份,而不是候选的空值。 +旧 discovery 调用 `(∅, name)` 命中后,**必须**用描述符**声明的** `namespace` 作为最终身份,而不是候选的空值。新的依赖 selector 已先填充默认 namespace,不使用 discovery;模板 selector 在切换到同一共享解析链后也必须遵守该规则。 若描述符本身未声明 `namespace`(上游裸包如 `opencv`),则**空命名空间就是它的合法身份**,不得强行填充。 @@ -347,18 +400,16 @@ asio = "1.38.1" **错误写法**:`[dependencies]` + `asio = "1.38.1"` —— 裸名不解析第三方命名空间(§4.2),报错并给出上述两种正确写法。 -### 9.2 裸名经 compat 搜索路径 +### 9.2 compat 必须显式选择 ```toml -[dependencies] +[dependencies.compat] gtest = "1.15.2" ``` ``` -选择器 gtest → 候选 ① (mcpplibs, gtest) ② (∅, gtest) -候选①:探测 gtest.lua / mcpplibs.gtest.lua / compat.gtest.lua - 命中 compat.gtest.lua,声明 (compat, compat.gtest) → 归一 (compat, gtest) - 校验:候选 ns 为默认命名空间 → 允许 ns ∈ {mcpplibs, compat} → ✓ +选择器 compat.gtest → 唯一身份 (compat, gtest) +探测 compat.gtest.lua,声明 (compat, compat.gtest) → 归一 (compat, gtest) → ✓ 身份 (compat, gtest) → store dir compat-x-compat.gtest ``` @@ -381,6 +432,7 @@ lua = "0.0.3" | 版本 | 日期 | 变更 | |---|---|---| +| 1.2 | 2026-08-09 | selector 收敛为唯一精确 PackageId:裸名只表示默认 mcpplibs,dotted 以最后一段为 name;移除 compat/空 namespace 隐式候选,加入 lock 保持与一个 release train 的双 selector 迁移 warning | | 1.1 | 2026-08-03 | 按当前实现复核:澄清文件名发现是快路径加身份回退扫描,修正 legacy `package.name` 的 wire key 示例,并将 0.0.106 明确为最低实现版本 | | 0.1 | 2026-07-25 | 首版草案。整合 #278 的双向验证结论:确立「身份 = `(namespace, name)`、层级归 `namespace`、`name` 为原子段」为规范形态,并如实标注 0.0.105 的过渡形态(强制 FQN)与全部待实现项 | | 1.0 | 2026-07-25 | **mcpp 0.0.106 全部实现**:身份归一化去 split-on-last-dot、target 用字面 `name`、store 目录、文件名自由(快路径+身份扫描)、`name` 形态校验反转。xlings 0.4.69 修好 #381 后 §3.3 的 `(namespace, name)` 唯一自然成立。状态 草案 → 评审中 | @@ -392,5 +444,5 @@ lua = "0.0.3" --- -> **本规范所需行为自 mcpp 0.0.106 起已全部实现,当前实现仍符合**,状态为「评审中」。 +> 描述符身份规则自 mcpp 0.0.106 起实现;精确 selector 自 2026.8.10.1 起实现。当前实现符合本规范,状态为「评审中」。 > 英文版待补(`docs/spec/` 顶层按仓库惯例为英文,本文档先以中文成稿)。 diff --git a/docs/zh/00-getting-started.md b/docs/zh/00-getting-started.md index 3d5024d6..c6014209 100644 --- a/docs/zh/00-getting-started.md +++ b/docs/zh/00-getting-started.md @@ -69,6 +69,24 @@ int main() { } ``` +### 从包模板创建项目 + +`mcpp new --template` 与 `mcpp add` 使用同一种精确包 selector 风格: + +```bash +mcpp new gui-demo --template ocornut.imgui@1.92.8:docking +mcpp new --list-templates ocornut.imgui@1.92.8 +``` + +文法是 `[namespace.]name[@version][:template]`,namespace、version 与模板名可分别 +省略。省略 namespace 只表示唯一默认命名空间 `mcpplibs`,不会按短名扫描整个索引。 +省略模板名时,mcpp 使用唯一的 `default = true`;若包只有一个模板且未显式声明 +default,该单模板自动成为默认。多个模板却没有 default 会明确报错并提示 +`--list-templates`。不再引入另一套 `--variant` 词汇。 + +包身份、版本与模板会在提交目标目录前全部解析完成;下载、渲染、hook 或校验失败时, +不会留下半成品项目目录。 + ## 构建与运行 ```bash @@ -143,8 +161,9 @@ mcpp pack --mode self-contained # 打包 loader、libc 与依赖 ## 更多入口 -- GUI 起步:`mcpp new myapp --template imgui`(模板随 imgui 库分发、版本自动对齐; - `mcpp new --list-templates imgui` 查看库提供的全部模板,`--template imgui:docking` 选指定模板)。 +- GUI 起步:`mcpp new myapp --template ocornut.imgui@1.92.8:docking`(模板随包分发; + 省略 `:docking` 使用已声明 default/唯一模板,或运行 + `mcpp new --list-templates ocornut.imgui@1.92.8`)。 - 解释默认决策:`mcpp why [toolchain|runtime|deps]`;主机能力体检:`mcpp self doctor`; 机器可读解析清单:构建产物 `target///resolution.json`。 - 离线运行:`mcpp --offline` 或 `MCPP_OFFLINE=1` 可阻止索引刷新、下载和工具链安装。 diff --git a/docs/zh/05-mcpp-toml.md b/docs/zh/05-mcpp-toml.md index c8c109b0..a964e893 100644 --- a/docs/zh/05-mcpp-toml.md +++ b/docs/zh/05-mcpp-toml.md @@ -340,18 +340,17 @@ path = "src/capi/lua.cppm" # 覆盖默认的 lib-root 位置 ```toml # 默认包空间(mcpplibs)下的包 [dependencies] -gtest = "1.15.2" # 精确版本 -mbedtls = "3.6.1" -ftxui = "6.1.9" +cmdline = "0.0.2" # 精确版本 +templates = "0.0.1" -# dotted selector: 先匹配 mcpplibs., 找不到再匹配同级 peer root。 -# 例如 imgui.core 会按顺序尝试 mcpplibs.imgui/core, imgui/core。 -[dependencies] -capi.lua = "0.0.3" +# dotted selector 是单一精确身份:最后一段是包名,之前所有段都是 namespace。 compat.gtest = "1.15.2" imgui.core = "0.0.1" imgui.backend.glfw_opengl3 = "0.0.1" +mcpplibs.capi.lua = "0.0.3" +``` +```toml # 命名空间子表写法 [dependencies.mcpplibs] cmdline = "0.0.2" @@ -359,17 +358,23 @@ tinyhttps = "0.2.2" llmapi = "0.2.5" [dependencies.compat] -glfw = "3.4" # 显式 namespace, 不走 mcpplibs 优先候选 +glfw = "3.4" # 显式 namespace,无回退搜索 +``` +```toml # 路径依赖(本地开发) [dependencies] mylib = { path = "../mylib" } +``` +```toml # Git 依赖 —— tag / branch / rev 三选一 [dependencies] mylib = { git = "https://github.com/user/mylib.git", tag = "v1.0.0" } applib = { git = "https://github.com/user/applib.git", branch = "develop" } +``` +```toml # 长式 dep spec:features 与 backend 旋钮 [dependencies] imgui = { version = "0.0.3", features = ["docking"] } # 请求该依赖的 feature @@ -408,22 +413,18 @@ qux = ">=1.0, <2.0" # 范围组合 #### 命名空间解析规则 -每个包的身份是**命名空间 + 名字**二元组。依赖 key 的写法决定 mcpp 到哪些命名空间里找。 +每个包的身份是**命名空间 + 名字**二元组。每个 selector 都只规范化成一个身份: -**裸名只在三个地方解析**,按序: - -| # | 命名空间 | 示例 | -|---|---|---| -| 1 | `mcpplibs` — 默认命名空间 | `cmdline = "0.0.2"` | -| 2 | `compat` — 第三方 C/C++ 库的包装命名空间 | `gtest = "1.15.2"` → `compat.gtest` | -| 3 | 完全没有声明命名空间的上游包 | `opencv = "4.10.0"` | +- `cmdline` → `(mcpplibs, cmdline)`;省略 namespace 只表示默认 `mcpplibs`。 +- `compat.gtest` → `(compat, gtest)`。 +- `mcpplibs.capi.lua` → `(mcpplibs.capi, lua)`。 -**其他命名空间一律必须写全。** 不存在按短名的全索引模糊搜索: +不存在有序回退或按短名的全索引模糊搜索: ```toml # ✅ 正确 —— 点式选择器 [dependencies] -"chriskohlhoff.asio" = "1.38.1" +chriskohlhoff.asio = "1.38.1" # ✅ 正确 —— 命名空间子表(同一组织有多个包时更推荐) [dependencies.chriskohlhoff] @@ -434,9 +435,33 @@ asio = "1.38.1" asio = "1.38.1" ``` -第三种写法会明确报错,并列出搜索过的命名空间;若该短名的包存在于别处,错误信息会直接给出应当改写成的那一行。 +第三种写法会明确报错,指出实际尝试的 `(mcpplibs, asio)`;若该短名存在于别处,错误信息会给出可直接复制的显式 selector。 + +##### 裸名过渡期(`2026.8.10.1` 起,`2026.9` 移除) + +索引里已发布的 `compat.*` 包与既有 manifest **全部**写成裸名(`gtest = "1.15.2"`)。 +升级后直接失败,等于让一次程序发布把**已经发布、且无法追溯修改**的数据作废, +所以有一个版本的过渡期:裸名在 `mcpplibs` 未命中时仍可到达 `compat.`, +不声明 namespace 的 descriptor 也仍可被裸名解析。 + +但它不再静默: + +``` +warning: dependency 'gtest' resolved to 'compat.gtest' through the deprecated +bare-name search; namespace omission means `mcpplibs` only. Write the exact +package: + [dependencies.compat] + gtest = "1.15.2" + (or run `mcpp add compat.gtest@1.15.2`). This fallback is removed in 2026.9. +``` + +写进 `mcpp.lock`、install 与 cache 的是**规范身份**,歧义拼写只存在于你的 +`mcpp.toml` 里,直到你改它;`mcpp add gtest@1.15.2` 会替你改。 -**为什么不让裸名跨所有命名空间去找?** 因为依赖解析必须可复现。全域短名搜索意味着:(a) 两个命名空间拥有同名包时,胜负由索引顺序决定;(b) **新增一个索引可能悄悄改变某个既有依赖解析到的包**。要求写出命名空间,才能让同一份 `mcpp.toml` 在每台机器上解析到相同的包。 +过渡期**不适用于**写明 namespace 的 selector(`mcpplibs.gtest` 未命中就是未命中), +裸名也**仍然**到不了第三方 namespace。 + +**为什么只允许一个身份?** 因为依赖解析必须可复现。候选搜索会让同短名包受索引状态影响,新增索引还可能悄悄重定向既有依赖。 **给 xpkg 作者:** 索引描述符里,身份是 `(package.namespace, package.name)` 二元组。命名空间是点分路径,**`name` 是单一原子段**: @@ -454,7 +479,13 @@ package = { 文件名只是提示 —— 描述符按声明的身份被发现,所以 `pkgs/c/chriskohlhoff.asio.lua` 与 `pkgs/z/anything.lua` 解析结果完全相同。推荐 `.lua` 或 `..lua`(命中 mcpp 的快路径),但不强制。 -旧的完全限定拼写(`name = "chriskohlhoff.asio"`)仍被接受,已发布的描述符无需改动。`mcpp xpkg parse` 会校验该规则,请在索引 CI 里跑它。需要 mcpp >= 0.0.106 与 xlings >= 0.4.69;规范全文见 `docs/spec/package-identity.md`。 +旧的完全限定拼写(`name = "chriskohlhoff.asio"`)仍被接受,已发布的描述符无需改动。`mcpp xpkg parse` 会校验该规则,请在索引 CI 里跑它。描述符身份需要 mcpp >= 0.0.106,精确 selector 需要 mcpp >= 2026.8.10.1,两者使用 xlings >= 0.4.69;规范全文见 `docs/spec/package-identity.md`。 + +`mcpp new --template` 刻意复用同一身份模型,而不是另造包文法: +`[ns.]name[@version][:tname]`。其中裸名同样只表示 `mcpplibs`,version 与模板名可分别 +省略。省略 `tname` 时选择唯一显式 default;若未写 `default = true` 且只有一个模板, +该单模板自动成为默认。多个未标默认的模板会报错,绝不按目录顺序选择。规范表见 +`docs/spec/package-identity.md` §4.4。 #### 表形式 —— 让 feature 贡献的不止是隐含 feature `[features]` 的条目除了写成数组,还可写成**表**,从而让该 feature 在隐含 feature @@ -670,27 +701,69 @@ mcpp cache clean [--deps|--std|--all|--legacy] `mcpp cache verify` 另外会报告任何逃出条目的记录地址, 使这条不变量可以离线审计。 -### 2.11 `[runtime]` — 主机运行时能力 +### 2.11 `[runtime]` — provider-neutral 运行时契约 ```toml [runtime] -library_dirs = ["vendor/lib"] # 烤进产物 RUNPATH 的目录(相对包根) -dlopen_libs = ["libGL.so.1"] # 运行期 dlopen 的 soname(doctor 校验) -capabilities = ["opengl.glx.driver"] # 需要的主机能力(开放命名空间) -provides = ["opengl.glx.driver"] # 显式声明本包兑现的能力(强 provider) - -# 显式 provider 覆盖(三档旋钮的"显式"档) -[runtime."opengl.glx.driver"] -provider = "compat.glx-runtime" +requirements = [ + { kind = "capability", value = "display.present", phase = "run", required = true }, + { kind = "soname", value = "libwidget.so.1", phase = "link", required = false }, +] +provides = ["display.present"] +artifacts = [ + { role = "library", path = "runtime/libwidget.so.1", provenance = "payload", abi = "elf-x86_64", digest = "sha256:...", host_fingerprint = "host-1" }, +] + +# 平台无关 LinkIntent;路径相对本包根目录。 +libraries = ["widget"] +link_library_dirs = ["lib"] +transitive_needed_dirs = ["runtime/closure"] +runtime_search_dirs = ["runtime"] +frameworks = ["WindowKit"] +deploy_files = ["bin/widget.dll"] + +# 多 provider 时使用精确 canonical identity。 +[runtime."display.present"] +provider = "acme.widget-runtime@2.0.0" ``` -- **provider 选择**:声明 `provides` 的包(强)优先于仅在 `capabilities` 列出 - 能力的包(弱,向后兼容);`[runtime.] provider=` 显式覆盖最优先, - 指向依赖图中不存在的 provider 时给出 warning。 -- 解析结果可经 `mcpp why runtime`、`mcpp self doctor` 与构建产物 - `target///resolution.json` 查看(默认不是魔法)。 -- 能力命名约定:分层小写 `domain.sub.role`(如 `opengl.glx.driver`、 - `x11.display`)与前缀类 `abi:`(如 `abi:glibc`,参与工具链 ABI 强制)。 +`requirements` 记录非空 `kind`/`value`、`link` 或 `run` 阶段,以及是否强制 +(`required` 默认 `true`)。`artifacts` 必须含 `role`、`path`、`provenance`; +可选 requirement 仍保留为 provenance,但不会进入硬 ABI/doctor 输入。 +`libraries` 中显式的相对文件路径按声明包根目录解析;裸逻辑名仍按目标平台拼成库名。 +`abi`、`digest`、`host_fingerprint` 是可选证据。requester/provider 身份不由描述符 +填写:resolver 会用含 namespace、version、source/index provenance 的精确 PackageId +给 requirement 和 artifact 盖章。因此描述符不能冒充别的包, +`alpha.backend` 也不会与 `beta.backend` 混同。 + +只有 `provides` 会创建描述符侧 provider fact;需要某能力绝不会让 requester 自动 +成为 provider。显式 `[runtime.] provider=` 接受 canonical +`namespace.name@version`(或唯一无歧义的兼容拼写);不存在或同短名歧义都会 hard error。 +xlings SubOS 已选择的 provider/artifact fact 排在描述符 fallback 前。图形栈、driver、 +ICD、WSL 与 host provenance 选择由 xlings/xim 负责;mcpp 只记录、消费通用结果, +不探测 GPU 硬件。 + +LinkIntent 把不同发现阶段分开: + +| 字段 | ELF | Mach-O | PE/Windows | +|---|---|---|---| +| `link_library_dirs` | `-L` | `-L` | `-L` 或 `/LIBPATH:` | +| `transitive_needed_dirs` | `-Wl,-rpath-link` | 无 flag | 无 flag | +| `runtime_search_dirs` | 只进 RUNPATH/rpath,绝不进 `-L` | 只进 rpath | 无 flag | +| `frameworks` | 无 flag | `-framework` | 无 flag | +| `deploy_files` | copy edge | copy edge | 复制到产物旁,绝不成为 linker flag | + +一个兼容发布周期内仍读取旧字段:`library_dirs` 只映射到运行期搜索; +`dlopen_libs` 映射为必需的 run-phase soname requirement;`capabilities` 映射为必需的 +run-phase capability requirement。这些旧字段都不会创建 provider。 + +`target///resolution.json` schema 2 持久化 RuntimeBinding、canonical +requirements/providers/artifacts、LinkIntent、平台搜索机制与链接后 verdict。 +`mcpp why runtime` 只是最新存储文件的纯解释器:不重新解析 manifest,也不启动图形/ +硬件 probe。需要重新诊断所选 host provider 时使用 `xlings doctor`。 + +能力名使用分层小写 `domain.sub.role`(如 `display.present`)和前缀类 +`abi:`(如 `abi:glibc`,参与工具链 ABI 强制)。 ### 2.12 `[package] platforms` — 平台声明 @@ -724,12 +797,15 @@ OPENBLAS_NUM_THREADS = "1" host 工具(`make`/`cmake`/`protoc`…)、按项目固定工具版本、或设构建期环境变量——无需手改 `.xlings.json`。`[toolchain]`(§2.7)仍是编译器的便捷简写;`[xlings.workspace]` 是其通用形式。 -在 Linux 上 `subos` 还有第二重含义:它决定**构建绑定到哪个 C 运行时**。subos 会自述其 -runtime(xlings 2026.8.5.1 起),mcpp 以此为权威,而不是去四处找一个 libc——所以同一台机器 -上 `subos = "el8"` 与 `subos = "trixie"` 两个项目各自产出面向自己 glibc 的产物,且编译期与 -运行期保证一致。该绑定计入工具链指纹,故切换它会重新构建,而不会复用另一个 subos 的目标文件。 -更旧的 subos(或没有 subos)则回落到工具链自身安装时所对应的运行时;`mcpp build -v` 会打印 -实际走了哪一条。参见 docs/08-toolchain-internals.md §2.1。 +`subos` 选择根项目用于 build/run 的**本地开发 OS 环境**。未声明该键时固定使用 mcpp 已初始化、 +经 release 验证的 `McppDefault`;`subos = "default"` 则仍是显式的 +`NamedSubos("default")`。没有 CLI/环境变量 override,也不会隐式跟随 xlings active/current。 + +在 Linux 上,所选环境同时固定 loader/libc contract,所以 `el8`、`trixie` 可在同一机器共存, +并进入不同构建指纹。workspace 整体构建时由 workspace root 覆盖 member 声明;member/依赖中的 +SubOS 不传递——库只有作为独立 root 开发时才使用自己的声明,作为别人的源码依赖时使用消费者 +root 的环境。指定的命名 SubOS 不存在、缺少或使用不兼容 runtime contract 都会直接报错,不会 +回退 default/active/编译器烙入状态。参见 docs/08-toolchain-internals.md §2.1。 ### 2.14 依赖产出的 host 工具(mcpp 2026.8.5.1+) @@ -1022,7 +1098,7 @@ version = "1.0.0" [targets.mymath] kind = "lib" -[dev-dependencies] +[dev-dependencies.compat] gtest = "1.15.2" ``` @@ -1130,7 +1206,7 @@ mcpp build --target x86_64-linux-musl | 静态 stdlib | `true` | 便携二进制 | | 头文件 | `include/`(如果存在) | 自动加到 `-I` | | 测试 | `tests/**/*.cpp` | `mcpp test` 自动发现 | -| 依赖命名空间 | `mcpp`(默认) | 平铺写法走默认 ns | +| 依赖命名空间 | `mcpplibs`(默认) | 裸 selector 只表示该精确 ns | ### 4.1 旧 `[language]` 兼容层 diff --git a/docs/zh/06-workspace.md b/docs/zh/06-workspace.md index fb05a4bf..c8ac3fca 100644 --- a/docs/zh/06-workspace.md +++ b/docs/zh/06-workspace.md @@ -100,7 +100,7 @@ mbedtls.workspace = true # 根 mcpp.toml [workspace.dependencies] cmdline = "0.0.2" -capi.lua = "0.0.3" # dotted selector: mcpplibs.capi/lua, then capi/lua +mcpplibs.capi.lua = "0.0.3" # 精确 selector:(mcpplibs.capi, lua) [workspace.dependencies.compat] mbedtls = "3.6.1" diff --git a/docs/zh/08-toolchain-internals.md b/docs/zh/08-toolchain-internals.md index b6b41b83..790776ec 100644 --- a/docs/zh/08-toolchain-internals.md +++ b/docs/zh/08-toolchain-internals.md @@ -7,14 +7,13 @@ ## 1. 一张图看全模型 ``` -mcpp.toml [toolchain] / 全局默认 / `mcpp toolchain install` - │ (三条入口路径 —— 共享同一条管线) +mcpp.toml [xlings].subos / mcpp 管理的默认运行时 ▼ -解析 payload(沙箱里的 xim:gcc / xim:llvm / xim:musl-gcc xpkg) +解析 runtime binding ← 产物将加载哪个 libc(§2.1)——是答案,不是搜索 ▼ -ensure_post_install_fixup() ← 幂等收敛(marker 闸门) +解析工具链 payload ← 项目/default/install 入口共享同一管线 ▼ -解析 runtime binding ← 产物将加载哪个 libc(§2.1)——是答案,不是搜索 +ensure_post_install_fixup() ← 精确 glibc@version、marker 闸门,不取 readdir 首项 ▼ detect / probe ← triple、sysroot、payload 路径(glibc、linux-headers) ▼ @@ -25,6 +24,8 @@ ToolchainLinkModel(C 库轴的唯一解析器) └──► cfg 再生 (供人类直接使用的 clang++.cfg) ▼ hermetic 链接校验(`-###` 干跑) ← 校验沙箱 CRT/loader 的解析结果 + ▼ +链接 → 内部 ELF 物理校验 ← 校验真实产物及解析闭包(§6.1) ``` 贯穿一切的两条原则: @@ -63,15 +64,27 @@ xlings 后端解析/自动安装到沙箱 ### 2.1 runtime binding:绑哪个 libc,只决定一次 -payload-first 的构建会链接到某个具体的 glibc,而**是哪一个**是关于环境的事实, -不该靠推断。mcpp 按顺序解析: +payload-first 的构建会链接到某个具体的 glibc,而**是哪一个**是根项目本地开发 OS 的事实, +不该从编译器路径或 shell 状态推断。mcpp 只有两种选择: -1. `[xlings] subos = ""` —— 该 subos(活动 subos 的兄弟)在自己 - `.xlings.json` 的 `subos_info` 块里自述 runtime(xlings 2026.8.5.1 起)。 -2. 活动 subos,同一个块。 -3. *兼容路径。* 在该块出现之前创建的 subos 无法作答,此时以工具链自身烙入的值 - 顶上——gcc 的 specs、clang 的 cfg。这正是产物**将会**加载的那个值,故编译期 - 与运行期仍然一致;一旦 subos 能作答,这条自动退场。 +1. 未声明 `[xlings].subos`:使用全局 mcpp 配置所选 xlings home 中已初始化的 + `McppDefault` (`subos/default`)。 +2. `[xlings] subos = ""`:使用 `NamedSubos(name)`;显式 `"default"` + 仍保留命名选择身份,其他名称解析到根项目的本地 xlings scope。 + +workspace 构建由 workspace root 选择;member 与 dependency 声明不合并、不传递。 +`XLINGS_ACTIVE_SUBOS`、current、编译器 owner home 以及 CLI/环境 override 都不是第三层。 +所选 SubOS 必须提供受支持的 `subos_info`:环境不存在或 contract 缺失/不兼容会 hard error, +绝不回退 default/active/编译器烙入状态。该 contract 只读取一次形成 `RuntimeBinding` snapshot, +由 configure/link/run/test 与 fast-path cache 共同使用。 +Linux snapshot 还记录所选 loader/libc 目录的规范路径及可选的创建宿主 glibc floor; +它们是链接后校验的证据,不是新的选择入口。 + +当解析后的 SubOS view 可规范化到一个受管 glibc payload 时,该真实 view 是权威事实。 +旧 xlings 状态可能在 view 已原子切到受管 2.44 后仍保留 `runtime = "glibc@2.39"`;此时 +mcpp 在同一 binding 中记录真实的 2.44 身份与路径。若旧 view 是断链,mcpp 只允许解析 +`runtime` 精确指名的 payload;不会枚举已安装版本,也不会挑“最近/最新”版本。两条路径 +都只消费 xlings 给出的事实,不引入 mcpp 自己的运行时选择策略。 没有 binding 是**拒绝**而不是取默认值:`CLibMode::PayloadFirst` 会被放弃,而不是 去挑一个 libc。 @@ -83,12 +96,52 @@ payload-first 的构建会链接到某个具体的 glibc,而**是哪一个**是 引用了 `GLIBC_2.42` 符号,却跑在没有这些符号的运行时上,且报错落在与"拉进第二个 glibc 的那条依赖"毫无关系的包上。目录顺序不是决策依据。 -因为 binding 决定产物加载什么,它计入工具链指纹(11 个字段,不是 10 个)——只在 -runtime 上不同的两次构建绝不能共用同一条缓存。 +因为 binding 决定产物加载什么,完整规范化 contract hash 是工具链指纹第 11 个字段—— +即使两个命名 SubOS 都使用同一 glibc,只要 provider 或环境声明不同也绝不能共用缓存。 注意:probe 已**不再**从 clang cfg 挖 `--sysroot`——cfg 是这套机制的输出, 不是输入(见 §5)。 +### 2.2 通用运行时 provider 与 artifact + +`RuntimeBinding` 还携带 xlings 为所选开发 OS 解析好的 provider-neutral fact。 +`subos_info.runtime_contract` 是 schema 1 上可选、可加的 block: + +```json +{ + "providers": [ + {"capability": "display.present", "provider": { + "namespace": "xim", "name": "display-runtime", "version": "1.0.0", + "source": "xim-pkgindex@"}} + ], + "artifacts": [ + {"role": "driver", "provider": { + "namespace": "xim", "name": "display-runtime", "version": "1.0.0", + "source": "xim-pkgindex@"}, + "path": "${subosdir}/lib/runtime/provider.so", + "provenance": "subos_view", "abi": "elf-x86_64", + "digest": "sha256:...", "host_fingerprint": "..."} + ] +} +``` + +binding parser 只解析一次 `${subosdir}` 和相对 artifact 路径,排序 fact,并把它们纳入 +contract hash 与 cache snapshot。进入 BuildPlan 后,所选 provider fact 排在描述符 +fallback 前。描述符 requirement/artifact 则由 resolver 分别用 requester/provider 的 +canonical PackageId 盖章,所以不同 namespace 下相同短名不会在任何环节碰撞。 + +所有权边界是刻意的:mcpp-index 表达通用 runtime requirement;xlings/xim 选择并诊断 +宿主图形/运行时栈;mcpp 只消费已选 provider/artifact fact 与通用 LinkIntent。 +mcpp 不含硬件、driver vendor、WSL 或 ICD 选择路径;源码 gate 会拒绝引入这类 +provider-specific 分支,也拒绝把相关词汇与外部 probe 启动耦合。 + +`LinkIntent` 分开 `linkLibraryDirs`、`transitiveNeededDirs` 和 +`runtimeSearchDirs`;最后一类绝不渲染为 `-L`。ELF 只为 runtime 目录发 rpath, +且仅为 transitive 类发 `-rpath-link`;Mach-O 发 rpath/framework;PE 发链接库路径并 +用显式 deploy-file copy edge。精确 RuntimeBinding、canonical identity、LinkIntent、 +搜索机制与链接后 verdict 写入 `resolution.json` schema 2。 +`mcpp why runtime` 只解释该存储文件;重新诊断由 `xlings doctor` 负责。 + ## 3. 链接模型(`src/toolchain/linkmodel.cppm`) `ToolchainLinkModel` 只回答一个问题——*如何对该工具链的 C 库编译与链接*—— @@ -225,6 +278,62 @@ CI 用一个**完全没有宿主工具链**的 job(`debian:stable-slim`,无 gcc `Scrt1.o`)守住这一切——那是唯一能忠实复现干净机器故障模式的环境类;另有 e2e `86_llvm_hermetic_link.sh` 在任何机器上复核 `-###` 的解析结果。 +### 6.1 链接后的 Linux 运行时物理校验(`elf_runtime.cppm`) + +hermetic 校验回答的是链接前问题:driver 看起来会解析到什么。运行时物理校验回答更强的 +链接后问题:新 ELF 真正记录了什么,其闭包实际会加载什么。 +`[build] allow_host_libs = true` 会有意放宽前者,但不会屏蔽后者已经可证明的物理矛盾。 + +对每个新链接的 Linux executable/shared object,mcpp 在进程内解析 ELF64 little-endian +program/dynamic/GNU version 表;构建路径不启动 `readelf`、`patchelf` 或 `ldd`。读取内容包括 +`PT_INTERP`、`DT_RPATH`/`DT_RUNPATH`、`DT_NEEDED` 以及所需/导出的 `GLIBC_*` 版本。 +随后按产物搜索路径、所选 runtime/toolchain 目录和已知宿主库目录解析闭包,并执行: + +- **规则 B(同源):** `PT_INTERP` 和闭包中每个 `libc.so.6` 必须都来自 + `RuntimeBinding` 选定的规范 payload。宿主 loader + 私有 libc,或两个私有 libc payload, + 都是可证明的 pre-main 故障。 +- **规则 A(版本 floor):** 闭包请求的每个 `GLIBC_x.y` 都不得高于所选 libc 的 GNU + version definitions。满足条件时链接宿主 DSO 完全允许;mcpp 校验的是物理事实,不是 + 强加“禁止宿主库”策略。 + +判定是类型化的:`Pass`、`ProvenMismatch`、`Inconclusive`。可证明的 A/B 冲突会 hard fail, +诊断给出规范化 requester/provider/artifact 路径与可复制的 SubOS 修复步骤。无法取得 +loader cache/硬件闭包时明确报告 inconclusive,绝不伪装成绿色。macOS/Windows 复用同一 +类型接口但为 no-op,不会套用 ELF/glibc 规则。 + +verdict 以 `.mcpp-runtime-verdicts.json` 存在 `build.ninja` 旁,键包含产物 stat 指纹与 +完整 runtime contract hash。热 no-op 必须已有当前 `pass` 记录,比较 Ninja 前后产物 stat, +并执行零次 ELF 解析;若 Ninja 意外重链则先退回完整路径重新校验,之后才允许成功/运行。 +`mcpp self doctor` 复用同一存档 verdict,不会拿已经变化的当前宿主重新猜一次。 + +装后 fixup 同样按精确身份收敛:`glibc@2.44` 只解析 +`/xim-x-glibc/2.44/{lib64,lib}`。精确 payload 缺失/陈旧就是错误,其他已安装版本 +永远不是回退项。 + +### 6.2 一条运行时搜索路径可以住在哪里(`runtime_env_contract.cppm`) + +告诉 loader「去哪找」有两条通道,差别不在便利性,而在**波及范围**: + +| | 波及到 | | +| --- | --- | --- | +| `DT_RUNPATH` | 携带它的那**一个**对象,及其 `dlopen()` | 逐二进制 | +| `LD_LIBRARY_PATH` | 本进程**以及它派生的每一个进程** | 继承,且一直传下去 | + +第二行就是私有 libc 目录必须是 **binary 作用域**的原因。glibc 的 `libc.so.6` 与它的 +`ld.so` 通过 `GLIBC_PRIVATE` 版本锁死:2.44 的 libc 里 `__pointer_chk_guard` 是未定义 +引用,只有 2.44 自己的 loader 导出它。mcpp 构建出来的程序没事——`PT_INTERP` 指向私有 +loader;`/bin/sh` 有事:它的 `PT_INTERP` 指向**宿主** loader,而且任何环境变量都改不了, +于是 `popen()`/`system()` 的子进程在重定位阶段就死掉,连 `main` 都进不去,也没有任何 +输出(#401;#291 是同一形状往内一跳,杀掉的是 mcpp 自己的嵌套宿主工具)。 + +所以 mcpp 不会把私有 libc 目录发布到环境里。也不需要:只要存在 payload,link model +就已经在 `--dynamic-linker` 旁发了 `-Wl,-rpath,`,恰好覆盖这个目录存在的唯一 +理由——某个 `dlopen()` 的 `DT_NEEDED` 闭包看不到可执行文件的 RUNPATH。 + +这是**作用域**,不是条件判断。「只在依赖可能 dlopen 时才导出」仍然是导出,而死掉的 +子进程不关心原因。普通依赖运行时目录保留环境作用域:它们没有 loader 耦合,宿主二进制 +撞上去最多是困惑,不会死。 + ## 7. 扩充指南 ### 7.1 新增一个工具链(新编译器家族或发行版) @@ -291,6 +400,9 @@ mcpp 把运行时 DLL 部署到产物 exe 旁,这正是该平台对 §3–§4 | 链接模型 + loader 解析 | `src/toolchain/linkmodel.cppm` | | 统一 fixup 管线(patchelf/specs/cfg、marker)| `src/toolchain/post_install.cppm` | | install/lifecycle 入口 | `src/toolchain/lifecycle.cppm`;auto-install 入口在 `src/build/prepare.cppm` | +| root runtime 选择/binding | `src/xlings/runtime_selection.cppm`、`src/platform/runtime_binding.cppm`、`src/xlings/subos_info.cppm` | +| 通用 runtime contract + LinkIntent | `src/manifest/types.cppm`、`src/build/plan.cppm`、`src/build/flags.cppm` | +| 存储 resolution 解释 | `src/build/prepare.cppm`、`src/build/runtime_validation.cppm`、`src/doctor.cppm` | | flag 组装(主构建)| `src/build/flags.cppm` | | `import std;` 预编译 | `src/toolchain/stdmod.cppm` | | build.mcpp 宿主 flags | `src/build/build_program.cppm` | diff --git a/docs/zh/09-release.md b/docs/zh/09-release.md index 0c1577ec..86af4477 100644 --- a/docs/zh/09-release.md +++ b/docs/zh/09-release.md @@ -37,11 +37,44 @@ POSIX `sh`/dash 解析不了 —— `sh check_version_pins.sh` 会在第 95 行 ``` 四平台构建(linux x86_64 / linux aarch64 / macOS ARM64 / Windows x64) → GitHub Release v,含 tarball 与 .sha256 边车文件 + → 重新计算每个载荷的哈希并发布不可变 mcpp-release.json → 镜像到 xlings-res/mcpp 的 GitHub 与 GitCode 双端 → 向 openxlings/xim-pkgindex 开版本 bump PR → workflow_run 钩子触发 ci-fresh-install ``` +### 2.1 不可变 release manifest + +`release-manifest` 会等待四个平台上传 job 全部结束,然后下载最终的非 draft、 +非 prerelease GitHub Release,重新计算每个带版本平台载荷的 SHA256,校验对应的 +`.sha256` 边车文件,并发布 schema 1 的 `mcpp-release.json`: + +```json +{ + "schema": 1, + "version": "", + "tag": "v", + "commit": "", + "assets": [ + { + "platform": "linux", + "arch": "x86_64", + "name": "mcpp--linux-x86_64.tar.gz", + "sha256": "<重新计算的-sha256>" + } + ] +} +``` + +数组按平台、架构、名称排序,包含所有带版本的平台载荷;其中 Linux x86_64、 +Linux aarch64、macOS ARM64、Windows x86_64 四项是硬性要求。无版本别名和源码包 +刻意不进入 desired-state 行。 + +workflow 随后会再次下载公开 release、重新生成 manifest,并要求逐字节一致。 +重跑 workflow 时,已有 manifest 只有在字节完全相同时才会被接受;同一 tag 下 +绝不以不同内容覆盖。下游发布消费者(尤其 `mcpp-bin` AUR reconciler)必须消费 +该 manifest,不能从会变化的工作区或部分 release 资产猜测发布是否完整。 + 两步**没有**自动化: - **合并 xim-pkgindex 的 bump PR** —— 由维护者完成。在它落地之前,发布出来的版本 @@ -66,6 +99,27 @@ curl -fsSL -o /tmp/p.tgz "https://github.com/xlings-res/mcpp/releases/download/$ sha256sum /tmp/p.tgz # 与 xim-pkgindex 的 pkgs/m/mcpp.lua 对照 ``` +要在本地针对 GitHub 公开资产重放 release gate: + +```bash +V= +TAG="v$V" +AUDIT=$(mktemp -d) +mkdir -p "$AUDIT/assets" +gh api "repos/mcpp-community/mcpp/releases/tags/$TAG" > "$AUDIT/release.json" +gh release download "$TAG" -R mcpp-community/mcpp --dir "$AUDIT/assets" +python3 scripts/release/generate_manifest.py \ + --release-json "$AUDIT/release.json" \ + --assets-dir "$AUDIT/assets" \ + --version "$V" \ + --tag "$TAG" \ + --commit "$(git rev-list -n 1 "$TAG")" \ + --output "$AUDIT/expected.json" +cmp "$AUDIT/assets/mcpp-release.json" "$AUDIT/expected.json" +``` + +这个命令会重新计算载荷哈希,不会从已发布 manifest 抄哈希,也不会盲信边车文件。 + 然后在 **clean-room `XLINGS_HOME`** 里真装一次 —— 绝不要用本机的 `~/.xlings`, 它的缓存状态会把一个坏掉的索引掩盖过去: @@ -133,7 +187,8 @@ $(find "$XLINGS_HOME" -name mcpp -type f -path '*/bin/*' | head -1) --version [ ] `bash .github/tools/check_version_pins.sh` 通过(校验 `mcpp.toml` = `MCPP_VERSION`,且 `.xlings.json` 未领先) [ ] 合入 main,CI 全绿 [ ] gh workflow run release.yml --ref main -[ ] release.yml 全绿(4 个构建 + publish-ecosystem) +[ ] release.yml 全绿(4 个构建 + 不可变 manifest + publish-ecosystem) +[ ] 下载的 mcpp-release.json 能从公开资产逐字节重新生成 [ ] 双端都服务四个平台,sha256 重新算过 [ ] 合并 xim-pkgindex 的 bump PR [ ] clean-room XLINGS_HOME:xlings install mcpp@ 成功 diff --git a/mcpp.toml b/mcpp.toml index 7555910c..d4685526 100644 --- a/mcpp.toml +++ b/mcpp.toml @@ -1,6 +1,6 @@ [package] name = "mcpp" -version = "2026.8.8.4" +version = "2026.8.10.1" description = "Modern C++ build & package management tool" license = "Apache-2.0" authors = ["mcpp-community"] @@ -47,7 +47,8 @@ linkage = "static" mcpplibs.cmdline = "0.0.1" # `mcpp build` ignores [dev-dependencies]; only `mcpp test` resolves them. -[dev-dependencies] +# Namespace omission means exactly mcpplibs, so the compat wrapper is explicit. +[dev-dependencies.compat] gtest = "1.15.2" # Everything else uses M5.0 conventions: diff --git a/scripts/aur/README.md b/scripts/aur/README.md index 83b49c4a..557ef750 100644 --- a/scripts/aur/README.md +++ b/scripts/aur/README.md @@ -37,7 +37,7 @@ under a root-owned system prefix. mcpp resolves `MCPP_HOME` from the running binary's *real* path (`/proc/self/exe`, which resolves symlinks). A plain `/usr/bin/mcpp` symlink would therefore make `MCPP_HOME` resolve into the read-only install dir and -every command would fail to write. So both packages split the tree: +every command would fail to write. So all three packages split the tree: | Path | Contents | Mode | | --- | --- | --- | @@ -74,29 +74,47 @@ between releases — at the cost of tracking bleeding-edge master. ``` scripts/aur/ README.md this file - update.sh bump the release-pinned packages (mcpp-bin, mcpp-m) + update.sh render + verify mcpp-bin from mcpp-release.json + render_mcpp_bin.py pure manifest-to-PKGBUILD renderer + reconcile_mcpp_bin.py desired/observed-state reconciler + aur.archlinux.org.known_hosts pinned production host key mcpp-bin/{PKGBUILD, .SRCINFO, mcpp.sh} mcpp-m/{PKGBUILD, .SRCINFO, mcpp.sh} mcpp-git/{PKGBUILD, .SRCINFO, mcpp.sh} ``` -`mcpp.sh` is identical in all three dirs (each AUR repo must be self-contained); -`update.sh` keeps `mcpp-bin`/`mcpp-m` in sync. `mcpp-git` has no checksums to -bump (VCS source) and is not touched by `update.sh`. +`mcpp.sh` is currently identical in all three dirs because each AUR repo must +be self-contained. Automation in this phase owns **only `mcpp-bin`**. +`mcpp-m` is frozen and is neither read, rendered, staged, nor pushed by the +reconciler; `mcpp-git` is also outside this release workflow. ## Releasing a new version -After a GitHub release is published (and mirrored), bump the release-pinned -packages (`mcpp-bin`, `mcpp-m`; `mcpp-git` tracks master and needs no bump): +Every complete stable GitHub release publishes immutable `mcpp-release.json`. +Render the checked-in `mcpp-bin` package from that desired state (Docker is +required so `.SRCINFO` comes from a non-root Arch `makepkg`, not a handwritten +fallback): ```sh -scripts/aur/update.sh # uses [package].version from mcpp.toml -# or pin: scripts/aur/update.sh 0.0.66 +scripts/aur/update.sh # latest complete stable release +scripts/aur/update.sh 2026.8.10.1 # accepted only if it is that exact latest tag ``` -`update.sh` pulls the per-arch `.sha256` sidecars (for `mcpp-bin`) and hashes -the source archive (for `mcpp-m`), rewrites `pkgver` + checksums, resets -`pkgrel=1`, and regenerates both `.SRCINFO` files. +The renderer downloads both Linux payloads and sidecars, recomputes their +SHA256 values against the manifest, rewrites only `mcpp-bin`, runs +`makepkg --printsrcinfo`, and then runs `makepkg --verifysource`. There is no +version-from-worktree fallback and no downgrade override. + +For a read-only live reconciliation (including AUR RPC and HTTPS git state): + +```sh +python3 scripts/aur/reconcile_mcpp_bin.py \ + --trigger local \ + --report-json /tmp/mcpp-aur-report.json +``` + +Without `--publish`, an upgrade is rendered, source-verified, and shown as an +exact dry-run diff but never pushed. ### Test locally (on Arch) @@ -109,10 +127,31 @@ mcpp --version ## Automated publishing (CI) [`.github/workflows/aur-publish.yml`](../../.github/workflows/aur-publish.yml) -publishes both packages automatically. It runs when the `release` workflow -**completes successfully** (not on `release: published` — the aarch64 asset the -`mcpp-bin` checksum needs is uploaded by a later release job), refreshes the -PKGBUILDs via `update.sh`, and pushes each package to its AUR git repo over SSH. +reconciles `mcpp-bin` only. It runs after a successful `release` workflow, every +six hours to recover from transient AUR outages, and on manual dispatch. Manual +dispatch defaults to dry-run and requires `publish=true` to push; the two +automatic triggers plan and report but do **not** push until the repository +variable `AUR_AUTOPUBLISH` is set — see +[Arming the automatic triggers](#arming-the-automatic-triggers). + +Every trigger follows the same state machine: + +1. Select the latest complete, non-draft, non-prerelease manifest. +2. Validate both Linux payloads and sidecars and render in a clean directory. +3. Generate `.SRCINFO` and verify sources as a non-root user in Arch. +4. Query AUR RPC and clone the known `mcpp-bin` repo over HTTPS. +5. Compare versions with Arch `vercmp` and print the exact diff before the SSH + secret is loaded. +6. Refuse downgrades; otherwise use a normal fast-forward SSH push with bounded + maintenance retry and a pinned AUR host key. +7. Verify public git HEAD, bounded-poll RPC, then install and run + `mcpp --version` in a clean Arch container. + +The result is classified as `noop`, `updated`, `transient`, `permanent`, or +`refused-downgrade`, and the Actions summary records versions, hashes, commit, +retry count, and drift age. AUR runs are downstream workflows: a failure is +visible and retried by schedule but cannot rewrite the completed GitHub release +conclusion. > The AUR has no "watch upstream" feature — packages only update when their git > repo is pushed. This workflow is that push. @@ -120,7 +159,7 @@ PKGBUILDs via `update.sh`, and pushes each package to its AUR git repo over SSH. ### One-time setup you need to do 1. **AUR account** — sign in at with the account - that will own `mcpp-bin` / `mcpp-m`. + that owns `mcpp-bin`. 2. **Generate a dedicated SSH key** (no passphrase, it's for CI): @@ -139,22 +178,54 @@ PKGBUILDs via `update.sh`, and pushes each package to its AUR git repo over SSH. Then delete the local `aur_ci` / `aur_ci.pub` files. -That's the only secret required — AUR auth is SSH-key based, there is **no API -token**. The default `GITHUB_TOKEN` is *not* used (we push to the AUR, not to -GitHub). +That's the only publishing secret required — AUR auth is SSH-key based, there +is **no API token**. GitHub's read-only token is used only to fetch the release +manifest/assets. The private key is not loaded during the inspect/dry-run step. +The server host key is checked against the vendored ED25519 key sourced from +Arch Linux's infrastructure repository; the workflow never uses +`ssh-keyscan` as a trust decision. + +### Arming the automatic triggers + +`schedule` fires every six hours off the default branch. That means merging +this workflow is, by itself, enough to start writing to a third-party service +unattended — potentially before anyone has watched the reconciler complete a +real push even once. Merging is a decision about code; publishing to the AUR is +a decision about the outside world, and the two should not be the same act. + +So both automatic triggers (`workflow_run` after a release, and `schedule`) +stop after the plan-and-report step unless the repository variable +`AUR_AUTOPUBLISH` is set to `true`. Dry runs still validate payloads, render +`.SRCINFO`, query the AUR and print the exact diff, so the reporting value is +unchanged — only the push is withheld. + +To arm it, once: + +1. Run the workflow manually with `publish=false` and read the summary: it must + show the intended version and a clean diff. +2. Run it manually with `publish=true` and confirm the push, the AUR RPC row, + and a clean install in an Arch container. +3. Only then set *Settings → Secrets and variables → Actions → Variables →* + `AUR_AUTOPUBLISH = true`. + +Unsetting the variable is the kill switch: automatic runs immediately fall back +to reporting without publishing, with no code change and no revert. ### First publish -The first time, the package names must be free. The workflow auto-creates each -AUR repo on first push (AUR does this for a valid, available name). If you -prefer to claim them by hand first, push an initial commit manually: +The reconciler treats `mcpp-bin` as a known existing package. A failed clone or +missing RPC row is an error and **never** becomes an empty-repository first +publish. If a new package ever needs to be claimed, do that explicitly and +review the initial history by hand: ```sh git clone ssh://aur@aur.archlinux.org/mcpp-bin.git cp scripts/aur/mcpp-bin/{PKGBUILD,.SRCINFO,mcpp.sh} mcpp-bin/ && cd mcpp-bin -git add -A && git commit -m "initial mcpp-bin" && git push # repeat for mcpp-m +git add PKGBUILD .SRCINFO mcpp.sh +git commit -m "initial mcpp-bin" && git push ``` -After that, every release publishes both packages with no manual step. You can -also run it on demand from the Actions tab (*aur-publish → Run workflow*, -optional version input). +After the known repository exists, the reconciler never force-pushes or +reinitializes it. Manual recovery is available from *aur-publish → Run +workflow*: first leave `publish=false` to inspect, then set `publish=true` only +for the exact latest complete stable tag. diff --git a/scripts/aur/aur.archlinux.org.known_hosts b/scripts/aur/aur.archlinux.org.known_hosts new file mode 100644 index 00000000..64b5be4d --- /dev/null +++ b/scripts/aur/aur.archlinux.org.known_hosts @@ -0,0 +1,4 @@ +# Pinned AUR production ED25519 host key. +# Source: https://github.com/archlinux/infrastructure/blob/master/docs/ssh-known_hosts.txt +# Fingerprint: SHA256:RFzBCUItH9LZS0cKB5UE6ceAYhBD5C8GeOBip8Z11+4 +aur.archlinux.org ssh-ed25519 AAAAC3NzaC1lZDI1NTE5AAAAIEuBKrPzbawxA/k2g6NcyV5jmqwJ2s+zpgZGZ7tpLIcN diff --git a/scripts/aur/mcpp-bin/PKGBUILD b/scripts/aur/mcpp-bin/PKGBUILD index 1d0ba43f..dcc10e51 100644 --- a/scripts/aur/mcpp-bin/PKGBUILD +++ b/scripts/aur/mcpp-bin/PKGBUILD @@ -1,4 +1,4 @@ -# Maintainer: mcpp-community +# Maintainer: mcpp-community # # mcpp-bin — prebuilt release binaries from GitHub, the same artifacts the # upstream `install.sh` one-liner downloads. See scripts/aur/README.md for diff --git a/scripts/aur/reconcile_mcpp_bin.py b/scripts/aur/reconcile_mcpp_bin.py new file mode 100755 index 00000000..88748a49 --- /dev/null +++ b/scripts/aur/reconcile_mcpp_bin.py @@ -0,0 +1,1132 @@ +#!/usr/bin/env python3 +"""Reconcile AUR mcpp-bin to the latest complete stable mcpp release.""" + +from __future__ import annotations + +import argparse +import hashlib +import json +import os +import re +import shutil +import subprocess +import sys +import tempfile +import time +from dataclasses import dataclass +from datetime import datetime, timezone +from enum import Enum +from pathlib import Path +from typing import Any, Callable, NoReturn, TypeVar + +from render_mcpp_bin import ( + DesiredState, + RenderError, + load_desired_state, + materialize_package, +) + + +GITHUB_REPOSITORY = "mcpp-community/mcpp" +AUR_PACKAGE = "mcpp-bin" +AUR_HTTPS_URL = "https://aur.archlinux.org/mcpp-bin.git" +AUR_SSH_URL = "ssh://aur@aur.archlinux.org/mcpp-bin.git" +AUR_RPC_URL = "https://aur.archlinux.org/rpc/v5/info?arg[]=mcpp-bin" +ARCH_IMAGE = "archlinux:base-devel" +MANAGED_FILES = ("PKGBUILD", ".SRCINFO", "mcpp.sh") +AUR_COMMIT_NAME = "speak-agent" +AUR_COMMIT_EMAIL = "speak-agent@users.noreply.github.com" +SIDECAR_RE = re.compile(r"^([0-9a-fA-F]{64})[ \t]+\*?(\S+)$") +T = TypeVar("T") + + +class Classification(str, Enum): + NOOP = "noop" + UPDATED = "updated" + TRANSIENT = "transient" + PERMANENT = "permanent" + REFUSED_DOWNGRADE = "refused-downgrade" + + +class PlanAction(str, Enum): + NOOP = "noop" + UPGRADE = "upgrade" + REPAIR = "repair" + WAIT_RPC = "wait-rpc" + REFUSE_DOWNGRADE = "refuse-downgrade" + + +class ReconcileFailure(RuntimeError): + def __init__( + self, + message: str, + classification: Classification, + *, + retryable: bool = False, + ) -> None: + super().__init__(message) + self.classification = classification + self.retryable = retryable + + +class CommandFailure(RuntimeError): + def __init__( + self, + command: tuple[str, ...] | list[str], + returncode: int, + stdout: str, + stderr: str, + ) -> None: + self.command = tuple(command) + self.returncode = returncode + self.stdout = stdout + self.stderr = stderr + super().__init__( + f"command failed ({returncode}): {' '.join(self.command)}\n{stderr.strip()}" + ) + + +@dataclass(frozen=True) +class CommandResult: + stdout: str + stderr: str + returncode: int + + +class CommandRunner: + def run( + self, + args: list[str] | tuple[str, ...], + *, + cwd: Path | None = None, + env: dict[str, str] | None = None, + timeout: float | None = None, + ) -> CommandResult: + command = tuple(str(arg) for arg in args) + try: + completed = subprocess.run( + command, + cwd=cwd, + env=env, + timeout=timeout, + check=False, + capture_output=True, + text=True, + ) + except subprocess.TimeoutExpired as exc: + raise CommandFailure( + command, + 124, + exc.stdout or "", + f"command timed out after {timeout}s: {exc.stderr or ''}", + ) from exc + except OSError as exc: + raise CommandFailure(command, 127, "", str(exc)) from exc + result = CommandResult(completed.stdout, completed.stderr, completed.returncode) + if completed.returncode != 0: + raise CommandFailure( + command, + result.returncode, + result.stdout, + result.stderr, + ) + return result + + +@dataclass(frozen=True) +class ObservedState: + rpc_version: str | None + git_version: str + git_head: str + content_matches: bool + + +@dataclass(frozen=True) +class ReconcilePlan: + action: PlanAction + reason: str + + +def permanent(message: str) -> NoReturn: + raise ReconcileFailure(message, Classification.PERMANENT) + + +def classify_command_failure(error: CommandFailure) -> ReconcileFailure: + detail = f"{error.stdout}\n{error.stderr}".lower() + auth_markers = ( + "permission denied", + "publickey", + "authentication failed", + "host key verification failed", + "invalid format", + ) + transient_markers = ( + "maintenance", + "temporarily unavailable", + "service unavailable", + "timed out", + "timeout", + "connection reset", + "connection closed", + "remote end hung up", + "ssh_exchange_identification", + "http 502", + "http 503", + "http 504", + "502 bad gateway", + "503 service unavailable", + "504 gateway timeout", + ) + if any(marker in detail for marker in auth_markers): + return ReconcileFailure(str(error), Classification.PERMANENT) + if "non-fast-forward" in detail or "fetch first" in detail: + return ReconcileFailure( + str(error), Classification.TRANSIENT, retryable=False + ) + if any(marker in detail for marker in transient_markers): + return ReconcileFailure(str(error), Classification.TRANSIENT, retryable=True) + return ReconcileFailure(str(error), Classification.PERMANENT) + + +def retry_transient( + operation: Callable[[], T], + *, + max_attempts: int, + base_delay: float, + sleep: Callable[[float], None] = time.sleep, +) -> tuple[T, int]: + if max_attempts < 1: + permanent("retry max_attempts must be at least 1") + retries = 0 + for attempt in range(1, max_attempts + 1): + try: + return operation(), retries + except CommandFailure as error: + failure = classify_command_failure(error) + except ReconcileFailure as error: + failure = error + if not failure.retryable or attempt == max_attempts: + raise failure + delay = base_delay * (2 ** (attempt - 1)) + sleep(delay) + retries += 1 + raise AssertionError("retry loop must return or raise") + + +def _sha256_file(path: Path) -> str: + digest = hashlib.sha256() + try: + with path.open("rb") as stream: + for chunk in iter(lambda: stream.read(1024 * 1024), b""): + digest.update(chunk) + except OSError as exc: + permanent(f"cannot read release asset {path}: {exc}") + return digest.hexdigest() + + +def _sidecar_digest(path: Path, primary_name: str) -> str: + try: + lines = [ + line.strip() + for line in path.read_text(encoding="utf-8").splitlines() + if line.strip() + ] + except (OSError, UnicodeError) as exc: + permanent(f"missing or unreadable release sidecar {path}: {exc}") + if len(lines) != 1: + permanent(f"release sidecar must have one checksum record: {path.name}") + match = SIDECAR_RE.fullmatch(lines[0]) + if match is None or match.group(2) != primary_name: + permanent(f"invalid release sidecar record: {path.name}") + return match.group(1).lower() + + +def validate_release_assets( + desired: DesiredState, assets_dir: Path +) -> dict[str, str]: + validated: dict[str, str] = {} + for arch in ("x86_64", "aarch64"): + asset = desired.asset("linux", arch) + payload = assets_dir / asset.name + sidecar = assets_dir / f"{asset.name}.sha256" + if not payload.is_file(): + permanent(f"missing release payload: {asset.name}") + if not sidecar.is_file(): + permanent(f"missing release sidecar: {sidecar.name}") + sidecar_digest = _sidecar_digest(sidecar, asset.name) + actual_digest = _sha256_file(payload) + if sidecar_digest != asset.sha256: + permanent( + f"sidecar/manifest SHA256 mismatch for {asset.name}: " + f"{sidecar_digest} != {asset.sha256}" + ) + if actual_digest != asset.sha256: + permanent( + f"payload/manifest SHA256 mismatch for {asset.name}: " + f"{actual_digest} != {asset.sha256}" + ) + validated[arch] = actual_digest + return validated + + +def select_complete_release( + releases: Any, requested_tag: str | None = None +) -> dict[str, Any]: + if not isinstance(releases, list): + permanent("GitHub releases response must be an array") + selected: dict[str, Any] | None = None + for release in releases: + if not isinstance(release, dict): + permanent("GitHub release row must be an object") + if release.get("draft") is not False or release.get("prerelease") is not False: + continue + assets = release.get("assets") + if not isinstance(assets, list): + continue + names = { + item.get("name") + for item in assets + if isinstance(item, dict) and isinstance(item.get("name"), str) + } + if "mcpp-release.json" in names: + selected = release + break + if selected is None: + raise ReconcileFailure( + "no complete stable release with mcpp-release.json is available", + Classification.TRANSIENT, + retryable=True, + ) + selected_tag = selected.get("tag_name") + if not isinstance(selected_tag, str) or not selected_tag: + permanent("selected GitHub release has no valid tag_name") + if requested_tag and requested_tag != selected_tag: + raise ReconcileFailure( + f"requested tag {requested_tag!r} is not latest complete stable " + f"tag {selected_tag!r}", + Classification.REFUSED_DOWNGRADE, + ) + return selected + + +def plan_reconciliation( + desired_version: str, + observed: ObservedState, + compare: Callable[[str, str], int], +) -> ReconcilePlan: + git_comparison = compare(desired_version, observed.git_version) + rpc_comparison = ( + compare(desired_version, observed.rpc_version) + if observed.rpc_version is not None + else 1 + ) + if git_comparison < 0 or rpc_comparison < 0: + return ReconcilePlan( + PlanAction.REFUSE_DOWNGRADE, + "AUR already contains a version newer than desired state", + ) + if git_comparison == 0 and observed.content_matches: + if rpc_comparison == 0: + return ReconcilePlan(PlanAction.NOOP, "git, RPC, and desired content agree") + return ReconcilePlan( + PlanAction.WAIT_RPC, + "AUR git is desired state while RPC metadata is still behind", + ) + if git_comparison > 0: + return ReconcilePlan(PlanAction.UPGRADE, "desired version is newer than AUR git") + return ReconcilePlan( + PlanAction.REPAIR, + "AUR version matches desired state but tracked package content differs", + ) + + +def poll_rpc_version( + fetch_version: Callable[[], str | None], + *, + desired_version: str, + compare: Callable[[str, str], int], + max_attempts: int, + delay: float, + sleep: Callable[[float], None] = time.sleep, +) -> tuple[str, int]: + if max_attempts < 1: + permanent("RPC max_attempts must be at least 1") + last: str | None = None + for attempt in range(1, max_attempts + 1): + last = fetch_version() + if last is not None: + comparison = compare(desired_version, last) + if comparison == 0: + return last, attempt + if comparison < 0: + raise ReconcileFailure( + f"AUR RPC advanced to {last}, newer than desired {desired_version}", + Classification.REFUSED_DOWNGRADE, + ) + if attempt < max_attempts: + sleep(delay) + raise ReconcileFailure( + f"AUR RPC did not converge to {desired_version}; last={last!r}", + Classification.TRANSIENT, + retryable=True, + ) + + +def clone_observed_repo( + runner: Any, + destination: Path, + url: str = AUR_HTTPS_URL, +) -> None: + try: + runner.run(["git", "clone", "--", url, str(destination)]) + except CommandFailure as error: + raise classify_command_failure(error) from error + if not (destination / ".git").is_dir(): + permanent(f"known AUR package clone produced no git repository: {destination}") + + +def _run_or_classify( + runner: CommandRunner, + args: list[str], + *, + cwd: Path | None = None, + timeout: float | None = None, +) -> CommandResult: + try: + return runner.run(args, cwd=cwd, timeout=timeout) + except CommandFailure as error: + raise classify_command_failure(error) from error + + +def _parse_json(text: str, label: str) -> Any: + try: + return json.loads(text) + except json.JSONDecodeError as exc: + lower = text.lower() + if "maintenance" in lower or "service unavailable" in lower: + raise ReconcileFailure( + f"{label} returned transient non-JSON response", + Classification.TRANSIENT, + retryable=True, + ) from exc + permanent(f"{label} returned invalid JSON: {exc}") + + +def _release_asset_names(release: dict[str, Any]) -> set[str]: + raw_assets = release.get("assets") + if not isinstance(raw_assets, list): + permanent("selected GitHub release assets must be an array") + names: set[str] = set() + for item in raw_assets: + if not isinstance(item, dict) or not isinstance(item.get("name"), str): + permanent("selected GitHub release has an invalid asset row") + name = item["name"] + if name in names: + permanent(f"selected GitHub release has duplicate asset name: {name}") + names.add(name) + return names + + +def _download_release_asset( + runner: CommandRunner, + *, + repository: str, + tag: str, + name: str, + destination: Path, +) -> Path: + destination.mkdir(parents=True, exist_ok=True) + try: + runner.run( + [ + "gh", + "release", + "download", + tag, + "--repo", + repository, + "--dir", + str(destination), + "--pattern", + name, + ], + timeout=300, + ) + except CommandFailure as error: + failure = classify_command_failure(error) + if failure.classification is Classification.PERMANENT: + permanent(f"cannot download required release asset {name}: {error}") + raise failure from error + path = destination / name + if not path.is_file(): + permanent(f"GitHub download omitted required release asset: {name}") + return path + + +@dataclass(frozen=True) +class DesiredSnapshot: + desired: DesiredState + manifest_path: Path + assets_dir: Path + release: dict[str, Any] | None + + +def acquire_desired_snapshot( + runner: CommandRunner, + *, + work_dir: Path, + repository: str, + requested_tag: str | None, + local_manifest: Path | None, + local_assets_dir: Path | None, +) -> DesiredSnapshot: + release: dict[str, Any] | None = None + if local_manifest is not None: + desired = load_desired_state(local_manifest) + if requested_tag and requested_tag != desired.tag: + raise ReconcileFailure( + f"requested tag {requested_tag!r} does not match local manifest " + f"tag {desired.tag!r}", + Classification.REFUSED_DOWNGRADE, + ) + manifest_path = local_manifest + else: + response = _run_or_classify( + runner, + ["gh", "api", f"repos/{repository}/releases?per_page=20"], + timeout=60, + ) + release = select_complete_release( + _parse_json(response.stdout, "GitHub releases API"), requested_tag + ) + tag = release["tag_name"] + manifest_dir = work_dir / "manifest" + manifest_path = _download_release_asset( + runner, + repository=repository, + tag=tag, + name="mcpp-release.json", + destination=manifest_dir, + ) + desired = load_desired_state(manifest_path) + if desired.tag != tag: + permanent( + f"release/manifest tag mismatch: release={tag!r}, " + f"manifest={desired.tag!r}" + ) + + if local_assets_dir is not None: + assets_dir = local_assets_dir + else: + assets_dir = work_dir / "assets" + release_names = _release_asset_names(release) if release is not None else None + for arch in ("x86_64", "aarch64"): + asset = desired.asset("linux", arch) + for name in (asset.name, f"{asset.name}.sha256"): + if release_names is not None and name not in release_names: + permanent(f"selected GitHub release inventory is missing {name}") + _download_release_asset( + runner, + repository=repository, + tag=desired.tag, + name=name, + destination=assets_dir, + ) + validate_release_assets(desired, assets_dir) + return DesiredSnapshot(desired, manifest_path, assets_dir, release) + + +def generate_srcinfo_and_verify( + runner: CommandRunner, + *, + package_dir: Path, + image: str, +) -> None: + script = """ +set -euo pipefail +useradd --create-home builder +cp -a /input /tmp/package +chown -R builder:builder /tmp/package +runuser -u builder -- env HOME=/home/builder bash -c 'cd /tmp/package && makepkg --printsrcinfo' > /output/SRCINFO +runuser -u builder -- env HOME=/home/builder bash -c 'cd /tmp/package && makepkg --verifysource --noconfirm' +""".strip() + with tempfile.TemporaryDirectory(prefix="mcpp-aur-arch-output-") as raw_output: + generated_dir = Path(raw_output) + _run_or_classify( + runner, + [ + "docker", + "run", + "--rm", + "--volume", + f"{package_dir.resolve()}:/input:ro", + "--volume", + f"{generated_dir.resolve()}:/output", + image, + "bash", + "-euc", + script, + ], + timeout=900, + ) + generated = generated_dir / "SRCINFO" + if not generated.is_file(): + permanent("Arch makepkg did not generate .SRCINFO") + try: + content = generated.read_bytes() + (package_dir / ".SRCINFO").write_bytes(content) + except OSError as exc: + permanent(f"cannot install generated .SRCINFO: {exc}") + + +def _srcinfo_package_version(path: Path) -> str: + try: + text = path.read_text(encoding="utf-8") + except (OSError, UnicodeError) as exc: + permanent(f"cannot read AUR .SRCINFO {path}: {exc}") + version_match = re.search(r"^\s*pkgver\s*=\s*(\S+)\s*$", text, re.MULTILINE) + release_match = re.search(r"^\s*pkgrel\s*=\s*(\S+)\s*$", text, re.MULTILINE) + if version_match is None or release_match is None: + permanent(f"AUR .SRCINFO lacks pkgver/pkgrel: {path}") + return f"{version_match.group(1)}-{release_match.group(1)}" + + +class VersionComparator: + def __init__(self, runner: CommandRunner, image: str) -> None: + self.runner = runner + self.image = image + self.command = shutil.which("vercmp") + self.cache: dict[tuple[str, str], int] = {} + + def __call__(self, left: str, right: str) -> int: + key = (left, right) + if key in self.cache: + return self.cache[key] + args = ( + [self.command, left, right] + if self.command + else ["docker", "run", "--rm", self.image, "vercmp", left, right] + ) + result = _run_or_classify(self.runner, args, timeout=120) + try: + value = int(result.stdout.strip()) + except ValueError: + permanent(f"Arch vercmp returned invalid output: {result.stdout!r}") + value = (value > 0) - (value < 0) + self.cache[key] = value + return value + + +@dataclass(frozen=True) +class RpcState: + version: str + last_modified: int | None + + +def fetch_rpc_state(runner: CommandRunner) -> RpcState: + result = _run_or_classify( + runner, + [ + "curl", + "--fail", + "--silent", + "--show-error", + "--location", + "--connect-timeout", + "15", + "--max-time", + "30", + AUR_RPC_URL, + ], + timeout=45, + ) + payload = _parse_json(result.stdout, "AUR RPC") + if not isinstance(payload, dict) or not isinstance(payload.get("results"), list): + permanent("AUR RPC response has no results array") + rows = payload["results"] + if len(rows) != 1 or not isinstance(rows[0], dict): + permanent( + "known AUR package mcpp-bin is missing or ambiguous; refusing first publish" + ) + version = rows[0].get("Version") + modified = rows[0].get("LastModified") + if not isinstance(version, str) or not version: + permanent("AUR RPC mcpp-bin row has no valid Version") + if modified is not None and type(modified) is not int: + modified = None + return RpcState(version, modified) + + +def inspect_observed_repo( + runner: CommandRunner, + *, + clone_dir: Path, + desired_package_dir: Path, + rpc_state: RpcState, +) -> ObservedState: + clone_observed_repo(runner, clone_dir) + git_version = _srcinfo_package_version(clone_dir / ".SRCINFO") + git_head = _run_or_classify( + runner, ["git", "rev-parse", "HEAD"], cwd=clone_dir + ).stdout.strip() + if re.fullmatch(r"[0-9a-fA-F]{40,64}", git_head) is None: + permanent(f"AUR git returned invalid HEAD: {git_head!r}") + tracked_text = _run_or_classify( + runner, ["git", "ls-files", "-z"], cwd=clone_dir + ).stdout + tracked = {item for item in tracked_text.split("\0") if item} + content_matches = tracked == set(MANAGED_FILES) + for name in MANAGED_FILES: + observed_path = clone_dir / name + desired_path = desired_package_dir / name + if not observed_path.is_file() or not desired_path.is_file(): + content_matches = False + continue + if observed_path.read_bytes() != desired_path.read_bytes(): + content_matches = False + return ObservedState( + rpc_version=rpc_state.version, + git_version=git_version, + git_head=git_head.lower(), + content_matches=content_matches, + ) + + +def stage_desired_repo( + runner: CommandRunner, *, clone_dir: Path, desired_package_dir: Path +) -> str: + tracked_text = _run_or_classify( + runner, ["git", "ls-files", "-z"], cwd=clone_dir + ).stdout + tracked = {item for item in tracked_text.split("\0") if item} + for name in sorted(tracked - set(MANAGED_FILES)): + relative = Path(name) + if relative.is_absolute() or ".." in relative.parts: + permanent(f"unsafe tracked path in AUR repository: {name!r}") + _run_or_classify(runner, ["git", "rm", "-f", "--", name], cwd=clone_dir) + for name in MANAGED_FILES: + try: + shutil.copyfile(desired_package_dir / name, clone_dir / name) + except OSError as exc: + permanent(f"cannot stage desired AUR file {name}: {exc}") + _run_or_classify( + runner, ["git", "add", "--", *MANAGED_FILES], cwd=clone_dir + ) + return _run_or_classify( + runner, + ["git", "diff", "--cached", "--no-ext-diff", "--"], + cwd=clone_dir, + ).stdout + + +def publish_staged_repo( + runner: CommandRunner, + *, + clone_dir: Path, + desired_version: str, + max_attempts: int, + base_delay: float, +) -> tuple[str, int]: + _run_or_classify( + runner, + ["git", "config", "user.name", AUR_COMMIT_NAME], + cwd=clone_dir, + ) + _run_or_classify( + runner, + ["git", "config", "user.email", AUR_COMMIT_EMAIL], + cwd=clone_dir, + ) + _run_or_classify( + runner, + ["git", "commit", "-m", f"mcpp-bin {desired_version}"], + cwd=clone_dir, + ) + local_head = _run_or_classify( + runner, ["git", "rev-parse", "HEAD"], cwd=clone_dir + ).stdout.strip() + + def push() -> CommandResult: + return runner.run( + ["git", "push", AUR_SSH_URL, "HEAD:master"], + cwd=clone_dir, + timeout=120, + ) + + _, retries = retry_transient( + push, + max_attempts=max_attempts, + base_delay=base_delay, + ) + remote = _run_or_classify( + runner, + ["git", "ls-remote", AUR_HTTPS_URL, "refs/heads/master"], + timeout=60, + ).stdout.split() + if not remote or remote[0].lower() != local_head.lower(): + raise ReconcileFailure( + f"AUR public git head does not match pushed commit {local_head}", + Classification.TRANSIENT, + retryable=True, + ) + return local_head.lower(), retries + + +def clean_arch_install_smoke( + runner: CommandRunner, + *, + package_dir: Path, + desired_version: str, + image: str, +) -> None: + script = """ +set -euo pipefail +pacman -Syu --noconfirm --needed git +useradd --create-home builder +cp -a /input /tmp/package +chown -R builder:builder /tmp/package +runuser -u builder -- bash -lc 'cd /tmp/package && makepkg --noconfirm' +pacman -U --noconfirm /tmp/package/mcpp-bin-*.pkg.tar.zst +install -d -o builder -g builder /tmp/mcpp-home +actual=$(runuser -u builder -- env HOME=/home/builder MCPP_HOME=/tmp/mcpp-home mcpp --version) +printf '%s\n' "$actual" +grep -F -- "$EXPECTED_VERSION" <<<"$actual" +""".strip() + env_args = ["--env", f"EXPECTED_VERSION={desired_version}"] + _run_or_classify( + runner, + [ + "docker", + "run", + "--rm", + *env_args, + "--volume", + f"{package_dir.resolve()}:/input:ro", + image, + "bash", + "-euc", + script, + ], + timeout=1800, + ) + + +def _release_age_seconds(release: dict[str, Any] | None) -> int | None: + if release is None: + return None + value = release.get("published_at") + if not isinstance(value, str): + return None + try: + published = datetime.fromisoformat(value.replace("Z", "+00:00")) + except ValueError: + return None + return max(0, int((datetime.now(timezone.utc) - published).total_seconds())) + + +def write_report(report: dict[str, Any], json_path: Path | None, summary: Path | None) -> None: + rendered = json.dumps(report, indent=2, sort_keys=True) + "\n" + if json_path is not None: + json_path.parent.mkdir(parents=True, exist_ok=True) + json_path.write_text(rendered, encoding="utf-8") + if summary is not None: + rows = [ + ("trigger", report.get("trigger")), + ("classification", report.get("classification")), + ("action", report.get("action")), + ("desired", report.get("desired_version")), + ("observed RPC", report.get("observed_rpc_version")), + ("observed git", report.get("observed_git_version")), + ("manifest SHA256", report.get("manifest_sha256")), + ("x86_64 SHA256", report.get("asset_sha256", {}).get("x86_64")), + ("aarch64 SHA256", report.get("asset_sha256", {}).get("aarch64")), + ("remote commit", report.get("remote_commit")), + ("push retries", report.get("push_retries")), + ("RPC polls", report.get("rpc_polls")), + ("drift age seconds", report.get("drift_age_seconds")), + ("published", report.get("published")), + ("message", report.get("message")), + ] + def cell(value: Any) -> str: + return str(value).replace("|", "\\|").replace("`", "'").replace("\n", " ") + + lines = ["## mcpp-bin AUR reconciliation", "", "| Field | Value |", "|---|---|"] + lines.extend(f"| {key} | `{cell(value)}` |" for key, value in rows) + with summary.open("a", encoding="utf-8") as stream: + stream.write("\n".join(lines) + "\n") + + +def reconcile_once( + args: argparse.Namespace, + *, + runner: CommandRunner, + work_dir: Path, + report: dict[str, Any], +) -> None: + snapshot = acquire_desired_snapshot( + runner, + work_dir=work_dir, + repository=args.repository, + requested_tag=args.tag, + local_manifest=args.manifest, + local_assets_dir=args.assets_dir, + ) + desired = snapshot.desired + asset_hashes = validate_release_assets(desired, snapshot.assets_dir) + report.update( + { + "desired_version": desired.package_version, + "release_version": desired.version, + "tag": desired.tag, + "release_commit": desired.commit, + "manifest_sha256": desired.manifest_sha256, + "asset_sha256": asset_hashes, + "drift_age_seconds": _release_age_seconds(snapshot.release), + } + ) + + package_dir = args.output_dir if args.render_only else work_dir / "desired-package" + if package_dir is None: + permanent("--render-only requires --output-dir") + materialize_package( + desired=desired, + template_dir=args.template_dir, + output_dir=package_dir, + ) + generate_srcinfo_and_verify( + runner, + package_dir=package_dir, + image=args.arch_image, + ) + rendered_version = _srcinfo_package_version(package_dir / ".SRCINFO") + if rendered_version != desired.package_version: + permanent( + f"generated .SRCINFO version {rendered_version!r} does not match " + f"desired {desired.package_version!r}" + ) + + if args.render_only: + report.update( + { + "classification": Classification.UPDATED.value, + "action": "render-only", + "needs_publish": False, + "published": False, + "message": f"rendered and verified {package_dir}", + } + ) + return + + rpc_state = fetch_rpc_state(runner) + clone_dir = work_dir / "aur-git" + observed = inspect_observed_repo( + runner, + clone_dir=clone_dir, + desired_package_dir=package_dir, + rpc_state=rpc_state, + ) + report.update( + { + "observed_rpc_version": observed.rpc_version, + "observed_git_version": observed.git_version, + "remote_commit": observed.git_head, + } + ) + compare = VersionComparator(runner, args.arch_image) + plan = plan_reconciliation(desired.package_version, observed, compare) + report["action"] = plan.action.value + report["message"] = plan.reason + + if plan.action is PlanAction.REFUSE_DOWNGRADE: + raise ReconcileFailure(plan.reason, Classification.REFUSED_DOWNGRADE) + if plan.action is PlanAction.NOOP: + report.update( + { + "classification": Classification.NOOP.value, + "needs_publish": False, + "published": False, + } + ) + return + if plan.action is PlanAction.WAIT_RPC: + state: RpcState | None = None + + def fetch_version() -> str: + nonlocal state + state = fetch_rpc_state(runner) + return state.version + + _, polls = poll_rpc_version( + fetch_version, + desired_version=desired.package_version, + compare=compare, + max_attempts=args.rpc_poll_attempts, + delay=args.rpc_poll_delay, + ) + report.update( + { + "classification": Classification.NOOP.value, + "observed_rpc_version": state.version if state else observed.rpc_version, + "rpc_polls": polls, + "needs_publish": False, + "published": False, + "message": "AUR RPC converged to the already-published git state", + } + ) + return + + diff = stage_desired_repo( + runner, clone_dir=clone_dir, desired_package_dir=package_dir + ) + if not diff.strip(): + permanent(f"{plan.action.value} plan produced no staged AUR diff") + print("--- mcpp-bin desired-state diff ---") + print(diff, end="" if diff.endswith("\n") else "\n") + report["needs_publish"] = True + if not args.publish: + report.update( + { + "classification": Classification.UPDATED.value, + "published": False, + "message": f"{plan.action.value} dry-run validated; publish required", + } + ) + return + + remote_commit, retries = publish_staged_repo( + runner, + clone_dir=clone_dir, + desired_version=desired.version, + max_attempts=args.push_attempts, + base_delay=args.push_base_delay, + ) + state: RpcState | None = None + + def fetch_version_after_push() -> str: + nonlocal state + state = fetch_rpc_state(runner) + return state.version + + _, polls = poll_rpc_version( + fetch_version_after_push, + desired_version=desired.package_version, + compare=compare, + max_attempts=args.rpc_poll_attempts, + delay=args.rpc_poll_delay, + ) + clean_arch_install_smoke( + runner, + package_dir=package_dir, + desired_version=desired.version, + image=args.arch_image, + ) + report.update( + { + "classification": Classification.UPDATED.value, + "observed_rpc_version": state.version if state else desired.package_version, + "remote_commit": remote_commit, + "push_retries": retries, + "rpc_polls": polls, + "needs_publish": False, + "published": True, + "message": "AUR git, RPC, and clean Arch install converged", + } + ) + + +def parse_args(argv: list[str] | None = None) -> argparse.Namespace: + script_dir = Path(__file__).resolve().parent + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("--repository", default=GITHUB_REPOSITORY) + parser.add_argument("--tag", help="optional exact latest complete stable tag") + parser.add_argument("--publish", action="store_true", help="push an approved diff") + parser.add_argument("--trigger", default="manual") + parser.add_argument("--manifest", type=Path, help="local audit manifest") + parser.add_argument("--assets-dir", type=Path, help="local audited release assets") + parser.add_argument("--template-dir", type=Path, default=script_dir / "mcpp-bin") + parser.add_argument("--render-only", action="store_true") + parser.add_argument("--output-dir", type=Path) + parser.add_argument("--work-dir", type=Path) + parser.add_argument("--arch-image", default=ARCH_IMAGE) + parser.add_argument("--push-attempts", type=int, default=4) + parser.add_argument("--push-base-delay", type=float, default=5.0) + parser.add_argument("--rpc-poll-attempts", type=int, default=12) + parser.add_argument("--rpc-poll-delay", type=float, default=5.0) + parser.add_argument("--report-json", type=Path) + parser.add_argument("--summary", type=Path) + args = parser.parse_args(argv) + if args.assets_dir is not None and args.manifest is None: + parser.error("--assets-dir requires --manifest") + if args.render_only and args.output_dir is None: + parser.error("--render-only requires --output-dir") + if args.publish and args.render_only: + parser.error("--publish and --render-only are mutually exclusive") + return args + + +EXIT_CODES = { + Classification.NOOP: 0, + Classification.UPDATED: 0, + Classification.TRANSIENT: 75, + Classification.PERMANENT: 2, + Classification.REFUSED_DOWNGRADE: 3, +} + + +def main(argv: list[str] | None = None) -> int: + args = parse_args(argv) + report: dict[str, Any] = { + "schema": 1, + "package": AUR_PACKAGE, + "trigger": args.trigger, + "classification": Classification.PERMANENT.value, + "action": None, + "desired_version": None, + "observed_rpc_version": None, + "observed_git_version": None, + "manifest_sha256": None, + "asset_sha256": {}, + "remote_commit": None, + "push_retries": 0, + "rpc_polls": 0, + "drift_age_seconds": None, + "needs_publish": False, + "published": False, + "message": None, + } + failure: ReconcileFailure | None = None + runner = CommandRunner() + try: + if args.work_dir is not None: + args.work_dir.mkdir(parents=True, exist_ok=True) + reconcile_once(args, runner=runner, work_dir=args.work_dir, report=report) + else: + with tempfile.TemporaryDirectory(prefix="mcpp-aur-reconcile-") as raw: + reconcile_once(args, runner=runner, work_dir=Path(raw), report=report) + except RenderError as exc: + failure = ReconcileFailure(str(exc), Classification.PERMANENT) + except ReconcileFailure as exc: + failure = exc + except Exception as exc: # defensive boundary: always emit classification/report + failure = ReconcileFailure( + f"unexpected reconciler failure: {type(exc).__name__}: {exc}", + Classification.PERMANENT, + ) + + if failure is not None: + report["classification"] = failure.classification.value + report["message"] = str(failure) + try: + write_report(report, args.report_json, args.summary) + except OSError as exc: + print(f"cannot write AUR reconciliation report: {exc}", file=sys.stderr) + return EXIT_CODES[Classification.PERMANENT] + print(json.dumps(report, sort_keys=True)) + classification = Classification(report["classification"]) + if failure is not None: + print(f"mcpp-bin reconcile error: {failure}", file=sys.stderr) + return EXIT_CODES[classification] + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/scripts/aur/render_mcpp_bin.py b/scripts/aur/render_mcpp_bin.py new file mode 100755 index 00000000..d7f769c4 --- /dev/null +++ b/scripts/aur/render_mcpp_bin.py @@ -0,0 +1,224 @@ +#!/usr/bin/env python3 +"""Render the mcpp-bin AUR package from an immutable release manifest.""" + +from __future__ import annotations + +import argparse +import hashlib +import json +import re +import sys +from dataclasses import dataclass +from pathlib import Path +from typing import Any, NoReturn + + +VERSION_RE = re.compile(r"[0-9][0-9A-Za-z.+-]*\Z") +COMMIT_RE = re.compile(r"(?:[0-9a-fA-F]{40}|[0-9a-fA-F]{64})\Z") +TOKEN_RE = re.compile(r"[A-Za-z0-9_]+\Z") +SHA256_RE = re.compile(r"[0-9a-fA-F]{64}\Z") + + +class RenderError(RuntimeError): + """The release manifest or package template violates its contract.""" + + +def fail(message: str) -> NoReturn: + raise RenderError(message) + + +@dataclass(frozen=True) +class ReleaseAsset: + platform: str + arch: str + name: str + sha256: str + + +@dataclass(frozen=True) +class DesiredState: + schema: int + version: str + tag: str + commit: str + assets: tuple[ReleaseAsset, ...] + manifest_sha256: str + + @property + def package_version(self) -> str: + return f"{self.version}-1" + + def asset(self, platform: str, arch: str) -> ReleaseAsset: + matches = [ + asset + for asset in self.assets + if asset.platform == platform and asset.arch == arch + ] + if len(matches) != 1: + fail( + f"manifest requires exactly one {platform}/{arch} asset, " + f"found {len(matches)}" + ) + return matches[0] + + +def _string_field(value: Any, field: str) -> str: + if not isinstance(value, str) or not value: + fail(f"manifest field {field!r} must be a non-empty string") + return value + + +def load_desired_state(path: Path) -> DesiredState: + try: + raw_bytes = path.read_bytes() + manifest = json.loads(raw_bytes) + except (OSError, UnicodeError, json.JSONDecodeError) as exc: + fail(f"cannot read release manifest {path}: {exc}") + if not isinstance(manifest, dict): + fail("release manifest root must be an object") + schema = manifest.get("schema") + if type(schema) is not int or schema != 1: + fail(f"unsupported release manifest schema: {schema!r}") + + version = _string_field(manifest.get("version"), "version") + tag = _string_field(manifest.get("tag"), "tag") + commit = _string_field(manifest.get("commit"), "commit") + if VERSION_RE.fullmatch(version) is None: + fail(f"invalid manifest version: {version!r}") + if tag != f"v{version}": + fail(f"manifest tag/version mismatch: {tag!r} versus {version!r}") + if COMMIT_RE.fullmatch(commit) is None: + fail("manifest commit must be a full 40- or 64-hex object ID") + + raw_assets = manifest.get("assets") + if not isinstance(raw_assets, list): + fail("manifest assets must be an array") + assets: list[ReleaseAsset] = [] + identities: set[tuple[str, str]] = set() + names: set[str] = set() + for index, row in enumerate(raw_assets): + if not isinstance(row, dict): + fail(f"manifest asset #{index} must be an object") + platform = _string_field(row.get("platform"), f"assets[{index}].platform") + arch = _string_field(row.get("arch"), f"assets[{index}].arch") + name = _string_field(row.get("name"), f"assets[{index}].name") + digest = _string_field(row.get("sha256"), f"assets[{index}].sha256") + if TOKEN_RE.fullmatch(platform) is None or TOKEN_RE.fullmatch(arch) is None: + fail(f"invalid platform/arch token in manifest asset #{index}") + if Path(name).name != name: + fail(f"manifest asset name must be a basename: {name!r}") + if SHA256_RE.fullmatch(digest) is None: + fail(f"invalid SHA256 in manifest asset #{index}") + identity = (platform, arch) + if identity in identities: + fail(f"duplicate manifest platform/arch: {platform}/{arch}") + if name in names: + fail(f"duplicate manifest asset name: {name}") + identities.add(identity) + names.add(name) + assets.append(ReleaseAsset(platform, arch, name, digest.lower())) + + desired = DesiredState( + schema=schema, + version=version, + tag=tag, + commit=commit.lower(), + assets=tuple(assets), + manifest_sha256=hashlib.sha256(raw_bytes).hexdigest(), + ) + for arch in ("x86_64", "aarch64"): + asset = desired.asset("linux", arch) + expected_name = f"mcpp-{version}-linux-{arch}.tar.gz" + if asset.name != expected_name: + fail( + f"manifest Linux {arch} asset must be {expected_name!r}, " + f"got {asset.name!r}" + ) + return desired + + +def _replace_once(text: str, pattern: str, replacement: str, label: str) -> str: + rendered, count = re.subn(pattern, replacement, text, count=1, flags=re.MULTILINE) + if count != 1: + fail(f"mcpp-bin PKGBUILD must contain exactly one {label}") + return rendered + + +def render_pkgbuild(template: str, desired: DesiredState) -> str: + x86 = desired.asset("linux", "x86_64") + arm = desired.asset("linux", "aarch64") + rendered = _replace_once( + template, r"^pkgver=.*$", f"pkgver={desired.version}", "pkgver" + ) + rendered = _replace_once(rendered, r"^pkgrel=.*$", "pkgrel=1", "pkgrel") + rendered = _replace_once( + rendered, + r"^sha256sums_x86_64=.*$", + f"sha256sums_x86_64=('{x86.sha256}')", + "sha256sums_x86_64", + ) + rendered = _replace_once( + rendered, + r"^sha256sums_aarch64=.*$", + f"sha256sums_aarch64=('{arm.sha256}')", + "sha256sums_aarch64", + ) + return rendered + + +def materialize_package( + *, desired: DesiredState, template_dir: Path, output_dir: Path +) -> None: + if template_dir.name != "mcpp-bin": + fail(f"refusing non-mcpp-bin template directory: {template_dir}") + pkgbuild_path = template_dir / "PKGBUILD" + launcher_path = template_dir / "mcpp.sh" + try: + template = pkgbuild_path.read_text(encoding="utf-8") + launcher = launcher_path.read_bytes() + output_dir.mkdir(parents=True, exist_ok=True) + (output_dir / "PKGBUILD").write_text( + render_pkgbuild(template, desired), encoding="utf-8" + ) + (output_dir / "mcpp.sh").write_bytes(launcher) + except OSError as exc: + fail(f"cannot materialize mcpp-bin package: {exc}") + + +def parse_args(argv: list[str] | None = None) -> argparse.Namespace: + default_template = Path(__file__).resolve().parent / "mcpp-bin" + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("--manifest", required=True, type=Path) + parser.add_argument("--template-dir", type=Path, default=default_template) + parser.add_argument("--output-dir", required=True, type=Path) + return parser.parse_args(argv) + + +def main(argv: list[str] | None = None) -> int: + args = parse_args(argv) + try: + desired = load_desired_state(args.manifest) + materialize_package( + desired=desired, + template_dir=args.template_dir, + output_dir=args.output_dir, + ) + except RenderError as exc: + print(f"mcpp-bin render error: {exc}", file=sys.stderr) + return 2 + print( + json.dumps( + { + "version": desired.version, + "package_version": desired.package_version, + "manifest_sha256": desired.manifest_sha256, + "output_dir": str(args.output_dir), + }, + sort_keys=True, + ) + ) + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/scripts/aur/update.sh b/scripts/aur/update.sh index a48452ec..330ca010 100755 --- a/scripts/aur/update.sh +++ b/scripts/aur/update.sh @@ -1,136 +1,38 @@ #!/usr/bin/env bash -# scripts/aur/update.sh — bump BOTH AUR packages to a release version. +# Compatibility entrypoint for rendering the mcpp-bin AUR package. # -# mcpp-bin/ prebuilt release binaries (per-arch tarball checksums) -# mcpp-m/ built from source via mcpp-bin (source-archive checksum) -# -# Pulls checksums straight from the GitHub release / archive, rewrites each -# PKGBUILD's pkgver + sums, resets pkgrel to 1, regenerates both .SRCINFO -# files, and keeps the shared mcpp.sh wrapper in sync across both dirs. -# Run after a release is published, then publish each package (see README.md). -# -# Usage: -# scripts/aur/update.sh [VERSION] # default: [package].version from mcpp.toml +# Desired state comes only from the latest complete stable release's immutable +# mcpp-release.json. An optional VERSION or vVERSION must match that exact +# release; this wrapper has no downgrade override and publishes nothing. set -euo pipefail -cd "$(dirname "$0")" -REPO="mcpp-community/mcpp" -ARCHES=(linux-x86_64 linux-aarch64) +SCRIPT_DIR=$(cd -- "$(dirname -- "${BASH_SOURCE[0]}")" && pwd) -# Resolve version: explicit arg, else mcpp.toml at repo root. -if [[ $# -ge 1 ]]; then - VER="$1" -else - VER=$(grep -m1 -E '^\s*version\s*=' ../../mcpp.toml | sed -E 's/.*"([^"]+)".*/\1/') +if (( $# > 1 )); then + echo "usage: scripts/aur/update.sh [VERSION|vVERSION]" >&2 + exit 2 fi -[[ -n "$VER" ]] || { echo "error: could not determine version" >&2; exit 1; } -echo ":: targeting mcpp v${VER}" - -reldl="https://github.com/${REPO}/releases/download/v${VER}" -archive_url="https://github.com/${REPO}/archive/v${VER}.tar.gz" - -# --- prebuilt binary checksums (mcpp-bin) ---------------------------------- -declare -A SUMS -for plat in "${ARCHES[@]}"; do - url="${reldl}/mcpp-${VER}-${plat}.tar.gz.sha256" - echo ":: fetching ${url}" - line=$(curl -fsSL --connect-timeout 15 "$url") \ - || { echo "error: cannot fetch sha256 for ${plat} (is v${VER} released?)" >&2; exit 1; } - SUMS[$plat]=$(awk '{print $1}' <<<"$line") - [[ -n "${SUMS[$plat]}" ]] || { echo "error: empty sha256 for ${plat}" >&2; exit 1; } -done -x86=${SUMS[linux-x86_64]} -arm=${SUMS[linux-aarch64]} - -# --- source-archive checksum (mcpp-m) ---------------------------------------- -echo ":: hashing source archive ${archive_url}" -src=$(curl -fsSL --connect-timeout 30 "$archive_url" | sha256sum | awk '{print $1}') -[[ -n "$src" ]] || { echo "error: cannot hash source archive" >&2; exit 1; } -# --- keep the wrapper in sync ---------------------------------------------- -cp -f mcpp-bin/mcpp.sh mcpp-m/mcpp.sh 2>/dev/null || true +args=( + --render-only + --output-dir "$SCRIPT_DIR/mcpp-bin" + --trigger local-update +) -# --- rewrite mcpp-bin/PKGBUILD --------------------------------------------- -sed -i -E \ - -e "s/^pkgver=.*/pkgver=${VER}/" \ - -e "s/^pkgrel=.*/pkgrel=1/" \ - -e "s/^sha256sums_x86_64=\('[^']*'\)/sha256sums_x86_64=('${x86}')/" \ - -e "s/^sha256sums_aarch64=\('[^']*'\)/sha256sums_aarch64=('${arm}')/" \ - mcpp-bin/PKGBUILD - -# --- rewrite mcpp-m/PKGBUILD --------------------------------------------- -sed -i -E \ - -e "s/^pkgver=.*/pkgver=${VER}/" \ - -e "s/^pkgrel=.*/pkgrel=1/" \ - -e "s/^sha256sums=\('[^']*'\)/sha256sums=('${src}')/" \ - mcpp-m/PKGBUILD - -# --- regenerate .SRCINFO files --------------------------------------------- -# Prefer makepkg's own generator on an Arch host; fall back to templates so -# this also works when bumping from a non-Arch machine (e.g. CI / dev box). -_srcinfo() { # $1 = package dir - # CI runs as root where makepkg refuses to run; MCPP_AUR_NO_MAKEPKG=1 - # forces the template path (byte-identical output, no makepkg needed). - if [[ -z "${MCPP_AUR_NO_MAKEPKG:-}" ]] && command -v makepkg >/dev/null 2>&1; then - ( cd "$1" && makepkg --printsrcinfo > .SRCINFO ) - return - fi - case "$1" in - mcpp-bin) - cat > mcpp-bin/.SRCINFO < mcpp-m/.SRCINFO <&2 + exit 2 + } + args+=(--manifest "$MCPP_AUR_MANIFEST" --assets-dir "$MCPP_AUR_ASSETS_DIR") +fi -echo ":: updated to v${VER}" -echo " mcpp-bin x86_64 ${x86}" -echo " mcpp-bin aarch64 ${arm}" -echo " mcpp-m source ${src}" -echo ":: review, then publish each package (see scripts/aur/README.md)" +exec python3 "$SCRIPT_DIR/reconcile_mcpp_bin.py" "${args[@]}" diff --git a/scripts/release/generate_manifest.py b/scripts/release/generate_manifest.py new file mode 100644 index 00000000..0446dfae --- /dev/null +++ b/scripts/release/generate_manifest.py @@ -0,0 +1,228 @@ +#!/usr/bin/env python3 +"""Generate and validate the immutable mcpp GitHub release manifest.""" + +from __future__ import annotations + +import argparse +import hashlib +import json +import re +import sys +from pathlib import Path +from typing import Any, NoReturn + + +SCHEMA_VERSION = 1 +COMMIT_RE = re.compile(r"(?:[0-9a-fA-F]{40}|[0-9a-fA-F]{64})\Z") +VERSION_RE = re.compile(r"[0-9][0-9A-Za-z.+-]*\Z") +PRIMARY_ASSET_RE = re.compile( + r"^mcpp-(?P[0-9][0-9A-Za-z.+-]*)-" + r"(?P[A-Za-z][A-Za-z0-9_]*)-" + r"(?P[A-Za-z0-9_]+)\." + r"(?:tar\.gz|zip)$" +) +SIDECAR_RE = re.compile(r"^(?P[0-9a-fA-F]{64})[ \t]+\*?(?P\S+)$") + + +class ManifestError(RuntimeError): + """A release violates the public manifest contract.""" + + +def fail(message: str) -> NoReturn: + raise ManifestError(message) + + +def load_release(path: Path) -> dict[str, Any]: + try: + release = json.loads(path.read_text(encoding="utf-8")) + except (OSError, UnicodeError, json.JSONDecodeError) as exc: + fail(f"cannot read release JSON {path}: {exc}") + if not isinstance(release, dict): + fail("release JSON root must be an object") + return release + + +def release_asset_names(release: dict[str, Any]) -> set[str]: + raw_assets = release.get("assets") + if not isinstance(raw_assets, list): + fail("release JSON assets must be an array") + + names: set[str] = set() + for index, raw_asset in enumerate(raw_assets): + if not isinstance(raw_asset, dict): + fail(f"release asset #{index} must be an object") + name = raw_asset.get("name") + if not isinstance(name, str) or not name: + fail(f"release asset #{index} has no valid name") + if name in names: + fail(f"duplicate release asset name: {name}") + names.add(name) + return names + + +def sha256_file(path: Path) -> str: + digest = hashlib.sha256() + try: + with path.open("rb") as stream: + for chunk in iter(lambda: stream.read(1024 * 1024), b""): + digest.update(chunk) + except OSError as exc: + fail(f"cannot read release asset {path}: {exc}") + return digest.hexdigest() + + +def read_sidecar(path: Path, primary_name: str) -> str: + try: + lines = [ + line.strip() + for line in path.read_text(encoding="utf-8").splitlines() + if line.strip() + ] + except (OSError, UnicodeError) as exc: + fail(f"cannot read sidecar {path}: {exc}") + if len(lines) != 1: + fail(f"sidecar must contain exactly one checksum record: {path.name}") + match = SIDECAR_RE.fullmatch(lines[0]) + if match is None: + fail(f"invalid SHA256 sidecar syntax: {path.name}") + if match.group("name") != primary_name: + fail( + f"sidecar filename target mismatch: {path.name} names " + f"{match.group('name')!r}, expected {primary_name!r}" + ) + return match.group("sha256").lower() + + +def required_primary_assets(version: str) -> set[str]: + return { + f"mcpp-{version}-linux-x86_64.tar.gz", + f"mcpp-{version}-linux-aarch64.tar.gz", + f"mcpp-{version}-macosx-arm64.tar.gz", + f"mcpp-{version}-windows-x86_64.zip", + } + + +def generate_manifest( + *, + release: dict[str, Any], + assets_dir: Path, + version: str, + tag: str, + commit: str, +) -> dict[str, Any]: + if VERSION_RE.fullmatch(version) is None: + fail(f"invalid version: {version!r}") + if tag != f"v{version}": + fail(f"tag/version mismatch: expected v{version!s}, got {tag!r}") + if COMMIT_RE.fullmatch(commit) is None: + fail("commit must be a full 40- or 64-hex object ID") + if release.get("tag_name") != tag: + fail( + f"release tag mismatch: release has {release.get('tag_name')!r}, " + f"expected {tag!r}" + ) + if release.get("draft") is not False: + fail("release must have draft=false") + if release.get("prerelease") is not False: + fail("release must have prerelease=false") + + names = release_asset_names(release) + discovered: list[tuple[str, re.Match[str]]] = [] + for name in sorted(names): + match = PRIMARY_ASSET_RE.fullmatch(name) + if match is None: + continue + if match.group("version") != version: + fail( + f"asset version mismatch: {name!r} has " + f"{match.group('version')!r}, expected {version!r}" + ) + discovered.append((name, match)) + + required = required_primary_assets(version) + missing = sorted(required - {name for name, _ in discovered}) + if missing: + fail(f"missing required primary asset(s): {', '.join(missing)}") + + manifest_assets: list[dict[str, str]] = [] + identities: dict[tuple[str, str], str] = {} + for name, match in discovered: + platform = "macos" if match.group("platform") == "macosx" else match.group("platform") + arch = match.group("arch") + identity = (platform, arch) + if identity in identities: + fail( + f"duplicate platform/arch {platform}/{arch}: " + f"{identities[identity]} and {name}" + ) + identities[identity] = name + + sidecar_name = f"{name}.sha256" + if sidecar_name not in names: + fail(f"missing sidecar release asset: {sidecar_name}") + primary_path = assets_dir / name + sidecar_path = assets_dir / sidecar_name + if not primary_path.is_file(): + fail(f"missing downloaded primary asset: {primary_path}") + if not sidecar_path.is_file(): + fail(f"missing downloaded sidecar: {sidecar_path}") + + expected_digest = read_sidecar(sidecar_path, name) + actual_digest = sha256_file(primary_path) + if actual_digest != expected_digest: + fail( + f"SHA256 mismatch for {name}: sidecar has {expected_digest}, " + f"payload has {actual_digest}" + ) + manifest_assets.append( + { + "platform": platform, + "arch": arch, + "name": name, + "sha256": actual_digest, + } + ) + + manifest_assets.sort(key=lambda item: (item["platform"], item["arch"], item["name"])) + return { + "schema": SCHEMA_VERSION, + "version": version, + "tag": tag, + "commit": commit.lower(), + "assets": manifest_assets, + } + + +def parse_args(argv: list[str] | None = None) -> argparse.Namespace: + parser = argparse.ArgumentParser(description=__doc__) + parser.add_argument("--release-json", required=True, type=Path) + parser.add_argument("--assets-dir", required=True, type=Path) + parser.add_argument("--version", required=True) + parser.add_argument("--tag", required=True) + parser.add_argument("--commit", required=True) + parser.add_argument("--output", required=True, type=Path) + return parser.parse_args(argv) + + +def main(argv: list[str] | None = None) -> int: + args = parse_args(argv) + try: + release = load_release(args.release_json) + manifest = generate_manifest( + release=release, + assets_dir=args.assets_dir, + version=args.version, + tag=args.tag, + commit=args.commit, + ) + rendered = json.dumps(manifest, indent=2, ensure_ascii=False) + "\n" + args.output.parent.mkdir(parents=True, exist_ok=True) + args.output.write_text(rendered, encoding="utf-8") + except (ManifestError, OSError) as exc: + print(f"release manifest error: {exc}", file=sys.stderr) + return 2 + return 0 + + +if __name__ == "__main__": + raise SystemExit(main()) diff --git a/src/build/execute.cppm b/src/build/execute.cppm index 960c45d4..48192537 100644 --- a/src/build/execute.cppm +++ b/src/build/execute.cppm @@ -15,6 +15,7 @@ import mcpp.diag; import mcpp.build.plan; import mcpp.build.backend; import mcpp.build.ninja; +import mcpp.build.runtime_validation; import mcpp.bmi_cache; import mcpp.manifest; import mcpp.modgraph.scanner; @@ -22,6 +23,7 @@ import mcpp.toolchain.post_install; import mcpp.toolchain.stdmod; import mcpp.xlings; import mcpp.xlings.subos_info; +import mcpp.platform.runtime_binding; import mcpp.log; import mcpp.platform; import mcpp.fetcher.progress; @@ -88,6 +90,9 @@ struct BuildCacheEntry { // said not to use — and ruling the cache out is `local`'s entire purpose. // Same back-compat contract as `profile`: empty ⇒ miss. std::string cacheMode; + // Exact immutable snapshot used by the build. Optional distinguishes a + // current cache from one written by an older mcpp (or a corrupt payload). + std::optional runtimeBinding; }; std::vector read_build_cache(const std::filesystem::path& projectRoot) { @@ -164,6 +169,12 @@ std::vector read_build_cache(const std::filesystem::path& proje e.subosRecorded = true; haveNextLine = static_cast(std::getline(f, line)); } + if (haveNextLine && line.starts_with("runtimeBinding=")) { + auto decoded = mcpp::platform::runtime::deserialize_runtime_binding( + line.substr(15)); + if (decoded) e.runtimeBinding = std::move(*decoded); + haveNextLine = static_cast(std::getline(f, line)); + } // Optional profile line. Same back-compat contract as the two blocks // above: absent ⇒ e.profile stays empty ⇒ every fast path treats the // entry as a miss and falls through to prepare_build. @@ -193,7 +204,7 @@ void write_build_cache(const std::filesystem::path& projectRoot, const std::string& runEnvValue = "", const std::string& profile = "", const std::string& cacheMode = "", - const std::string& subosDir = "") { + const mcpp::platform::runtime::RuntimeBinding& runtimeBinding = {}) { auto path = projectRoot / kBuildCacheFile; auto entries = read_build_cache(projectRoot); @@ -208,8 +219,10 @@ void write_build_cache(const std::filesystem::path& projectRoot, // Insert at front (MRU). BuildCacheEntry newEntry{targetTriple, outputDir.string(), ninjaProgram, fingerprintHex, runtimeEnvKey, runtimeEnvValue, std::move(runTargets), - runEnvKey, runEnvValue, subosDir, /*subosRecorded=*/true, + runEnvKey, runEnvValue, runtimeBinding.subosDir.string(), + /*subosRecorded=*/true, profile, cacheMode}; + newEntry.runtimeBinding = runtimeBinding; entries.insert(entries.begin(), std::move(newEntry)); // Trim to LRU capacity. @@ -238,6 +251,10 @@ void write_build_cache(const std::filesystem::path& projectRoot, f << "runEnv=" << e.runEnvKey << '\n'; f << e.runEnvValue << '\n'; f << "subos=" << e.subosDir << '\n'; + if (e.runtimeBinding) + f << "runtimeBinding=" + << mcpp::platform::runtime::serialize_runtime_binding(*e.runtimeBinding) + << '\n'; f << "profile=" << e.profile << '\n'; f << "cacheMode=" << e.cacheMode << '\n'; } @@ -315,8 +332,7 @@ compute_run_env(const mcpp::build::BuildPlan& plan) { return {key, value}; } -// The environment the active subos declares for the programs it hosts -// (mcpp#352). +// The environment captured by this build's selected RuntimeBinding (mcpp#352). // // A GL application needs three things and mcpp only ever supplied two: the // binary links (bootstrap), it finds its libraries (RPATH), and then it has to @@ -326,46 +342,18 @@ compute_run_env(const mcpp::build::BuildPlan& plan) { // launched — `xlings subos use` applied them, `mcpp run` did not. Hence a // binary that links fine and exits 255 with no output. // -// Resolved at RUN time, deliberately not cached with the build: these values -// belong to the subos, not to the build, and a user who switches subos between -// `mcpp build` and `mcpp run` must get the new one. It is a file read. +// The declarations are snapshotted with the build and serialized into the +// fast-path cache. A changed SubOS manifest invalidates that cache and causes a +// fresh prepare; no invocation mixes newly read run state with old objects. // // mcpp does not know what any of these variables MEAN, and that is the design: // when the ecosystem gains a Vulkan loader or a new driver bridge, the // declaration changes and this code does not. -// The subos a RUN should use: an explicit override if the caller set one, -// otherwise the subos this build belongs to. -// -// The override lives HERE and not in the derivation, because the derivation's -// answer is cached and this one must not be: MCPP_SUBOS_DIR says "for this -// invocation". It exists so tests can exercise this path without touching a -// developer's real subos — an earlier e2e wrote through a symlink and -// permanently broke a real toolchain — and so a user can point one run at -// another subos without switching the active one. -std::filesystem::path subos_dir_for_run(const std::filesystem::path& buildSubos) { - if (const char* e = std::getenv("MCPP_SUBOS_DIR"); e && *e) - return std::filesystem::path(e); - return buildSubos; -} - std::vector> compute_subos_env(const mcpp::build::BuildPlan& plan) { - auto built = mcpp::xlings::paths::subos_dir_of(plan.toolchain.binaryPath); - auto dir = subos_dir_for_run(built ? *built : std::filesystem::path{}); - if (dir.empty()) return {}; - auto info = mcpp::xlings::subos::read(dir); - // The note is a `verbose` line rather than a warning: a subos with no - // self-description is the normal state of every machine whose subos - // predates the block, and a warning on every run would train people to - // ignore it. It becomes loud only where it explains a failure — the GL - // diagnostic path in doctor. - if (!info.note.empty()) - mcpp::log::verbose("subos", info.note); - // Resolved AGAINST the caller's environment, not in a vacuum: these - // entries replace the variable in the child, so a `set` that ignores an - // exported value overwrites it and a `prepend` that ignores it drops it. - return mcpp::xlings::subos::resolve_env( - info, dir, [](std::string_view v) -> std::optional { + return mcpp::platform::runtime::resolve_runtime_environment( + plan.runtimeBinding, + [](std::string_view v) -> std::optional { if (const char* e = std::getenv(std::string(v).c_str())) return std::string(e); return std::nullopt; @@ -500,14 +488,13 @@ export int run_build_plan(BuildContext& ctx, bool verbose, bool no_cache, auto fpHex = ctx.outputDir.filename().string(); auto runTargets = compute_run_targets(ctx.plan); auto [runEnvKey, runEnvValue] = compute_run_env(ctx.plan); - auto subosDir = mcpp::xlings::paths::subos_dir_of(ctx.plan.toolchain.binaryPath); write_build_cache(ctx.projectRoot, ctx.outputDir, r->ninjaProgram, std::string(targetOverride), fpHex, r->runtimeEnvKey.empty() ? "-" : r->runtimeEnvKey, r->runtimeEnvValue, std::move(runTargets), runEnvKey, runEnvValue, ctx.profile, std::string(cache_mode_name(ctx.cacheMode)), - subosDir ? subosDir->string() : std::string{}); + ctx.plan.runtimeBinding); } // The one place the --strict policy is settled. Degradations reported by @@ -724,6 +711,7 @@ export std::optional try_fast_build(const std::filesystem::path& projectRoo } } if (!match) return std::nullopt; + if (!match->runtimeBinding) return std::nullopt; auto outputDirStr = match->outputDir; auto ninjaProgram = match->ninjaProgram; @@ -754,6 +742,10 @@ export std::optional try_fast_build(const std::filesystem::path& projectRoo auto ninjaTime = std::filesystem::last_write_time(ninjaPath, ec); if (ec) return std::nullopt; + auto runtimeManifest = match->runtimeBinding->subosDir / ".xlings.json"; + auto runtimeTime = std::filesystem::last_write_time(runtimeManifest, ec); + if (ec || runtimeTime > ninjaTime) return std::nullopt; + // Check mcpp.toml auto tomlPath = projectRoot / "mcpp.toml"; auto tomlTime = std::filesystem::last_write_time(tomlPath, ec); @@ -763,12 +755,20 @@ export std::optional try_fast_build(const std::filesystem::path& projectRoo // instead of a hand-rolled recursive_directory_iterator over src/. if (sources_newer_than(projectRoot, ninjaTime, want->resourceScripts)) return std::nullopt; + auto validatedBefore = + mcpp::build::runtime_validation::validated_artifact_snapshot( + outputDir, *match->runtimeBinding); + if (!validatedBefore) return std::nullopt; + // All inputs are older than build.ninja → fast-path: just run ninja. std::chrono::milliseconds elapsed{}; auto rc = run_ninja_fast(ninjaProgram, outputDir, ninjaPath, verbose, runtimeEnvKey, runtimeEnvValue, &elapsed); if (!rc) return std::nullopt; if (*rc != 0) return rc; + if (!mcpp::build::runtime_validation::artifact_snapshot_unchanged( + *validatedBefore)) + return std::nullopt; // relinked: full path reconstructs + validates closure mcpp::ui::finished(want->profile, elapsed); return 0; @@ -817,8 +817,8 @@ std::optional try_fast_run(const std::filesystem::path& projectRoot, // mcpp wrote the entry, so without this line an upgraded mcpp would reuse a // pre-upgrade build until something else happened to invalidate it. Measured // on a real upgrade from 2026.8.7.1, not reasoned about. - if (!match->subosRecorded) - return std::nullopt; // predates `subos=`; rebuild once, then it is there + if (!match->runtimeBinding) + return std::nullopt; // predates the immutable snapshot; rebuild once // P1: verify fingerprint matches the outputDir basename. if (!match->fingerprint.empty()) { @@ -844,18 +844,30 @@ std::optional try_fast_run(const std::filesystem::path& projectRoot, auto ninjaTime = std::filesystem::last_write_time(ninjaPath, ec); if (ec) return std::nullopt; + auto runtimeManifest = match->runtimeBinding->subosDir / ".xlings.json"; + auto runtimeTime = std::filesystem::last_write_time(runtimeManifest, ec); + if (ec || runtimeTime > ninjaTime) return std::nullopt; + auto tomlPath = projectRoot / "mcpp.toml"; auto tomlTime = std::filesystem::last_write_time(tomlPath, ec); if (ec || tomlTime > ninjaTime) return std::nullopt; if (sources_newer_than(projectRoot, ninjaTime, want->resourceScripts)) return std::nullopt; + auto validatedBefore = + mcpp::build::runtime_validation::validated_artifact_snapshot( + outputDir, *match->runtimeBinding); + if (!validatedBefore) return std::nullopt; + // Fresh → run ninja (picks up any incremental object/link work) then // exec the cached exe path directly. auto rc = run_ninja_fast(ninjaProgram, outputDir, ninjaPath, /*verbose=*/false, match->runtimeEnvKey, match->runtimeEnvValue); if (!rc) return std::nullopt; if (*rc != 0) return rc; + if (!mcpp::build::runtime_validation::artifact_snapshot_unchanged( + *validatedBefore)) + return std::nullopt; // never execute an artifact not validated for this binding auto exe = outputDir / chosen->second; auto pathCtx = mcpp::fetcher::make_path_ctx(/*cfg=*/nullptr, projectRoot); @@ -870,31 +882,18 @@ std::optional try_fast_run(const std::filesystem::path& projectRoot, std::vector> childEnv; if (!match->runEnvKey.empty() && !match->runEnvValue.empty()) childEnv.emplace_back(match->runEnvKey, match->runEnvValue); - // ...and the subos's declared environment, re-READ here rather than taken - // from the cache. Which subos is a build property (cached above); what it - // declares is the subos's own, and a user who installs a graphics stack - // between two runs must get it without rebuilding. - // - // This is the half that a fast path is most likely to lose, and losing it - // would be invisible in the worst way: the first `mcpp run` after a build - // takes the full path and works, every later one takes this path and does - // not. A GL program would run once and then stop finding its driver. - { - // Same rule as the full path, through the same helper: an override - // for this invocation, else the subos this build was recorded against. - auto subosDir = subos_dir_for_run(std::filesystem::path(match->subosDir)); - if (!subosDir.empty()) { - auto info = mcpp::xlings::subos::read(subosDir); - for (auto& kv : mcpp::xlings::subos::resolve_env( - info, subosDir, - [](std::string_view v) -> std::optional { - if (const char* e = std::getenv(std::string(v).c_str())) - return std::string(e); - return std::nullopt; - })) - childEnv.push_back(std::move(kv)); - } - } + // ...and exactly the environment declaration snapshot used by the build. + // Installing/changing a provider invalidates the fast path via the SubOS + // manifest mtime check; this invocation never mixes a new run contract + // with objects built under the old one. + for (auto& kv : mcpp::platform::runtime::resolve_runtime_environment( + *match->runtimeBinding, + [](std::string_view v) -> std::optional { + if (const char* e = std::getenv(std::string(v).c_str())) + return std::string(e); + return std::nullopt; + })) + childEnv.push_back(std::move(kv)); return mcpp::platform::process::run_exec(argv, childEnv) == 0 ? 0 : 1; } diff --git a/src/build/flags.cppm b/src/build/flags.cppm index d2aed189..ac025eba 100644 --- a/src/build/flags.cppm +++ b/src/build/flags.cppm @@ -14,6 +14,7 @@ export module mcpp.build.flags; import std; import mcpp.build.distribution; import mcpp.build.plan; +import mcpp.manifest.types; import mcpp.modgraph.scanner; import mcpp.platform; import mcpp.toolchain.clang; @@ -64,6 +65,15 @@ struct CompileFlags { } }; +enum class LinkIntentFlavor { Elf, MachO, PeGnu, PeMsvc }; + +// Spell a provider-neutral LinkIntent for one output format. Kept pure so +// every platform contract can be asserted on every CI host. deployFiles are +// intentionally absent: the backend emits copy edges, never linker flags. +std::string render_link_intent_flags( + const mcpp::manifest::LinkIntent& intent, + LinkIntentFlavor flavor); + CompileFlags compute_flags(const BuildPlan& plan); // The kind → role map. One line of policy, in one place: a test binary runs on @@ -268,6 +278,62 @@ std::string shell_quote_arg(std::string_view arg) { } } +std::string render_link_intent_flags( + const mcpp::manifest::LinkIntent& intent, + LinkIntentFlavor flavor) { + std::string out; + auto token = [](std::string value) { + return shell_quote_arg(escape_ninja_chars(value)); + }; + auto path_token = [&](std::string_view prefix, + const std::filesystem::path& path) { + return token(std::string(prefix) + path.string()); + }; + + for (auto const& dir : intent.linkLibraryDirs) { + out += ' '; + out += path_token(flavor == LinkIntentFlavor::PeMsvc + ? "/LIBPATH:" : "-L", dir); + } + if (flavor == LinkIntentFlavor::Elf) { + for (auto const& dir : intent.transitiveNeededDirs) { + out += ' '; + out += path_token("-Wl,-rpath-link,", dir); + } + } + if (flavor == LinkIntentFlavor::Elf + || flavor == LinkIntentFlavor::MachO) { + for (auto const& dir : intent.runtimeSearchDirs) { + out += ' '; + out += path_token("-Wl,-rpath,", dir); + } + } + + for (auto const& library : intent.libraries) { + if (library.empty()) continue; + out += ' '; + const std::filesystem::path asPath(library); + const bool explicitToken = library.starts_with('-') + || library.starts_with('/') || asPath.has_parent_path() + || asPath.has_extension(); + if (explicitToken) { + out += token(library); + } else if (flavor == LinkIntentFlavor::PeMsvc) { + out += token(library + ".lib"); + } else { + out += token("-l" + library); + } + } + if (flavor == LinkIntentFlavor::MachO) { + for (auto const& framework : intent.frameworks) { + if (framework.empty()) continue; + out += " -framework "; + out += token(framework); + } + } + return out; +} + CompileFlags compute_flags(const BuildPlan& plan) { CompileFlags f; @@ -415,6 +481,28 @@ CompileFlags compute_flags(const BuildPlan& plan) { // is only assemblable by its own x86_64-w64-mingw32-as. bool isMuslTc = mcpp::toolchain::is_musl_target(plan.toolchain); bool isMingwTc = mcpp::toolchain::is_mingw_target(plan.toolchain); + const auto linkIntentFlavor = [&] { + if (isMingwTc) return LinkIntentFlavor::PeGnu; + if (isMsvcDialect) return LinkIntentFlavor::PeMsvc; + auto triple = plan.toolchain.targetTriple; + std::ranges::transform(triple, triple.begin(), + [](unsigned char c) { return std::tolower(c); }); + if (triple.find("darwin") != std::string::npos + || triple.find("apple") != std::string::npos) + return LinkIntentFlavor::MachO; + if (triple.find("windows") != std::string::npos + || triple.find("mingw") != std::string::npos) + return LinkIntentFlavor::PeGnu; + if (triple.empty()) { + if constexpr (mcpp::platform::is_windows) + return LinkIntentFlavor::PeGnu; + if constexpr (mcpp::platform::needs_explicit_libcxx) + return LinkIntentFlavor::MachO; + } + return LinkIntentFlavor::Elf; + }(); + const std::string link_intent_ld = + render_link_intent_flags(plan.linkIntent, linkIntentFlavor); std::filesystem::path binutilsBin; if (!isMuslTc && !isMingwTc && caps.stdlib_id == "libstdc++") { auto ar = mcpp::toolchain::archive_tool(plan.toolchain); @@ -713,14 +801,17 @@ CompileFlags compute_flags(const BuildPlan& plan) { runtime_dirs += " -L" + escape_path(dir); runtime_dirs += " -Wl,-rpath," + escape_path(dir); } - // ...plus dependency packages' [runtime] library_dirs (e.g. - // compat.glx-runtime's host-GL passthrough), so dlopen()'d host libs - // (libGL/libGLX) are reachable at run time. Only the dep dirs — NOT the - // glibc payload dir — so static/musl links stay clean. - for (auto& dir : plan.depRuntimeLibraryDirs) { - runtime_dirs += " -L" + escape_path(dir); - runtime_dirs += " -Wl,-rpath," + escape_path(dir); - } + // Dependency/runtime provider search comes from LinkIntent and is + // rendered separately. In particular runtimeSearchDirs contributes + // RUNPATH only; it must never become a link-time -L path. + // + // The private libc directory is NOT emitted here: the link model + // already puts it in the artifact's RUNPATH wherever a payload exists + // (`-L -Wl,-rpath,` next to --dynamic-linker), and this + // link line has a hard 128KiB ceiling that real workspaces already + // spend 43% of. `208_private_libc_stays_in_the_binary.sh` asserts that + // coverage, so if a toolchain ever stops providing it the failure is a + // red test rather than a silent loss of dlopen() resolution. } // For Clang with payload paths: the payload C runtime — -B so the driver @@ -778,7 +869,8 @@ CompileFlags compute_flags(const BuildPlan& plan) { std::string mingw_stdexp; if (caps.stdlib_id == "libstdc++") mingw_stdexp = " -lstdc++exp"; - f.ld = std::format("{}{}{}", user_ldflags, mingw_stdexp, link_extra); + f.ld = std::format("{}{}{}{}", link_intent_ld, user_ldflags, + mingw_stdexp, link_extra); return f; } @@ -789,10 +881,7 @@ CompileFlags compute_flags(const BuildPlan& plan) { // ldflags pass through verbatim; GNU link_extra (-flto/-s) does // not apply. f.ldBinary = mcpp::toolchain::link_tool(plan.toolchain); - std::string libpaths; - for (auto& dir : plan.depRuntimeLibraryDirs) - libpaths += " /LIBPATH:" + escape_path(dir); - f.ld = libpaths + user_ldflags; + f.ld = link_intent_ld + user_ldflags; return f; } // PE link, MSVC-ABI Clang (native MinGW is handled by the target-keyed @@ -832,7 +921,8 @@ CompileFlags compute_flags(const BuildPlan& plan) { // // Native cl.exe (isMsvcDialect, returned above) keeps link.exe: there // the response file is ours, and 2026.8.5.3 already fixed it. - f.ld = std::format(" -fuse-ld=lld{}{}", user_ldflags, link_extra); + f.ld = std::format(" -fuse-ld=lld{}{}{}", link_intent_ld, + user_ldflags, link_extra); } else if constexpr (mcpp::platform::needs_explicit_libcxx) { // macOS. The C++ runtime itself is decided by the contract table above // (dist::Format::MachO) and rides unit_ldflags; what is left here is @@ -871,8 +961,9 @@ CompileFlags compute_flags(const BuildPlan& plan) { std::string macos_sdk; if (auto sdk = mcpp::platform::macos::sdk_path()) macos_sdk = " -isysroot " + escape_path(*sdk); - f.ld = std::format("{}{}{} -fuse-ld=lld{}{}{}", full_static, - b_flag, macos_sdk, version_min, user_ldflags, link_extra); + f.ld = std::format("{}{}{} -fuse-ld=lld{}{}{}{}", full_static, + b_flag, macos_sdk, version_min, link_intent_ld, + user_ldflags, link_extra); } else { // libatomic: 16-byte / oversized std::atomic needs the out-of-line // __atomic_* libcalls from libatomic, which the driver won't add on @@ -881,8 +972,10 @@ CompileFlags compute_flags(const BuildPlan& plan) { // actually being present (see atomic_link_flag). std::string atomic_ld = atomic_link_flag(plan.toolchain.linkRuntimeDirs, !full_static.empty()); - f.ld = std::format("{}{}{}{}{}{}{}{}", full_static, link_toolchain_flags, b_flag, - runtime_dirs, atomic_ld, payload_ld, user_ldflags, link_extra); + f.ld = std::format("{}{}{}{}{}{}{}{}{}", full_static, + link_toolchain_flags, b_flag, runtime_dirs, + link_intent_ld, atomic_ld, payload_ld, + user_ldflags, link_extra); } return f; diff --git a/src/build/ninja_backend.cppm b/src/build/ninja_backend.cppm index 4c78eb44..588b3b57 100644 --- a/src/build/ninja_backend.cppm +++ b/src/build/ninja_backend.cppm @@ -24,6 +24,8 @@ import mcpp.build.distribution; import mcpp.build.plan; import mcpp.build.flags; import mcpp.build.hermetic; +import mcpp.build.runtime_validation; +import mcpp.platform.elf_runtime; import mcpp.build.compile_commands; import mcpp.build.cmdlimits; import mcpp.diag; @@ -1521,6 +1523,11 @@ std::optional check_inline_command_lengths(const std::string& manif std::expected NinjaBackend::build(const BuildPlan& plan, const BuildOptions& opts) { auto t0 = std::chrono::steady_clock::now(); + // Captured before ninja touches any link output. The post-build runtime + // validator compares this snapshot, so a hot no-op performs zero ELF + // parses and an output rebuilt behind an unchanged build.ninja is caught. + auto runtimeBefore = + mcpp::build::runtime_validation::snapshot_link_artifacts(plan); std::error_code ec; std::filesystem::create_directories(plan.outputDir, ec); @@ -1685,6 +1692,33 @@ std::expected NinjaBackend::build(const BuildPlan& plan std::chrono::steady_clock::now() - t0); if (ok) { + auto runtimeReport = + mcpp::build::runtime_validation::validate_changed_artifacts( + plan, runtimeBefore); + std::string runtimeFailure; + std::filesystem::path runtimeFailureArtifact; + for (auto const& checked : runtimeReport.artifacts) { + using Status = mcpp::platform::elf::RuntimeVerdict::Status; + auto explanation = checked.verdict.explain(); + if (checked.verdict.status == Status::ProvenMismatch) { + if (runtimeFailure.empty()) { + runtimeFailureArtifact = checked.artifact; + runtimeFailure = std::move(explanation); + } else if (!explanation.empty()) { + runtimeFailure += "\n" + explanation; + } + } else if (checked.verdict.status == Status::Inconclusive) { + mcpp::ui::warning(std::format( + "runtime closure validation is inconclusive for {}{}{}", + checked.artifact.string(), explanation.empty() ? "" : ":\n", + explanation)); + } + } + if (!runtimeFailure.empty()) { + return std::unexpected(BuildError{ + "runtime closure validation failed (proven Linux ELF mismatch)", + runtimeFailureArtifact, std::move(runtimeFailure)}); + } if (opts.verbose && !out.empty()) std::fputs(out.c_str(), stdout); std::set want(opts.ninjaTargets.begin(), opts.ninjaTargets.end()); diff --git a/src/build/plan.cppm b/src/build/plan.cppm index 795cc5b0..8100bf90 100644 --- a/src/build/plan.cppm +++ b/src/build/plan.cppm @@ -15,6 +15,9 @@ import mcpp.toolchain.dialect; import mcpp.toolchain.fingerprint; import mcpp.toolchain.triple; import mcpp.platform; +import mcpp.platform.runtime_binding; +import mcpp.platform.runtime_env_contract; +import mcpp.xlings.subos_info; export namespace mcpp::build { @@ -94,10 +97,32 @@ struct ResourceUnit { std::vector implicitInputs; }; +struct RuntimeCapabilityProvider { + std::string capability; + mcpp::manifest::PackageId provider; +}; + +struct ResolvedRuntimeContract { + std::vector requirements; + std::vector artifacts; + mcpp::manifest::LinkIntent linkIntent; + std::vector providers; +}; + +// Normalize structured and legacy runtime metadata and stamp every fact with +// the exact package identity that supplied it. Exported as a pure seam for +// contract tests and machine-readable tooling; it performs no host probing. +ResolvedRuntimeContract resolve_runtime_contract( + const std::vector& packages); + struct BuildPlan { mcpp::manifest::Manifest manifest; mcpp::toolchain::Toolchain toolchain; mcpp::toolchain::Fingerprint fingerprint; + // One immutable snapshot selected before workspace member substitution. + // Build/run/test and cache fast paths consume this value; none may re-read + // xlings active/current state. + mcpp::platform::runtime::RuntimeBinding runtimeBinding; std::string cppStandard = "c++23"; std::string cppStandardFlag = "-std=c++23"; // Module-graph-global dialect flags (issue #210), pre-joined with a @@ -150,6 +175,9 @@ struct BuildPlan { // binary's RUNPATH (e.g. compat.glx-runtime). Kept separate so static/musl // links don't pull the glibc payload dir. std::vector depRuntimeLibraryDirs; + std::vector runtimeRequirements; + std::vector runtimeArtifacts; + mcpp::manifest::LinkIntent linkIntent; // Windows runtime-DLL deployment. On PE (`supports_rpath` is false) a // directly-launched .exe cannot RUNPATH-locate a dependency's DLL, so each // *.dll found in a dependency's [runtime] library_dir is copied beside the @@ -171,13 +199,16 @@ struct BuildPlan { // (capability, provider package). A named aggregate instead of std::pair: // musl-gcc 15.1 modules failed to emit vector>'s // move-ctor instantiation across the module boundary (release link error). - struct CapabilityProvider { - std::string capability; - std::string provider; - }; - std::vector runtimeProviders; + std::vector runtimeProviders; }; +// Merge the generic facts exported by the already-selected xlings +// RuntimeBinding. This is data ingestion only: xlings has already selected +// providers and materialized artifacts before mcpp sees this snapshot. +void merge_runtime_binding_contract( + BuildPlan& plan, + const mcpp::platform::runtime::RuntimeBinding& binding); + // Is `p` inside one of `roots`, judged LEXICALLY? // // Lexical is the whole point (mcpp#344). std::filesystem::relative() runs @@ -452,6 +483,167 @@ void append_unique_path(std::vector& out, } // namespace +ResolvedRuntimeContract resolve_runtime_contract( + const std::vector& packages) +{ + ResolvedRuntimeContract out; + auto append_string = [](std::vector& values, std::string value) { + if (!value.empty() && std::ranges::find(values, value) == values.end()) + values.push_back(std::move(value)); + }; + auto absolute_from = [](const std::filesystem::path& root, + const std::filesystem::path& value) { + return (value.is_absolute() ? value : root / value).lexically_normal(); + }; + auto append_requirement = [&](mcpp::manifest::RuntimeRequirement value) { + const bool duplicate = std::ranges::any_of(out.requirements, + [&](auto const& existing) { + return existing.kind == value.kind + && existing.value == value.value + && existing.phase == value.phase + && existing.requester == value.requester + && existing.required == value.required; + }); + if (!duplicate) out.requirements.push_back(std::move(value)); + }; + + for (auto const& package : packages) { + const auto id = mcpp::manifest::package_id(package.manifest.package); + auto const& runtime = package.manifest.runtimeConfig; + + for (auto requirement : runtime.requirements) { + requirement.requester = id; + append_requirement(std::move(requirement)); + } + // One compatibility train: old soname/capability lists are normalized + // into the structured requirement shape. They do NOT imply provider + // ownership; only `provides` below creates a provider fact. + for (auto const& soname : runtime.dlopenLibs) { + append_requirement({ + .kind = "soname", .value = soname, .phase = "run", + .requester = id, .required = true, + }); + } + for (auto const& capability : runtime.capabilities) { + append_requirement({ + .kind = "capability", .value = capability, .phase = "run", + .requester = id, .required = true, + }); + } + + for (auto artifact : runtime.artifacts) { + artifact.provider = id; + artifact.path = absolute_from(package.root, artifact.path); + const bool duplicate = std::ranges::any_of(out.artifacts, + [&](auto const& existing) { + return existing.role == artifact.role + && existing.provider == artifact.provider + && existing.path == artifact.path + && existing.provenance == artifact.provenance + && existing.abi == artifact.abi + && existing.digest == artifact.digest + && existing.hostFingerprint == artifact.hostFingerprint; + }); + if (!duplicate) out.artifacts.push_back(std::move(artifact)); + } + + for (auto const& capability : runtime.provides) { + const bool duplicate = std::ranges::any_of(out.providers, + [&](auto const& existing) { + return existing.capability == capability + && existing.provider == id; + }); + if (!duplicate) out.providers.push_back({capability, id}); + } + + for (auto const& library : runtime.linkIntent.libraries) { + const std::filesystem::path asPath(library); + const bool explicitPath = !library.starts_with('-') + && (asPath.is_absolute() || asPath.has_parent_path() + || asPath.has_extension()); + append_string(out.linkIntent.libraries, + explicitPath ? absolute_from(package.root, asPath).string() + : library); + } + for (auto const& framework : runtime.linkIntent.frameworks) + append_string(out.linkIntent.frameworks, framework); + auto append_paths = [&](auto const& input, auto& output) { + for (auto const& path : input) + append_unique_path(output, absolute_from(package.root, path)); + }; + append_paths(runtime.linkIntent.linkLibraryDirs, + out.linkIntent.linkLibraryDirs); + append_paths(runtime.linkIntent.transitiveNeededDirs, + out.linkIntent.transitiveNeededDirs); + append_paths(runtime.linkIntent.runtimeSearchDirs, + out.linkIntent.runtimeSearchDirs); + append_paths(runtime.linkIntent.deployFiles, + out.linkIntent.deployFiles); + // Legacy library_dirs means run-time discovery only. It deliberately + // does not enter linkLibraryDirs; callers that need -L must opt into + // the structured field. + append_paths(runtime.libraryDirs, out.linkIntent.runtimeSearchDirs); + } + return out; +} + +void merge_runtime_binding_contract( + BuildPlan& plan, + const mcpp::platform::runtime::RuntimeBinding& binding) { + auto package_id = [](const mcpp::xlings::subos::PackageIdentity& value) { + return mcpp::manifest::PackageId{ + .namespace_ = value.namespace_, + .name = value.name, + .version = value.version, + .sourceProvenance = value.source, + }; + }; + + std::vector selected; + for (auto const& provider : binding.runtimeProviders) { + RuntimeCapabilityProvider value{ + .capability = provider.capability, + .provider = package_id(provider.provider), + }; + if (std::ranges::none_of(selected, [&](auto const& existing) { + return existing.capability == value.capability + && existing.provider == value.provider; + })) + selected.push_back(std::move(value)); + } + // xlings' selected providers are authoritative and therefore precede + // descriptor-declared fallback/provider facts. + for (auto it = selected.rbegin(); it != selected.rend(); ++it) { + if (std::ranges::none_of(plan.runtimeProviders, [&](auto const& existing) { + return existing.capability == it->capability + && existing.provider == it->provider; + })) + plan.runtimeProviders.insert(plan.runtimeProviders.begin(), *it); + } + + for (auto const& artifact : binding.runtimeArtifacts) { + mcpp::manifest::RuntimeArtifact value{ + .role = artifact.role, + .provider = package_id(artifact.provider), + .path = artifact.path.lexically_normal(), + .provenance = artifact.provenance, + .abi = artifact.abi, + .digest = artifact.digest, + .hostFingerprint = artifact.hostFingerprint, + }; + if (std::ranges::none_of(plan.runtimeArtifacts, [&](auto const& existing) { + return existing.role == value.role + && existing.provider == value.provider + && existing.path == value.path + && existing.provenance == value.provenance + && existing.abi == value.abi + && existing.digest == value.digest + && existing.hostFingerprint == value.hostFingerprint; + })) + plan.runtimeArtifacts.push_back(std::move(value)); + } +} + // True if `src` defines a top-level `int main(` / `auto main(` entry point. // Comments and string/char/raw-string literals are stripped first, so test // fixtures that embed `"int main() {...}"` or R"(int main(){})" don't @@ -628,56 +820,65 @@ make_plan(const mcpp::manifest::Manifest& manifest, plan.stdBmiPath = stdBmiPath; plan.stdObjectPath = stdObjectPath; - for (auto const& package : packages) { - for (auto const& dir : package.manifest.runtimeConfig.libraryDirs) { - auto abs = dir.is_absolute() ? dir : package.root / dir; - append_unique_path(plan.runtimeLibraryDirs, abs); - append_unique_path(plan.depRuntimeLibraryDirs, abs); - // Windows runtime-DLL deployment: stage each *.dll from this dir - // beside the produced executable (bin/). The *.dll filter — not a - // platform guard — keeps this inert for real .so/.dylib deps, so - // non-Windows builds are unchanged. See BuildPlan::DeployFile. - std::error_code dirEc; - if (std::filesystem::is_directory(abs, dirEc)) { - for (auto const& entry : - std::filesystem::directory_iterator(abs, dirEc)) { - if (!entry.is_regular_file()) continue; - auto ext = entry.path().extension().string(); - std::ranges::transform(ext, ext.begin(), - [](unsigned char c){ return std::tolower(c); }); - if (ext != ".dll") continue; - std::filesystem::path dest = - std::filesystem::path("bin") / entry.path().filename(); - if (std::ranges::none_of(plan.runtimeDeployFiles, - [&](auto const& d){ return d.dest == dest; })) - plan.runtimeDeployFiles.push_back({entry.path(), dest}); - } - } - } - for (auto const& lib : package.manifest.runtimeConfig.dlopenLibs) { - if (std::ranges::find(plan.runtimeDlopenLibs, lib) == plan.runtimeDlopenLibs.end()) - plan.runtimeDlopenLibs.push_back(lib); - } - for (auto const& cap : package.manifest.runtimeConfig.capabilities) { - if (std::ranges::find(plan.runtimeCapabilities, cap) == plan.runtimeCapabilities.end()) - plan.runtimeCapabilities.push_back(cap); + auto runtimeContract = resolve_runtime_contract(packages); + plan.runtimeRequirements = std::move(runtimeContract.requirements); + plan.runtimeArtifacts = std::move(runtimeContract.artifacts); + plan.linkIntent = std::move(runtimeContract.linkIntent); + plan.runtimeProviders = std::move(runtimeContract.providers); + + for (auto const& dir : plan.linkIntent.runtimeSearchDirs) { + append_unique_path(plan.runtimeLibraryDirs, dir); + append_unique_path(plan.depRuntimeLibraryDirs, dir); + } + for (auto const& requirement : plan.runtimeRequirements) { + // Optional requirements remain first-class provenance in + // resolution.json, but they cannot become hard ABI/doctor inputs. + if (!requirement.required) continue; + if (requirement.kind == "soname") { + if (std::ranges::find(plan.runtimeDlopenLibs, requirement.value) + == plan.runtimeDlopenLibs.end()) + plan.runtimeDlopenLibs.push_back(requirement.value); + } else if (requirement.kind == "capability") { + if (std::ranges::find(plan.runtimeCapabilities, requirement.value) + == plan.runtimeCapabilities.end()) + plan.runtimeCapabilities.push_back(requirement.value); } } - // Provider mapping (capability -> package), strongest first: packages - // that explicitly `provides` a capability win over packages that merely - // list it in `capabilities` (weak/back-compat providers). Downstream - // lookups take the first match. - for (auto const& package : packages) { - for (auto const& cap : package.manifest.runtimeConfig.provides) - plan.runtimeProviders.push_back({cap, package.manifest.package.name}); + + auto add_deploy = [&](const std::filesystem::path& source) + -> std::optional { + const auto normalized = source.lexically_normal(); + const auto dest = std::filesystem::path("bin") / source.filename(); + auto existing = std::ranges::find_if(plan.runtimeDeployFiles, + [&](auto const& value) { return value.dest == dest; }); + if (existing != plan.runtimeDeployFiles.end()) { + if (existing->source.lexically_normal() != normalized) { + return std::format( + "runtime deploy collision: '{}' and '{}' both target '{}'", + existing->source.string(), normalized.string(), dest.string()); + } + return std::nullopt; + } + plan.runtimeDeployFiles.push_back({normalized, dest}); + return std::nullopt; + }; + // Structured deploy files are explicit and platform-neutral. Legacy + // library_dirs keeps its one-train DLL discovery behavior below. + for (auto const& source : plan.linkIntent.deployFiles) { + if (auto collision = add_deploy(source)) + return std::unexpected(std::move(*collision)); } - for (auto const& package : packages) { - for (auto const& cap : package.manifest.runtimeConfig.capabilities) { - bool dup = false; - for (auto& pr : plan.runtimeProviders) - if (pr.capability == cap - && pr.provider == package.manifest.package.name) { dup = true; break; } - if (!dup) plan.runtimeProviders.push_back({cap, package.manifest.package.name}); + for (auto const& dir : plan.linkIntent.runtimeSearchDirs) { + std::error_code dirEc; + if (!std::filesystem::is_directory(dir, dirEc)) continue; + for (auto const& entry : std::filesystem::directory_iterator(dir, dirEc)) { + if (!entry.is_regular_file()) continue; + auto ext = entry.path().extension().string(); + std::ranges::transform(ext, ext.begin(), + [](unsigned char c){ return std::tolower(c); }); + if (ext != ".dll") continue; + if (auto collision = add_deploy(entry.path())) + return std::unexpected(std::move(*collision)); } } // The same private runtime directories embedded as executable RUNPATH are @@ -687,29 +888,36 @@ make_plan(const mcpp::manifest::Manifest& manifest, for (auto const& dir : tc.linkRuntimeDirs) { append_unique_path(plan.runtimeLibraryDirs, dir); } - // The private glibc payload is the ONE entry that is not also in the - // executable's RUNPATH (flags.cppm excludes it deliberately, so static and - // musl links stay clean). It is here purely so a dlopen()'d library — whose - // own DT_NEEDED closure never consults the main executable's RUNPATH — can - // still resolve the same libc the executable was linked against. + // The private glibc payload exists here for ONE reason: a dlopen()'d + // library, whose own DT_NEEDED closure never consults the main + // executable's RUNPATH, must still resolve the same libc the executable + // was linked against. So it is only published when this build actually has + // such a library. // - // So add it ONLY when this build actually has such a library. mcpp#291: + // It is published through the ARTIFACT — the link model already emits + // `-Wl,-rpath,` alongside --dynamic-linker wherever a payload + // exists — and never through the environment. // LD_LIBRARY_PATH is inherited by the whole process subtree, and a child - // that is a HOST binary (/bin/sh, reached via a provider's popen()) loads - // the HOST loader — PT_INTERP is baked into the executable and no - // environment variable can override it — while this variable hands it the - // payload libc.so.6. libc and ld.so are version-locked to each other - // through GLIBC_PRIVATE, so on any host whose glibc differs from the - // payload's the shell dies of SIGSEGV inside the dynamic linker, before - // main, with empty stdout and no diagnostic. (It does NOT reproduce when - // host and payload glibc happen to match, which is why this survived.) + // that is a HOST binary (/bin/sh, reached via popen()) loads the HOST + // loader — PT_INTERP is baked in and no environment variable overrides it + // — while the variable hands it the payload libc.so.6. libc and ld.so are + // version-locked through GLIBC_PRIVATE, so the shell dies during + // relocation, before main: glibc 2.44's libc.so.6 needs + // `__pointer_chk_guard` from its own loader (mcpp#401), and older payloads + // segfault in the linker instead (mcpp#291). It does NOT reproduce when + // host and payload glibc happen to match, which is how it survived. // // process.cppm's strip_private_glibc already removes this entry from - // mcpp's OWN children. It cannot help one hop further out: mcpp sets the - // variable for the target deliberately, and what the target then spawns is - // beyond mcpp's reach. Not emitting it unless it is needed is. - if (tc.payloadPaths && !plan.depRuntimeLibraryDirs.empty()) { - append_unique_path(plan.runtimeLibraryDirs, tc.payloadPaths->glibcLib); + // mcpp's OWN children. It cannot help one hop further out — what the + // target spawns is beyond mcpp's reach. Putting the directory in the + // artifact's RUNPATH instead is: DT_RUNPATH reaches the object that + // carries it and the dlopen() it performs, and nothing else. + if constexpr (mcpp::platform::publishes_via_environment( + mcpp::platform::kPrivateLibcSearchScope)) { + if (tc.payloadPaths && !plan.depRuntimeLibraryDirs.empty()) { + append_unique_path(plan.runtimeLibraryDirs, + tc.payloadPaths->glibcLib); + } } // 1a. Object addressing. diff --git a/src/build/prepare.cppm b/src/build/prepare.cppm index 3a7d2649..f38f4600 100644 --- a/src/build/prepare.cppm +++ b/src/build/prepare.cppm @@ -45,6 +45,8 @@ import mcpp.lockfile; import mcpp.config; import mcpp.xlings; import mcpp.xlings.subos_info; +import mcpp.xlings.runtime_selection; +import mcpp.platform.runtime_binding; import mcpp.toolchain.post_install; import mcpp.platform; import mcpp.fetcher; @@ -57,6 +59,7 @@ import mcpp.pm.index_refresh; import mcpp.pm.mangle; import mcpp.pm.compat; import mcpp.pm.dep_spec; +import mcpp.pm.dependency_selector; import mcpp.pm.lock_io; import mcpp.version_req; import mcpp.ui; @@ -305,6 +308,55 @@ std::string canonical_package_build_metadata( s += pkg.manifest.package.name; s += "@"; s += pkg.manifest.package.version; + s += " source="; + s += pkg.manifest.package.sourceProvenance; + auto const& runtime = pkg.manifest.runtimeConfig; + for (auto const& requirement : runtime.requirements) { + s += " runtime-need:"; + s += requirement.kind; + s += ':'; + s += requirement.value; + s += ':'; + s += requirement.phase; + s += requirement.required ? ":required" : ":optional"; + } + for (auto const& artifact : runtime.artifacts) { + s += " runtime-artifact:"; + s += artifact.role; + s += ':'; + s += artifact.path.generic_string(); + s += ':'; + s += artifact.provenance; + s += ':'; + s += artifact.abi; + s += ':'; + s += artifact.digest; + s += ':'; + s += artifact.hostFingerprint; + } + for (auto const& value : runtime.linkIntent.libraries) + s += " link-library:" + value; + for (auto const& value : runtime.linkIntent.linkLibraryDirs) + s += " link-dir:" + value.generic_string(); + for (auto const& value : runtime.linkIntent.transitiveNeededDirs) + s += " needed-dir:" + value.generic_string(); + for (auto const& value : runtime.linkIntent.runtimeSearchDirs) + s += " runtime-dir:" + value.generic_string(); + for (auto const& value : runtime.linkIntent.frameworks) + s += " framework:" + value; + for (auto const& value : runtime.linkIntent.deployFiles) + s += " deploy:" + value.generic_string(); + // Legacy fields remain fingerprinted while they are readable. + for (auto const& value : runtime.libraryDirs) + s += " legacy-runtime-dir:" + value.generic_string(); + for (auto const& value : runtime.dlopenLibs) + s += " legacy-soname:" + value; + for (auto const& value : runtime.capabilities) + s += " legacy-capability:" + value; + for (auto const& value : runtime.provides) + s += " legacy-provides:" + value; + for (auto const& [capability, provider] : runtime.providerOverrides) + s += " provider-override:" + capability + '=' + provider; if (!pkg.manifest.buildConfig.cStandard.empty()) { s += " c_standard="; s += pkg.manifest.buildConfig.cStandard; @@ -712,6 +764,8 @@ export struct BuildContext { mcpp::manifest::Manifest manifest; mcpp::toolchain::Toolchain tc; mcpp::toolchain::Fingerprint fp; + mcpp::xlings::runtime::RuntimeSelection runtimeSelection; + mcpp::platform::runtime::RuntimeBinding runtimeBinding; std::filesystem::path projectRoot; std::filesystem::path outputDir; std::filesystem::path stdBmi; @@ -838,6 +892,12 @@ export struct BuildOverrides { // mcpp.build.execute imported it). A pointer keeps the exported layout // trivial, and it also avoids copying the manifest per tool build. std::shared_ptr preloaded_manifest; + // Nested source/tool builds inherit the consumer root's local development + // OS. A dependency's own [xlings].subos is never consulted or propagated. + std::shared_ptr + inherited_runtime_selection; + std::shared_ptr + inherited_runtime_binding; std::string target_triple; // empty = host triple, fall through to [toolchain] bool force_static = false; // --static (or implied by musl target) std::string package_filter; // -p : only build this workspace member @@ -910,84 +970,6 @@ std::string with_index_cause(std::string msg) { msg += "\n" + hint; return msg; } -// The runtime this build targets, in xlings's own spelling ("glibc@2.39"). -// -// A degradation chain with every step explicit, and deliberately NO final -// "otherwise pick something". Payload resolution declines without an -// authority, because the guess it used to make is what let the compile side -// and the artifact's interpreter name different glibc versions -- invisibly, -// until the binary met a library built against the other one. -// -// 1. [xlings] subos -> that subos's subos_info.runtime -// 2. the active subos's subos_info.runtime -// 3. "" -- no authority, therefore no PayloadFirst -// -// `compilerBin` may be empty on the first call: the compiler has not been -// probed yet, and an inherited toolchain resolves its subos from its OWNER -// home. The caller re-resolves once it knows where the compiler is. -std::string resolve_runtime_binding(const mcpp::manifest::Manifest& m, - const std::filesystem::path& compilerBin) { - auto runtime_of = [](const std::filesystem::path& dir) -> std::string { - return mcpp::xlings::subos::read(dir).runtime; - }; - // A declaration can only be honoured while the payload it names is still - // installed. Subos descriptions are written once and the payloads beneath - // them are replaced independently -- upgraded, garbage-collected -- so a - // subos can go on saying `glibc@2.39` on a machine that now has only 2.44. - // - // Taking that at face value is not conservative, it is worse than any - // substitution: the exact-match probe finds nothing, no loader reaches the - // link line, and the artifact ends up on the HOST loader -- outside the - // sandbox entirely. Measured on CI, three rounds running. - // - // So a declaration naming an absent payload is passed over rather than - // returned, and resolution carries on to something that can be honoured. - auto usable = [&](const std::string& binding) { - if (binding.empty()) return false; - if (compilerBin.empty()) return true; // cannot check yet; caller re-asks - if (mcpp::toolchain::probe_payload_paths(compilerBin, binding)) - return true; - mcpp::log::verbose("probe", std::format( - "subos declares runtime {}, but that payload is not installed here " - "— looking for one that is", binding)); - return false; - }; - if (auto active = mcpp::xlings::paths::subos_dir_of(compilerBin)) { - // 1 — the project names a subos; it is a sibling of the active one. - if (!m.xlings.subos.empty()) { - auto named = active->parent_path() / m.xlings.subos; - if (auto r = runtime_of(named); usable(r)) return r; - } - // 2 — whatever is active. - if (auto r = runtime_of(*active); usable(r)) return r; - } - - // 3 — COMPATIBILITY: the value this toolchain already has baked in. - // - // A subos created before xlings grew `subos_info` cannot answer, and that - // is the state of every machine installed before 2026.8.5.1 -- including - // mcpp's own sandbox, whose vendored xlings is never upgraded. Refusing - // there would break every existing user, so the toolchain's own baked - // value stands in. - // - // This is NOT the guess this design removed. The guess picked a version by - // directory order, unrelated to what the artifact would load. This reads - // the value the artifact WILL use -- gcc's specs, clang's cfg -- so the - // invariant that matters, compile side == run side, still holds. It is a - // migration path, and it goes away on its own: once the subos describes - // itself, step 1 or 2 answers first. - if (!compilerBin.empty()) { - if (auto r = mcpp::toolchain::baked_runtime_binding(compilerBin); - !r.empty()) { - mcpp::log::verbose("probe", std::format( - "subos does not describe itself; using the runtime this " - "toolchain was installed against ({}). `xlings self update` " - "and a fresh subos make this authoritative", r)); - return r; - } - } - return {}; -} } // namespace export std::expected @@ -1027,6 +1009,7 @@ prepare_build(bool print_fingerprint, // If the manifest has [workspace] and is a virtual workspace (no [package]), // or if -p filter is set, switch to the target member's manifest. std::optional wsManifest; // keep workspace manifest alive + std::filesystem::path runtimeWorkspaceRoot; if (m->workspace.present) { std::string targetMember; @@ -1070,6 +1053,7 @@ prepare_build(bool print_fingerprint, return std::unexpected(std::format( "workspace member '{}' has no mcpp.toml", targetMember)); } + runtimeWorkspaceRoot = *root; wsManifest = std::move(*m); // preserve workspace manifest m = mcpp::manifest::load(memberDir / "mcpp.toml"); if (!m) return std::unexpected(std::format( @@ -1105,23 +1089,37 @@ prepare_build(bool print_fingerprint, if (!wsRoot.empty()) { auto wsm = mcpp::manifest::load(wsRoot / "mcpp.toml"); if (wsm && wsm->workspace.present) { + runtimeWorkspaceRoot = wsRoot; + wsManifest = std::move(*wsm); // #224: anchor relative `path`/`[indices].path` to the // workspace root, not this member's own directory. - mcpp::project::merge_workspace_deps(*m, *wsm, wsRoot); + mcpp::project::merge_workspace_deps(*m, *wsManifest, wsRoot); if (m->toolchain.byPlatform.empty()) { - m->toolchain = wsm->toolchain; + m->toolchain = wsManifest->toolchain; } - for (auto& [triple, entry] : wsm->targetOverrides) { + for (auto& [triple, entry] : wsManifest->targetOverrides) { if (!m->targetOverrides.contains(triple)) { m->targetOverrides[triple] = entry; } } // Inherit workspace indices if member doesn't define any - mcpp::project::inherit_workspace_indices(*m, *wsm, wsRoot); + mcpp::project::inherit_workspace_indices(*m, *wsManifest, wsRoot); } } } + mcpp::xlings::runtime::RuntimeSelection runtimeSelection; + if (overrides.inherited_runtime_selection) { + runtimeSelection = *overrides.inherited_runtime_selection; + } else { + std::optional> wsRef; + if (wsManifest) wsRef = std::cref(*wsManifest); + auto selected = mcpp::xlings::runtime::select_runtime( + *m, wsRef, *root, runtimeWorkspaceRoot); + if (!selected) return std::unexpected(selected.error()); + runtimeSelection = std::move(*selected); + } + // Where mcpp WRITES — derived here because `root` is only final now: the // workspace block above may have moved it to the selected member. Defaults // to the project root, so every existing invocation is byte-for-byte @@ -1133,6 +1131,10 @@ prepare_build(bool print_fingerprint, std::error_code wdEc; std::filesystem::create_directories(workRoot, wdEc); } + if (m->package.sourceProvenance.empty()) { + m->package.sourceProvenance = + "path+" + root->lexically_normal().generic_string(); + } // A `compat`-form (Form B) package's sources live under a wrap directory // inside the version dir, which is why its descriptor writes globs like @@ -1174,13 +1176,18 @@ prepare_build(bool print_fingerprint, // fetched. Keyed by package name — the same key the writer at the end of // this function emits, both taken from the root manifest's [dependencies]. std::map gitLockAnchors; + std::map packageIdentityLockAnchors; { auto lockPath = workRoot / "mcpp.lock"; if (std::filesystem::exists(lockPath)) { if (auto lock = mcpp::pm::load(lockPath); lock) { - for (auto const& p : lock->packages) + for (auto const& p : lock->packages) { + if (!p.namespace_.empty()) + packageIdentityLockAnchors.emplace( + p.name, p.namespace_); if (auto parsed = mcpp::pm::parse_git_source(p.source); parsed) gitLockAnchors.emplace(p.name, std::move(*parsed)); + } } else { // Degraded, not a plain warning: the engine silently does less // than asked — every git branch dep falls back to `ls-remote` @@ -1250,6 +1257,25 @@ prepare_build(bool print_fingerprint, return &*cfg_opt; }; + // Resolve one exact runtime contract before resolving/fixing a toolchain. + // The fixup is itself a consumer of RuntimeBinding: doing it first would + // recreate #392 by letting directory order choose a libc and only later + // discovering what the project selected. + mcpp::platform::runtime::RuntimeBinding runtimeBindingSnapshot; + if (overrides.inherited_runtime_binding) { + runtimeBindingSnapshot = *overrides.inherited_runtime_binding; + } else { + auto cfgRuntime = get_cfg(); + if (!cfgRuntime) return std::unexpected(cfgRuntime.error()); + auto resolved = mcpp::platform::runtime::resolve_runtime_binding( + runtimeSelection, {}, **cfgRuntime); + if (!resolved) return std::unexpected(resolved.error()); + runtimeBindingSnapshot = std::move(*resolved); + } + const auto runtimePayload = runtimeBindingSnapshot.libc.value_or(""); + const auto runtimeLibDir = runtimeBindingSnapshot.libraryDirs.empty() + ? std::filesystem::path{} : runtimeBindingSnapshot.libraryDirs.front(); + constexpr std::string_view kCurrentPlatform = mcpp::platform::name; // M5.5: toolchain resolution priority: @@ -1545,7 +1571,11 @@ prepare_build(bool print_fingerprint, // Same post-install fixup as `mcpp toolchain install` — this manifest // [toolchain] path previously ran none, so a freshly auto-installed // payload kept its stale install-time cfg / unpatched runtime libs. - mcpp::toolchain::ensure_post_install_fixup(**cfg, payload->root, pkg); + if (auto fixed = mcpp::toolchain::ensure_post_install_fixup( + **cfg, payload->root, pkg, + runtimeBindingSnapshot.runtimeId, runtimeLibDir); !fixed) + return std::unexpected(std::format( + "toolchain post-install fixup: {}", fixed.error())); // Canonical rendering, whatever spelling the manifest/config used: // "Resolved gcc@16.1.0 → x86_64-linux-musl → ". mcpp::ui::info("Resolved", @@ -1690,7 +1720,11 @@ prepare_build(bool print_fingerprint, // `mcpp toolchain install` performs — without it a fresh sandbox // gcc cannot find the C library (stdlib.h: No such file or // directory) and a fresh llvm keeps its stale install-time cfg. - mcpp::toolchain::ensure_post_install_fixup(**cfg, payload->root, defaultPkg); + if (auto fixed = mcpp::toolchain::ensure_post_install_fixup( + **cfg, payload->root, defaultPkg, + runtimeBindingSnapshot.runtimeId, runtimeLibDir); !fixed) + return std::unexpected(std::format( + "default toolchain post-install fixup: {}", fixed.error())); // Persist the default so we don't ask again next time. if (auto wr = mcpp::config::write_default_toolchain(**cfg, defaultSpec); wr) { @@ -1723,27 +1757,9 @@ prepare_build(bool print_fingerprint, tcOrigin = TcOrigin::FirstRun; } - // The authority for "which libc", resolved before the toolchain is - // probed: payload paths are addresses for a version this names. - auto runtimeBinding = resolve_runtime_binding(*m, {}); - auto tc = mcpp::toolchain::detect(explicit_compiler, runtimeBinding); + auto tc = mcpp::toolchain::detect( + explicit_compiler, runtimePayload, runtimeBindingSnapshot.contractHash); if (!tc) return std::unexpected(tc.error().message); - // The first pass has no compiler, so it can neither discover a binding - // that only the compiler's own home knows (an inherited toolchain resolves - // into its owner) nor check that a declared one is still installed. Both - // failures look identical from here: no payload paths. Re-ask with the - // compiler in hand whenever that is what happened. - // - // The old condition was `runtimeBinding.empty()`, which meant a subos - // naming a payload that had since been upgraded away won on the first pass - // and the second pass never ran. - if (!tc->payloadPaths) { - auto fromCompiler = resolve_runtime_binding(*m, tc->binaryPath); - if (!fromCompiler.empty() && fromCompiler != runtimeBinding) { - tc = mcpp::toolchain::detect(explicit_compiler, fromCompiler); - if (!tc) return std::unexpected(tc.error().message); - } - } // ── Targeting the MSVC ABI without a usable MSVC ───────────────────── // @@ -1828,7 +1844,11 @@ prepare_build(bool print_fingerprint, "MinGW-w64 payload {} has no known C++ frontend in {}", gnuPkg.target(), payloadR->binDir.string())); } - mcpp::toolchain::ensure_post_install_fixup(**cfgR, payloadR->root, gnuPkg); + if (auto fixed = mcpp::toolchain::ensure_post_install_fixup( + **cfgR, payloadR->root, gnuPkg, + runtimeBindingSnapshot.runtimeId, runtimeLibDir); !fixed) + return std::unexpected(std::format( + "MinGW toolchain post-install fixup: {}", fixed.error())); // Persist both axes so the repair happens once, not on every build. if (mcpp::config::write_default_toolchain(**cfgR, pins::kFirstRunWinGnu)) @@ -1838,7 +1858,9 @@ prepare_build(bool print_fingerprint, tcSpec = std::string(pins::kFirstRunWinGnu); tcOrigin = TcOrigin::FirstRun; - tc = mcpp::toolchain::detect(explicit_compiler, runtimeBinding); + tc = mcpp::toolchain::detect( + explicit_compiler, runtimePayload, + runtimeBindingSnapshot.contractHash); if (!tc) return std::unexpected(tc.error().message); } @@ -1932,7 +1954,11 @@ prepare_build(bool print_fingerprint, "host toolchain payload '{}' has no known C++ frontend in {}", pkg.target(), payload->binDir.string())); } - mcpp::toolchain::ensure_post_install_fixup(**cfgH, payload->root, pkg); + if (auto fixed = mcpp::toolchain::ensure_post_install_fixup( + **cfgH, payload->root, pkg, + runtimeBindingSnapshot.runtimeId, runtimeLibDir); !fixed) + return std::unexpected(std::format( + "host toolchain post-install fixup: {}", fixed.error())); auto htc = mcpp::toolchain::detect(frontend); if (!htc) return std::unexpected(htc.error().message); mcpp::ui::info("Resolved", std::format( @@ -1998,15 +2024,32 @@ prepare_build(bool print_fingerprint, // [xlings] build environment (L-1). This creates .mcpp/.xlings.json with // custom non-builtin index entries (so xlings can clone them) plus the // [xlings] deps/workspace/subos/envs materialized verbatim. - if (!m->indices.empty() || !m->xlings.empty()) { + const auto& runtimeOwnerManifest = wsManifest ? *wsManifest : *m; + const bool materializeRootRuntime = + !overrides.inherited_runtime_binding && !runtimeOwnerManifest.xlings.empty(); + if (!m->indices.empty() || materializeRootRuntime) { auto cfg2 = get_cfg(); if (cfg2) { mcpp::xlings::ProjectEnv penv; - penv.deps = m->xlings.deps; - penv.subos = m->xlings.subos; - for (auto const& [k, v] : m->xlings.workspace) penv.workspace.emplace_back(k, v); - for (auto const& [k, v] : m->xlings.envs) penv.envs.emplace_back(k, v); - mcpp::config::ensure_project_index_dir(**cfg2, workRoot, m->indices, penv); + if (materializeRootRuntime) { + penv.deps = runtimeOwnerManifest.xlings.deps; + penv.subos = runtimeOwnerManifest.xlings.subos; + for (auto const& [k, v] : runtimeOwnerManifest.xlings.workspace) + penv.workspace.emplace_back(k, v); + for (auto const& [k, v] : runtimeOwnerManifest.xlings.envs) + penv.envs.emplace_back(k, v); + } + if (runtimeSelection.ownerRoot == workRoot) { + mcpp::config::ensure_project_index_dir( + **cfg2, workRoot, m->indices, penv); + } else { + if (!m->indices.empty()) + mcpp::config::ensure_project_index_dir( + **cfg2, workRoot, m->indices, {}); + if (materializeRootRuntime) + mcpp::config::ensure_project_index_dir( + **cfg2, runtimeSelection.ownerRoot, {}, penv); + } // On first build, the project index data root may be empty because // ensure_project_index_dir only writes .xlings.json but does not @@ -2186,14 +2229,21 @@ prepare_build(bool print_fingerprint, }; auto xpkgLuaMatchesCandidate = - [](const mcpp::pm::DependencyCoordinate& coord, - std::string_view luaContent, - bool allowLegacyBareDefault) { + [&](const mcpp::pm::DependencyCoordinate& coord, + std::string_view luaContent, + bool allowLegacyBareDefault) { // Single source of truth: the descriptor identity gate lives in - // mcpp.manifest and is shared with the read_xpkg_lua family. + // mcpp.manifest and is shared with the read_xpkg_lua family. A + // descriptor served by a declared project index inherits that + // index's namespace when package.namespace is omitted; preserve + // the same owner context during this second, stricter check. + const auto route = index_route(); + const auto* owner = route.find_for_ns(coord.namespace_); + const std::string_view ownerNs = owner + ? std::string_view{owner->name} : std::string_view{}; return mcpp::manifest::xpkg_lua_identity_matches( luaContent, coord.namespace_, coord.shortName, - allowLegacyBareDefault); + allowLegacyBareDefault, ownerNs); }; auto dependencyCoordinates = @@ -2210,6 +2260,8 @@ prepare_build(bool print_fingerprint, return out; }; + std::set selectorMigrationWarnings; + auto selectDependencyCandidate = [&](mcpp::manifest::DependencySpec& spec, const std::string& depName) -> std::expected @@ -2221,12 +2273,74 @@ prepare_build(bool print_fingerprint, "dependency '{}' has no lookup candidates", depName))); } + // One release train of migration support for the former dotted + // candidate search. A lockfile records the identity an existing + // project already selected, so keep that identity stable until the + // user rewrites the selector explicitly. Without a lock anchor, never + // fall back: only diagnose a valid old-primary package and continue + // with the new exact coordinate. + if (spec.legacyCandidateSearch) { + const auto exact = candidates.front(); + bool lockExpressesIntent = false; + if (auto locked = packageIdentityLockAnchors.find(depName); + locked != packageIdentityLockAnchors.end()) { + lockExpressesIntent = true; + if (locked->second != exact.namespace_) { + mcpp::pm::DependencyCoordinate lockedCoordinate{ + .namespace_ = locked->second, + .shortName = exact.shortName, + }; + if (selectorMigrationWarnings.insert(depName).second) { + mcpp::ui::warning(std::format( + "dependency selector '{}' now means exact package " + "'{}', but mcpp.lock records '{}'; keeping the " + "locked identity for this migration release. " + "Write '{}' to keep it explicitly, or remove the " + "lock and keep '{}' to migrate", + depName, + mcpp::pm::format_package_selector(exact), + mcpp::pm::format_package_selector(lockedCoordinate), + mcpp::pm::format_package_selector(lockedCoordinate), + mcpp::pm::format_package_selector(exact))); + } + candidates.assign(1, std::move(lockedCoordinate)); + } + } + + if (!lockExpressesIntent && spec.isVersion()) { + if (auto old = mcpp::pm::legacy_prefixed_coordinate(exact)) { + auto oldLua = readStrictLuaForCandidate(*old); + if (oldLua && xpkgLuaMatchesCandidate( + *old, *oldLua, + /*allowLegacyBareDefault=*/false) + && selectorMigrationWarnings.insert(depName).second) { + mcpp::ui::warning(std::format( + "dependency selector '{}' now resolves exactly to " + "'{}'; an older mcpp would select the existing " + "package '{}'. Write '{}' to keep the old identity " + "or keep '{}' for the new exact identity", + depName, + mcpp::pm::format_package_selector(exact), + mcpp::pm::format_package_selector(*old), + mcpp::pm::format_package_selector(*old), + mcpp::pm::format_package_selector(exact))); + } + } + } + } + auto selected = candidates.front(); bool matched = false; - if (spec.isVersion() && candidates.size() > 1) { + if (spec.isVersion()) { for (auto& candidate : candidates) { auto lua = readStrictLuaForCandidate(candidate); - if (!lua || !xpkgLuaMatchesCandidate( + if (!lua) continue; + if (auto violation = mcpp::manifest:: + xpkg_name_form_violation_from_lua(*lua)) { + return std::unexpected(std::format( + "dependency '{}': {}", depName, *violation)); + } + if (!xpkgLuaMatchesCandidate( candidate, *lua, /*allowLegacyBareDefault=*/false)) { continue; } @@ -2267,6 +2381,70 @@ prepare_build(bool print_fingerprint, break; } + // One-release bare-name migration. Namespace omission means exactly + // `mcpplibs`, but every published `compat.*` package and every user + // manifest written before this release spells the dependency bare — + // `gtest = "1.15.2"`, `ftxui = "6.1.9"`. Making that an immediate + // hard error means an mcpp upgrade breaks builds against data that + // is already published and cannot be edited retroactively; the + // symmetric rule ("published data must not break the program") is + // why the index floor degrades instead of bricking. + // + // The defect #278 removed was the SILENCE, not the reach: mcpp used + // to continue with a namespace the user never wrote and never say + // so. A hit here is announced, is recorded downstream under its + // canonical identity, and names the exact edit that removes the + // warning. Only a selector whose namespace was OMITTED is eligible — + // `mcpplibs.gtest` states an identity and must still miss. + if (!matched && spec.isVersion() && spec.namespaceOmitted) { + for (auto& legacy : + mcpp::pm::legacy_bare_candidates(candidates.front())) { + auto lua = readStrictLuaForCandidate(legacy); + if (!lua) continue; + if (mcpp::manifest::xpkg_name_form_violation_from_lua(*lua)) + continue; + if (!xpkgLuaMatchesCandidate( + legacy, *lua, /*allowLegacyBareDefault=*/false)) + continue; + auto declaredNs = + mcpp::manifest::extract_xpkg_namespace(*lua); + // Same narrowing as the exact loop: the namespace-less rung + // is "upstream package that declares no namespace", not a + // cross-namespace wildcard. + if (legacy.namespace_.empty() && !declaredNs.empty()) + continue; + + selected = legacy; + if (selected.namespace_.empty()) + selected.namespace_ = declaredNs; + matched = true; + // Downstream — lock, install, cache label — must see the + // canonical identity, so the ambiguous spelling survives in + // exactly one place: the user's manifest, until they edit it. + candidates.assign(1, selected); + + if (selectorMigrationWarnings.insert(depName).second) { + mcpp::ui::warning(std::format( + "dependency '{}' resolved to '{}' through the " + "deprecated bare-name search; namespace omission " + "means `{}` only. Write the exact package:" + "\n [dependencies.{}]" + "\n {} = \"{}\"" + "\n (or run `mcpp add {}@{}`). This fallback is " + "removed in {}.", + depName, + mcpp::pm::format_package_selector(selected), + mcpp::pm::kDefaultNamespace, + selected.namespace_, selected.shortName, + spec.version, + mcpp::pm::format_package_selector(selected), + spec.version, + mcpp::pm::kBareNameFallbackRemovedIn)); + } + break; + } + } + // A custom GIT index is cloned lazily by xlings during install, so // at selection time its descriptors may legitimately not be on disk // yet. "Not found" is therefore not conclusive for those namespaces @@ -2279,11 +2457,9 @@ prepare_build(bool print_fingerprint, return index_route().lazy_git(c.namespace_); }); - // T9 (#278) — no candidate resolved. This used to fall through to - // `candidates.front()` SILENTLY, so mcpp carried on with a namespace - // it had invented, and the user met the failure much later (during - // download/install) wrapped around that invented name. Fail here, - // and say exactly which identities were tried. + // An exact coordinate that a readable index cannot serve fails at + // resolution. Never carry it into install-time compatibility + // retries, which would reintroduce cross-namespace guessing. if (!matched && !anyLazyGitIndex) { std::string tried; for (auto& c : candidates) { @@ -2304,21 +2480,19 @@ prepare_build(bool print_fingerprint, hint += "\n a package with this name exists under " "another namespace:"; for (auto& fqn : fqns) hint += "\n " + fqn; - hint += std::format( - "\n bare names only resolve to the `{}` / `{}` " - "namespaces. write it out:" - "\n [dependencies]" - "\n \"{}\" = \"{}\"" - "\n or:" - "\n [dependencies.{}]" - "\n {} = \"{}\"", - mcpp::pm::kDefaultNamespace, - mcpp::pm::kCompatNamespace, - fqns.front(), spec.version.empty() ? "" - : spec.version, - fqns.front().substr(0, fqns.front().rfind('.')), - fqns.front().substr(fqns.front().rfind('.') + 1), - spec.version.empty() ? "" : spec.version); + if (auto suggested = mcpp::pm::parse_package_selector( + fqns.front()); suggested + && suggested->namespace_) { + hint += std::format( + "\n namespace omission means `{}`. write the " + "exact package:" + "\n [dependencies.{}]" + "\n {} = \"{}\"", + mcpp::pm::kDefaultNamespace, + *suggested->namespace_, suggested->name, + spec.version.empty() ? "" + : spec.version); + } } } @@ -2333,10 +2507,10 @@ prepare_build(bool print_fingerprint, mcpp::pm::staleness_note( mcpp::config::make_xlings_env(**cfgA))); } - return std::unexpected(std::format( - "dependency '{}': no package found under the namespaces " - "mcpp searched\n tried: {}{}", - depName, tried, hint)); + return std::unexpected(with_index_cause(std::format( + "dependency '{}': no package found for exact selector" + "\n tried: {}{}", + depName, tried, hint))); } } @@ -2413,7 +2587,7 @@ prepare_build(bool print_fingerprint, } if (field.kind == mcpp::manifest::McppField::TableBody) { auto dm = mcpp::manifest::synthesize_from_xpkg_lua( - *luaContent, depName, version, targetPlatform); + *luaContent, shortName, version, targetPlatform); if (!dm) return false; for (auto const& [generatedPath, _] : dm->buildConfig.generatedFiles) { if (!generatedPath.empty()) return true; @@ -2477,7 +2651,7 @@ prepare_build(bool print_fingerprint, auto field = mcpp::manifest::extract_mcpp_field(*luaContent); if (field.kind == mcpp::manifest::McppField::TableBody) { auto depManifest = mcpp::manifest::synthesize_from_xpkg_lua( - *luaContent, depName, version, targetPlatform); + *luaContent, shortName, version, targetPlatform); if (!depManifest) { return std::unexpected(std::format( "dependency '{}': {}", depName, depManifest.error().format())); @@ -2713,7 +2887,7 @@ prepare_build(bool print_fingerprint, if (auto r = loadFrom(matches.front()); !r) return std::unexpected(r.error()); } else if (field.kind == mcpp::manifest::McppField::TableBody) { auto dm = mcpp::manifest::synthesize_from_xpkg_lua( - *luaContent, depName, version, targetPlatform); + *luaContent, shortName, version, targetPlatform); if (!dm) return std::unexpected(std::format( "dependency '{}': {}", depName, dm.error().format())); warn_unknown_xpkg_keys(*dm, depName); @@ -3184,25 +3358,11 @@ prepare_build(bool print_fingerprint, } }; - // Stage a dep's source files into a fresh directory, rewriting their - // module / import declarations against `rename`. Used by the multi- - // version mangling fallback (Level 1) so two cross-major copies of - // the same package can coexist with distinct module names. - // - // Headers (referenced via `[build].include_dirs`) are NOT staged — - // those keep pointing at the original install dir via absolutized - // include paths. - auto stage_with_rewrite = [](const std::filesystem::path& srcRoot, - const std::filesystem::path& dstRoot, - const mcpp::manifest::Manifest& depManifest, - const std::map& rename) - -> std::expected + auto package_source_files = []( + const std::filesystem::path& srcRoot, + const mcpp::manifest::Manifest& depManifest) + -> std::expected, std::string> { - std::error_code ec; - std::filesystem::create_directories(dstRoot, ec); - if (ec) return std::unexpected(std::format( - "stage: cannot create '{}': {}", dstRoot.string(), ec.message())); - // Resolve the source globs against the original root, falling // back to the convention default if the manifest didn't set any. std::vector globs = depManifest.modules.sources; @@ -3228,8 +3388,32 @@ prepare_build(bool print_fingerprint, "stage: no source files found under '{}' (globs={})", srcRoot.string(), globs.size())); } + return sourceFiles; + }; + + // Stage a dep's source files into a fresh directory, rewriting their + // module / import declarations against `rename`. Used by the multi- + // version mangling fallback (Level 1) so two cross-major copies of + // the same package can coexist with distinct module names. + // + // Headers (referenced via `[build].include_dirs`) are NOT staged — + // those keep pointing at the original install dir via absolutized + // include paths. + auto stage_with_rewrite = [&](const std::filesystem::path& srcRoot, + const std::filesystem::path& dstRoot, + const mcpp::manifest::Manifest& depManifest, + const std::map& rename) + -> std::expected + { + std::error_code ec; + std::filesystem::create_directories(dstRoot, ec); + if (ec) return std::unexpected(std::format( + "stage: cannot create '{}': {}", dstRoot.string(), ec.message())); + + auto sources = package_source_files(srcRoot, depManifest); + if (!sources) return std::unexpected(sources.error()); - for (auto const& f : sourceFiles) { + for (auto const& f : *sources) { auto rel = std::filesystem::relative(f, srcRoot, ec); if (ec) return std::unexpected(std::format( "stage: cannot relativize '{}': {}", f.string(), ec.message())); @@ -3251,6 +3435,29 @@ prepare_build(bool print_fingerprint, return {}; }; + auto declared_modules_for = [&](const std::filesystem::path& srcRoot, + const mcpp::manifest::Manifest& depManifest) + -> std::expected, std::string> + { + auto sources = package_source_files(srcRoot, depManifest); + if (!sources) return std::unexpected(sources.error()); + std::vector modules; + for (auto const& file : *sources) { + std::ifstream is(file); + if (!is) return std::unexpected(std::format( + "mangle: cannot read '{}'", file.string())); + std::stringstream buf; buf << is.rdbuf(); + for (auto& name : mcpp::pm::declared_module_roots(buf.str())) { + if (std::ranges::find(modules, name) == modules.end()) + modules.push_back(std::move(name)); + } + } + if (modules.empty()) return std::unexpected(std::format( + "mangle: package '{}' declares no named C++ module to rewrite", + depManifest.package.name)); + return modules; + }; + // Stage 2a — feature-activated optional dependencies. Defined as local // lambdas (NOT file-scope functions): keeping their std::map instantiations // inside this implementation unit avoids polluting the exported module BMI, @@ -3502,14 +3709,22 @@ prepare_build(bool print_fingerprint, spec.version)); } - // Module names in the source files use the dep's full - // [package].name (e.g. "mcpplibs.cmdline"), not the - // namespaced-subtable shortName. Use that for the - // rename key so the rewriter actually matches what the - // .cppm sources declare. - const std::string moduleName = secondaryManifest.package.name; - std::string mangled = - mcpp::pm::mangle_name(moduleName, spec.version); + // Module names are authored API and are not required to + // mirror package identity. Discover every provided module + // root from the secondary's source text, then rewrite the + // same map in both the secondary and its consumer. + auto moduleNames = declared_modules_for( + secondaryRoot, secondaryManifest); + if (!moduleNames) return std::unexpected(moduleNames.error()); + std::map rename; + for (auto const& module : *moduleNames) { + rename.emplace(module, + mcpp::pm::mangle_name(module, spec.version)); + } + const auto& moduleName = moduleNames->front(); + const auto& mangledModule = rename.at(moduleName); + const std::string mangledPackage = mcpp::pm::mangle_name( + key.shortName, spec.version); // Stage layout: // /target/.mangled//__/ ← rewritten secondary source @@ -3519,10 +3734,9 @@ prepare_build(bool print_fingerprint, auto stageBase = *root / "target" / ".mangled" / consumerManifest.package.name; auto secStage = stageBase - / std::format("{}__{}", moduleName, spec.version); + / std::format("{}__{}", key.shortName, spec.version); auto consumerStage = stageBase / "__self__"; - std::map rename{ {moduleName, mangled} }; if (auto r = stage_with_rewrite(secondaryRoot, secStage, secondaryManifest, rename); !r) return std::unexpected(r.error()); @@ -3539,11 +3753,16 @@ prepare_build(bool print_fingerprint, // exact (ns, mangled) pair dedup cleanly. The original // primary entry (it->second) is untouched. auto stagedManifest = secondaryManifest; - // Update [package].name to the mangled module name so - // the modgraph validator (which checks "exported module - // must be prefixed by package name") accepts the - // rewritten sources. - stagedManifest.package.name = mangled; + // Give the staged package a distinct atomic identity too; + // authored module names remain independent and are carried + // exclusively by the rename map above. + stagedManifest.package.name = mangledPackage; + if (stagedManifest.package.namespace_.empty()) { + stagedManifest.package.namespace_ = key.ns.empty() + ? std::string(mcpp::pm::kDefaultNamespace) : key.ns; + } + stagedManifest.package.sourceProvenance = std::format( + "index+{}@{}", cache_index_name(key.ns), spec.version); // Absolutize secondary's include_dirs against its original // install root so the staged copy still finds headers. for (auto& inc : stagedManifest.buildConfig.includeDirs) { @@ -3557,7 +3776,7 @@ prepare_build(bool print_fingerprint, std::make_unique(std::move(stagedManifest))); dep_cache_identities.push_back({ .indexName = cache_index_name(key.ns), - .packageName = mangled, + .packageName = mangledPackage, .version = spec.version, .sourceKind = "version", }); @@ -3566,7 +3785,7 @@ prepare_build(bool print_fingerprint, recordDependencyEdge(item.consumerDepIndex, depPackageIndex, spec); auto linkFlagsAdded = propagateLinkFlags(secStage, *dep_manifests.back()); - ResolvedKey mangledKey{key.ns, mangled}; + ResolvedKey mangledKey{key.ns, mangledPackage}; resolved[mangledKey] = ResolvedRecord{ .version = spec.version, .constraint = item.originalConstraint, @@ -3579,7 +3798,8 @@ prepare_build(bool print_fingerprint, mcpp::ui::info("Mangled", std::format("{} v{} ↔ v{} → {} (cross-major fallback)", - moduleName, it->second.version, spec.version, mangled)); + moduleName, it->second.version, spec.version, + mangledModule)); continue; } @@ -3638,6 +3858,12 @@ prepare_build(bool print_fingerprint, expectedShort)); } } + if (newManifest.package.namespace_.empty()) { + newManifest.package.namespace_ = key.ns.empty() + ? std::string(mcpp::pm::kDefaultNamespace) : key.ns; + } + newManifest.package.sourceProvenance = std::format( + "index+{}@{}", cache_index_name(key.ns), *merged); removeLinkFlags(it->second.linkFlagsAdded); auto linkFlagsAdded = propagateLinkFlags(newRoot, newManifest); @@ -3913,6 +4139,26 @@ prepare_build(bool print_fingerprint, } } + // Stamp the identity with the resolver's exact coordinate and source. + // A descriptor that omitted namespace inherits the coordinate that + // answered it; otherwise two indices containing the same short name + // collapse in runtime provenance even though resolution distinguished + // them correctly. + if (dep_manifest->package.namespace_.empty()) { + dep_manifest->package.namespace_ = key.ns.empty() + ? std::string(mcpp::pm::kDefaultNamespace) : key.ns; + } + if (sourceKind == "version") { + dep_manifest->package.sourceProvenance = std::format( + "index+{}@{}", cache_index_name(key.ns), spec.version); + } else if (sourceKind == "git") { + dep_manifest->package.sourceProvenance = std::format( + "git+{}#{}={}", spec.git, spec.gitRefKind, spec.gitRev); + } else { + dep_manifest->package.sourceProvenance = + "path+" + dep_root.lexically_normal().generic_string(); + } + // Stage 2a: merge this dependency's active feature-deps into its own // dependency set before its children are pushed, so a dep's feature can // transitively pull a provider. `spec.features` = features the consumer @@ -4471,6 +4717,12 @@ prepare_build(bool print_fingerprint, sub.preloaded_manifest = std::make_shared( *dep_manifests[depIdx - 1]); + sub.inherited_runtime_selection = std::make_shared< + const mcpp::xlings::runtime::RuntimeSelection>( + runtimeSelection); + sub.inherited_runtime_binding = std::make_shared< + const mcpp::platform::runtime::RuntimeBinding>( + runtimeBindingSnapshot); sub.tool_chain = overrides.tool_chain.empty() ? std::format("root → {}:{}", depName, toolName) : std::format("{} → {}:{}", overrides.tool_chain, depName, @@ -4983,6 +5235,8 @@ prepare_build(bool print_fingerprint, ctx.manifest = *m; ctx.tc = *tc; ctx.fp = fp; + ctx.runtimeSelection = runtimeSelection; + ctx.runtimeBinding = runtimeBindingSnapshot; ctx.profile = effectiveProfile; ctx.cacheMode = cacheMode; ctx.projectRoot= *root; @@ -5010,6 +5264,9 @@ prepare_build(bool print_fingerprint, stdBmiPath, stdObjectPath, storeRoots); if (!planResult) return std::unexpected(planResult.error()); ctx.plan = std::move(*planResult); + ctx.plan.runtimeBinding = runtimeBindingSnapshot; + mcpp::build::merge_runtime_binding_contract( + ctx.plan, runtimeBindingSnapshot); ctx.plan.compileDbPath = workRoot / "compile_commands.json"; // GCC: a clean `*link:` for this build, so the payload's specs cannot // inject other homes' rpath entries into the artifact. AFTER the plan is @@ -5934,23 +6191,47 @@ prepare_build(bool print_fingerprint, } } - // Apply [runtime.] provider = "" overrides: prefer the - // named provider for matching capabilities (capability name prefix match). - // Warn if the named provider isn't in the dependency graph. + // Apply [runtime.] provider = "" overrides. Canonical + // identity wins; the old short spelling is accepted only when it denotes + // exactly one provider. A same-short-name collision is never guessed. for (auto& [capKey, prov] : ctx.manifest.runtimeConfig.providerOverrides) { - bool found = false; + std::vector candidates; + for (auto const& entry : ctx.plan.runtimeProviders) { + if (!entry.capability.starts_with(capKey)) continue; + const auto withoutVersion = entry.provider.namespace_.empty() + ? entry.provider.name + : entry.provider.namespace_ + "." + entry.provider.name; + if (entry.provider.canonical() == prov || withoutVersion == prov) + candidates = {entry.provider}; + } + if (candidates.empty()) { + for (auto const& entry : ctx.plan.runtimeProviders) { + if (entry.capability.starts_with(capKey) + && entry.provider.name == prov) + candidates.push_back(entry.provider); + } + } + std::ranges::sort(candidates); + candidates.erase(std::ranges::unique(candidates).begin(), candidates.end()); + if (candidates.empty()) { + return std::unexpected(std::format( + "[runtime.{}] provider = \"{}\" does not name a provider in " + "the resolved dependency graph", capKey, prov)); + } + if (candidates.size() != 1) { + std::string choices; + for (auto const& candidate : candidates) + choices += (choices.empty() ? "" : ", ") + candidate.canonical(); + return std::unexpected(std::format( + "[runtime.{}] provider = \"{}\" is ambiguous; use one exact " + "canonical identity: [{}]", capKey, prov, choices)); + } + const auto selected = candidates.front(); std::stable_partition(ctx.plan.runtimeProviders.begin(), ctx.plan.runtimeProviders.end(), [&](const auto& pr) { - bool match = pr.capability.rfind(capKey, 0) == 0 && pr.provider == prov; - found = found || match; - return match; + return pr.capability.starts_with(capKey) && pr.provider == selected; }); - if (!found) { - std::println(stderr, - "warning: [runtime.{}] provider = \"{}\" — no such provider in the " - "dependency graph for that capability", capKey, prov); - } } // Capability-driven ABI enforcement, dimensional (see src/toolchain/abi.cppm @@ -5969,7 +6250,7 @@ prepare_build(bool print_fingerprint, for (auto& cap : ctx.plan.runtimeCapabilities) { std::string provider; for (auto& [c, p] : ctx.plan.runtimeProviders) - if (c == cap) { provider = p; break; } + if (c == cap) { provider = p.canonical(); break; } if (auto con = mcpp::toolchain::parse_abi_capability( cap, provider.empty() ? std::string_view{"?"} : std::string_view{provider})) constraints.push_back(std::move(*con)); @@ -5988,35 +6269,132 @@ prepare_build(bool print_fingerprint, } } - // Per-build resolution manifest artifact: a machine-readable record of the - // resolved plan (toolchain/abi, runtime closure, capabilities+providers, - // deps) written next to the build outputs. Same data as `mcpp why`; usable - // by CI/tooling. (capability -> plan, serialized.) + // Per-build resolution manifest: the durable, provider-neutral facts that + // `mcpp why runtime` interprets without resolving again or probing the + // current host. The post-link validator replaces `validation.pending` + // with the exact artifact verdict produced at the link seam. { const std::string tcAbi = ctx.tc.targetTriple.find("musl") != std::string::npos ? "musl" : ctx.tc.stdlibId == "libc++" ? "libc++" : ctx.tc.compiler == mcpp::toolchain::CompilerId::MSVC ? "msvc" : "glibc"; + auto package_json = [](const mcpp::manifest::PackageId& id) { + return nlohmann::json{ + {"canonical", id.canonical()}, + {"namespace", id.namespace_}, + {"name", id.name}, + {"version", id.version}, + {"source", id.sourceProvenance}, + }; + }; + auto path_array = [](auto const& paths) { + nlohmann::json values = nlohmann::json::array(); + for (auto const& path : paths) + values.push_back(path.lexically_normal().generic_string()); + return values; + }; nlohmann::json j; + j["schema_version"] = 2; j["toolchain"] = { {"spec", ctx.tc.label()}, {"abi", tcAbi}, {"triple", ctx.tc.targetTriple}, {"stdlib", ctx.tc.stdlibId}, }; nlohmann::json dirs = nlohmann::json::array(); for (auto& d : ctx.plan.runtimeLibraryDirs) dirs.push_back(d.string()); - nlohmann::json caps = nlohmann::json::array(); + nlohmann::json legacyCaps = nlohmann::json::array(); + nlohmann::json providers = nlohmann::json::array(); for (auto& [cap, prov] : ctx.plan.runtimeProviders) - caps.push_back({{"capability", cap}, {"provider", prov}}); + { + legacyCaps.push_back({{"capability", cap}, + {"provider", prov.canonical()}}); + providers.push_back({{"capability", cap}, + {"provider", package_json(prov)}}); + } + nlohmann::json requirements = nlohmann::json::array(); + for (auto const& requirement : ctx.plan.runtimeRequirements) { + requirements.push_back({ + {"kind", requirement.kind}, + {"value", requirement.value}, + {"phase", requirement.phase}, + {"requester", package_json(requirement.requester)}, + {"required", requirement.required}, + }); + } + nlohmann::json artifacts = nlohmann::json::array(); + for (auto const& artifact : ctx.plan.runtimeArtifacts) { + artifacts.push_back({ + {"role", artifact.role}, + {"provider", package_json(artifact.provider)}, + {"path", artifact.path.lexically_normal().generic_string()}, + {"provenance", artifact.provenance}, + {"abi", artifact.abi}, + {"digest", artifact.digest}, + {"host_fingerprint", artifact.hostFingerprint}, + }); + } + nlohmann::json binding = nlohmann::json::parse( + mcpp::platform::runtime::serialize_runtime_binding( + ctx.plan.runtimeBinding), nullptr, false); + if (binding.is_discarded()) binding = nlohmann::json::object(); + + auto triple = ctx.tc.targetTriple; + std::ranges::transform(triple, triple.begin(), + [](unsigned char c) { return std::tolower(c); }); + const bool pe = triple.find("windows") != std::string::npos + || triple.find("mingw") != std::string::npos; + const bool macho = triple.find("darwin") != std::string::npos + || triple.find("apple") != std::string::npos; + std::string format = pe ? "pe" : macho ? "macho" : "elf"; + nlohmann::json search = { + {"format", format}, + {"link_library", pe ? "libpath" : "library_path"}, + {"transitive_needed", format == "elf" ? "rpath_link" : "none"}, + {"runtime", format == "pe" ? "deploy" + : format == "macho" ? "loader_rpath" : "runpath"}, + }; j["runtime"] = { {"library_dirs", dirs}, {"dlopen_libs", ctx.plan.runtimeDlopenLibs}, - {"capabilities", caps}, + {"capabilities", legacyCaps}, + {"binding", binding}, + {"requirements", requirements}, + {"artifacts", artifacts}, + {"providers", providers}, + {"link_intent", { + {"libraries", ctx.plan.linkIntent.libraries}, + {"link_library_dirs", + path_array(ctx.plan.linkIntent.linkLibraryDirs)}, + {"transitive_needed_dirs", + path_array(ctx.plan.linkIntent.transitiveNeededDirs)}, + {"runtime_search_dirs", + path_array(ctx.plan.linkIntent.runtimeSearchDirs)}, + {"frameworks", ctx.plan.linkIntent.frameworks}, + {"deploy_files", path_array(ctx.plan.linkIntent.deployFiles)}, + }}, + {"search", search}, + {"validation", { + {"status", format == "elf" ? "pending" : "not_exercised"}, + {"source", "post_link"}, + {"artifacts", nlohmann::json::array()}, + }}, }; std::error_code ec; std::filesystem::create_directories(ctx.plan.outputDir, ec); - if (std::ofstream js(ctx.plan.outputDir / "resolution.json"); js) + auto path = ctx.plan.outputDir / "resolution.json"; + auto tmp = path; + tmp += ".tmp"; + if (std::ofstream js(tmp); js) { js << j.dump(2) << "\n"; + js.close(); + std::filesystem::rename(tmp, path, ec); + if (ec) { + ec.clear(); + std::filesystem::remove(path, ec); + ec.clear(); + std::filesystem::rename(tmp, path, ec); + } + } } return ctx; diff --git a/src/build/runtime_validation.cppm b/src/build/runtime_validation.cppm new file mode 100644 index 00000000..24cd3876 --- /dev/null +++ b/src/build/runtime_validation.cppm @@ -0,0 +1,433 @@ +// mcpp.build.runtime_validation — validate only freshly linked Linux ELFs. +// +// The backend snapshots link outputs before ninja and compares their stat +// fingerprints afterwards. An unchanged no-op build therefore performs zero +// ELF parses. Verdicts are persisted beside build.ninja and keyed by artifact +// stat + RuntimeBinding contract so doctor can explain the last result without +// probing the host again. + +export module mcpp.build.runtime_validation; + +import std; +import mcpp.build.plan; +import mcpp.libs.json; +import mcpp.platform; +import mcpp.platform.elf_runtime; +import mcpp.platform.runtime_binding; + +export namespace mcpp::build::runtime_validation { + +struct ArtifactStamp { + bool exists = false; + std::uintmax_t size = 0; + std::int64_t mtime = 0; + + bool operator==(const ArtifactStamp&) const = default; +}; + +using ArtifactSnapshot = std::map; + +struct ValidatedArtifact { + std::filesystem::path artifact; + mcpp::platform::elf::RuntimeVerdict verdict; + bool cacheHit = false; +}; + +struct ValidationReport { + std::vector artifacts; + + bool has_proven_mismatch() const { + return std::ranges::any_of(artifacts, [](auto const& artifact) { + return artifact.verdict.status + == mcpp::platform::elf::RuntimeVerdict::Status::ProvenMismatch; + }); + } +}; + +struct StoredRuntimeSummary { + std::filesystem::path artifact; + mcpp::platform::elf::RuntimeVerdict verdict; + std::string contractHash; +}; + +ArtifactSnapshot snapshot_link_artifacts(const mcpp::build::BuildPlan& plan); + +ValidationReport validate_changed_artifacts( + const mcpp::build::BuildPlan& plan, + const ArtifactSnapshot& before); + +std::optional +latest_stored_verdict(const std::filesystem::path& targetRoot); + +// Fast paths may run ninja without reconstructing BuildPlan. They are allowed +// only when every stored artifact still has the stat/contract fingerprint that +// was validated. The returned snapshot can be compared after ninja; any +// change drops to the full path, which reconstructs the search closure and +// validates before reporting success/running the program. +std::optional validated_artifact_snapshot( + const std::filesystem::path& outputDir, + const mcpp::platform::runtime::RuntimeBinding& binding); + +bool artifact_snapshot_unchanged(const ArtifactSnapshot& snapshot); + +} // namespace mcpp::build::runtime_validation + +namespace mcpp::build::runtime_validation { +namespace { + +constexpr std::string_view kCacheFile = ".mcpp-runtime-verdicts.json"; + +ArtifactStamp stamp(const std::filesystem::path& path) { + ArtifactStamp out; + std::error_code ec; + if (!std::filesystem::is_regular_file(path, ec)) return out; + out.exists = true; + out.size = std::filesystem::file_size(path, ec); + if (ec) { out.exists = false; return out; } + auto time = std::filesystem::last_write_time(path, ec); + if (ec) { out.exists = false; return out; } + out.mtime = static_cast(time.time_since_epoch().count()); + return out; +} + +std::string fingerprint(const std::filesystem::path& artifact, + const ArtifactStamp& value, + std::string_view contractHash) { + auto input = std::format("{}\n{}\n{}\n{}", + artifact.generic_string(), value.size, value.mtime, contractHash); + std::uint64_t hash = 0xcbf29ce484222325ull; + for (unsigned char c : input) { + hash ^= c; + hash *= 0x100000001b3ull; + } + return std::format("{:016x}", hash); +} + +std::string status_name(mcpp::platform::elf::RuntimeVerdict::Status status) { + using Status = mcpp::platform::elf::RuntimeVerdict::Status; + switch (status) { + case Status::Pass: return "pass"; + case Status::ProvenMismatch: return "proven_mismatch"; + case Status::Inconclusive: return "inconclusive"; + } + return "inconclusive"; +} + +mcpp::platform::elf::RuntimeVerdict::Status +parse_status(std::string_view value) { + using Status = mcpp::platform::elf::RuntimeVerdict::Status; + if (value == "pass") return Status::Pass; + if (value == "proven_mismatch") return Status::ProvenMismatch; + return Status::Inconclusive; +} + +nlohmann::json read_cache(const std::filesystem::path& outputDir) { + std::ifstream input(outputDir / kCacheFile); + if (!input) return nlohmann::json::object(); + auto doc = nlohmann::json::parse(input, nullptr, false); + if (doc.is_discarded() || !doc.is_object()) return nlohmann::json::object(); + return doc; +} + +void write_cache(const std::filesystem::path& outputDir, + const nlohmann::json& doc) { + std::error_code ec; + std::filesystem::create_directories(outputDir, ec); + auto path = outputDir / kCacheFile; + auto tmp = outputDir / (std::string(kCacheFile) + ".tmp"); + { + std::ofstream output(tmp, std::ios::trunc); + if (!output) return; + output << doc.dump(2) << '\n'; + if (!output) return; + } + std::filesystem::rename(tmp, path, ec); + if (ec) { + ec.clear(); + std::filesystem::remove(path, ec); + ec.clear(); + std::filesystem::rename(tmp, path, ec); + } + if (ec) std::filesystem::remove(tmp, ec); +} + +std::string cache_key(const mcpp::build::BuildPlan& plan, + const std::filesystem::path& artifact) { + std::error_code ec; + auto relative = std::filesystem::relative(artifact, plan.outputDir, ec); + return ec ? artifact.lexically_normal().generic_string() + : relative.lexically_normal().generic_string(); +} + +std::vector +runtime_search_dirs(const mcpp::build::BuildPlan& plan) { + std::vector out; + auto append = [&](auto const& dirs) { + for (auto const& dir : dirs) { + if (dir.empty() || std::ranges::find(out, dir) != out.end()) continue; + out.push_back(dir); + } + }; + append(plan.runtimeLibraryDirs); + append(plan.depRuntimeLibraryDirs); + append(plan.toolchain.compilerRuntimeDirs); + append(plan.runtimeBinding.libraryDirs); + return out; +} + +std::optional cached_artifact( + const nlohmann::json& doc, + std::string_view key, + std::string_view expectedFingerprint, + const std::filesystem::path& path) { + try { + auto artifacts = doc.find("artifacts"); + if (artifacts == doc.end() || !artifacts->is_object()) return std::nullopt; + auto it = artifacts->find(std::string(key)); + if (it == artifacts->end() || !it->is_object() + || it->value("fingerprint", "") != expectedFingerprint) + return std::nullopt; + ValidatedArtifact out; + out.artifact = path; + out.cacheHit = true; + out.verdict.status = parse_status(it->value("status", "inconclusive")); + out.verdict.diagnostics = it->value( + "diagnostics", std::vector{}); + return out; + } catch (...) { + return std::nullopt; + } +} + +void store_artifact(nlohmann::json& doc, + std::string_view key, + std::string_view artifactFingerprint, + const ValidatedArtifact& value) { + if (!doc.contains("artifacts") || !doc["artifacts"].is_object()) + doc["artifacts"] = nlohmann::json::object(); + doc["artifacts"][std::string(key)] = { + {"fingerprint", artifactFingerprint}, + {"status", status_name(value.verdict.status)}, + {"diagnostics", value.verdict.diagnostics}, + }; +} + +void sync_resolution_verdict(const mcpp::build::BuildPlan& plan, + const nlohmann::json& cache) { + const auto path = plan.outputDir / "resolution.json"; + std::ifstream input(path); + auto resolution = nlohmann::json::parse(input, nullptr, false); + if (resolution.is_discarded() || !resolution.is_object()) return; + auto runtime = resolution.find("runtime"); + if (runtime == resolution.end() || !runtime->is_object()) return; + + nlohmann::json checked = nlohmann::json::array(); + using Status = mcpp::platform::elf::RuntimeVerdict::Status; + Status summary = Status::Pass; + bool any = false; + if (auto artifacts = cache.find("artifacts"); + artifacts != cache.end() && artifacts->is_object()) { + for (auto it = artifacts->begin(); it != artifacts->end(); ++it) { + if (!it.value().is_object()) continue; + any = true; + auto status = parse_status(it.value().value("status", "inconclusive")); + if (status == Status::ProvenMismatch + || (status == Status::Inconclusive && summary == Status::Pass)) + summary = status; + checked.push_back({ + {"path", (plan.outputDir / it.key()).lexically_normal().generic_string()}, + {"status", status_name(status)}, + {"diagnostics", it.value().value( + "diagnostics", std::vector{})}, + {"fingerprint", it.value().value("fingerprint", "")}, + }); + } + } + const bool hasCheckableOutput = std::ranges::any_of( + plan.linkUnits, [](auto const& unit) { + return unit.kind != mcpp::build::LinkUnit::StaticLibrary; + }); + (*runtime)["validation"] = { + {"status", any ? status_name(summary) + : hasCheckableOutput ? "pending" : "not_exercised"}, + {"source", "post_link"}, + {"contract_hash", plan.runtimeBinding.contractHash}, + {"artifacts", std::move(checked)}, + }; + + std::error_code ec; + auto tmp = path; + tmp += ".tmp"; + if (std::ofstream output(tmp); output) { + output << resolution.dump(2) << '\n'; + output.close(); + std::filesystem::rename(tmp, path, ec); + if (ec) { + ec.clear(); + std::filesystem::remove(path, ec); + ec.clear(); + std::filesystem::rename(tmp, path, ec); + } + } +} + +} // namespace + +ArtifactSnapshot snapshot_link_artifacts(const mcpp::build::BuildPlan& plan) { + ArtifactSnapshot out; + for (auto const& unit : plan.linkUnits) { + if (unit.kind == mcpp::build::LinkUnit::StaticLibrary) continue; + auto artifact = plan.outputDir / unit.output; + out.emplace(artifact, stamp(artifact)); + } + return out; +} + +ValidationReport validate_changed_artifacts( + const mcpp::build::BuildPlan& plan, + const ArtifactSnapshot& before) { + ValidationReport report; + if constexpr (!mcpp::platform::is_linux) return report; + if (plan.runtimeBinding.platform != "linux" + || !plan.runtimeBinding.runtimeId.starts_with("glibc@")) + return report; + + auto doc = read_cache(plan.outputDir); + bool changedCache = false; + if (doc.value("schema", 0) != 1 + || doc.value("contract_hash", "") != plan.runtimeBinding.contractHash) { + doc = nlohmann::json::object(); + changedCache = true; + } + doc["schema"] = 1; + doc["contract_hash"] = plan.runtimeBinding.contractHash; + auto searchDirs = runtime_search_dirs(plan); + std::set currentKeys; + for (auto const& [artifact, ignored] : before) { + (void)ignored; + currentKeys.insert(cache_key(plan, artifact)); + } + if (auto artifacts = doc.find("artifacts"); + artifacts != doc.end() && artifacts->is_object()) { + for (auto it = artifacts->begin(); it != artifacts->end();) { + if (!currentKeys.contains(it.key())) { + it = artifacts->erase(it); + changedCache = true; + } else { + ++it; + } + } + } + for (auto const& [artifact, oldStamp] : before) { + auto now = stamp(artifact); + if (!now.exists) continue; + + auto key = cache_key(plan, artifact); + auto fp = fingerprint(artifact, now, plan.runtimeBinding.contractHash); + if (auto cached = cached_artifact(doc, key, fp, artifact)) { + // Same stat before/after and a current stored verdict is the hot + // no-op: do not parse. Only PASS may also stay silent; a stored + // mismatch must keep failing and an inconclusive result must keep + // explaining itself on every invocation. + if (!(now == oldStamp) + || cached->verdict.status + != mcpp::platform::elf::RuntimeVerdict::Status::Pass) + report.artifacts.push_back(std::move(*cached)); + continue; + } + + ValidatedArtifact validated; + validated.artifact = artifact; + auto resolution = mcpp::platform::elf::resolve_runtime_closure( + artifact, plan.runtimeBinding, searchDirs); + validated.verdict = mcpp::platform::elf::validate_runtime_artifact( + artifact, plan.runtimeBinding, resolution); + store_artifact(doc, key, fp, validated); + changedCache = true; + report.artifacts.push_back(std::move(validated)); + } + if (changedCache) write_cache(plan.outputDir, doc); + sync_resolution_verdict(plan, doc); + return report; +} + +std::optional validated_artifact_snapshot( + const std::filesystem::path& outputDir, + const mcpp::platform::runtime::RuntimeBinding& binding) { + auto doc = read_cache(outputDir); + if (doc.value("schema", 0) != 1 + || doc.value("contract_hash", "") != binding.contractHash) + return std::nullopt; + auto artifacts = doc.find("artifacts"); + if (artifacts == doc.end() || !artifacts->is_object() || artifacts->empty()) + return std::nullopt; + + ArtifactSnapshot out; + for (auto it = artifacts->begin(); it != artifacts->end(); ++it) { + if (!it.value().is_object()) return std::nullopt; + if (parse_status(it.value().value("status", "inconclusive")) + != mcpp::platform::elf::RuntimeVerdict::Status::Pass) + return std::nullopt; + auto artifact = outputDir / it.key(); + auto current = stamp(artifact); + if (!current.exists) return std::nullopt; + auto expected = fingerprint(artifact, current, binding.contractHash); + if (it.value().value("fingerprint", "") != expected) + return std::nullopt; + out.emplace(std::move(artifact), current); + } + return out; +} + +bool artifact_snapshot_unchanged(const ArtifactSnapshot& snapshot) { + return std::ranges::all_of(snapshot, [](auto const& entry) { + return stamp(entry.first) == entry.second; + }); +} + +std::optional +latest_stored_verdict(const std::filesystem::path& targetRoot) { + std::error_code ec; + if (!std::filesystem::is_directory(targetRoot, ec)) return std::nullopt; + std::filesystem::path newest; + std::filesystem::file_time_type newestTime{}; + bool found = false; + for (auto it = std::filesystem::recursive_directory_iterator( + targetRoot, std::filesystem::directory_options::skip_permission_denied, ec); + !ec && it != std::filesystem::recursive_directory_iterator{}; + it.increment(ec)) { + if (!it->is_regular_file(ec) || it->path().filename() != kCacheFile) continue; + auto time = it->last_write_time(ec); + if (ec) { ec.clear(); continue; } + if (!found || time > newestTime) { + found = true; + newest = it->path(); + newestTime = time; + } + } + if (!found) return std::nullopt; + auto doc = read_cache(newest.parent_path()); + auto artifacts = doc.find("artifacts"); + if (artifacts == doc.end() || !artifacts->is_object()) return std::nullopt; + + StoredRuntimeSummary summary; + summary.contractHash = doc.value("contract_hash", ""); + using Status = mcpp::platform::elf::RuntimeVerdict::Status; + summary.verdict.status = Status::Pass; + for (auto it = artifacts->begin(); it != artifacts->end(); ++it) { + if (!it.value().is_object()) continue; + auto status = parse_status(it.value().value("status", "inconclusive")); + const bool worse = status == Status::ProvenMismatch + || (status == Status::Inconclusive && summary.verdict.status == Status::Pass); + if (!worse && !summary.artifact.empty()) continue; + summary.artifact = newest.parent_path() / it.key(); + summary.verdict.status = status; + summary.verdict.diagnostics = it.value().value( + "diagnostics", std::vector{}); + if (status == Status::ProvenMismatch) break; + } + if (summary.artifact.empty()) return std::nullopt; + return summary; +} + +} // namespace mcpp::build::runtime_validation diff --git a/src/cli.cppm b/src/cli.cppm index 2290fb5c..8ad3912b 100644 --- a/src/cli.cppm +++ b/src/cli.cppm @@ -57,8 +57,8 @@ void print_usage() { std::println(" mcpp run [target] [-- args...] Build + run a binary target"); std::println(" mcpp test [pattern] [-- args...] Build + run tests/**/*.cpp (--list, --timeout, --build-timeout, --message-format json)"); std::println(" mcpp clean [--bmi-cache] Remove target/ (and optionally the build cache)"); - std::println(" mcpp add [@] Add a dependency to mcpp.toml"); - std::println(" mcpp remove Remove a dependency from mcpp.toml"); + std::println(" mcpp add [ns.]pkg@ver Add an exact dependency to mcpp.toml"); + std::println(" mcpp remove [ns.]pkg Remove an exact dependency from mcpp.toml"); std::println(" mcpp update [pkg] Re-resolve deps and rewrite mcpp.lock"); std::println(" mcpp search Search packages in registries"); std::println(" mcpp publish [--dry-run] Publish package to default registry"); @@ -82,7 +82,7 @@ void print_usage() { std::println("Build options:"); std::println(" --verbose, -v Verbose compiler output"); std::println(" --quiet, -q Suppress status output"); - std::println(" --print-fingerprint Show toolchain fingerprint and 10 inputs"); + std::println(" --print-fingerprint Show toolchain fingerprint and 11 inputs"); std::println(" --cache Dependency cache: global (default) | local | off"); std::println(" --no-cache Deprecated alias for --cache=off (clears the build dir)"); std::println(" --no-color Disable colored output"); @@ -229,14 +229,14 @@ int run(int argc, char** argv) { // (cmd_new validates presence for project creation itself). .arg(cl::Arg("name").help("Package directory name")) .option(cl::Option("template").short_name('t').takes_value().value_name("SPEC") - .help("bin (default) | [@ver][: