Skip to content

Commit 3c70cff

Browse files
committed
perf(build): 让「接口没变就不级联重编」真正生效 + bench 改测真实二进制
## 1. mcpp 侧:一个从未生效过的机制 `cxx_module` 规则保留上一份 BMI、重编、内容相同则换回旧文件,让 ninja 的 restat 判定输出未变、从而不重建导入者。这套机制 2026-05-12 就设计并实现了,判据是 `cmp -s`。 **它一次都没走通过。** GCC 把 wall-clock 写进 BMI 的内容: buildtime: 2026/08/12 02:25:01 UTC localtime: 2026/08/12 02:25:01 UTC 同一份源码相隔一秒的两次编译,BMI 差恰好 4 个字节,`cmp` 永远报「变了」。当年的 设计说明只预见到 GCC 会重写文件(mtime 抖动)并据此开出内容比较的药方,没有预见到 时间戳本身就是内容 —— 所以药方按原样写出来就不可能生效。 新增 `mcpp bmi-equal`(内部子命令,由 ninja 规则调用),比较时掩掉这两个字段。 刻意不用 `SOURCE_DATE_EPOCH`:那会把整个编译的 epoch 钉死,从而改变**用户代码**里 `__DATE__` / `__TIME__` 的展开;掩码只改变 mcpp 认为「什么算相等」,别的都不动。 构造上保守:找不到预期字段、或两份文件对字段位置判断不一致时回落为严格比较 —— 可以把等价的判成不同,但绝不会把不同的判成相同。 实测(`bench --project` 测 mcpp 构建 mcpp 自身,touch 一个 46 导入者、内容未变的文件): scenario 2026.8.11.3 2026.8.12.1 noop 0.27s 0.19s touch-hub 73.99s 0.45s ~164x 单测从两侧钉死(8 例):只测「等价的判相等」会放过一个恒返回 true 的实现,而那比 原缺陷更糟 —— 它会静默吞掉所有真实的级联。 ## 2. bench 侧:测二进制,不测模拟 删掉 `mcpp-opt` 引擎。它靠在构建前后设 `SOURCE_DATE_EPOCH` 来**模拟**优化 —— 在 harness 里模拟一个改动,测的是 harness 对该改动的理解,而且一旦真实实现与之 分叉就会静默地不再跟踪。 改为**按二进制参数化**:`--engines mcpp=<路径A>,mcpp=<路径B>` 注册两个引擎,各自 向自己的二进制询问版本并据此标注(`mcpp@2026.8.11.3` / `mcpp@2026.8.12.1`), 两行永远不会塌成一行。上面那张对比表就是这么测出来的。 新增 `--project <dir>`:就地测量一个已存在的工程,mcpp 自身即基础用例。该模式下 variant 轴坍缩为 `native`(工程就是它现在的样子,在它之上生成会毁掉被测对象); 需要扰动文件的场景必须显式指定 `--hub/--leaf/--body`,否则报 `skipped` **并说明 原因**,而不是挑一个文件产出一个看着有效的数字。 `edit-body` 会改源文件。项目模式下那是用户的文件,因此逐字节保存并在退出时恢复 —— 包括构建失败的路径,那正是遗留改动最容易被漏掉的时候。 ## 3. 顺带修掉的两个真实问题 - **生成的 fixture 没钉工具链**,依赖机器的全局默认。本机绿、CI 红(`seed build exited 1`)。现在与本仓库其他 mcpp 工程一样显式 pin。 - **e2e 失败时只打印子进程日志的路径**,而那个 tmpdir 在 trap 里已被删除 —— 在 CI 上等于没有信息。现在直接转储内容。 验证:`mcpp build` ✅ · 新增 8 个单测 ✅ · e2e 230 ✅ · 六引擎本机实测 ✅
1 parent 48de5d2 commit 3c70cff

19 files changed

Lines changed: 627 additions & 105 deletions

.agents/docs/2026-08-12-bench-suite-architecture-and-plan.md

Lines changed: 5 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -131,21 +131,23 @@ fixtures/synth-<N>x<D>/
131131

132132
第三种变体直接对应上一轮分析的 **F4 / §6.3**:把实现移出接口单元。有了它,"改一行函数体"的代价差异就是**测出来的**,不是推断的。
133133

134-
同时保留 `self` fixture —— 即 mcpp 自身(137 模块),因为真实工程的依赖形状不是合成器能编出来的
134+
同时提供 **`--project <dir>` 模式**:直接就地测量一个已存在的工程(mcpp 自身即基础用例),因为真实工程的依赖形状不是合成器能编出来的。该模式下 variant 轴坍缩为 `native`,并且 `edit-body` 会在测量前后**逐字节保存并恢复**被改的源文件 —— 包括构建失败的路径,那正是遗留改动最容易被忽略的时候
135135

136136
---
137137

138138
## 5. 场景矩阵
139139

140140
| 维度 | 取值 |
141141
|---|---|
142-
| engine | mcpp, mcpp-opt(优化后), cmake, xmake, meson, bazel |
142+
| engine | `mcpp=<binary>`(可给多个,自动按版本标注)、cmakexmakemesonbazel |
143143
| variant | headers, modules, modules-impl |
144144
| profile | release, debug |
145145
| scenario | cold, noop, touch-hub, edit-body, touch-leaf |
146146
| compiler | gcc, clang, msvc(平台可用者) |
147147

148-
**`mcpp` vs `mcpp-opt`**:同一份源码、同一编译器,区别只在是否启用上一轮验证过的优化(BMI 时间戳归一 + BMI 落盘即释放)。这让"优化前后"成为矩阵里的**一个正交维度**,而不是另做一次实验。
148+
**"优化前后"用两个真实二进制表达,不用模拟。** `--engines mcpp=<旧>,mcpp=<新>` 会注册两个引擎,各自向自己的二进制询问版本并据此标注(`mcpp@2026.8.11.3` / `mcpp@2026.8.12.1`)。
149+
150+
早期设计里有一个 `mcpp-opt` 引擎,靠在构建前后设 `SOURCE_DATE_EPOCH`**模拟**优化。已删除:**在 harness 里模拟一个改动,测的是 harness 对该改动的理解**,而且一旦真实实现与之分叉,它会静默地不再跟踪。优化属于 mcpp,基准测的是二进制。
149151

150152
矩阵是笛卡尔积但**不是全跑**:`spec.cppm` 用显式的 include/exclude 规则裁剪,CI 默认跑一个小集合,`workflow_dispatch` 可放开。
151153

.agents/docs/2026-08-12-modular-build-performance-deep-analysis.md

Lines changed: 10 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -333,11 +333,11 @@ GCC 的 BMI 携带函数体(为了跨模块内联),所以任何编辑都会改
333333

334334
**收益 / 成本** 排序。每条都给出判据与验证方式。
335335

336-
### 6.1 【L0·一行改动 · 仅 GCC】固化 BMI 时间戳,让级联抑制真正生效
336+
### 6.1 ✅【已实施 · 2026.8.12.1】让级联抑制真正生效(仅 GCC)
337337

338338
**问题**:F3**仅适用于 GCC** —— Clang 的 `.pcm` 实测字节稳定(§7.2),那边的级联抑制本来就在工作。
339339

340-
**方案 A(推荐,零语义影响)**——把"BMI 是否相等"的判据从裸 `cmp` 换成**时间戳无关比较**mcpp 已有 helper 子命令模式(`mcpp stage` / `mcpp dyndep`),新增 `mcpp bmi-equal <a> <b>`,跳过 BMI 内的 `buildtime:` / `localtime:` 字段。`cxx_module` 规则里把
340+
**已按方案 A 实施**(2026.8.12.1):把"BMI 是否相等"的判据从裸 `cmp` 换成**时间戳无关比较**。新增 `mcpp bmi-equal <a> <b>`(内部子命令),跳过 BMI 内的 `buildtime:` / `localtime:` 字段。`cxx_module` 规则里把
341341

342342
```sh
343343
cmp -s "$bmi_out" "$bmi_out.bak"
@@ -351,7 +351,14 @@ $mcpp bmi-equal "$bmi_out" "$bmi_out.bak"
351351

352352
**副作用**:方案 B 会改变 `__DATE__` / `__TIME__` 的值。用户代码可能依赖,因此不宜作为默认。**方案 A 无此问题,应作为默认;方案 B 作为可选项。**
353353

354-
**实测收益**:touch 一个 46 导入者的模块,**73.0s → 0.22s(332×)**。正确性不变(真实接口变更仍完整级联)。
354+
**实测收益(落地后,由 `bench --project` 测 mcpp 构建自身)**:
355+
356+
| 场景 | 2026.8.11.3 | 2026.8.12.1 |
357+
|---|---|---|
358+
| `noop` | 0.27s | 0.19s |
359+
| **`touch-hub`**(46 导入者,内容未变) | **73.99s** | **0.45s** |
360+
361+
正确性由单测从**两侧**钉死(`tests/unit/test_bmi_equivalent.cpp`):真实接口变更仍完整级联。
355362

356363
**验证方式**:`bench/run.sh --scenario touch-hub`,并**必须同时验证**接口变更场景仍然级联——只测 touch 分不清"级联被正确抑制"和"级联坏了"。
357364

CHANGELOG.md

Lines changed: 38 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -5,6 +5,44 @@
55
66
## [2026.8.12.1] — 2026-08-12
77

8+
### 性能
9+
10+
- **⚠️ 「接口没变就不级联重编」的机制从设计之日起从未生效过 —— 现已修复。**
11+
12+
`cxx_module` 规则会保留上一份 BMI、重编、然后在内容相同时把旧文件换回去,让
13+
ninja 的 `restat` 判定输出未变、从而**不重建导入者**。这套机制 2026-05-12 就设计
14+
并实现了,判据是 `cmp -s`
15+
16+
但 GCC 把 wall-clock **写进了 BMI 的内容**:
17+
18+
```
19+
buildtime: 2026/08/12 02:25:01 UTC
20+
localtime: 2026/08/12 02:25:01 UTC
21+
```
22+
23+
同一份源码相隔一秒的两次编译,BMI 差**恰好 4 个字节** —— `cmp` 于是永远报「变了」,
24+
这条快路径**一次都没有走通过**。当年的设计说明只预见到 GCC 会重写文件(mtime 抖动)
25+
并据此开出内容比较的药方,没有预见到时间戳本身就是内容,所以药方按原样写出来就不
26+
可能生效。
27+
28+
新增 `mcpp bmi-equal`(内部子命令,由 ninja 规则调用),比较时掩掉这两个字段。
29+
刻意**不用 `SOURCE_DATE_EPOCH`**:那会把整个编译的 epoch 钉死,从而改变**用户代码**
30+
`__DATE__` / `__TIME__` 的展开结果;掩码只改变 mcpp 认为「什么算相等」,别的什么都不动。
31+
32+
构造上保守:找不到预期字段、或两份文件对字段位置的判断不一致时,回落为严格比较 ——
33+
它可以把等价的 BMI 判成不同,但**绝不会**把不同的 BMI 判成相同。
34+
35+
实测(用 `bench/` 套件测 mcpp 构建 mcpp 自身,touch 一个被 46 个模块导入、**内容未变**
36+
的文件):
37+
38+
| 场景 | 2026.8.11.3 | 2026.8.12.1 |
39+
|---|---|---|
40+
| `noop` | 0.27s | 0.19s |
41+
| **`touch-hub`** | **73.99s** | **0.45s** |
42+
43+
~164×。正确性由单测从**两侧**钉死:只测「等价的判相等」会放过一个恒返回 true 的实现,
44+
而那比原缺陷更糟 —— 它会静默吞掉所有真实的级联。
45+
846
### 新增
947

1048
- **`bench/` —— 构建引擎基准套件(顶层目录,用 mcpp 自己写)。**

bench-child.log

Lines changed: 7 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -0,0 +1,7 @@
1+
Resolving toolchain
2+
Resolved gcc@16.1.0 → @mcpp/registry/data/xpkgs/xim-x-gcc/16.1.0/bin/g++
3+
Inferred sources [src/**/*.{cppm,cpp,cc,c,S,s,asm}]
4+
Inferred target mcpp (bin from src/main.cpp)
5+
Compiling mcpp v2026.8.12.1 (.)
6+
Cached mcpplibs.cmdline v0.0.1 (3 units)
7+
Finished release [optimized] in 0.22s

bench/README.md

Lines changed: 33 additions & 2 deletions
Original file line numberDiff line numberDiff line change
@@ -8,13 +8,24 @@ Written in C++23 and built by mcpp, so it runs identically on Linux, macOS and
88
Windows — a shell-based harness cannot, and this suite replaced one that could
99
only run on Linux.
1010

11-
```
12-
bench --engines mcpp,mcpp-opt,cmake,xmake,meson,bazel \
11+
```bash
12+
# generated fixtures, across engines and source forms
13+
bench --engines mcpp,cmake,xmake,meson,bazel \
1314
--variants headers,modules,modules-impl \
1415
--scenarios cold,noop,touch-hub,edit-body \
1516
--compiler /path/to/g++ --jobs 32 --out report.json
17+
18+
# a REAL project, measured in place — e.g. mcpp building itself,
19+
# comparing two mcpp binaries
20+
bench --project . --engines mcpp=/usr/bin/mcpp,mcpp=./target/*/*/bin/mcpp \
21+
--scenarios noop,touch-hub --hub src/platform/platform.cppm
1622
```
1723

24+
Each `mcpp=<path>` engine labels itself from the version that binary reports
25+
(`mcpp@2026.8.12.1`), so two releases never collapse into one row. That is how
26+
"did this release get faster?" is answered — by running both, not by emulating
27+
one of them in the harness.
28+
1829
---
1930

2031
## 1. What is measured
@@ -44,6 +55,7 @@ claim gets a number instead of an argument.
4455
| # | Invariant | How it is enforced |
4556
|---|---|---|
4657
| I1 | Identical compiler **binary** across engines | `--compiler <path>` is threaded into cmake (`-DCMAKE_CXX_COMPILER`), meson & xmake (`CXX`), bazel (`CC` + `--action_env`). mcpp uses its hermetic payload — a **declared asymmetry**, see §5. |
58+
| I0 | Optimisations are measured, never emulated | Engines are parameterised by BINARY (`mcpp=<path>`). The harness contains no "what if we also set X" mode: emulating a change measures the harness's idea of it and silently stops tracking the implementation. |
4759
| I2 | Identical source set | All variants come from one generator; no engine globs its own inputs. |
4860
| I3 | Identical language level | C++23 everywhere; `import std;` is **absent from every fixture** (see §5). |
4961
| I4 | Same parallelism | `--jobs N` is passed to every engine that accepts one. |
@@ -52,6 +64,25 @@ claim gets a number instead of an argument.
5264

5365
---
5466

67+
## 2b. Two modes
68+
69+
| mode | fixture | when |
70+
|---|---|---|
71+
| **generated** (default) | `--units/--fanin/--weight` synthesise the same project in three source forms | comparing **source forms**, and engines against each other on identical input |
72+
| **project** (`--project DIR`) | an existing tree, measured **in place** | comparing **engine binaries** on a real codebase — mcpp building itself is the base case |
73+
74+
In project mode the variant axis collapses to `native`: the project is whatever
75+
it already is, and generating over it would destroy the thing being measured.
76+
Scenarios that perturb a file need to be told which one (`--hub`, `--leaf`,
77+
`--body`); without it they report `skipped` **with the reason** rather than
78+
picking a file and producing a number that looks valid.
79+
80+
`edit-body` rewrites a source file. In project mode that file belongs to the
81+
user, so its exact bytes are captured before and restored afterwards — including
82+
when the build fails, which is precisely when a leftover edit would be missed.
83+
84+
---
85+
5586
## 3. Scenarios
5687

5788
| Scenario | Perturbation | What it exercises |

bench/src/engines/mcpp.cppm

Lines changed: 57 additions & 34 deletions
Original file line numberDiff line numberDiff line change
@@ -1,10 +1,16 @@
1-
// bench.engines.mcpp — mcpp as a measured engine, including its optimised form.
1+
// bench.engines.mcpp — mcpp as a measured engine.
22
//
3-
// Two engines live here because they differ by CONFIGURATION, not by code:
4-
// `mcpp` is the shipped behaviour, `mcpp-opt` additionally applies the
5-
// optimisations validated in the 2026-08-12 analysis. Keeping them as two
6-
// registry entries makes "before vs after" an ordinary axis of the matrix
7-
// instead of a separate experiment run by hand.
3+
// PARAMETERISED BY BINARY, not by a simulated flag. `--engines mcpp=/path/to/A,
4+
// mcpp=/path/to/B` registers two engines that differ only in which mcpp runs, and
5+
// each labels itself with the version it reports. That is how "did this release
6+
// get faster?" is answered: by running both releases, not by approximating one
7+
// of them.
8+
//
9+
// An earlier revision had an `mcpp-opt` engine that set SOURCE_DATE_EPOCH around
10+
// the build to emulate an optimisation. It was removed: emulating a change in
11+
// the harness measures the harness's idea of the change, and it silently stops
12+
// tracking the real implementation the moment the two diverge. Optimisations
13+
// belong in mcpp; the bench measures binaries.
814
export module bench.engines.mcpp;
915

1016
import std;
@@ -17,16 +23,26 @@ namespace bench::engines {
1723

1824
class McppEngine : public Engine {
1925
public:
20-
explicit McppEngine(bool optimised) : optimised_(optimised) {}
26+
// `program` may be a bare name resolved through PATH or an absolute path to
27+
// a specific build. `label` is what appears in results; empty means "ask the
28+
// binary", which is what makes a two-version comparison self-describing.
29+
explicit McppEngine(std::string program = "mcpp", std::string label = {})
30+
: program_(std::move(program)), label_(std::move(label)) {}
2131

22-
std::string_view name() const override { return optimised_ ? "mcpp-opt" : "mcpp"; }
32+
std::string_view name() const override {
33+
if (label_.empty()) label_ = discover_label();
34+
return label_;
35+
}
2336

2437
Availability probe() const override {
25-
return probe_program("mcpp", {"mcpp", "--version"});
38+
const auto v = version_string();
39+
if (v.empty())
40+
return {false, std::format("{} not runnable", program_)};
41+
return {true, v};
2642
}
2743

28-
// mcpp compiles plain .cpp as readily as modules, so every fixture variant
29-
// is in scope.
44+
// mcpp compiles plain .cpp as readily as modules, and a Native project is
45+
// whatever it already is, so every variant is in scope.
3046
bool supports(Variant) const override { return true; }
3147
std::string unsupported_reason(Variant) const override { return {}; }
3248

@@ -36,40 +52,47 @@ public:
3652

3753
platform::RunResult build(const Job& job) const override {
3854
const std::vector<std::string> argv{
39-
"mcpp", "build", job.profile == "debug" ? "--dev" : "--release"};
40-
41-
// The optimisation under test is `SOURCE_DATE_EPOCH`. GCC stamps a wall
42-
// clock into every BMI, so mcpp's content-comparison cascade
43-
// suppression can never fire; pinning the epoch makes BMIs byte-stable
44-
// and it fires — measured 73.0 s -> 0.22 s on touch-hub.
45-
//
46-
// A FIXED constant, not "now": the whole point is that two builds a
47-
// minute apart produce identical bytes. The value is arbitrary but must
48-
// not change within a comparison.
49-
//
50-
// Scoped, so the variable never leaks into the next cell — an
51-
// unoptimised `mcpp` measurement running after an `mcpp-opt` one would
52-
// otherwise silently inherit the optimisation and the two would tie.
53-
if (optimised_) {
54-
platform::ScopedEnv epoch("SOURCE_DATE_EPOCH", "1700000000");
55-
return platform::run(argv, job.project_dir, job.log_path);
56-
}
55+
program_, "build", job.profile == "debug" ? "--dev" : "--release"};
5756
return platform::run(argv, job.project_dir, job.log_path);
5857
}
5958

6059
void clean(const Job& job) const override {
6160
// Artifacts only. ~/.mcpp holds the toolchain and the dependency cache;
62-
// deleting those would measure provisioning, which is a different
63-
// question and would make "cold" mean something else for this engine
61+
// removing those would measure provisioning, which is a different
62+
// question, and would make "cold" mean something else for this engine
6463
// than for the others.
6564
platform::remove_tree(job.project_dir / "target");
6665
}
6766

6867
private:
69-
bool optimised_;
68+
std::string program_;
69+
mutable std::string label_;
70+
71+
// `mcpp --version` prints "mcpp <version>". Empty means the binary could not
72+
// be run at all — which probe() reports as unavailable rather than failed.
73+
std::string version_string() const {
74+
const auto out = platform::run_capture({program_, "--version"});
75+
if (!out) return {};
76+
auto line = *out;
77+
if (const auto nl = line.find('\n'); nl != std::string::npos) line.resize(nl);
78+
while (!line.empty() && (line.back() == '\r' || line.back() == ' ')) line.pop_back();
79+
return line;
80+
}
81+
82+
std::string discover_label() const {
83+
const auto v = version_string(); // e.g. "mcpp 2026.8.12.1"
84+
if (v.empty()) return "mcpp";
85+
const auto sp = v.rfind(' ');
86+
if (sp == std::string::npos) return "mcpp";
87+
// "mcpp@2026.8.12.1" — distinct per version, so two binaries never
88+
// collapse into one row of the result table.
89+
return std::format("mcpp@{}", v.substr(sp + 1));
90+
}
7091
};
7192

72-
export std::unique_ptr<Engine> make_mcpp() { return std::make_unique<McppEngine>(false); }
73-
export std::unique_ptr<Engine> make_mcpp_opt() { return std::make_unique<McppEngine>(true); }
93+
export std::unique_ptr<Engine> make_mcpp(std::string program = "mcpp",
94+
std::string label = {}) {
95+
return std::make_unique<McppEngine>(std::move(program), std::move(label));
96+
}
7497

7598
} // namespace bench::engines

bench/src/fixture/buildfiles.cppm

Lines changed: 11 additions & 4 deletions
Original file line numberDiff line numberDiff line change
@@ -76,10 +76,17 @@ inline void emit_mcpp(const std::filesystem::path& root, Variant variant, const
7676
"\n"
7777
"[build]\n"
7878
"default-profile = \"release\"\n";
79-
if (variant != Variant::Headers)
80-
toml += "include_dirs = [\"src\"]\n";
81-
else
82-
toml += "include_dirs = [\"include\"]\n";
79+
toml += variant == Variant::Headers ? "include_dirs = [\"include\"]\n"
80+
: "include_dirs = [\"src\"]\n";
81+
// The toolchain is PINNED, matching every other mcpp project in this repo.
82+
// Relying on the machine's global default makes the fixture build depend on
83+
// ambient state — it works on a developer box that has one and fails on a
84+
// fresh CI sandbox that does not, which is exactly how this surfaced: green
85+
// locally, "seed build exited 1" on the runner.
86+
toml += "\n[toolchain]\n"
87+
"default = \"gcc@16.1.0\"\n"
88+
"macos = \"llvm@22.1.8\"\n"
89+
"windows = \"llvm@20.1.7\"\n";
8390
detail::write(root / "mcpp.toml", toml);
8491
}
8592

0 commit comments

Comments
 (0)