You signed in with another tab or window. Reload to refresh your session.You signed out in another tab or window. Reload to refresh your session.You switched accounts on another tab or window. Reload to refresh your session.Dismiss alert
Each `mcpp=<path>` engine labels itself from the version that binary reports
25
+
(`mcpp@2026.8.12.1`), so two releases never collapse into one row. That is how
26
+
"did this release get faster?" is answered — by running both, not by emulating
27
+
one of them in the harness.
28
+
18
29
---
19
30
20
31
## 1. What is measured
@@ -44,6 +55,7 @@ claim gets a number instead of an argument.
44
55
| # | Invariant | How it is enforced |
45
56
|---|---|---|
46
57
| I1 | Identical compiler **binary** across engines |`--compiler <path>` is threaded into cmake (`-DCMAKE_CXX_COMPILER`), meson & xmake (`CXX`), bazel (`CC` + `--action_env`). mcpp uses its hermetic payload — a **declared asymmetry**, see §5. |
58
+
| I0 | Optimisations are measured, never emulated | Engines are parameterised by BINARY (`mcpp=<path>`). The harness contains no "what if we also set X" mode: emulating a change measures the harness's idea of it and silently stops tracking the implementation. |
47
59
| I2 | Identical source set | All variants come from one generator; no engine globs its own inputs. |
48
60
| I3 | Identical language level | C++23 everywhere; `import std;` is **absent from every fixture** (see §5). |
49
61
| I4 | Same parallelism |`--jobs N` is passed to every engine that accepts one. |
@@ -52,6 +64,25 @@ claim gets a number instead of an argument.
52
64
53
65
---
54
66
67
+
## 2b. Two modes
68
+
69
+
| mode | fixture | when |
70
+
|---|---|---|
71
+
|**generated** (default) |`--units/--fanin/--weight` synthesise the same project in three source forms | comparing **source forms**, and engines against each other on identical input |
72
+
|**project** (`--project DIR`) | an existing tree, measured **in place**| comparing **engine binaries** on a real codebase — mcpp building itself is the base case |
73
+
74
+
In project mode the variant axis collapses to `native`: the project is whatever
75
+
it already is, and generating over it would destroy the thing being measured.
76
+
Scenarios that perturb a file need to be told which one (`--hub`, `--leaf`,
77
+
`--body`); without it they report `skipped`**with the reason** rather than
78
+
picking a file and producing a number that looks valid.
79
+
80
+
`edit-body` rewrites a source file. In project mode that file belongs to the
81
+
user, so its exact bytes are captured before and restored afterwards — including
82
+
when the build fails, which is precisely when a leftover edit would be missed.
0 commit comments