docs(design): m-coverage-cross-package-attribution — the -coverpkg decision, and the answer is NO - #724
Merged
Conversation
…cision, and the answer is NO The charter queued `-coverpkg` as a decision-with-tradeoffs after iteration 204 found `pkg.ParseManifestFile` reading 0.0% on Sonar's new-code gate despite being exercised by cmd/ailang. This doc decides it: keep own-package (LOCALITY) semantics for the gated, badged and Sonar-fed metric, and add a separate non-gating `test-coverage-xpkg` diagnostic. The crux, and the reason the intuitive answer is wrong: iteration 204's actual defect was a function with NO own-package test. `-coverpkg` would have painted those lines green and hidden exactly the gap class that motivated the item. Sonar's 0% was a TRUE POSITIVE that forced the right fix. Two of the charter row's own premises are REFUTED by first-party measurement (3 replicates per arm, 105 test-bearing packages, at 376e192): - "would slow a 120k-line suite materially" -> 89/78/82s vs 92/79/83s, ~+1-4%. - "moves every number in the repo" -> total 45.5% -> 48.1%, i.e. UP, both far above the 29% gate. The risk was never the gate breaking; it is the gate silently LOOSENING. The real cost was unnamed by the row: the merged profile goes 5.7 MB -> 599 MB (~105x, one full-repo profile per test binary), and the SonarCloud step is `continue-on-error: true`, so an ingest failure would be silent (ci.yml:258). Also named, independent of the decision: CI runs the coverage suite TWICE (492s of a 1127s critical-path job) because test-coverage-gate's dependency and the badge step each run it. Quorum: two rounds, both reviewers present in each, BLOCKED both times; round 1's fixes were accepted and did not recur. Round 2's two surviving objections were resolved under Gate 2's narrow-refinement carve-out (each carried the reviewer's own proposed_fix; neither disputed the design direction; first use ratified by Mark, charter:739). Both were MEASURED rather than forwarded, and both were confirmed: - gpt5-6-sol: the "and by whom" capability claim is unsupported. Confirmed (V17) — the 599 MB profile has one `mode:` line and field counts of {2,3}; there is no originating-test column. Contract narrowed to "executed by any test elsewhere"; provenance moved to Future Work. - gemini-3-1-pro: the git-ignore premise cited V10, a Sonar-config read. Confirmed as a mis-sourced citation; the conclusion survives measurement (V16: check-ignore rc=0, control on Makefile rc=1). Risks table re-pointed V10 -> V16. Doc lands as Planned. No sprint runs until D-COV-1 (does the coverage number mean LOCALITY or EXECUTION?) is answered by Mark; the doc recommends LOCALITY and carries a decomposed 3-4 day Option-A sprint should he answer EXECUTION. Designer: claude-fable-5 (rotation). Quorum: gpt5-6-sol + gemini-3-1-pro, metered $0.1454. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
|
sunholo-voight-kampff
added a commit
that referenced
this pull request
Aug 15, 2026
… it would have hidden the defect that raised the question Charter STATUS rotated (205 in, 202 archived; arithmetic asserted before the write, and the archive asserted to have GAINED what the charter lost), queue row updated, dashboard overwritten, log entry 206 appended. The pick was the queue head, and the iteration's finding is that turning `-coverpkg` on would have painted iteration 204's actual defect green — a function with no own-package test — so Sonar's 0.0% was a true positive that forced the right fix. Two of the queue row's own premises are refuted by first-party A/B (105 packages, 3 replicates per arm): runtime 89/78/82s -> 92/79/83s (~+1-4%, not "material"), and total 45.5% -> 48.1%, i.e. UP, both arms far above the 29% gate. The real cost was one the row never named: the merged profile goes 5.7 MB -> 599 MB (~105x), and the Sonar step is continue-on-error, so an ingest failure would be silent. Quorum blocked twice with both reviewers present in each round; round 2 closed under the narrow-refinement carve-out with both objections measured rather than forwarded, and both confirmed. Gate 3b green: PR #724 -> c095f1f, 21 checks, 4/4 required. Gate 5: no skill edit — three candidate gaps, each at instance 1 against the >=2 bar, recorded as watch-items. One is FLAGGED as a live rule tension: the Fable one-run-per-iteration discipline has no stated exception for the revision pass Gate 2's own one-revision flow requires. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
sunholo-voight-kampff
added a commit
that referenced
this pull request
Aug 15, 2026
… it would have hidden the defect that raised the question (#725) Charter STATUS rotated (205 in, 202 archived; arithmetic asserted before the write, and the archive asserted to have GAINED what the charter lost), queue row updated, dashboard overwritten, log entry 206 appended. The pick was the queue head, and the iteration's finding is that turning `-coverpkg` on would have painted iteration 204's actual defect green — a function with no own-package test — so Sonar's 0.0% was a true positive that forced the right fix. Two of the queue row's own premises are refuted by first-party A/B (105 packages, 3 replicates per arm): runtime 89/78/82s -> 92/79/83s (~+1-4%, not "material"), and total 45.5% -> 48.1%, i.e. UP, both arms far above the 29% gate. The real cost was one the row never named: the merged profile goes 5.7 MB -> 599 MB (~105x), and the Sonar step is continue-on-error, so an ingest failure would be silent. Quorum blocked twice with both reviewers present in each round; round 2 closed under the narrow-refinement carve-out with both objections measured rather than forwarded, and both confirmed. Gate 3b green: PR #724 -> c095f1f, 21 checks, 4/4 required. Gate 5: no skill edit — three candidate gaps, each at instance 1 against the >=2 bar, recorded as watch-items. One is FLAGGED as a live rule tension: the Fable one-run-per-iteration discipline has no stated exception for the revision pass Gate 2's own one-revision flow requires. Co-authored-by: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
A design doc deciding the
-coverpkgquestion the charter queued after iteration 204, plusnothing else. Doc only — no production code, no
make/, nosonar-project.properties.The decision: NO
Keep own-package (LOCALITY) semantics for the gated / badged / Sonar-fed metric. Add a
separate, non-gating
test-coverage-xpkgdiagnostic target for triage.The crux, and why the intuitive answer is wrong. Iteration 204's defect was a function with
no own-package test (
pkg.ParseManifestFile, exercised only viacmd/ailang).-coverpkgwould have painted those lines green — hiding exactly the gap class that motivated this item.
Sonar's 0.0% was a true positive that forced the right fix.
Two of the charter row's own premises are REFUTED by measurement
First-party, 3 replicates per arm, the exact
make test-coveragepackage list (105 test-bearingpackages), at
376e19284, darwin/arm64, go1.26.6:-coverpkg=./...)total:with wide headroom. The risk was never the gate breaking; it is the gate silently loosening.
The real cost was unnamed by the row: the merged profile goes 5.7 MB → 599 MB (~105×, one
full-repo profile per test binary). And the SonarCloud step is
continue-on-error: true(
ci.yml:258), so an ingest failure on a 599 MB profile would be silent.Independent finding, named but kept separate: CI runs the coverage suite twice — 492s of a
1127s critical-path
testjob (run31858629366).Narrowing that travels with the runtime result: measured on darwin/arm64 with a warm Go build
cache. Steady-state, not a cold first build.
Quorum: 2 rounds, both reviewers present in each, BLOCKED both times
Round 1's fixes were accepted and did not recur. Round 2's two surviving objections were resolved
under Gate 2's narrow-refinement carve-out — each carried the reviewer's own
proposed_fix,neither disputed the design direction, and first use was ratified by Mark (charter:739). Both were
measured rather than forwarded, and both were confirmed:
CONFIRMED (V17): the 599 MB profile has exactly one
mode:line and field counts of only{2,3}—file:range numStatements count, no originating-test column. Contract narrowed to"executed by any test elsewhere"; the reviewer's arm-2 redesign recorded in Future Work rather
than dropped.
sonar-project.propertiesread;Sonar exclusions do not govern Git. CONFIRMED as a mis-sourced citation; the conclusion
survives measurement (V16:
git check-ignore -v coverage/coverage-xpkg.out→ rc=0 via.gitignore:19, controlgit check-ignore -v Makefile→ rc=1). Risks table re-pointed V10→V16.Right conclusion, wrong evidence — the class this doc is otherwise about, caught in the doc itself.
Round 1 also produced the sharpest finding: the original
XC1was vacuous. Measured on thereal profile,
internal/mcp_client(1024 lines) andinternal/auth/gcp(288) are present withevery counter zero, while
XC1's own canaryinternal/versionis genuinely exercised (56/168nonzero) — so it passed by luck, and would have passed identically and falsely had it named
mcp_client.XC1is now two-sided withmcp_clientas a named negative control.Status
Lands as Planned. No sprint runs until
D-COV-1is answered — does the coverage numbermean LOCALITY or EXECUTION? The doc recommends LOCALITY and carries a decomposed 3–4 day Option-A
sprint should the answer be EXECUTION.
Designer
claude-fable-5(rotation) · quorumgpt5-6-sol+gemini-3-1-pro· metered $0.1454.🤖 Generated with Claude Code