[feat](advisor): weakness-averse second opinion + repair 3 live mismatches in the disclosure - #71
Merged
Merged
Conversation
…thout compensation? Step 2 of the formula work, and a correction to my own plan: I had proposed this as a separate button. That was wrong twice over — a control the user must first discover is the exact discoverability failure fixed elsewhere in this app, and a second warning panel would compete with the compensation disclosure rather than complete it. It is now one line attached to that disclosure, which turns "there is a weakness here" into the question that actually matters: does the DECISION depend on it? "Microservices scores 2/5 on Cost efficiency, which you weighted 13.2% … Score weakness more heavily and Serverless (FaaS) wins instead — this pick leans on offsetting." Computed only when a compensation exists; on its own the verdict has nothing to qualify. Additive stays the recommendation — this is a second opinion, and a test asserts running it cannot change what rank() returns. I OVERCLAIMED, AND THE TEST CAUGHT ME. The first doc comment said a geometric mean means "no amount of strength elsewhere fully recovers" a weakness. That is false, and my own test failed proving it: with twelve equally-weighted attributes, one fit of 1 among eleven 5s scores 4.37 — comfortably above a flat row of 3s. Geometric aggregation REDUCES compensability; it does not remove it. Strength still buys off weakness, it simply pays a worse rate. Genuine non-compensation needs a veto or threshold rule (ELECTRE-style), which would need its own ADR. The comment now says this plainly and two tests pin it: one proves the aggregators disagree inside the band where they genuinely do (~32–50% weight on the weak attribute), the other asserts the equal-weight case where the weakness IS bought off, so the overclaim cannot come back. That correction also applies to PR #70's description, which called geometric a proper fix for compensability. It is a partial one. GUARD FIX, NOT A GUARD DODGE. check-language-purity flagged the new Indonesian string for containing the English word "other" — from the `{other}` placeholder. Placeholders are substituted at runtime: they are code, not prose, and reading them as language is simply wrong. The guard now strips `{…}` tokens before analysis rather than the string being reworded to slip past it. Re-verified it still bites. BUNDLE 286→287 total (measured 286.2). Being explicit about which promise applies: the note on the INITIAL budget says a third raise there should trigger splitting the below-the-fold Advisor results into their own chunk. That is a first-paint concern and this is not it — initial sits at 122.5/123 and barely moved. This is the NFR ceiling guard (cap 300), measuring the whole app rather than what loads first. Gates: build clean, lint 0, 265/265 unit, 15/15 E2E, 7/7 guards, 0 axe in both languages and both themes at 390px. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Step 2 of the formula work, plus a correction to my own plan: I had proposed this
as a separate button. Wrong twice — a control the user must first discover is the
exact discoverability failure fixed elsewhere in this app, and a second warning
panel would compete with the compensation disclosure rather than complete it. It
is one line attached to that disclosure, turning "there is a weakness here" into
the question that matters: does the DECISION depend on it?
"Microservices scores 2/5 on Cost efficiency, which you weighted 13.2% …
Score weakness more heavily and Serverless (FaaS) wins instead —
this pick leans on offsetting."
Computed only when a compensation exists. Additive stays the recommendation; a
test asserts running this cannot change what rank() returns.
I OVERCLAIMED AND MY OWN TEST CAUGHT ME. The first draft said a geometric mean
means "no amount of strength elsewhere fully recovers" a weakness. False: with
twelve equal-weight attributes, one fit of 1 among eleven 5s scores 4.37, above a
flat row of 3s. Geometric aggregation REDUCES compensability, it does not remove
it — strength still buys off weakness, at a worse rate. Real non-compensation
needs a veto/threshold rule, which would need its own ADR. Two tests pin this: one
proves the aggregators disagree inside the band where they genuinely do (~32–50%
weight on the weak attribute), the other asserts the equal-weight case where the
weakness IS bought off, so the overclaim cannot return.
THREE DEFECTS FOUND IN PRE-PR REVIEW, none of which any gate would have caught:
1. The function was named `nonCompensatoryWinner` while its own doc comment, three
lines above, said it is not non-compensatory. Names outlive comments: someone
reads the name in six months, assumes the property holds, and cites it. Renamed
`weaknessAverseWinner` — averse is true, prohibitive is not.
2. The EN and ID strings did not say the same thing. EN claimed weakness "counts
double" — a specific numeric claim that is simply false — where ID said
"dihitung lebih berat" (weighted more heavily), which is accurate. EN now
matches. The purity guard cannot catch this: it checks language mixing, not
whether the two languages make the same claim.
3. The 28.2% and 12.4% figures were prose only. The frozen-model guards catch an
ACCIDENTAL model change, but a deliberate one updates both copies, passes CI,
and leaves these numbers quietly wrong — and they are load-bearing, since they
are the whole argument for disclosing rather than switching aggregator. Both are
now measured in a test against a seeded sample, asserted exactly (5634/20000 and
2484/20000) so any model move fails and names the comments to rewrite. ~250ms.
Also verified comment against code rather than trusting either: the tie-break is
the identical comparator `rankWith` uses; `ln` is safe because real fits are 2–5;
the verdict really is gated on a compensation existing.
GUARD FIX, NOT A GUARD DODGE. check-language-purity flagged the new Indonesian
string for the English word "other" — from the `{other}` placeholder. Placeholders
are substituted at runtime: code, not prose. The guard now strips `{…}` before
analysis instead of the string being reworded to slip past it. Re-verified it bites.
BUNDLE 286→287 total (measured 286.2). Which promise applies: the note on the
INITIAL budget says a third raise there should split the below-the-fold Advisor
results into their own chunk. That is a first-paint concern and this is not it —
initial is 122.5/123 and barely moved. This guard is the NFR ceiling (cap 300).
Gates: build clean, lint 0, 266/266 unit, 15/15 E2E (run three times — an earlier
single failure was webserver flake, not code), 7/7 guards, 0 axe in both languages
and both themes at 390px.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
…ation disclosure Pre-PR review of the disclosure merged in #70. Three defects, all live in production, none of which any existing gate would have caught. 1. THE SAME NUMBER, RENDERED TWO WAYS. Seven places in this app display quality weights through roundWeights() — Priorities, the override panel, Fitness functions, chat, exports, the Manual. The disclosure was the only one printing the raw float, so a session showed "Cost efficiency 13%" in the Priorities panel and "which you weighted 13.2%" in the disclosure, for one attribute at one moment. Two renderings of one number reads as the tool contradicting itself, which is corrosive for a product whose whole pitch is that its arithmetic is auditable. Now routed through roundWeights like everything else. 2. IT TOLD THE READER THEY HAD DONE SOMETHING THEY HAD NOT. The copy said "which you weighted 13%". In Guided mode — the default — the reader never sets a weight: they answer 14 questions and the model DERIVES the weights. The app is careful about this everywhere else ('prio.derived' reads "Worked out from your answers"), so this sentence contradicted the product's own framing and misattributed the model's derivation to the user. Reworded to "sits at 13% in your priorities", which is true whether the weights were derived or edited in Expert mode. 3. THE CAVEAT VANISHED AT EXPORT. The Advisor warns on screen when the pick is weak on a highly-ranked attribute, but executiveSummary() — the MADR/report text that actually gets circulated and archived — carried anti-patterns and close-call status and not this. So the warning disappeared at precisely the moment the decision became permanent: someone reads the record a year later with no idea the pick leaned on offsetting. The summary now records it, in both languages, quoting the same rounded weight the screen shows. It caveats the SELECTED option rather than the top-ranked one, because a decision record documents the decision that was made — including a deliberate choice against the recommendation. Tests: the export caveat is asserted in both languages; one test pins that its weight matches roundWeights so the two can never drift apart again; one covers choosing against the recommendation. Writing them caught a bad premise of my own — an earlier "stays silent" case changed the levels but left Microservices selected while 100% of the weight sat on time-to-market, where it scores 2/5. The caveat fired correctly and the test was wrong; it now uses Monolith and says why. Also swept the rest of the app for the same class of defect rather than assuming this was the only one: every score display already routes through displayScore (RadarPanel and ScenarioCompare looked like exceptions and are not), and isCloseCall has a single definition. Those were checked and found clean. Separately, PR #70's own description has been corrected on GitHub: it called a geometric mean a proper fix for compensability, which the later work disproved. Gates: build clean, lint 0, 270/270 unit, 15/15 E2E, 7/7 guards, both bundle budgets within limit with no raise. Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Two things: the second opinion on the recommendation (step 2 of the formula work), and three defects it uncovered in the compensation disclosure already merged in #70 and live in production.
Part 1 — the weakness-averse second opinion
I proposed this as a separate button. Wrong twice: a control the user must first discover is the exact discoverability failure fixed elsewhere in this app, and a second warning panel would compete with the compensation disclosure rather than complete it.
It is one line attached to that disclosure, turning "there is a weakness here" into the question that matters — does the decision depend on it?
Additive stays the recommendation; a test asserts running this cannot change what
rank()returns.I overclaimed and my own test caught me
The first draft said a geometric mean means "no amount of strength elsewhere fully recovers" a weakness.
The spike still wins. Geometric aggregation reduces compensability; it does not remove it. Real non-compensation needs a veto/threshold rule (ELECTRE-style). Two tests pin the accurate claim so it cannot drift back — and PR #70's description has been corrected on GitHub, since it made the same overclaim.
The function was also named
nonCompensatoryWinnerwhile its own doc comment, three lines above, said it is not that. Names outlive comments — someone reads it in six months, assumes the property holds, and cites it. RenamedweaknessAverseWinner.The documented rates are now machine-checked
28.2%and12.4%are load-bearing — they are the entire argument for disclosing rather than switching aggregator. The frozen-model guards catch an accidental model change, but a deliberate one updates both copies, passes CI, and leaves these numbers quietly wrong. Both are now measured against a seeded sample and asserted exactly (5634/20000,2484/20000), so any model move fails and names the comments to rewrite. ~250ms.Part 2 — three mismatches in the shipped disclosure
All live in production since #70. None would have been caught by any existing gate.
1 · The same number, rendered two ways
Seven places display weights through
roundWeights(). The disclosure was the only one printing the raw float:One attribute, one session, two numbers. For a product whose pitch is that its arithmetic is auditable, that reads as the tool contradicting itself.
2 · It told the reader they had done something they had not
The copy said "which you weighted 13%". In Guided mode — the default — the reader never sets a weight: they answer 14 questions and the model derives them. The app is careful about this everywhere else (
prio.derivedreads "Worked out from your answers"), so the sentence contradicted the product's own framing. Now "sits at 13% in your priorities" — true whether derived or edited in Expert mode.3 · The caveat vanished at export
The Advisor warns on screen, but
executiveSummary()— the MADR/report text that actually gets circulated and archived — carried anti-patterns and close-call status and not this.So the warning disappeared at precisely the moment the decision became permanent. Someone reads the record a year later with no idea the pick leaned on offsetting.
Now recorded in both languages, quoting the same rounded weight the screen shows, and caveating the selected option rather than the top-ranked one — a decision record documents the decision that was made, including a deliberate choice against the recommendation.
Checked and found clean
I did not stop at the first finding. Every score display already routes through
displayScore(RadarPanelandScenarioComparelooked like exceptions and are not), andisCloseCallhas a single definition.A test premise of mine was wrong, not the code
An early "stays silent" case changed the levels but left Microservices selected while 100% of the weight sat on time-to-market, where it scores 2/5. The caveat fired correctly; the test was wrong. It now uses Monolith and says why in the comment.
Guard fix, not a guard dodge
check-language-purityflagged the new Indonesian string for the English word "other" — from the{other}placeholder. Placeholders are code, not prose. The guard now strips{…}before analysis rather than the string being reworded to slip past it. Re-verified it still bites.Gates
build clean · lint 0 · 270/270 unit · 15/15 E2E · 7/7 guards · 0 axe in both languages and both themes at 390px · both bundle budgets within limit with no raise.
🤖 Generated with Claude Code