Skip to content

fix(engine): recover calibration composers to the default blend on invalid weights - #8724

Merged
JSONbored merged 2 commits into
JSONbored:mainfrom
shin-core:fix/calibration-composers-invalid-weight-8643
Jul 26, 2026
Merged

fix(engine): recover calibration composers to the default blend on invalid weights#8724
JSONbored merged 2 commits into
JSONbored:mainfrom
shin-core:fix/calibration-composers-invalid-weight-8643

Conversation

@shin-core

Copy link
Copy Markdown
Contributor

What & why

Closes #8643.

pairwise-calibration.ts's normalizePairwiseWeights distinguishes two total <= 0 situations via
isInvalidWeight(): a caller that passed NaN/negative weight(s) by mistake recovers to the
documented default blend, while a caller that explicitly zeroed every weight gets real zeros (the
objective-only fallback). The three sibling composers claimed convergence with that behavior in their
comments but only replicated the explicit-all-zero half — so calling any of them with a NaN/negative
weight alongside otherwise-valid weights silently collapsed to a 100%-objective-anchor composite
instead of the documented 45/35/20 default. Because this module is shared by both the miner (AMS) and
review (ORB) sides, that was a real double-blast-radius miscalibration.

Change

Added the same isInvalidWeight() distinction to normalizeCompositeWeights in all three files,
mirroring pairwise-calibration.ts:58-83 exactly:

  • gate-verdict-calibration.ts
  • reviewer-consensus-calibration.ts
  • finding-severity-calibration.ts

When the clamped total is empty and any input weight was NaN/negative, each function now returns
its own DEFAULT_COMPOSITE_WEIGHTS; the explicit-all-zero case still returns real zeros unchanged.

Validation

  • New invalid-weight test in each of the three files' own suites (mirrors
    pairwise-calibration.test.ts:146-155): one NaN weight + one negative weight, real dependent scores
    present, asserts the result matches that file's DEFAULT_COMPOSITE_WEIGHTS, not an objective-only
    collapse.
  • All three files' existing explicit-all-zero tests pass unchanged (no behavior regression).
  • The Codecov-graded vitest convergence suite (test/unit/engine-calibration-convergence.test.ts)
    gains the matching invalid-weight cases for all three composers, so the changed engine-src branches
    are covered under vitest.
  • Full local gate green (npm run test:ci); bug-catch verified (reverting the fix fails exactly the
    three new invalid-weight assertions).

…valid weights

The three sibling calibration composers claimed convergence with
pairwise-calibration.ts's invalid-weight handling but only replicated the
explicit-all-zero half. Calling computeGateVerdictCompositeCalibrationScore,
computeReviewerConsensusCompositeCalibrationScore, or
computeFindingSeverityCompositeCalibrationScore with a NaN/negative weight
alongside otherwise-valid weights silently collapsed to a 100%-objective-anchor
composite instead of the documented 45/35/20 default -- a real miscalibration on
both the miner (AMS) and review (ORB) sides that share this module.

Add the same isInvalidWeight() distinction to each normalizeCompositeWeights,
mirroring pairwise-calibration.ts:58-83: an empty clamped total recovers to
DEFAULT_COMPOSITE_WEIGHTS when any input weight was NaN/negative, while the
explicit-all-zero case still returns real zeros for the objective-only fallback.

Adds an invalid-weight test to each engine suite (mirroring
pairwise-calibration.test.ts) and the matching cases to the Codecov-graded
vitest convergence suite so the changed engine-src branches are covered.

Closes JSONbored#8643
@shin-core
shin-core requested a review from JSONbored as a code owner July 26, 2026 01:07
@superagent-security

Copy link
Copy Markdown
Contributor

🚨 Contributor flagged. Click here for more info: Superagent Dashboard

@superagent-security superagent-security Bot added the contributor:flagged Contributor flagged for review by trust analysis. label Jul 26, 2026
@superagent-security

Copy link
Copy Markdown
Contributor

Superagent didn't find any vulnerabilities or security issues in this PR.

@codecov

codecov Bot commented Jul 26, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 93.80%. Comparing base (cdd760d) to head (ba04fa4).

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #8724      +/-   ##
==========================================
- Coverage   93.80%   93.80%   -0.01%     
==========================================
  Files         797      797              
  Lines       79493    79508      +15     
  Branches    24085    24094       +9     
==========================================
+ Hits        74567    74580      +13     
  Misses       3555     3555              
- Partials     1371     1373       +2     
Flag Coverage Δ
backend 95.08% <100.00%> (-0.01%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
...oopover-engine/src/finding-severity-calibration.ts 34.94% <100.00%> (+2.34%) ⬆️
...es/loopover-engine/src/gate-verdict-calibration.ts 41.37% <100.00%> (+2.32%) ⬆️
...pover-engine/src/reviewer-consensus-calibration.ts 100.00% <100.00%> (ø)

... and 1 file with indirect coverage changes

@loopover-orb loopover-orb Bot added the gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier. label Jul 26, 2026
@loopover-orb

loopover-orb Bot commented Jul 26, 2026

Copy link
Copy Markdown
Contributor

Tip

✅ LoopOver review result - approve/merge recommended

Review updated: 2026-07-26 05:25:09 UTC

7 files · 1 AI reviewer · no blockers · readiness 95/100 · CI green · unknown

✅ Suggested Action - Approve/Merge

  • safe to merge

Review summary
The AI review returned non-blocking notes for this change but did not include a separate narrative summary. Review the nits below before deciding this PR.

Nits — 4 non-blocking
  • The Contributor trust check failed with no detail provided in CI STATUS, and this branch is 6 commits behind default — that's a plausible pre-existing cause rather than a defect in this diff, but worth confirming isn't masking something new.
  • packages/loopover-engine/src/reviewer-consensus-calibration.ts is flagged as slightly over the 400-line file-size threshold per the external brief; consider whether this file is due for a split in a follow-up, though this PR's addition (17 lines) is not the primary driver.
  • The PR description states an issue link (fix(engine): three calibration composers silently collapse to 100%-objective-anchor on NaN/negative weight input #8643) is closed by this change, which satisfies the issue-scope requirement — confirm fix(engine): three calibration composers silently collapse to 100%-objective-anchor on NaN/negative weight input #8643 is in fact an open, maintainer-acknowledged issue and not self-filed to satisfy the linking requirement.
  • Nothing structural to change — the mirrored implementation is correct and the new tests target the real total<=0 && isInvalidWeight branch in each file, matching pairwise-calibration.ts's existing pattern exactly.

Decision drivers

  • ✅ Code review — No blockers (1 reviewer)
  • ✅ Gate result — Passing (No configured blocker found.)
Context & advisory signals — never blocks the verdict
Signal Result Evidence
Linked issue ✅ Linked #8643
Related work ✅ No active overlap found No same-issue or scoped active PR overlap found.
Change scope ✅ 20/20 Low review scope from cached public metadata (1 linked issue).
Validation posture ✅ 25/25 PR body includes validation/test evidence.
Contributor workload ✅ 10/10 Author activity: 45 registered-repo PR(s), 25 merged, 0 issue(s).
Contributor context ✅ Confirmed Gittensor contributor shin-core; Gittensor profile; 45 PR(s), 0 issue(s).
Improvement ✅ Minor risk: clean · value: minor
Linked issue satisfaction

Addressed
The PR adds the isInvalidWeight() distinction to normalizeCompositeWeights in all three files (gate-verdict, reviewer-consensus, finding-severity), mirroring pairwise-calibration.ts exactly, and adds new tests in each file's own suite plus the convergence suite that verify NaN/negative weights recover to DEFAULT_COMPOSITE_WEIGHTS while existing explicit-all-zero tests remain unchanged.

Review context
  • Author: shin-core
  • Role context: outside_contributor
  • Public audience mode: oss maintainer
  • Lane context: Repository is configured for direct PR review.
  • Public profile languages: TypeScript, JavaScript, Solidity, Dart, Python, CSS, PHP, Rust
  • Official Gittensor activity: 45 PR(s), 0 issue(s).
  • PR-specific overlap: none found.
Contributor next steps
  • Start here: Triage stale or unlinked PRs.
Signal definitions
  • Related work = same linked issue, overlapping active PRs, or title/path similarity.
  • Change scope = cached public metadata such as size labels, draft state, and review-burden hints.
  • Validation posture = whether the PR provides enough public validation/test evidence for maintainer review.
  • Contributor workload = public contributor activity and cleanup pressure, not a repo-wide quality failure.
  • Contributor context = public GitHub/Gittensor identity context; non-Gittensor status is not a blocker.
🧪 Chat with LoopOver

Ask LoopOver a question about this PR directly in a comment — grounded only in the same cached, public-safe facts shown above, never a new claim.

  • @loopover ask &lt;question&gt; answers contribution-quality Q&A with source citations and freshness.
  • @loopover chat &lt;question&gt; answers in natural prose from cached decision-pack facts via local inference (maintainer/collaborator; read-only).
  • A plain-language @loopover mention with a real question is routed to the closest matching read-only command automatically — no exact syntax required.

Full command reference: https://loopover.ai/docs/loopover-commands

🧪 Experimental — new and may change.

🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed


💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →.

Checked by LoopOver, a quiet PR intelligence layer for OSS maintainers.

  • Re-run LoopOver review

@JSONbored
JSONbored merged commit 6735ef1 into JSONbored:main Jul 26, 2026
7 of 8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

contributor:flagged Contributor flagged for review by trust analysis. gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix(engine): three calibration composers silently collapse to 100%-objective-anchor on NaN/negative weight input

2 participants