Skip to content

fix(engine): dedup provider-track-record aggregation per provider+target - #8955

Closed
jeffrey701 wants to merge 1 commit into
JSONbored:mainfrom
jeffrey701:fix-8876-provider-track-record-dedup
Closed

fix(engine): dedup provider-track-record aggregation per provider+target#8955
jeffrey701 wants to merge 1 commit into
JSONbored:mainfrom
jeffrey701:fix-8876-provider-track-record-dedup

Conversation

@jeffrey701

Copy link
Copy Markdown
Contributor

What

computeProviderTrackRecords
(packages/loopover-engine/src/calibration/provider-track-record.ts) builds a stancesByTarget
map that correctly collapses multiple same-provider signals for one targetKey to the latest vote,
but the main aggregation loop then iterated the raw signals array — incrementing
signals/decided/shared/consensus/agreed once per raw row rather than once per distinct
(provider, targetKey) pair. Since the real caller loadLiveProviderTrackRecords reads raw
audit-event rows with no dedup, a provider re-reviewing the same PR (common after a new push) had its
precision and agreement stats inflated proportionally to how many times it re-reviewed.

Change

Collapse repeated (provider, targetKey) signals to the latest row before aggregating (a
Map keyed by provider + targetKey, keeping the last-seen row — matching stancesByTarget's own
latest-vote .set dedup), then iterate that deduped set. One distinct pair counts once, scored on
its latest vote.

Validation

  • New test in test/unit/provider-track-record-engine.test.ts (imports the engine source, so Codecov
    grades it): a provider with two rows for the same target (pass then a re-review fail) now
    produces signals: 1 scored on the latest fail vote — was signals: 2 / agreementRate: 0.5
    before. Existing multi-provider consensus/split/precision assertions are unaffected.
  • npx vitest run test/unit/provider-track-record-engine.test.ts → 8/8 pass; 100% line+branch
    coverage on every changed line.

Closes #8876

@jeffrey701
jeffrey701 requested a review from JSONbored as a code owner July 26, 2026 14:19
@superagent-security

Copy link
Copy Markdown
Contributor

Superagent didn't find any vulnerabilities or security issues in this PR.

@jeffrey701 jeffrey701 closed this Jul 26, 2026
@jeffrey701

Copy link
Copy Markdown
Contributor Author

Closing as a duplicate of #8954 (opened ~70s earlier for the same #8876 fix). Deferring to that PR.

@codecov

codecov Bot commented Jul 26, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 93.86%. Comparing base (32ee72e) to head (ab729bb).
⚠️ Report is 9 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #8955      +/-   ##
==========================================
+ Coverage   90.56%   93.86%   +3.30%     
==========================================
  Files          96      807     +711     
  Lines       22490    80334   +57844     
  Branches     3884    24343   +20459     
==========================================
+ Hits        20367    75408   +55041     
- Misses       1945     3561    +1616     
- Partials      178     1365    +1187     
Flag Coverage Δ
backend 95.15% <100.00%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
...er-engine/src/calibration/provider-track-record.ts 100.00% <100.00%> (ø)

... and 710 files with indirect coverage changes

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix(engine): computeProviderTrackRecords double-counts a provider's repeated votes on the same PR

1 participant