Skip to content

test(stats): assert the totals/parity invariant instead of describing it - #9972

Merged
loopover-orb[bot] merged 1 commit into
mainfrom
test/cross-surface-stats-parity-invariant-9971
Jul 31, 2026
Merged

test(stats): assert the totals/parity invariant instead of describing it#9972
loopover-orb[bot] merged 1 commit into
mainfrom
test/cross-surface-stats-parity-invariant-9971

Conversation

@JSONbored

Copy link
Copy Markdown
Owner

What this adds

A test, and nothing else. #9967 already fixed #9963.

#9963's third acceptance criterion asked for a regression test pinning that totals.handled and reviewParity.verdicts cannot disagree on the same payload. #9967's tests cover the new aggregate read thoroughly, but the cross-surface invariant itself appears only in a comment:

//   reviewParity.verdicts = 2123   (read from decision_records, ungated)

Why the gap is worth closing specifically here

This is the one defect class per-surface testing structurally cannot catch, and verify-public-claims.ts says so in as many words:

the two are computed by different code over the same ledger, which is the divergence no amount of per-surface testing catches, because each surface is individually self-consistent.

Both halves were individually correct and individually green while production served handled: 0 beside verdicts: 2123. getPublicStats and loadReviewParityRollups are still separate reads with separate gating, so the next change to either can reintroduce the disagreement with every existing test passing.

How it asserts

Both halves are built over one seeded ledger on a real migrated D1, then compared by calling the verifier's own checkStatsParity — the function that printed the original FAIL against the Orb — rather than restating its rule. A hand-written >= would drift from the tool; this cannot.

Four cases:

Case Why it is here
Self-hosted Orb, populated ledger The regression. handled=12, verdicts=12, claim passes. Before #9967: handled=0.
A re-decided pull request The case a naive equality assertion gets wrong — a re-evaluation appends a verdict row, so parity counts 3 while handled counts 2 pull requests. Consistent, not contradictory; only parity exceeding all-time handled is the accounting error.
Empty ledger The hosted Worker's shape. 0 and 0 agree — a quiet deployment must not read as a broken one.
Zeroed handled beneath a populated rollup The failing direction, byte-for-byte the contradiction production published. Without it the three passing assertions could be vacuous.

Mutation testing

Reverting #9967's ledger fallback (ledgerTotals = null) fails exactly the two behavioural assertions, while the empty-ledger case and the failing-direction guard correctly stay green — so the suite is sensitive to the fix without being sensitive to unrelated noise.

Verification

  • npx tsc --noEmit clean
  • typecheck, fixture-clock-races:check, dead-source-files:check, dead-exports:check, coverage-boltons:check, import-specifiers:check green
  • No production code touched.

Closes #9971

@superagent-security

Copy link
Copy Markdown
Contributor

Superagent didn't find any vulnerabilities or security issues in this PR.

@loopover-orb

loopover-orb Bot commented Jul 30, 2026

Copy link
Copy Markdown
Contributor

Warning

⏸️ LoopOver review result - manual review recommended

Review updated: 2026-07-31 02:47:37 UTC

1 file · 1 AI reviewer · no blockers · CI green · clean

⏸️ Suggested Action - Manual Review

Review summary
This is a self-contained regression test file that seeds a decision_records ledger on a real D1 test env, builds both getPublicStats and loadReviewParityRollups off it, and asserts them through the verifier's own checkStatsParity across four cases (populated, re-decided PR, empty ledger, and a mutation-guard that manually zeroes handled to confirm the assertion is data-driven rather than vacuous). The re-decided-PR case correctly encodes that parity counting verdicts and totals counting distinct pull requests can differ without being a contradiction, which is the subtle edge the naive-equality approach in the PR's own critique would get wrong. The raw-SQL insert into decision_records bypasses the normal writer (persistDecisionRecord), which is reasonable for a read-path test but means the row shape is now a second place the schema must be kept in sync with.

Nits — 6 non-blocking
  • Verify the `Env` type used in `seedVerdict`'s signature is available (likely an ambient Workers type, not shown in this diff) — test/unit/public-stats-parity-invariant.test.ts:26.
  • The raw INSERT into `decision_records` (id, repo_full_name, pull_number, head_sha, action, reason_code, record_digest, record_json, created_at) duplicates the table's column list outside the normal writer path; confirm it stays in sync with the actual schema/migration rather than drifting silently.
  • The CI failures on `validate-tests` and `validate` have no detail provided; given the branch is 1 commit behind default, this is plausibly caused by something that landed on main after divergence rather than this diff — worth a rebase to confirm before merging.
  • Consider a comment or assertion noting the tolerance value (5) used implicitly by checkStatsParity in the re-decided case, since the test's margin (3 vs 2) doesn't meaningfully exercise the tolerance boundary itself.
  • If persistDecisionRecord already validates row shape, consider adding a thin schema-shape assertion or importing shared column constants to reduce the two-places-to-update risk noted above.
  • PR author also opened the linked issue — Link an issue that was opened by a different contributor, or provide a rationale for why this self-authored issue represents genuine discovery work.

Decision drivers

  • ✅ Code review — No blockers (1 reviewer)
  • ✅ Gate result — Passing (No configured blocker found.)
Context & advisory signals — never blocks the verdict
Signal Result Evidence
Linked issue ✅ Linked #9963, #9971
Related work ✅ No active overlap found No same-issue or scoped active PR overlap found.
Change scope ✅ 20/20 Low review scope from cached public metadata (2 linked issues).
Validation posture ✅ 25/25 PR body includes validation/test evidence.
Contributor workload ✅ 10/10 Author activity: 9 registered-repo PR(s), 8 merged, 274 issue(s).
Contributor context ✅ Confirmed Gittensor contributor JSONbored; Gittensor profile; 9 PR(s), 274 issue(s).
Improvement ℹ️ Insufficient signal risk: clean · value: insufficient-signal
Linked issue satisfaction

Partially addressed
The PR adds a solid regression test that exercises the totals/parity invariant through the verifier's own checkStatsParity, directly satisfying the issue's third acceptance criterion, but it explicitly relies on a prior PR (#9967) for the actual fix to totals.handled and does not itself modify getPublicStats or the underlying aggregate logic.

Review context
  • Author: JSONbored
  • Role context: owner (maintainer lane)
  • Public audience mode: oss maintainer
  • Lane context: Repository is configured for direct PR review.
  • Public profile languages: Python, TypeScript, Ruby, Go, MDX, Shell, Solidity, JavaScript
  • Official Gittensor activity: 9 PR(s), 274 issue(s).
  • PR-specific overlap: none found.
Contributor next steps
  • Start here: Treat this as maintainer-lane context rather than normal contributor-lane activity.
Signal definitions
  • Related work = same linked issue, overlapping active PRs, or title/path similarity.
  • Change scope = cached public metadata such as size labels, draft state, and review-burden hints.
  • Validation posture = whether the PR provides enough public validation/test evidence for maintainer review.
  • Contributor workload = public contributor activity and cleanup pressure, not a repo-wide quality failure.
  • Contributor context = public GitHub/Gittensor identity context; non-Gittensor status is not a blocker.
🧪 Chat with LoopOver

Ask LoopOver a question about this PR directly in a comment — grounded only in the same cached, public-safe facts shown above, never a new claim.

  • @loopover ask <question> answers contribution-quality Q&A with source citations and freshness.
  • @loopover chat <question> answers in natural prose from cached decision-pack facts via local inference (maintainer/collaborator; read-only).
  • A plain-language @loopover mention with a real question is routed to the closest matching read-only command automatically — no exact syntax required.

Full command reference: https://loopover.ai/docs/loopover-commands

🧪 Experimental — new and may change.

Decision record
  • action: hold · clause: success
  • config: 3d45500d4350f8ddbd1310ea6f6ee5f362396cb740901ea433441682463f8f6b · pack: oss-anti-slop · ci: failed
  • record: 62f3a54eb2013487f122096fe3fd36c611810f04485d55133d724fae25f2dd4a (schema v6, head d713746)

🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed


💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →.

Checked by LoopOver, a quiet PR intelligence layer for OSS maintainers.

  • Re-run LoopOver review

@codecov

codecov Bot commented Jul 30, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 91.00%. Comparing base (910fa59) to head (600312e).
⚠️ Report is 1 commits behind head on main.
✅ All tests successful. No failed tests found.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #9972      +/-   ##
==========================================
- Coverage   91.88%   91.00%   -0.88%     
==========================================
  Files         928      928              
  Lines      113722   113722              
  Branches    27436    27436              
==========================================
- Hits       104489   103495     -994     
- Misses       7936     9125    +1189     
+ Partials     1297     1102     -195     
Flag Coverage Δ
backend 94.11% <ø> (-1.56%) ⬇️

Flags with carried forward coverage won't be shown. Click here to find out more.
see 3 files with indirect coverage changes

@loopover-orb loopover-orb Bot added the manual-review Gittensor contributor context label Jul 30, 2026
@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Jul 31, 2026

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Updated (UTC)
❌ Deployment failed
View logs
loopover-ui d713746 Jul 31 2026, 12:30 AM

#9963 was an Orb publishing `totals.handled: 0` in the same payload as
`reviewParity.verdicts: 2123`, both derived from the same ledger. #9967 fixed
the aggregate and covered it well -- but per surface, which is the blind spot
the defect lived in. Both halves were individually correct and individually
green while production served the contradiction, and the verifier's own source
says why per-surface testing cannot catch this class:

  "the two are computed by different code over the same ledger, which is the
   divergence no amount of per-surface testing catches, because each surface is
   individually self-consistent."

`getPublicStats` and `loadReviewParityRollups` remain separate reads with
separate gating, so nothing stopped the next change to either from
reintroducing the disagreement with every existing test still green.

These build BOTH halves over one seeded ledger and compare them through the
verifier's own `checkStatsParity` -- the function that printed the original FAIL
against the Orb -- rather than restating its rule, so the test cannot drift from
the tool that decides whether production is publishing a contradiction.

Covers the case a hand-written equality assertion would get wrong: a re-decided
pull request appends a verdict row, so parity counts 3 while `handled` counts 2
pull requests. That is consistent, not a contradiction, and only parity
EXCEEDING all-time handled is the accounting error.

Mutation-tested against #9967's fix: reverting the ledger fallback fails exactly
the two behavioural assertions, while the empty-ledger case and the
failing-direction guard correctly stay green.

Closes #9971
@JSONbored
JSONbored force-pushed the test/cross-surface-stats-parity-invariant-9971 branch from d713746 to 600312e Compare July 31, 2026 02:28
@loopover-orb loopover-orb Bot removed the manual-review Gittensor contributor context label Jul 31, 2026

@loopover-orb loopover-orb Bot left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LoopOver approves — the gate is satisfied and CI is green.

@loopover-orb
loopover-orb Bot merged commit 244f3a9 into main Jul 31, 2026
8 checks passed
@loopover-orb
loopover-orb Bot deleted the test/cross-surface-stats-parity-invariant-9971 branch July 31, 2026 02:47
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

1 participant