Skip to content

fix(engine): weight local-scorer totalTokenScore with TEST_FILE_CONTRIBUTION_WEIGHT - #8977

Closed
philluiz2323 wants to merge 1 commit into
JSONbored:mainfrom
philluiz2323:fix/8875-local-scorer-test-weight-v2
Closed

fix(engine): weight local-scorer totalTokenScore with TEST_FILE_CONTRIBUTION_WEIGHT#8977
philluiz2323 wants to merge 1 commit into
JSONbored:mainfrom
philluiz2323:fix/8875-local-scorer-test-weight-v2

Conversation

@philluiz2323

Copy link
Copy Markdown
Contributor

Problem

Closes #8875.

computeLocalScorerTokens returned a raw unweighted line sum as totalTokenScore. When that value is fed back into buildScorePreview as an explicit total, preview honors it as-is and skips the TEST_FILE_CONTRIBUTION_WEIGHT (0.05×) discount it applies when deriving its own total — so a test-heavy local scorer preview over-counts test lines in the contribution-bonus ramp.

Re-filed after #8970: this repo is one-shot (follow-up commits auto-close). This PR includes the weight fix and the TS18048 ?? 0.05 typecheck narrowing in a single commit.

Fix

Compute totalTokenScore as source + TEST_FILE_CONTRIBUTION_WEIGHT * test + nonCode, matching preview.ts's derivedTotalTokenScore, using the shared DEFAULT_SCORING_CONSTANTS constant.

Tests

  • Existing classification fixtures updated to expect the weighted total.
  • New test-heavy fixture: local-scorer total equals the weighted formula, and feeding that total into buildScorePreview matches the derived-total path for contribution bonus / estimated merged score.
  • MCP loopover_run_local_scorer expectation updated.
  • Test reads the weight with ?? 0.05 so Record<string, number> indexing typechecks cleanly.

Reverting the weight fails the agreement test. git diff --check clean.

Scope

Validation

  • git diff --check
  • vitest run test/unit/local-scorer.test.ts (7 pass)
  • vitest run test/unit/mcp-run-local-scorer.test.ts (2 pass)
  • Engine/scorer-only; unrelated UI/workers checks not exercised.

Safety

  • No secrets / private scores exposed
  • No UI changes (N/A evidence)

…IBUTION_WEIGHT

computeLocalScorerTokens summed raw test lines into totalTokenScore, so feeding
that total back into buildScorePreview bypassed the 0.05x test-file discount
preview.ts applies when deriving its own total. Compute the weighted total the
same way as derivedTotalTokenScore so a test-heavy local preview matches the gate.

Tests use `?? 0.05` when reading DEFAULT_SCORING_CONSTANTS so strict typecheck
accepts the Record<string, number> index (TS18048).

Closes JSONbored#8875
@philluiz2323
philluiz2323 requested a review from JSONbored as a code owner July 26, 2026 14:48
@superagent-security

Copy link
Copy Markdown
Contributor

Superagent didn't find any vulnerabilities or security issues in this PR.

@codecov

codecov Bot commented Jul 26, 2026

Copy link
Copy Markdown

❌ 1 Tests Failed:

Tests completed Failed Passed Skipped
21945 1 21944 21
View the top 1 failed test(s) by shortest run time
test/unit/mcp-cli-plan-scorer-tools.test.ts > loopover-mcp plan-DAG + local-scorer + predict-gate tools (#6150) — pure tools > loopover_run_local_scorer computes source/test/non-code token scores from changed-file metadata
Stack Traces | 1.23s run time
AssertionError: expected 17.4 to be 25 // Object.is equality

- Expected
+ Received

- 25
+ 17.4

 ❯ test/unit/mcp-cli-plan-scorer-tools.test.ts:67:34

To view more test analytics, go to the Test Analytics Dashboard
📋 Got 3 mins? Take this short survey to help us improve Test Analytics.

@loopover-orb loopover-orb Bot added the gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier. label Jul 26, 2026
@loopover-orb

loopover-orb Bot commented Jul 26, 2026

Copy link
Copy Markdown
Contributor

Caution

🛑 LoopOver review result - fixes required

Review updated: 2026-07-26 15:15:45 UTC

4 files · 1 AI reviewer · no blockers · CI failing · unstable

🛑 Suggested Action - Fix Blockers

  • AI review did not produce public notes: The configured AI reviewer returned no usable public assessment for this PR head.

Review summary
AI review is unavailable for this PR head. LoopOver is holding this PR for manual review until the configured AI provider returns a usable public review summary.

Nits — 1 non-blocking
  • AI review did not produce public notes — Fix the configured AI provider, then re-run LoopOver review before relying on the result.

CI checks failing

  • validate
  • validate-tests

Decision drivers

  • ✅ Code review — No blockers (1 reviewer)
  • ⚠️ Gate result — Not blocking (Advisory; not blocking this PR.)
Context & advisory signals — never blocks the verdict
Signal Result Evidence
Linked issue ✅ Linked #8875
Related work ✅ No active overlap found No same-issue or scoped active PR overlap found.
Change scope ✅ 20/20 Low review scope from cached public metadata (1 linked issue).
Validation posture ✅ 25/25 PR body includes validation/test evidence.
Contributor workload ✅ 10/10 Author activity: 989 registered-repo PR(s), 565 merged, 125 issue(s).
Contributor context ✅ Confirmed Gittensor contributor philluiz2323; Gittensor profile; 989 PR(s), 125 issue(s).
Improvement ✅ Minor risk: clean · value: minor
Linked issue satisfaction

Addressed
The diff applies TEST_FILE_CONTRIBUTION_WEIGHT to totalTokenScore in computeLocalScorerTokens (matching preview.ts's derivedTotalTokenScore formula) and adds a test-heavy fixture that asserts both the weighted total and the resulting contributionBonus/estimatedMergedScore agree between the explicit-total and derived-total preview paths.

Review context
  • Author: philluiz2323
  • Role context: outside_contributor
  • Public audience mode: oss maintainer
  • Lane context: Repository is configured for direct PR review.
  • Public profile languages: Python, JavaScript, MDX, TypeScript, CSS, Cuda, HTML, Kotlin
  • Official Gittensor activity: 989 PR(s), 125 issue(s).
  • PR-specific overlap: none found.
Contributor next steps
  • Start here: Triage stale or unlinked PRs.
Signal definitions
  • Related work = same linked issue, overlapping active PRs, or title/path similarity.
  • Change scope = cached public metadata such as size labels, draft state, and review-burden hints.
  • Validation posture = whether the PR provides enough public validation/test evidence for maintainer review.
  • Contributor workload = public contributor activity and cleanup pressure, not a repo-wide quality failure.
  • Contributor context = public GitHub/Gittensor identity context; non-Gittensor status is not a blocker.
🧪 Chat with LoopOver

Ask LoopOver a question about this PR directly in a comment — grounded only in the same cached, public-safe facts shown above, never a new claim.

  • @loopover ask &lt;question&gt; answers contribution-quality Q&A with source citations and freshness.
  • @loopover chat &lt;question&gt; answers in natural prose from cached decision-pack facts via local inference (maintainer/collaborator; read-only).
  • A plain-language @loopover mention with a real question is routed to the closest matching read-only command automatically — no exact syntax required.

Full command reference: https://loopover.ai/docs/loopover-commands

🧪 Experimental — new and may change.

🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed


💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →.

Checked by LoopOver, a quiet PR intelligence layer for OSS maintainers.

  • Re-run LoopOver review

@loopover-orb

loopover-orb Bot commented Jul 26, 2026

Copy link
Copy Markdown
Contributor

LoopOver is closing this pull request on the maintainer's behalf (CI is failing (validate, validate-tests)). This is an automated maintenance action — to pursue this change, please open a new pull request with the issues resolved. Closed PRs may be analyzed later to improve review accuracy, but they are not automatically reopened or re-reviewed.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix(engine): local-scorer's totalTokenScore bypasses the TEST_FILE_CONTRIBUTION_WEIGHT discount when fed back into the score preview

1 participant