Skip to content

fix(engine): weight local-scorer totalTokenScore with TEST_FILE_CONTRIBUTION_WEIGHT - #8970

Closed
philluiz2323 wants to merge 2 commits into
JSONbored:mainfrom
philluiz2323:fix/8875-local-scorer-test-file-weight
Closed

fix(engine): weight local-scorer totalTokenScore with TEST_FILE_CONTRIBUTION_WEIGHT#8970
philluiz2323 wants to merge 2 commits into
JSONbored:mainfrom
philluiz2323:fix/8875-local-scorer-test-file-weight

Conversation

@philluiz2323

Copy link
Copy Markdown
Contributor

Problem

Closes #8875.

computeLocalScorerTokens returned a raw unweighted line sum as totalTokenScore. When that value is fed back into buildScorePreview as an explicit total, preview honors it as-is and skips the TEST_FILE_CONTRIBUTION_WEIGHT (0.05×) discount it applies when deriving its own total — so a test-heavy local scorer preview over-counts test lines in the contribution-bonus ramp.

Fix

Compute totalTokenScore as source + TEST_FILE_CONTRIBUTION_WEIGHT * test + nonCode, matching preview.ts's derivedTotalTokenScore, using the shared DEFAULT_SCORING_CONSTANTS constant.

Tests

  • Existing classification fixtures updated to expect the weighted total.
  • New test-heavy fixture: local-scorer total equals the weighted formula, and feeding that total into buildScorePreview matches the derived-total (no explicit total) path for contribution bonus / estimated merged score.
  • MCP loopover_run_local_scorer expectation updated accordingly.

Reverting the weight fails the agreement test. git diff --check clean.

Scope

Validation

  • git diff --check
  • vitest run test/unit/local-scorer.test.ts (7 pass)
  • vitest run test/unit/mcp-run-local-scorer.test.ts (2 pass)
  • Engine/scorer-only; unrelated UI/workers checks not exercised.

Safety

  • No secrets / private scores exposed
  • No UI changes (N/A evidence)

…IBUTION_WEIGHT

computeLocalScorerTokens summed raw test lines into totalTokenScore, so feeding
that total back into buildScorePreview bypassed the 0.05x test-file discount
preview.ts applies when deriving its own total. Compute the weighted total the
same way as derivedTotalTokenScore so a test-heavy local preview matches the gate.

Closes JSONbored#8875
@philluiz2323
philluiz2323 requested a review from JSONbored as a code owner July 26, 2026 14:37
@superagent-security

Copy link
Copy Markdown
Contributor

Superagent didn't find any vulnerabilities or security issues in this PR.

…ypecheck

DEFAULT_SCORING_CONSTANTS is Record<string, number>, so the indexed weight was
possibly undefined under strict typecheck. Default to 0.05 to satisfy TS18048.
@loopover-orb loopover-orb Bot closed this Jul 26, 2026
@loopover-orb

loopover-orb Bot commented Jul 26, 2026

Copy link
Copy Markdown
Contributor

This repository reviews pull requests one-shot: the PR must be correct as originally opened. Pushing an additional commit closes it automatically instead of restarting review — open a fresh pull request with every fix included.

@loopover-orb loopover-orb Bot added review-evasion Gittensor contributor context gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier. labels Jul 26, 2026
@codecov

codecov Bot commented Jul 26, 2026

Copy link
Copy Markdown

❌ 3 Tests Failed:

Tests completed Failed Passed Skipped
21941 3 21938 21
View the top 3 failed test(s) by shortest run time
test/unit/mcp-cli-plan-scorer-tools.test.ts > loopover-mcp plan-DAG + local-scorer + predict-gate tools (#6150) — pure tools > loopover_run_local_scorer computes source/test/non-code token scores from changed-file metadata
Stack Traces | 0.611s run time
AssertionError: expected 17.4 to be 25 // Object.is equality

- Expected
+ Received

- 25
+ 17.4

 ❯ test/unit/mcp-cli-plan-scorer-tools.test.ts:67:34
test/unit/miner-worktree-allocator-collisions.test.ts > loopover-miner worktree allocator collisions (#4298) > returns distinct worktree paths when multiple processes acquire simultaneously
Stack Traces | 0.853s run time
AssertionError: expected 2 to be 5 // Object.is equality

- Expected
+ Received

- 5
+ 2

 ❯ test/unit/miner-worktree-allocator-collisions.test.ts:148:41
test/unit/miner-worktree-allocator-collisions.test.ts > loopover-miner worktree allocator collisions (#4298) > rejects excess simultaneous cross-process acquire calls at the concurrency cap
Stack Traces | 0.926s run time
AssertionError: expected [ …(3) ] to have a length of 2 but got 3

- Expected
+ Received

- 2
+ 3

 ❯ test/unit/miner-worktree-allocator-collisions.test.ts:162:23

To view more test analytics, go to the Test Analytics Dashboard
📋 Got 3 mins? Take this short survey to help us improve Test Analytics.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier. review-evasion Gittensor contributor context

Projects

None yet

Development

Successfully merging this pull request may close these issues.

fix(engine): local-scorer's totalTokenScore bypasses the TEST_FILE_CONTRIBUTION_WEIGHT discount when fed back into the score preview

1 participant