Skip to content

orb(rewards): the Gittensor reward multiplier is chosen by contributor-authored PR title text #9077

Description

@JSONbored

ORB writes gittensor:bug / gittensor:feature / gittensor:priority onto PRs, and per our own manifest comments that label is the upstream reward multiplier (bug = 0.05x). Two ways a contributor controls it.

1. Title-derived classification (CONFIRMED). When linkedIssueLabelPropagation is off — the shipped default — or when no linked issue carries a mapped label, the type label falls through to deriveKindFromTitle(pr.title) (packages/loopover-engine/src/settings/pr-type-label.ts ~40-48, ~181). That is a pure regex over the PR title: prefix feat/feature, subject matching FEATURE_TITLE_ACTION_RE (add|create|enable|implement|support|wire|…) and not matching FEATURE_TITLE_DOWNGRADE_RE.

Nothing reads the diff, the changed files, the AI review, or the linked issue. A contributor picks their own multiplier by wording: feat: add support for X instead of fix: correct X.

The inverse hurts honest contributors just as much: FEATURE_TITLE_DOWNGRADE_RE contains incidental words like cache, format, rename, guard, normalize, classify. A genuine feature titled feat: add response caching is downgraded to gittensor:bug — a 20x reward cut caused by one word.

2. gittensor:priority from a merely-claimed issue (SUSPECTED). pr.linkedIssues comes from a regex over the PR body (extractLinkedIssueNumbersWithOverflow), which is author-controlled. isLinkedIssueTrustworthy (src/review/linked-issue-label-propagation-fetch.ts ~80-88) returns true unconditionally for any open issue, and this repo opts gittensor:priority into trustMaintainerAuthoredIssueForReward: true — so the unlock reduces to "the linked issue is maintainer-authored", which is true of essentially every open gittensor:priority issue by construction.

Writing Fixes #<any open priority issue> gets the priority label applied at review time. Nothing in the label path verifies the PR addresses it; the linked-issue satisfaction gate is a real mitigation but is a separate subsystem, runs after, and the label is never removed if the disposition later disagrees. The grey zone (a small real fix citing a large priority issue it partially touches) is where this bites.

Fix

  1. The multiplier-bearing label must not derive from author-controlled free text. Make issue-label propagation the only source for reward-bearing categories and fall back to no label rather than a guessed one — or classify from the diff/changed-path evidence the review already computes.
  2. Condition reward-bearing propagation on the satisfaction verdict, not merely on a closing-keyword reference.

Metadata

Metadata

Assignees

Labels

maintainer-onlyOwner-only work — yields no Gittensor points.orbGittensory Orb related - maintainer self-hosting analytics.

Projects

No projects

Relationships

None yet

Development

No branches or pull requests

Issue actions