Skip to content

feat(orb): salvageability axis — hold fixable, well-authored PRs instead of one-shot-closing - #8986

Merged
JSONbored merged 2 commits into
mainfrom
feat/salvageability-axis
Jul 26, 2026
Merged

feat(orb): salvageability axis — hold fixable, well-authored PRs instead of one-shot-closing#8986
JSONbored merged 2 commits into
mainfrom
feat/salvageability-axis

Conversation

@JSONbored

Copy link
Copy Markdown
Owner

Closes #8962. Advances #8828.

The decision audit's deepest finding: defect-confidence measures whether the defect is REAL, but close-correctness depends on whether closing is the right ACTION — 28 audited holds carried real findings at confidence 0.55–0.97 and still merged (17 as-is, 11 after the author fixed the flagged defect), while acted closes at the same confidences ran 96.0% correct. Same confidence, opposite close-correctness, split entirely by salvageability. That class is the road from ~96% to the 99.5% target.

The score (src/review/salvageability.ts, pure): deterministic and explainable — no AI call, every point named in factors:

  • defect-class fixability from the finding text (mechanical +45 / unknown +15 / structural +0 — and structural WINS when both match: no reputation makes a fabricated test or scope violation salvageable-in-place);
  • the author's realized MERGED history in THIS repo (+40 at ≥3, +25 at ≥1 — outcomes, not open PRs: the feat(orb): weekly stratified decision-audit sampling with a frozen adjudication rubric #8840 lesson);
  • in-flight iteration (+15 at ≥2 review cycles).

The wire (salvageability-wire.ts): two cheap reads (merged count via review_audit pr_outcome, cycles via decision_records), fail-OPEN — a DB blip degrades to today's behavior, never blocks or unblocks.

The gate seam (resolveAiReviewSalvageableHold): resolveAiReviewLowConfidenceHold's sibling for the OTHER side of the floor — fires only when the knob is set, the failure is AI-judgment-only, every blocker sits at/above the close-confidence floor, and the score clears gate.aiReview.salvageabilityMinScore. Routes into the existing held-for-review path with fix-it guidance; the low-confidence hold keeps precedence below the floor. Knob absent (default) short-circuits before any IO.

Auditability: decision records bump to schema v3 carrying salvageability: {score, factors} whenever an AI judgment shaped the decision (recomputed at finalize even when the knob is off, so the boundary accrues evidence BEFORE anyone enables it); the adjudication rubric gains the additive salvageable_close reason category (correct/incorrect semantics unchanged — version 1 labels stay comparable).

Config-as-code parity sweep: engine manifest type/parse/serializer/round-trip + src overlay + policy passthrough + OpenAPI schema (regenerated) + both example-YAML twins (the #8846 lesson).

Tests: 12 pure/resolver/wire tests (every factor tier, structural-wins, all stand-down arms, fail-open); manifest parse/round-trip/overlay; policy passthrough both arms; and an end-to-end queue test — an ABOVE-floor consensus defect on a salvageable PR (manifest knob 60, mechanical finding, 2 merged priors) is HELD not closed, with the v3 record carrying score 70 and its factors. Changed-line coverage: all statements and branches covered.

…ead of one-shot-closing

The 2026-07-26 decision audit proved defect confidence alone tops out
around 96% close precision: 28 contributor PRs held on real high-
confidence findings later merged — the residual error class is 'real
defect, salvageable PR'. This adds the second axis as a deterministic,
explainable score (fixable defect class + the author's realized merged
history here + in-flight iteration; no AI call, every point named in
factors) and a manifest-only gate.aiReview.salvageabilityMinScore knob:
when set and an AT/ABOVE-floor AI-judgment close would fire, a clearing
score routes it to the existing held-for-review path with fix-it
guidance. Unset (the default) is byte-identical to today. Decision
records bump to v3 carrying {score, factors} whenever an AI judgment
shaped the decision, so the new boundary is auditable from day one; the
rubric gains the additive salvageable_close reason category.
@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Jul 26, 2026

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Preview URL Updated (UTC)
✅ Deployment successful!
View logs
loopover-ui bc9e81a Commit Preview URL

Branch Preview URL
Jul 26 2026, 03:39 PM

@superagent-security

Copy link
Copy Markdown
Contributor

Superagent didn't find any vulnerabilities or security issues in this PR.

@github-actions

github-actions Bot commented Jul 26, 2026

Copy link
Copy Markdown
Contributor

Logic backtest

Replayed 0 historical case(s) for linked_issue_scope_mismatch through the base (904aeb9) and head (bc9e81a) versions of its detection logic (corpus checksum 4f53cda18c2b).

Backtest comparison: linked_issue_scope_mismatch

Verdict: unchanged — no comparable axis moved.

Advisory only — this check never blocks merge (#8105).

@codecov

codecov Bot commented Jul 26, 2026

Copy link
Copy Markdown

❌ 1 Tests Failed:

Tests completed Failed Passed Skipped
21993 1 21992 21
View the top 1 failed test(s) by shortest run time
test/unit/miner-worktree-allocator-collisions.test.ts > loopover-miner worktree allocator collisions (#4298) > returns distinct worktree paths when multiple processes acquire simultaneously
Stack Traces | 0.731s run time
AssertionError: expected 1 to be 5 // Object.is equality

- Expected
+ Received

- 5
+ 1

 ❯ test/unit/miner-worktree-allocator-collisions.test.ts:148:41

To view more test analytics, go to the Test Analytics Dashboard
📋 Got 3 mins? Take this short survey to help us improve Test Analytics.

@loopover-orb loopover-orb Bot added the gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier. label Jul 26, 2026
@loopover-orb

loopover-orb Bot commented Jul 26, 2026

Copy link
Copy Markdown
Contributor

Caution

🛑 LoopOver review result - fixes required

Review updated: 2026-07-26 15:34:45 UTC

19 files · 1 AI reviewer · no blockers · CI failing · unstable

🛑 Suggested Action - Manual Review

Review summary
The AI review returned non-blocking notes for this change but did not include a separate narrative summary. Review the nits below before deciding this PR.

Nits — 5 non-blocking
  • src/queue/processors.ts: computeSalvageabilityForTarget is invoked twice per PR when the knob is on and an AI-judgment blocker exists (once for the plan-site hold, once again unconditionally at decision-record finalize) — two extra DB reads that could be avoided by threading the first result through instead of recomputing.
  • src/review/salvageability.ts: the MECHANICAL_PATTERN/STRUCTURAL_PATTERN regexes are long and any future addition to either taxonomy requires touching a dense inline pattern — consider extracting the phrase lists into named arrays for easier maintenance.
  • src/review/salvageability.ts: the point weights (45/40/25/15/3/2) are explained in prose comments but not as named constants, making the audit-relevant numbers harder to grep for from callers.
  • The two `validate*` CI checks failed with no detail provided, and this branch is 6 commits behind the default branch — that's a plausible cause worth ruling out via rebase before assuming a content defect.
  • PR author also opened the linked issue — Link an issue that was opened by a different contributor, or provide a rationale for why this self-authored issue represents genuine discovery work.

CI checks failing

  • validate
  • validate-tests
  • validate-code

Decision drivers

  • ✅ Code review — No blockers (1 reviewer)
  • ✅ Gate result — Passing (No configured blocker found.)
Context & advisory signals — never blocks the verdict
Signal Result Evidence
Linked issue ✅ Linked #8962
Related work ✅ No active overlap found No same-issue or scoped active PR overlap found.
Change scope ✅ 20/20 Low review scope from cached public metadata (1 linked issue).
Validation posture ✅ 25/25 PR body includes validation/test evidence.
Contributor workload ✅ 10/10 Author activity: 13 registered-repo PR(s), 13 merged, 293 issue(s).
Contributor context ✅ Confirmed Gittensor contributor JSONbored; Gittensor profile; 13 PR(s), 293 issue(s).
Improvement ✅ Minor risk: clean · value: minor
Review context
  • Author: JSONbored
  • Role context: owner (maintainer lane)
  • Public audience mode: oss maintainer
  • Lane context: Repository is configured for direct PR review.
  • Public profile languages: Python, TypeScript, Ruby, Go, MDX, Shell, Solidity, JavaScript
  • Official Gittensor activity: 13 PR(s), 293 issue(s).
  • PR-specific overlap: none found.
Contributor next steps
  • Start here: Treat this as maintainer-lane context rather than normal contributor-lane activity.
  • Then work through the remaining 1 step in the Signals table above.
Signal definitions
  • Related work = same linked issue, overlapping active PRs, or title/path similarity.
  • Change scope = cached public metadata such as size labels, draft state, and review-burden hints.
  • Validation posture = whether the PR provides enough public validation/test evidence for maintainer review.
  • Contributor workload = public contributor activity and cleanup pressure, not a repo-wide quality failure.
  • Contributor context = public GitHub/Gittensor identity context; non-Gittensor status is not a blocker.
🧪 Chat with LoopOver

Ask LoopOver a question about this PR directly in a comment — grounded only in the same cached, public-safe facts shown above, never a new claim.

  • @loopover ask <question> answers contribution-quality Q&A with source citations and freshness.
  • @loopover chat <question> answers in natural prose from cached decision-pack facts via local inference (maintainer/collaborator; read-only).
  • A plain-language @loopover mention with a real question is routed to the closest matching read-only command automatically — no exact syntax required.

Full command reference: https://loopover.ai/docs/loopover-commands

🧪 Experimental — new and may change.

Decision record
  • action: hold · clause: success
  • config: 03a7f8b529a9 · pack: oss-anti-slop
  • record: e6cf91f658a0 (schema v2, head 2cb4e67)
Visual preview
Route Viewport Before (production) After (this PR's preview) Diff
/ desktop before /
before /
after /
after /
/ mobile before / (mobile)
before / (mobile)
after / (mobile)
after / (mobile)

Click any thumbnail to open the full-size screenshot. Before = production · After = this PR's preview deploy.

Scroll preview
Route Before (production) After (this PR's preview)
/ before / (scroll)
before / (scroll)
after / (scroll)
after / (scroll)

A short scroll-through clip (desktop) — click either thumbnail to open the full animation. Evidence for scroll-linked behavior a single screenshot can't show.

🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed


💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →.

Checked by LoopOver, a quiet PR intelligence layer for OSS maintainers.

  • Re-run LoopOver review

@loopover-orb loopover-orb Bot added the manual-review Gittensor contributor context label Jul 26, 2026
Engine version 3.14.1 -> 3.15.0 (the manifest gate type gained
aiReviewSalvageabilityMinScore — a public engine surface change — and
the advisory twin-pair guard accepts a bump where the new resolver
deliberately does not belong in gate-advisory.ts, matching its
low-confidence sibling). Sync packages/loopover-miner/expected-engine
.version, register the settings->yml alias row for salvageabilityMinScore
in check-docs-drift, and update the two decision-record schemaVersion
assertions to v3.
@JSONbored
JSONbored merged commit 7fd84ff into main Jul 26, 2026
7 of 9 checks passed
@JSONbored
JSONbored deleted the feat/salvageability-axis branch July 26, 2026 15:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier. manual-review Gittensor contributor context

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Close decision needs a salvageability axis — defect confidence alone does not predict close-correctness

1 participant