Skip to content

feat(miner): record feasibility-verdict reasons as rule-fired signals on the infeasible path#8569

Merged
JSONbored merged 1 commit into
JSONbored:mainfrom
RealDiligent:fix/critical-issue-feasibility-signals-8543
Jul 24, 2026
Merged

feat(miner): record feasibility-verdict reasons as rule-fired signals on the infeasible path#8569
JSONbored merged 1 commit into
JSONbored:mainfrom
RealDiligent:fix/critical-issue-feasibility-signals-8543

Conversation

@RealDiligent

Copy link
Copy Markdown
Contributor

Summary

AMS's calibration capture was one-sided: discovery records its four eligibility-exclusion reasons as rule-fired signals (recordEligibilityExclusionSignals), but the feasibility verdict — the other deterministic gate an attempt passes through — recorded nothing, so feasibility rules could not be precision-scored the same way (#8107).

Inside attempt-cli.ts's existing if (!codingTaskSpec.ready) branch only, this records one RuleFiredEvent per feasibility.avoidReasons entry (outcome: "avoid") and per raiseReasons entry (outcome: "raise"), with ruleId = the reason verbatim, targetKey = `${repoFullName}#issue-${issueNumber}` (byte-identical to discovery's format), an ISO occurredAt, and no metadata.

Everything goes through createSignalTrackingStore (canonical signal_rule_fired shape). Adds an initSignalTrackingStore seam — same name as discover's — defaulting to createSignalTrackingStore({ appendEvent, readEvents }) over the shared local event ledger. Best-effort discipline is mirrored verbatim from recordEligibilityExclusionSignals: store-init failure (try/catch) and each per-write failure (.catch(() => undefined)) are swallowed, so the branch's console output, JSON result, and exit-4 contract are unchanged. The pure modules (coding-task-spec.ts, feasibility.ts) are untouched, and the ready: true path records nothing.

Coverage

100% of the changed lines and branches — verified against the scoped simulation CI runs (vitest run --coverage --coverage.all=false), zero uncovered lines or branches on the diff.

Getting there surfaced a real inconsistency worth noting: four branches were initially unreachable because I passed runAttempt's already-resolved nowMs to the helper, whereas discover-cli passes the raw options so the helper's own nowMs ?? Date.now() default is exercised. I corrected the call site to forward the raw seam values — this both closes the coverage gap and makes the "mirror recordEligibilityExclusionSignals verbatim" requirement literally true.

Tests (root vitest, Codecov-visible via the miner-lib source path — the #8479 precedent):

  • both avoid and raise reasons → exact ruleId/outcome/targetKey/occurredAt asserted per event, order and counts matching the arrays;
  • ready: true → zero events recorded (capture is infeasible-only);
  • initSignalTrackingStore throws, and a store whose recordRuleFired rejects → exit code, JSON result, and console output identical to a run without the seam;
  • no seam injected → the default store + Date.now() fallbacks run;
  • empty reason arrays → the early return.

I verified the recording is genuinely wired: reverting the capture call makes the "records one signal per reason" test fail while the swallow tests still pass (a no-op also yields exit 4 — correct).

Closes #8543

Scope

  • The PR title follows type(scope): short summary Conventional Commit format, for example fix(api): restore profile access checks.
  • This PR is focused and does not mix unrelated backend, UI, MCP, docs, dependency, and deploy changes.
  • This follows CONTRIBUTING.md and does not reintroduce GitHub Pages, VitePress, site/, or CNAME.
  • I linked a currently open issue this PR resolves (e.g. Closes #123) — a linked open issue is required for every contributor PR.

Validation

  • git diff --check
  • npm run actionlint
  • npm run typecheck
  • npm run test:coverage locally; codecov/patch requires ≥99% coverage of the lines AND branches you changed (aim for 100% on your diff so CI variance does not fail near the threshold). Global coverage is a non-blocking trend with a loose 90% backstop, not the gate.
  • npm run test:workers
  • npm run build:mcp
  • npm run test:mcp-pack
  • npm run ui:openapi:check
  • npm run ui:lint
  • npm run ui:typecheck
  • npm run ui:build
  • npm audit --audit-level=moderate
  • New or changed behavior has unit/integration tests for new branches, fallback paths, and sanitizer boundaries

If any required check was skipped, explain why:

  • Diff is packages/loopover-miner/lib/attempt-cli.ts + its root test suite. No workflow, MCP, UI, worker, or dependency surface is touched, so actionlint, build:mcp/test:mcp-pack, ui:*, test:workers, and npm audit are not exercised by it.
  • Diff coverage verified at 100% (lines and branches) via the scoped run; the coverage is Codecov-visible because the root vitest suite imports the miner-lib source path directly (both packages/loopover-miner/lib/**/*.ts and *.js are in vitest.config.ts's coverage.include). Root tsc --noEmit and the miner package tsc are clean, and git diff --check passes.
  • Pre-existing unrelated failures: two tests in test/unit/miner-attempt-cli.test.ts (#5185 ... broken appendAttemptLogEvent and reports an unexpected allocator failure ...) fail on my Windows checkout because the sandbox cannot unlink an open sqlite handle (EBUSY). I verified they fail identically with my changes stashed (2 failed on clean origin/main too), so they are environmental and untouched by this PR; my six new cases pass.

Safety

  • No secrets, wallet details, hotkeys, coldkeys, user PATs, private keys, raw trust scores, private rankings, or private maintainer evidence are exposed.
  • Public GitHub text stays sanitized, low-noise, and does not imply compensation guarantees or optimization tactics.
  • Auth, cookie, CORS, GitHub App, Cloudflare, or session changes include negative-path tests.
  • API/OpenAPI/MCP behavior is updated and tested where needed.
  • UI changes use live API data or real empty/error/loading states, not production mock/demo fallbacks.
  • Visible UI changes include a UI Evidence section below with JPG/JPEG or PNG screenshots arranged as organized, captioned, clickable thumbnails. SVG screenshots are not used as review evidence. Review-only screenshots or recordings are not committed to the repository.
  • Public docs/changelogs are updated where needed; changelogs are only edited for release-prep PRs.

UI Evidence

Not applicable — a miner CLI signal-capture change and its tests; no visible UI, frontend, docs, or extension change.

Notes

  • Every ready: false occurrence records — no dedup across repeated attempts at the same issue, since each attempt is a distinct decision instance (matching the issue's requirement and discovery's own non-deduped capture).

… on the infeasible path

AMS calibration capture was one-sided: discovery records its eligibility
exclusions as rule-fired signals, but the feasibility verdict recorded nothing,
so feasibility rules could not be precision-scored (JSONbored#8107). Inside attempt-cli's
existing !codingTaskSpec.ready branch only, records one RuleFiredEvent per
avoidReasons entry (outcome "avoid") and per raiseReasons entry (outcome
"raise"), with targetKey byte-identical to discovery's
`${repoFullName}#issue-${issueNumber}` and no metadata.

Adds an initSignalTrackingStore seam and a default backed by the shared local
event ledger, mirroring discover-cli's recordEligibilityExclusionSignals
verbatim -- including its best-effort discipline: store-init and per-write
failures are swallowed so console output, JSON result, and the exit-4 contract
are unchanged. Pure modules stay side-effect free; the ready path records
nothing.

Closes JSONbored#8543
@RealDiligent
RealDiligent requested a review from JSONbored as a code owner July 24, 2026 21:19
@superagent-security

Copy link
Copy Markdown
Contributor

Superagent didn't find any vulnerabilities or security issues in this PR.

@codecov

codecov Bot commented Jul 24, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 89.70%. Comparing base (bd273c1) to head (db18ae5).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #8569      +/-   ##
==========================================
+ Coverage   89.58%   89.70%   +0.11%     
==========================================
  Files          97       98       +1     
  Lines       22706    22970     +264     
  Branches     3872     3960      +88     
==========================================
+ Hits        20341    20605     +264     
  Misses       2187     2187              
  Partials      178      178              
Flag Coverage Δ
backend 100.00% <100.00%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
packages/loopover-miner/lib/attempt-cli.ts 100.00% <100.00%> (ø)

@loopover-orb loopover-orb Bot added the gittensor:feature Gittensor-scored feature linked to a feature issue — scores a 0.25x multiplier. label Jul 24, 2026
@loopover-orb

loopover-orb Bot commented Jul 24, 2026

Copy link
Copy Markdown
Contributor

Tip

✅ LoopOver review result - approve/merge recommended

Review updated: 2026-07-24 21:23:46 UTC

2 files · 1 AI reviewer · no blockers · readiness 100/100 · CI green · clean

✅ Suggested Action - Approve/Merge

  • safe to merge

Review summary
This PR mirrors discovery's `recordEligibilityExclusionSignals` pattern for the feasibility-verdict path: `recordFeasibilityVerdictSignals` in attempt-cli.ts writes one `RuleFiredEvent` per avoid/raise reason via the shared `createSignalTrackingStore`, gated strictly inside the existing `!codingTaskSpec.ready` branch, with store-init and per-write failures swallowed so the branch's console output, JSON result, and exit-4 contract are provably unchanged. The targetKey format (`${repoFullName}#issue-${issueNumber}`) is byte-identical to discovery's sibling function as claimed, and the accompanying test suite exercises the happy path, the ready-path no-op, the default-store/`Date.now()` fallback, the empty-reasons early return, a throwing store-init, and a rejecting `recordRuleFired` — a thorough set that matches the described 100% coverage.

Nits — 3 non-blocking
  • attempt-cli.ts is already a large file (589 lines per the size-smell scan) and this PR adds another ~53 lines to it; worth flagging for a future split even though this addition itself is well-contained.
  • The conditional spread `...(options.initSignalTrackingStore ? {...} : {})` / `...(options.nowMs === undefined ? {} : {...})` at the call site (attempt-cli.ts) is a bit verbose but is consistent with the existing `fetchImpl` handling elsewhere in the file under `exactOptionalPropertyTypes`, so not worth changing.
  • None beyond the nits — the diff is narrowly scoped, mirrors an established sibling pattern verbatim, and is well covered by tests.

Decision drivers

  • ✅ Code review — No blockers (1 reviewer)
  • ✅ Gate result — Passing (No configured blocker found.)
Context & advisory signals — never blocks the verdict
Signal Result Evidence
Linked issue ✅ Linked #8543
Related work ✅ No active overlap found No same-issue or scoped active PR overlap found.
Change scope ✅ 20/20 Low review scope from cached public metadata (1 linked issue).
Validation posture ✅ 25/25 PR body includes validation/test evidence.
Contributor workload ✅ 10/10 Author activity: 370 registered-repo PR(s), 154 merged, 37 issue(s).
Contributor context ✅ Confirmed Gittensor contributor RealDiligent; Gittensor profile; 370 PR(s), 37 issue(s).
Improvement ✅ Minor risk: clean · value: minor · LLM: moderate
Linked issue satisfaction

Addressed
The diff adds recordFeasibilityVerdictSignals gated on !codingTaskSpec.ready, using createSignalTrackingStore with the exact ruleId/outcome/targetKey/occurredAt shape and no metadata, mirrors the initSignalTrackingStore seam and best-effort try/catch discipline from discover-cli, and includes tests for both-reasons-present, ready:true (zero events), init-throw, and write-rejection cases with exit

Review context
  • Author: RealDiligent
  • Role context: outside_contributor
  • Public audience mode: oss maintainer
  • Lane context: Repository is configured for direct PR review.
  • Public profile languages: Python, Ruby, TypeScript, Svelte, Cuda, JavaScript, Markdown, MDX
  • Official Gittensor activity: 370 PR(s), 37 issue(s).
  • PR-specific overlap: none found.
Contributor next steps
  • Keep the PR focused and include validation evidence before maintainer review.
Signal definitions
  • Related work = same linked issue, overlapping active PRs, or title/path similarity.
  • Change scope = cached public metadata such as size labels, draft state, and review-burden hints.
  • Validation posture = whether the PR provides enough public validation/test evidence for maintainer review.
  • Contributor workload = public contributor activity and cleanup pressure, not a repo-wide quality failure.
  • Contributor context = public GitHub/Gittensor identity context; non-Gittensor status is not a blocker.
🧪 Chat with LoopOver

Ask LoopOver a question about this PR directly in a comment — grounded only in the same cached, public-safe facts shown above, never a new claim.

  • @loopover ask &lt;question&gt; answers contribution-quality Q&A with source citations and freshness.
  • @loopover chat &lt;question&gt; answers in natural prose from cached decision-pack facts via local inference (maintainer/collaborator; read-only).
  • A plain-language @loopover mention with a real question is routed to the closest matching read-only command automatically — no exact syntax required.

Full command reference: https://loopover.ai/docs/loopover-commands

🧪 Experimental — new and may change.

🟩 Safe / merged · 🟦 Advisory · 🟨 Held for review · 🟥 Blocked / closed


💰 Earn for open-source contributions like this. Gittensor lets GitHub contributors earn for the work they already do — register to start earning →.

Checked by LoopOver, a quiet PR intelligence layer for OSS maintainers.

  • Re-run LoopOver review

@loopover-orb loopover-orb Bot added the manual-review Gittensor contributor context label Jul 24, 2026
@JSONbored
JSONbored merged commit 1c7c3a7 into JSONbored:main Jul 24, 2026
8 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gittensor:feature Gittensor-scored feature linked to a feature issue — scores a 0.25x multiplier. manual-review Gittensor contributor context

Projects

None yet

Development

Successfully merging this pull request may close these issues.

miner: record feasibility-verdict reasons as rule-fired signals on attempt's infeasible path

2 participants