Skip to content

feat(calibration): record advisory-mode rule-fired signals too, tagged with the resolved gate mode - #8784

Merged
JSONbored merged 2 commits into
mainfrom
feat/record-advisory-mode-signals
Jul 26, 2026
Merged

feat(calibration): record advisory-mode rule-fired signals too, tagged with the resolved gate mode#8784
JSONbored merged 2 commits into
mainfrom
feat/record-advisory-mode-signals

Conversation

@JSONbored

Copy link
Copy Markdown
Owner

Summary

The corpus-starvation fix from adversarial-audit epic #8757. recordConfiguredGateBlockerSignals only recorded findings whose gate resolved to "block" — but every gate defaults to advisory/off, so the majority of real traffic (advisory findings a human actually saw) recorded nothing, and every downstream backtest corpus, reliability curve, and knob-loosening decision was built from the biased block-mode-only minority sample.

Changes

Test plan

  • npm run typecheck — clean
  • Recorder suite: 25/25, including new tests for advisory-mode recording, off-mode non-recording, the sub-floor advisory_only demotion tag, and the stricter-of-two-gates composition
  • 1,583 tests green across all 20 suites touching evaluateGateCheck / the signal surface
  • Changed line range: 0 uncovered statements, 0 uncovered branches (verified via istanbul JSON)

Closes #8760

…d with the resolved gate mode (#8760)

recordConfiguredGateBlockerSignals only recorded findings whose gate RESOLVED
to "block" — but every gate defaults to advisory/off, so the majority of real
traffic (advisory findings a human actually saw) recorded nothing, and every
downstream backtest corpus, reliability curve, and knob-loosening decision was
built from the biased block-mode-only minority.

- isConfiguredGateBlocker is now a =="block" projection of a new
  resolveConfiguredGateMode(finding, policy) -> "block"|"advisory"|"off" —
  one source of truth, so recorder and gate can never drift; every mode
  default is byte-identical to the old switch.
- The recorder captures every non-"off" finding with metadata.gateMode so
  consumers can segment bot-actioned evidence from shown-to-a-human evidence.
  The advisory_only sub-floor AI demotion resolves "advisory" (a real finding
  a human saw), never "off". manifest_linked_issue_required composes the
  stricter of its two gates.
- Consumer-segmentation audit (in the recorder's doc comment): every current
  consumer is decided-case-driven (corpus pairing drops unpaired fires; the
  repeat alarm uses its own composite-ruleId namespace), so advisory fires sit
  inert until #8761's implicit confirmations start pairing them.
@superagent-security

Copy link
Copy Markdown
Contributor

Superagent didn't find any vulnerabilities or security issues in this PR.

@github-actions

github-actions Bot commented Jul 26, 2026

Copy link
Copy Markdown
Contributor

Logic backtest

Replayed 0 historical case(s) for linked_issue_scope_mismatch through the base (a93ae0a) and head (2aebb5e) versions of its detection logic (corpus checksum 4f53cda18c2b).

Backtest comparison: linked_issue_scope_mismatch

Verdict: unchanged — no comparable axis moved.

Advisory only — this check never blocks merge (#8105).

@cloudflare-workers-and-pages

cloudflare-workers-and-pages Bot commented Jul 26, 2026

Copy link
Copy Markdown

Deploying with  Cloudflare Workers  Cloudflare Workers

The latest updates on your project. Learn more about integrating Git with Workers.

Status Name Latest Commit Preview URL Updated (UTC)
✅ Deployment successful!
View logs
loopover-ui 2aebb5e Commit Preview URL

Branch Preview URL
Jul 26 2026, 05:36 AM

@JSONbored JSONbored self-assigned this Jul 26, 2026
@JSONbored
JSONbored merged commit f1be340 into main Jul 26, 2026
6 of 7 checks passed
@JSONbored
JSONbored deleted the feat/record-advisory-mode-signals branch July 26, 2026 05:43
@loopover-orb loopover-orb Bot added the gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier. label Jul 26, 2026
@codecov

codecov Bot commented Jul 26, 2026

Copy link
Copy Markdown

❌ 3 Tests Failed:

Tests completed Failed Passed Skipped
21179 3 21176 21
View the top 3 failed test(s) by shortest run time
test/unit/check-engine-parity-script.test.ts > check-engine-parity script > runEngineParityMain returns 0 for the real monorepo workspace
Stack Traces | 0.043s run time
AssertionError: expected 1 to be +0 // Object.is equality

- Expected
+ Received

- 0
+ 1

 ❯ test/unit/check-engine-parity-script.test.ts:798:48
test/unit/check-engine-parity-script.test.ts > check-engine-parity script > engine version skew > uses default version readers against the real monorepo workspace
Stack Traces | 0.0532s run time
AssertionError: expected [ Array(1) ] to deeply equal []

- Expected
+ Received

- []
+ [
+   "Gate-decision logic change in src/rules/advisory.ts requires either:
+   • a matching edit to the other twin (.../src/advisory/gate-advisory.ts), or
+   • a version bump in packages/loopover-engine/package.json (currently 3.14.1 vs base 3.14.1).",
+ ]

 ❯ test/unit/check-engine-parity-script.test.ts:785:31
test/unit/check-engine-parity-script.test.ts > check-engine-parity script > prints a clean summary and exits 0 for the real repo state when run as a subprocess
Stack Traces | 0.904s run time
Error: Command failed: .../loopover/node_modules/.bin/tsx scripts/check-engine-parity.ts
Engine-parity check found 1 issue(s):
Gate-decision logic change in src/rules/advisory.ts requires either:
  • a matching edit to the other twin (.../src/advisory/gate-advisory.ts), or
  • a version bump in packages/loopover-engine/package.json (currently 3.14.1 vs base 3.14.1).

 ❯ test/unit/check-engine-parity-script.test.ts:803:20

⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯⎯
Serialized Error: { status: 1, signal: null, output: [ null, '', 'Engine-parity check found 1 issue(s):\nGate-decision logic change in src/rules/advisory.ts requires either:\n  • a matching edit to the other twin (.../src/advisory/gate-advisory.ts), or\n  • a version bump in packages/loopover-engine/package.json (currently 3.14.1 vs base 3.14.1).\n' ], pid: 7145, stdout: '', stderr: 'Engine-parity check found 1 issue(s):\nGate-decision logic change in src/rules/advisory.ts requires either:\n  • a matching edit to the other twin (.../src/advisory/gate-advisory.ts), or\n  • a version bump in packages/loopover-engine/package.json (currently 3.14.1 vs base 3.14.1).\n' }

To view more test analytics, go to the Test Analytics Dashboard
📋 Got 3 mins? Take this short survey to help us improve Test Analytics.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

gittensor:bug Gittensor-scored bug fix — scores a 0.05x multiplier.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

calibration: record rule-fired signals for advisory/off gate modes too, tagged with the resolved mode

1 participant