fix(mcp,selfhost): audit mutating MCP tools, dedupe predict_gate agreement, guard repo privacy - #9173
Merged
Conversation
…ement, guard repo privacy Three related ORB gaps in one pass: - Every mutating MCP tool (agent pause, autonomy dial, private-config write, redeploy trigger) now records an audit_events row, mirroring the HTTP settings route's own audit shape. Previously none of them left any forensic trace of who flipped the autonomy dial or paused the agent. - computePredictedGateVerdict (backing predict_gate/explain_gate_disposition) now shares the sibling tools' per-actor rate limit, and the predicted-vs- live agreement report dedupes multiple predicted calls against the same real decision down to the latest one, so a contributor iterating on predict_gate can no longer inflate their own agreement sample. Gave predicted_gate_calls a 90-day retention window alongside the other append-only log tables. - PostHog no longer sends raw repo/PR/owner/SHA identifiers when reporting through the shared central key: they're HMAC'd with the same per-instance secret orb-collector.ts already uses. initPostHog now honors ORB_AIR_GAP and an explicit POSTHOG_DISABLED, and an empty POSTHOG_API_KEY means off instead of falling through to the central key. Self-hosted /metrics now pseudonymizes the repo label on the private-repo-scoped counters by default (with an explicit opt-out for an operator who wants raw labels on a verified private network), and observe() now routes through the same redaction path incr()/gaugeVector() already use. Fixed a scrubber bug where the vocabulary redaction pattern corrupted structured identifiers like repo names (e.g. "ossf/scorecard" -> "ossf/private context") by scoping vocabulary scrubbing away from identifier-shaped keys while keeping credential-shape scrubbing unconditional. Also fixed two stray NUL bytes in predicted-gate-agreement.ts's map-key join (pre-existing source corruption that made every past diff of that file render as binary) while touching the exact lines they sat on. Closes #9137 Closes #9138 Closes #9142
Contributor
|
Superagent didn't find any vulnerabilities or security issues in this PR. |
Deploying with
|
| Status | Name | Latest Commit | Preview URL | Updated (UTC) |
|---|---|---|---|---|
| ✅ Deployment successful! View logs |
loopover-ui | d93ea15 | Commit Preview URL Branch Preview URL |
Jul 27 2026, 05:34 AM |
❌ 1 Tests Failed:
View the top 1 failed test(s) by shortest run time
To view more test analytics, go to the Test Analytics Dashboard |
JSONbored
added a commit
that referenced
this pull request
Jul 27, 2026
#9173 introduced a comment referencing the deprecated pre-rename product name, which the branding-drift check on main was not catching -- breaking CI for every PR based on current main, including this one.
2 tasks
JSONbored
added a commit
that referenced
this pull request
Jul 27, 2026
…nd name every silent AI-review skip (#9180) * fix(review): finalize CI-stuck silence, post a waiting placeholder, and name every silent AI-review skip #9011: prReadyForReview used to defer a missing-required-context PR forever past the 2-minute cap instead of falling through to the existing finalize-past-cap block; a required context that never appears keeps ciState pending (never "passed"), so finalizing here can never produce a would-merge disposition. A new stuckReason ("missing_required_context" vs "ci_running") is threaded through the audit/log calls so the two causes are distinguishable. #9042: the silent wait before a review runs (measured live: 8.3min median, 21.9min p90, up to 50.9min) now gets an immediate "waiting" panel comment (renderWaitingForCiPlaceholder) upserted through the same PR_PANEL_COMMENT_MARKER the final verdict replaces, instead of leaving the PR blank until CI settles. #9000: root-causes the #8972 incident where a forced ("Re-run LoopOver review") retrigger completed with none of the events a forced pass should emit anywhere in the audit trail. shouldStartAiReviewForAdvisory folds in shouldRequirePublicAiReviewForAdvisory's hard gate (aiReviewMode off, an ineligible author, no head SHA, an AI kill-switch off, no AI binding) - a forced retrigger does not bypass this gate, and every one of its branches previously returned false with zero audit trail. resolvePublicAiReviewGateSkipReason names the exact reason, and a new catch-all audit event fires whenever aiReviewWillRun ends up false for a cause none of the other (already-audited) paths cover. The two adjacent silent branches in the frozen/paused/one-shot reuse chain - a hold condition true but nothing published to actually reuse - get their own named "unavailable" events for the same reason. This only closes the exhaustive-auditing and root-cause portion of #9000; the receipt-feedback / lost-webhook-click recovery sweep it also asks for is not yet implemented. Closes #9011 Closes #9042 * fix(review): correct stale 'gittensory' brand reference in a comment #9173 introduced a comment referencing the deprecated pre-rename product name, which the branding-drift check on main was not catching -- breaking CI for every PR based on current main, including this one.
JSONbored
added a commit
that referenced
this pull request
Jul 27, 2026
…s with the #9169 required-events expansion (#9203) #9169 promoted pull_request_review/check_run/check_suite to REQUIRED_INSTALLATION_EVENTS and switched the self-host setup wizard's manifest to derive default_events from that same canonical list. Three test fixtures (one in api.test.ts, two in backfill.test.ts) still mocked GitHub's live installation payload with the pre-#9169 event set, so their live-refresh assertions incorrectly expected a healthy status with no missing events. The self-hosting-github-app.mdx docs page's "Events: ..." sentence was similarly left describing the old 7-event list instead of the now- canonical 12-event set, failing the docs/manifest parity test. Also fixes an unrelated branding-drift regression surfaced by the same full test:ci run: a comment in predicted-gate-agreement.ts (added in #9173) still said "gittensory" instead of "loopover".
This was referenced Jul 27, 2026
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
audit_eventsrow. AddedrecordAuditEventcalls toloopover_set_agent_paused,loopover_set_action_autonomy(bothrepo.settings_updated, mirroringPUT /v1/repos/:owner/:repo/settings's own audit shape),loopover_admin_write_config(newconfig.private_writeevent type, audited on both success and failure), andloopover_admin_trigger_redeploy(newinstance.redeploy_triggered, audited on success, a failed run, and a companion connection error). I deliberately did not move the audit call down intoupsertRepositorySettingsitself (the "better" option the issue floats): that function has ~8 call sites acrosssrc/api/routes.ts,src/orb/apr-repo-transfer.ts, and several test files, and moving the audit there would require threading anactorparameter through all of them and re-auditing each call site for double-recording risk against the route layer's existing call — more invasive than a time-boxed pass justified. Added a table-driven invariant test (test/unit/mcp-mutating-tools-audit.test.ts) asserting all four tools produce the expected row, actor, and metadata.computePredictedGateVerdict(backing bothloopover_predict_gateandloopover_explain_gate_disposition) now callsenforceToolRateLimit, sharing the sibling slop-oracle tools' 20-per-300s cap instead of relying solely on the shared/mcproute class (120/min).computePredictedGateAgreementnow dedupes multiple predicted calls that pair against the same real decision down to just the latest one (keyed by object identity of the paired real-decision row), so N predictions before one real merge contribute exactly one paired sample instead of N — closing the lever that let a contributor drive the agreement metric toward 100%. Gavepredicted_gate_callsa 90-day retention window viaRETENTION_POLICY(the same scheduled prune job that already coversaudit_events/ai_usage_events/etc). Operational follow-up, not something computable from this PR: the currently-published agreement figure should be re-derived after this dedup lands, and whether the prior number was inflated should be stated separately.operationalPropertiesinsrc/selfhost/posthog.tsnow HMACsrepo/repository/owner/pull/pullNumber/pr/head_shawith the same per-instance secretorb-collector.ts'sexportOrbBatchalready generates and persists (system_flagskeyorb:anon_secret), but only when the active key is the sharedLOOPOVER_CENTRAL_POSTHOG_KEYfallback — an operator's ownPOSTHOG_API_KEYis unaffected (their own private project, no cross-tenant concern). Fails closed: if the secret hasn't been injected yet (an early-boot race), the identifying field is dropped, never sent raw.initPostHognow honorsORB_AIR_GAP=trueand an explicitPOSTHOG_DISABLED, and an explicitly-emptyPOSTHOG_API_KEY=""now means off instead of falling through to the central key. Self-hosted/metricsnow pseudonymizes (not strips, not raw) therepolabel onPRIVATE_REPO_LABEL_METRICSby default — the sameredacted-NschemeALWAYS_REDACT_REPO_LABEL_METRICSalready uses — since a self-hosted operator legitimately wants a stable per-repo series on their own dashboards, just not the real name leaving on an endpoint that may sit in front of application auth; an operator who has verified/metricsnever leaves their private network can opt back into raw labels withLOOPOVER_METRICS_REPO_LABELS=raw.observe()now routes throughpublicLabelsForMetric(previously the one metric-recording path that bypassed redaction entirely, including the always-redact set). Fixed the scrubber bug wherePUBLIC_UNSAFE_SCRUB's bare vocabulary match corrupted structured identifiers (ossf/scorecard→ossf/private context) by skipping vocabulary/phrase scrubbing forOPERATIONAL_TAG_KEYSwhile keeping credential-shape scrubbing (SECRET_VALUE/JWT_VALUE/QUERY_SECRET_VALUE) unconditional.predicted-gate-agreement.ts's map-key-join template literal (pre-existing corruption already onmain, sitting on the exact lines the orb(mcp): predict_gate writes an un-deduped row per call and all of them pair against one real decision — a contributor can drive the agreement metric to 100% #9138 fix touches) — both the writer and reader side were consistently corrupted the same way so it never affected runtime behavior, but it made every pastgit diff/GitHub PR view of that file render as "Binary files differ" instead of a real line diff. Becauseorigin/main's committed blob still has the NUL bytes, this PR's own diff for that one file will likely still render as binary (git's binary-vs-text check looks at both the old and new blob) — but the new blob is clean, so every diff of this file after this PR merges will render normally again.Closes #9137
Closes #9138
Closes #9142
Scope
type(scope): short summaryConventional Commit format.CONTRIBUTING.mdand does not touchsite/,CNAME, or**/lovable/**.Validation
git diff --checknpm run actionlintnpm run typechecknpm run test:coverage(full, unsharded) — not run locally, see note below; scoped coverage verification done instead.npm run test:workers— not run locally (notest/workers/**files touched).npm run build:mcp/npm run test:mcp-pack— not run locally (nopackages/loopover-mcp/**changes).npm run ui:openapi:check(no drift — no route/schema changes in this PR)npm run ui:lint/npm run ui:typecheck/npm run ui:build— not run (noapps/loopover-ui/**source changes beyond the regenerated env-reference doc below).npm audit --audit-level=moderate— not run locally, see note below.If any required check was skipped, explain why:
npm run test:ci,npm audit) was intentionally not run before pushing — GitHub CI runs it. In its place I ran and verified:npm run typecheck(clean),npm run db:migrations:check/db:schema-drift:check(no migration needed — no schema changes),npm run selfhost:env-reference:check(regeneratedapps/loopover-ui/src/lib/selfhost-env-reference.tsfor the two new self-host env vars this PR introduces:POSTHOG_DISABLEDandLOOPOVER_METRICS_REPO_LABELS),npm run cf-typegen:check(clean — no wrangler.jsonc changes),npm run ui:openapi:check(clean),npm run coverage-boltons:check(clean — new test files don't match the bolt-on pattern),git diff --check, andnpm run actionlint.mcp-mutating-tools-audit.test.ts,mcp-predict-gate.test.ts,mcp-automation-state.test.ts,mcp-admin-config-tools.test.ts,mcp-admin-redeploy-tool.test.ts,predicted-gate-agreement.test.ts,retention.test.ts,selfhost-posthog.test.ts,selfhost-metrics.test.ts,selfhost-redaction-scrub.test.ts), plus a fullnpm run test:changedpass (vitest's import-graph-based selection againstmain): 456 test files / 11,182 tests green, 0 failures.test:coveragerun, I ran scopedvitest --coveragepasses against just the relevant test files and inspected the per-file JSON coverage detail (statement/branch hit maps) cross-referenced line-by-line against this PR's actual diff. Every changed line/branch insrc/db/retention.ts,src/mcp/server.ts(all five touched regions),src/review/predicted-gate-agreement.ts,src/selfhost/metrics.ts,src/selfhost/posthog.ts, andsrc/selfhost/redaction-scrub.tsis covered — 100% lines/branches on every changed region, confirmed by diffing uncovered-line output against this PR's patch.src/server.ts's two changed call sites are incodecov.yml'signorelist (Docker-boot-only entrypoint, not unit-coverable) so they carry no coverage obligation.npm audit --audit-level=moderatereports 5 pre-existing high-severity advisories in theeslint/brace-expansion/minimatchdevDependency chain, unrelated to and unchanged by this PR (no new dependency added, nopackage.json/lockfile touched).Safety
UI Evidence
Not applicable — this PR is backend/MCP-only (no
apps/loopover-ui/**source changes beyond the regenerated, auto-generated env-reference doc).Notes
apps/loopover-ui/src/lib/selfhost-env-reference.ts) was regenerated vianpm run selfhost:env-referenceto add the two new operator-facing vars this PR introduces:POSTHOG_DISABLED(explicit kill switch,src/selfhost/posthog.ts) andLOOPOVER_METRICS_REPO_LABELS(raw-label opt-out,src/server.ts).audit_eventstable/recordAuditEventhelper, and orb(mcp): predict_gate writes an un-deduped row per call and all of them pair against one real decision — a contributor can drive the agreement metric to 100% #9138's retention entry reuses the existingpredicted_gate_callstable (already created by migration 0137) — just adding it to the in-codeRETENTION_POLICYarray.@loopover/mcppre-submit predictor CLI in this sandboxed environment (no interactive device-flow login available), soloopover_predict_gate/loopover_check_slop_risk/loopover_lint_pr_textwere not run against this PR before pushing — noting this per Phase 8's checklist rather than silently skipping it.