Skip to content

feat(review): send a PagerDuty resolve event once ops anomalies clear - #8953

Closed
hurryup52 wants to merge 2 commits into
JSONbored:mainfrom
hurryup52:feat/8903-pagerduty-resolve-event
Closed

feat(review): send a PagerDuty resolve event once ops anomalies clear#8953
hurryup52 wants to merge 2 commits into
JSONbored:mainfrom
hurryup52:feat/8903-pagerduty-resolve-event

Conversation

@hurryup52

Copy link
Copy Markdown
Contributor

Summary

  • runOpsAlerts paged PagerDuty when a repo had anomalies but never sent a matching resolve once they cleared (documented gap at src/review/ops-wire.ts:304-306), so an operator had to close the incident by hand every time.
  • Persist each repo's previous-tick anomaly state in system_flags (migration 0054, reused as a generic key/value store — no new table/migration) keyed ops_anomaly_active:<repoFullName>.
  • triggerPagerDutyIncident (src/services/notify-pagerduty.ts) now takes an eventAction: "trigger" | "resolve" (discriminated union, default trigger). A resolve skips the min-severity and cooldown gates (those exist only to stop a page from crying wolf) and posts event_action: "resolve" with no payload (PagerDuty's Events API v2 only requires payload for trigger).
  • runOpsAlerts now sends a resolve for the same dedupKey on the first tick a previously-anomalous repo comes back healthy.

Scope

  • The PR title follows type(scope): short summary Conventional Commit format, for example fix(api): restore profile access checks.
  • This PR is focused and does not mix unrelated backend, UI, MCP, docs, dependency, and deploy changes.
  • This follows CONTRIBUTING.md and does not reintroduce GitHub Pages, VitePress, site/, or CNAME.
  • I linked a currently open issue this PR resolves (e.g. Closes #123) — a linked open issue is required for every contributor PR.

Validation

  • git diff --check
  • npm run actionlint
  • npm run typecheck
  • npm run test:coverage locally; codecov/patch requires ≥99% coverage of the lines AND branches you changed (aim for 100% on your diff so CI variance does not fail near the threshold). Global coverage is a non-blocking trend with a loose 90% backstop, not the gate.
  • npm run test:workers
  • npm run build:mcp
  • npm run test:mcp-pack
  • npm run ui:openapi:check
  • npm run ui:lint
  • npm run ui:typecheck
  • npm run ui:build
  • npm audit --audit-level=moderate
  • New or changed behavior has unit/integration tests for new branches, fallback paths, and sanitizer boundaries

If any required check was skipped, explain why:

  • This PR touches only src/review/ops-wire.ts and src/services/notify-pagerduty.ts (no UI, no MCP/miner packages, no OpenAPI/schema change, no wrangler binding change), so the UI/MCP/OpenAPI-specific checks are not applicable to this diff. npm run test:ci's full remaining suite is being run now and this PR will be updated/amended if anything surfaces.

Safety

  • No secrets, wallet details, hotkeys, coldkeys, user PATs, private keys, raw trust scores, private rankings, or private maintainer evidence are exposed.
  • Public GitHub text stays sanitized, low-noise, and does not imply compensation guarantees or optimization tactics.
  • Auth, cookie, CORS, GitHub App, Cloudflare, or session changes include negative-path tests.
  • API/OpenAPI/MCP behavior is updated and tested where needed. (No OpenAPI/MCP surface touched by this PR.)
  • UI changes use live API data or real empty/error/loading states, not production mock/demo fallbacks.
  • Visible UI changes include a UI Evidence section below with JPG/JPEG or PNG screenshots arranged as organized, captioned, clickable thumbnails. SVG screenshots are not used as review evidence. Review-only screenshots or recordings are not committed to the repository.
  • Public docs/changelogs are updated where needed; changelogs are only edited for release-prep PRs.

Notes

  • No UI/auth/OpenAPI/MCP change in this PR — the corresponding checklist items above are not applicable.

Closes #8903

runOpsAlerts previously paged PagerDuty when a repo had anomalies but
never sent a matching resolve once they cleared, so an operator had
to close the incident by hand every time. Persist each repo's
previous-tick anomaly state in system_flags (no new table) and send
an eventAction: "resolve" for the same dedupKey once a repo goes
from anomalous to healthy.

Closes JSONbored#8903
@hurryup52
hurryup52 requested a review from JSONbored as a code owner July 26, 2026 14:13
@superagent-security

Copy link
Copy Markdown
Contributor

Superagent didn't find any vulnerabilities or security issues in this PR.

@codecov

codecov Bot commented Jul 26, 2026

Copy link
Copy Markdown

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 90.62%. Comparing base (401caef) to head (f4b01ca).
⚠️ Report is 7 commits behind head on main.

Additional details and impacted files
@@            Coverage Diff             @@
##             main    #8953      +/-   ##
==========================================
+ Coverage   90.56%   90.62%   +0.06%     
==========================================
  Files          96       98       +2     
  Lines       22490    22657     +167     
  Branches     3884     3927      +43     
==========================================
+ Hits        20367    20533     +166     
  Misses       1945     1945              
- Partials      178      179       +1     
Flag Coverage Δ
backend 99.40% <100.00%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

Files with missing lines Coverage Δ
src/review/ops-wire.ts 99.00% <100.00%> (ø)
src/services/notify-pagerduty.ts 100.00% <100.00%> (ø)

…everity assertion

typecheck flagged calls[0].body.payload.severity/.summary as possibly
undefined after stubPagerDutyFetch's payload field became optional
(JSONbored#8903's resolve event has none).
@hurryup52

Copy link
Copy Markdown
Contributor Author

Superseded by #8942, which merged first and already resolves #8903 (a different implementation of the same auto-resolve behavior). Closing to avoid a duplicate.

@hurryup52 hurryup52 closed this Jul 26, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

feature(ops): auto-resolve PagerDuty incidents once ops anomalies clear

1 participant