feat(review): send a PagerDuty resolve event once ops anomalies clear - #8953
Closed
hurryup52 wants to merge 2 commits into
Closed
feat(review): send a PagerDuty resolve event once ops anomalies clear#8953hurryup52 wants to merge 2 commits into
hurryup52 wants to merge 2 commits into
Conversation
runOpsAlerts previously paged PagerDuty when a repo had anomalies but never sent a matching resolve once they cleared, so an operator had to close the incident by hand every time. Persist each repo's previous-tick anomaly state in system_flags (no new table) and send an eventAction: "resolve" for the same dedupKey once a repo goes from anomalous to healthy. Closes JSONbored#8903
Contributor
|
Superagent didn't find any vulnerabilities or security issues in this PR. |
Codecov Report✅ All modified and coverable lines are covered by tests. Additional details and impacted files@@ Coverage Diff @@
## main #8953 +/- ##
==========================================
+ Coverage 90.56% 90.62% +0.06%
==========================================
Files 96 98 +2
Lines 22490 22657 +167
Branches 3884 3927 +43
==========================================
+ Hits 20367 20533 +166
Misses 1945 1945
- Partials 178 179 +1
Flags with carried forward coverage won't be shown. Click here to find out more.
|
…everity assertion typecheck flagged calls[0].body.payload.severity/.summary as possibly undefined after stubPagerDutyFetch's payload field became optional (JSONbored#8903's resolve event has none).
Contributor
Author
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
runOpsAlertspaged PagerDuty when a repo had anomalies but never sent a matchingresolveonce they cleared (documented gap atsrc/review/ops-wire.ts:304-306), so an operator had to close the incident by hand every time.system_flags(migration 0054, reused as a generic key/value store — no new table/migration) keyedops_anomaly_active:<repoFullName>.triggerPagerDutyIncident(src/services/notify-pagerduty.ts) now takes aneventAction: "trigger" | "resolve"(discriminated union, defaulttrigger). Aresolveskips the min-severity and cooldown gates (those exist only to stop a page from crying wolf) and postsevent_action: "resolve"with nopayload(PagerDuty's Events API v2 only requirespayloadfortrigger).runOpsAlertsnow sends aresolvefor the samededupKeyon the first tick a previously-anomalous repo comes back healthy.Scope
type(scope): short summaryConventional Commit format, for examplefix(api): restore profile access checks.CONTRIBUTING.mdand does not reintroduce GitHub Pages, VitePress,site/, orCNAME.Closes #123) — a linked open issue is required for every contributor PR.Validation
git diff --checknpm run actionlintnpm run typechecknpm run test:coveragelocally;codecov/patchrequires ≥99% coverage of the lines AND branches you changed (aim for 100% on your diff so CI variance does not fail near the threshold). Global coverage is a non-blocking trend with a loose 90% backstop, not the gate.npm run test:workersnpm run build:mcpnpm run test:mcp-packnpm run ui:openapi:checknpm run ui:lintnpm run ui:typechecknpm run ui:buildnpm audit --audit-level=moderateIf any required check was skipped, explain why:
src/review/ops-wire.tsandsrc/services/notify-pagerduty.ts(no UI, no MCP/miner packages, no OpenAPI/schema change, no wrangler binding change), so the UI/MCP/OpenAPI-specific checks are not applicable to this diff.npm run test:ci's full remaining suite is being run now and this PR will be updated/amended if anything surfaces.Safety
UI Evidencesection below with JPG/JPEG or PNG screenshots arranged as organized, captioned, clickable thumbnails. SVG screenshots are not used as review evidence. Review-only screenshots or recordings are not committed to the repository.Notes
Closes #8903