Skip to content

Proposal: Configurable Postmortems / Incident Reviews #60

Description

@Artesfrost

Summary

As a continuation of the previously proposed Incident Management functionality, it would be useful to add a Postmortem / Incident Review workflow to IncidentRelay.

Once an Incident is resolved or closed, users could generate a postmortem using information already collected during the Incident lifecycle, including related alerts, affected services, service dependencies, responders, activity history, and other available data.

The main idea is not to introduce one fixed postmortem format, but to provide configurable Postmortem Templates, since different organizations may have very different requirements for incident reviews.

Proposed Workflow

A possible workflow could be:

Incident
   ↓
Resolved / Closed
   ↓
Generate Postmortem
   ↓
Select Template
   ↓
Auto-populate available Incident data
   ↓
Review / Edit
   ↓
Finalize / Export

After closing an Incident, the user could select Generate Postmortem, choose one of the previously configured templates, and IncidentRelay would automatically populate as much information as possible.

The generated postmortem should remain editable before it is finalized.

Postmortem Template Builder

Instead of having one predefined format, administrators could create reusable Postmortem Templates.

A template could consist of configurable sections/modules such as:

  • Incident Summary
  • Impact
  • Timeline
  • Detection
  • Affected Services
  • Service Dependencies
  • Related Alerts / Alert Groups
  • Responders / Incident Commander
  • Root Cause
  • Contributing Factors
  • Resolution
  • Corrective Actions / Follow-up Tasks
  • Lessons Learned
  • External Ticket / Reference
  • Custom Text
  • Custom Fields

The administrator could choose which sections are included and define their order.

This would allow each organization to adapt the Postmortem format to its own incident-management process.

Services and Dependencies

It could be especially useful to include Services and their dependencies in the Postmortem.

For example, an Incident may directly affect one Service while causing degradation in several dependent Services.

If IncidentRelay already knows these relationships, the Postmortem could automatically include them.

For example:

Incident: Authentication Service Failure

Primary affected service:
- Authentication Service

Dependent / impacted services:
- Web Application
- Mobile Application
- API Gateway
- Customer Portal

Depending on the available Service model, this could potentially also be represented as a dependency chain:

Database
   ↓
Authentication Service
   ↓
API Gateway
   ↓
Web Application

This could make the Postmortem much more useful for understanding not only what failed, but also how the failure propagated through related services.

The user should still be able to manually add or remove affected Services before finalizing the Postmortem.

Automatic Data Population

Where possible, IncidentRelay could populate template sections using existing data.

For example:

Incident Summary      → Incident title / description
Timeline              → Incident activity / status changes
Affected Services     → Services linked to the Incident
Service Dependencies  → related/dependent Services
Related Alerts        → linked Alert Groups / Alerts
Responders            → Incident participants
Detection             → first related Alert / Alert Group
Started At            → Incident / first alert timestamp
Resolved At           → Incident resolution timestamp
Root Cause            → Incident root cause
Resolution            → Incident resolution

This would reduce the amount of information that has to be manually reconstructed after an incident.

Postmortem Lifecycle

A simple lifecycle could be:

Draft → Final

Optionally, IncidentRelay could support configurable rules such as:

Postmortem required for:
- Critical incidents
- P1 incidents
- P1/P2 incidents
- Incidents affecting specific Services

For example, an organization may require a Postmortem for every incident affecting a business-critical Service regardless of the Incident priority.

Export

It would be useful to support portable output formats such as:

  • Markdown
  • PDF
  • HTML

Markdown could be especially useful for self-hosted environments because the result could easily be stored in Git, Wiki/Confluence, or another documentation system.

Why This Could Be Useful

IncidentRelay already collects much of the information required for an incident review.

Combining Incident information with Alerts, Services, dependencies and activity history could provide a much more complete picture of what happened.

The complete lifecycle could then look like:

Service / Dependency
        ↓
Alert
        ↓
Alert Group
        ↓
Incident
        ↓
Resolution
        ↓
Postmortem

This could provide not only a record of the Incident itself, but also a reusable history of which services were affected, how the impact propagated, what caused it, and how it was resolved.

Would something like this fit the direction planned for the Incident Management functionality?

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions