Skip to content

Add alert triage SOAR closure evidence gates#1545

Open
minorstep wants to merge 1 commit into
UnitOneAI:mainfrom
minorstep:codex/alert-triage-soar-closure
Open

Add alert triage SOAR closure evidence gates#1545
minorstep wants to merge 1 commit into
UnitOneAI:mainfrom
minorstep:codex/alert-triage-soar-closure

Conversation

@minorstep
Copy link
Copy Markdown

Skill Improvement ($50-150 Bounty)

Skill Modified

Skill name: alert-triage
Skill path: skills/secops/alert-triage/

What Was Wrong

#1525 shows that the skill did not distinguish analyst-validated triage from SOAR or ticketing automation that auto-closes an alert, propagates duplicate status, or quietly suppresses future detections. That can turn a weak automation status into a false positive or benign true positive disposition without enough evidence.

What This PR Fixes

  • Bumps alert-triage to v1.0.1.
  • Adds explicit SOAR auto-disposition and case-closure integrity gates.
  • Requires closure actor, validation evidence, playbook branch, linked-case, and tuning side-effect review before accepting FP/BTP closure.
  • Extends the output schema with closure evidence and automation review fields.
  • Adds fixture scenarios for validated maintenance closure, unvalidated SOAR false positives, linked-case contamination, scoped suppression, and automation contradicted by correlated evidence.

Closes #1525.

Evidence

Before (skill misses this / false positive on this):

closed_by: soar-auto-close
closure_note: known benign
priority: P4
asset: finance-prod-03
validation_evidence: []
linked_case_effect: child alerts inherit duplicate closure

After (now correctly handled):

closure_evidence_gate: Automation closure not validated
expected_decision: not_evaluable_or_escalate
minimum_priority: P3
required_checks:
  - named closure actor or accountable automation owner
  - correlated telemetry and disposition rationale
  - scoped playbook logic
  - linked-case independence
  - separate tuning approval trail

Test Cases Added/Updated

  • Added calibration fixtures in skills/secops/alert-triage/tests/soar-closure-evidence-fixtures.md
  • Existing skill prompt-injection and allowed-tools constraints preserved
  • Existing markdown/frontmatter structure still validates locally

Bounty Tier

  • Minor ($50) — Doc update, small logic tweak, typo fix
  • Moderate ($100) — New edge case coverage, FP reduction with evidence
  • Substantial ($150) — Rewritten detection logic, major coverage expansion

Bounty Info

  • I have read and agree to the CONTRIBUTING.md bounty terms
  • Preferred payment method: available through the repository-approved payout flow; no private payout details are posted publicly.

Framework References

  • MITRE ATT&CK v16 remains the triage correlation vocabulary for alert tactics and techniques.
  • NIST SP 800-61 Rev 2 remains the incident analysis, prioritisation, documentation, and notification reference.

Testing

  • git diff --check
  • git diff --cached --check
  • Local validation for frontmatter version, markdown fence balance, ASCII-only content, and fixture IDs/YAML parse

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[REVIEW] alert-triage: add SOAR auto-disposition and playbook closure contamination gates

1 participant