AssureOps

Turn assurance claims into testable operating evidence.

AssureOps is a public-safe reference toolkit for building lightweight assurance cases around software and AI-enabled systems. It helps teams connect claims, assumptions, hazards, controls, runtime monitors, and verification evidence in a form that can be reviewed, tested, and versioned.

The project is intentionally standards-aware without claiming certification. It borrows practical ideas from assurance cases, hazard analysis, runtime assurance, NIST AI RMF-style risk management, and safety-critical software review practices.

What It Includes

• YAML assurance-case model for claims, evidence, hazards, controls, and monitors
• Validation checks for missing evidence, weak claim support, stale evidence, and monitor coverage gaps
• CLI for validating a case file and generating Markdown review reports
• Example assurance case for a retrieval service with runtime monitors
• Test suite covering validation rules, report generation, and CLI behavior
• Docker, Makefile, and GitHub Actions CI scaffolding

Why AssureOps

Assurance work often gets trapped in slide decks and static documents. AssureOps treats assurance artifacts as living engineering assets:

Claims -> Evidence -> Controls -> Runtime Monitors -> Review Report

That makes it easier to answer practical questions:

• What claim is this test result supporting?
• Which hazards still lack controls?
• Which controls depend on stale evidence?
• Which runtime monitors are expected to detect degraded behavior?
• What should be reviewed before a release?

Quick Start

git clone https://github.com/AIML-Solutions/assureops.git
cd assureops
python -m venv .venv
source .venv/bin/activate
pip install -e '.[dev]'
assureops validate examples/retrieval-service.case.yaml
assureops report examples/retrieval-service.case.yaml --output reports/retrieval-service.md

Example Validation Output

case: retrieval-service
status: pass
checks: 7
findings: 0

If a case has missing or stale evidence, validation returns a non-zero exit code and prints actionable findings.

Case Model

An assurance case is a YAML file with these top-level sections:

case_id: retrieval-service
system: Metadata-aware retrieval service
version: 0.1.0
claims:
  - id: claim.safe-retrieval
    statement: Retrieval responses use bounded, traceable context.
    evidence: [evidence.api-tests]
hazards:
  - id: hazard.stale-context
    description: Stale context is retrieved for operational decisions.
    controls: [control.metadata-filters]
controls:
  - id: control.metadata-filters
    description: Metadata filters constrain eligible context.
    monitors: [monitor.empty-result-rate]
monitors:
  - id: monitor.empty-result-rate
    signal: Empty or low-confidence retrieval result rate
    threshold: Alert when rate exceeds review threshold
evidence:
  - id: evidence.api-tests
    type: test
    path: tests/api/test_search.py
    status: passing

Validation Rules

AssureOps checks for:

• duplicate identifiers
• claims without evidence
• hazards without controls
• controls without linked hazards
• controls without monitors when runtime coverage is required
• missing evidence paths
• stale evidence based on configurable age limits
• non-passing evidence linked to active claims

Development

make lint
make test
make smoke

Production Notes

AssureOps is a review and engineering-support tool, not a certification authority. Teams using it for regulated, safety-critical, or high-impact systems should align outputs with their own governance, legal, security, and domain-specific review requirements.

License

MIT