Skip to content

fix(cyberbattlesim): reconcile SDL evidence requirements, simulator capture, and backend manifest #86

Description

@Brad-Edwards

Objective

Reconcile the authored CyberBattleSim SDL/task evidence requirements, the data the pinned simulator can actually expose, the data needed for equivalence testing, and the adapter's implemented capture and backend manifest.

Scope

  • Inventory every data-capture requirement in the CyberBattleSim SDL and task and map it to the pinned native simulator source.
  • Capture a complete per-step attacker action/outcome log sufficient to trace each episode and investigate outliers.
  • Capture the availability series and distinguish simulator termination, truncation/cutoff, and other terminal causes as far as the native source permits.
  • Capture and report stochastic controls and RNG bindings honestly; disclose every unbound source of randomness.
  • Ensure the resulting artifacts are sufficient to recompute the declared equivalence measures from preserved per-run data.
  • Correct authored requirements that overclaim simulator capability and explicitly disclose unavailable or lossy data.
  • Extend the adapter capture path and make its backend manifest truthful at field/artifact granularity.
  • Remove the false attacker-action-log satisfaction claim and every other hardcoded evidence-reference claim in this path.

Acceptance criteria

  • Every SDL/task capture requirement maps to a native source, an implemented adapter capture path, a manifest declaration, and an emitted portable artifact field.
  • Each episode can be traced through its attacker actions and outcomes to availability, termination/cutoff cause, reward, and equivalence inputs.
  • Availability and termination-cause comparisons can be recomputed from preserved evidence without trusting a summary claim.
  • RNG/control coverage and limitations are explicit and machine-readable.
  • Every unavailable or lossy requirement is explicit and cannot be admitted as satisfied.
  • Tests exercise the complete requirement-to-capability-to-artifact chain and fail on missing or falsely claimed data.
  • The corrected equivalence experiment is rerunnable from the preserved protocol and emits the required evidence.
  • The canonical verification graph passes.

References

Metadata

Metadata

Assignees

No one assigned

    Labels

    blockedBlocked on an upstream gate or dependency

    Type

    No type

    Projects

    No projects

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions