Executable Behavioral Validation (ASR-512)

[Brad-Edwards]

ASR-512 | FUNCTIONAL | SHOULD | Wave 3 | DRAFT

Statement

The ecosystem shall support executable validation probes for behavioral properties claimed by scenarios, participants, workflows, or experiments.

Status

Independent validation technique. Follows the layered-validation framework in #97. CI branch UID: ASR-512.

Requirements

• ASR-512

---

Tracks Ground Control requirement ASR-512

[Brad-Edwards]

🛠️ Picked up by /implement - driver codex, branch 260-behavioral-validation-probes, 2026-07-28T02:52:28.366Z.

[Brad-Edwards]

gc workflow phase recorded: preflight (issue #260). Posted by the MCP server to enforce ordering between workflow steps (issue #794 MVP-2). Do not edit or delete — used by downstream tools to gate phase prerequisites.

[Brad-Edwards]

gc workflow phase recorded: plan (issue #260). Posted by the MCP server to enforce ordering between workflow steps (issue #794 MVP-2). Do not edit or delete — used by downstream tools to gate phase prerequisites.

Implementation Plan

ASR-512's executable behavior already exists in the repository: target conformance drives admitted scenarios through provisioning, workflow/evaluation, and participant lifecycle surfaces; realization conformance executes positive and negative probes through an injected harness; truth-result and behavioral-claim contracts bind finite results to exact subjects, evidence, limitations, and nonclaims. This run will document and reconcile that existing coverage instead of adding a duplicate universal runner or schema.

Changes

1. Correct docs/decisions/issue-260-asr-512-executable-behavioral-validation-preflight.md so it records ASR-512 as the in-scope Ground Control requirement after the legacy issue body was migrated to a formal Requirements section.
2. Add docs/research/behavioral-validation/traceability-matrix-asr-512.md mapping each requirement clause to its existing scenario, participant, workflow, experiment/claim, conformance, evidence, and test surfaces, with explicit finite-evidence boundaries and nonclaims.
3. After merge, transition ASR-512 from DRAFT to ACTIVE and reconcile IMPLEMENTS, TESTS, and DOCUMENTS links to the mapped artifacts.

Documentation-only carve-out

The planned diff is static documentation only. Existing structural gates protect every requirement clause:

• Executable scenario/workflow/participant probes: test_runtime_conformance.py, test_reference_backend_conformance.py, and test_realization_honesty_conformance.py exercise the actual target/control-plane and injected-harness paths, including no-op, wrong-target, negative-probe, cleanup, and evidence behavior.
• Claim/evidence binding: test_truth_result_contracts.py fails closed when observed truth lacks a probe binding, evidence, or governed temporal context.
• Bounded behavioral-claim semantics: test_behavioral_relation_claims.py plus tools/check_behavioral_relation_claims.py enforce the revisioned relation catalog, finite quantifier/evidence boundaries, limitations, and nonclaims.
• Repository authority and drift: tools/check_repo_policy.py, tools/check_requirement_governance.py, generated-schema checks, and tools/verify_all.py keep package, contract, schema, documentation, and policy surfaces aligned.

No production line, schema, executable fixture, contract data, configuration, or runtime-consumed prose will change. A second issue comment will restate this carve-out before implementation proceeds.

Whole-repository review

• Security: no authentication, authorization, secret handling, config, argv, process, HTTP, evidence payload, or error-envelope path changes; the matrix records the incumbent capability, redaction, cleanup, and finite-evidence gates.
• Maintainability: reuse ADR-066/072/079/081, PropositionTruthResultModel, BehavioralClaimBindingModel, existing conformance runners, diagnostics, evidence carriers, and policy checks.
• Extensibility: preserve subject-specific adapters and carriers; future probe types add a governed case producer at the injected executor seam without introducing another truth algebra, registry, store, or runner.
• Surfaces: documentation only under approved authority roots; no changes to specs/, contracts/, Python packages, schemas, manifests, fixtures, versions, or changelog files.

Verification

Run with RAES_REQUIREMENT_UID=ASR-512 because the branch name contains no UID:

• focused conformance, truth-result, and behavioral-claim tests;
• implementations/python/.venv/bin/python tools/check_repo_policy.py;
• implementations/python/.venv/bin/python tools/check_requirement_governance.py;
• implementations/python/.venv/bin/python tools/verify_all.py;
• the configured completion and policy gates through the /implement workflow.

[Brad-Edwards]

Documentation-only TDD carve-out

The actual planned diff contains only static Markdown under docs/; it changes no production code, schema, contract data, executable fixture, configuration, or runtime-consumed prose.

Structural gates for ASR-512:

• executable scenario, workflow, and participant behavior: test_runtime_conformance.py, test_reference_backend_conformance.py, and test_realization_honesty_conformance.py;
• observed property truth and probe provenance: test_truth_result_contracts.py;
• bounded behavioral claims, evidence scope, limitations, and nonclaims: test_behavioral_relation_claims.py and tools/check_behavioral_relation_claims.py;
• repository authority and drift: tools/check_repo_policy.py, tools/check_requirement_governance.py, generated-schema checks, and tools/verify_all.py.

The documentation records existing executable behavior and its finite evidence boundary; it does not substitute prose for missing runtime behavior.

[Brad-Edwards]

gc_codex_review — cycle 1 of 1 (pre-push) on issue #260 (branch 260-behavioral-validation-probes)
Diff mode: inline — the complete diff was supplied in one prompt

Core review

Verdict: don't-ship

The preflight is shaped correctly: it preserves subject-specific claim semantics, reuses the repository’s truth, relation, conformance, evidence, and disclosure incumbents, and identifies a sensible subject-bound adapter seam without inventing a universal runner. The traceability matrix, however, crosses that seam by declaring ASR-512 already implemented even though its cited surfaces separately provide claim binding, apparatus execution, or evidence validation without demonstrating the required end-to-end join from an exact behavioral property through a bound executable probe to a property-specific result. That is a structural mismatch between the decision note and the claimed delivery state, and it risks foreclosing the actual adapter work the preflight says remains necessary.

Blocking findings (1):

1. [class] Matrix claims ASR-512 is implemented without demonstrating property-bound execution — docs/research/behavioral-validation/traceability-matrix-asr-512.md:11
   The cited incumbents establish useful pieces of the design, but the matrix repeatedly treats those pieces as an implemented executable behavioral-validation capability. A proposition binding contract does not execute its probe; target-conformance plan execution and participant lifecycle validation are not shown to be bound to the claimed scenario, workflow, or participant property; and the experiment row cites claim/catalog and evidence validation without any executable probe path. This conflicts with the preflight’s own boundary that apparatus success is not property truth and that a stable subject-bound probe case must join the exact claim, trusted executor, observation policy, and result carrier. Either provide and test that end-to-end join for each subject claimed as implemented, or recast the matrix as a gap/readiness analysis rather than requirement traceability asserting completion.

Security review

Verdict: ship

This is a documentation-only change that maps ASR-512 onto existing subject-specific claim, execution, evidence, authorization, redaction, and cleanup boundaries. Its design seam—trusted, capability-checked executors receiving bounded, subject-bound probe cases—preserves separation between untrusted contract data and executable behavior and leaves room for additional governed adapters without introducing a generic command, import, URL, storage, or authentication surface. The diff adds no runtime code, configuration, credentials, or externally reachable behavior, and introduces no concrete exploitable security issue.

No blocking findings.

[Brad-Edwards]

gc_codex_review pre-push cycle 1 of 1 complete for issue #260 on branch '260-behavioral-validation-probes'. Posted by the MCP server to enforce the pre-push hard-cap-1 contract (issues #796, #804, #906). Do not edit or delete — used by the next gc_codex_review (uncommitted) invocation to count cycles.