Counterfactual And Necessity Validation (ASR-513)

[Brad-Edwards]

ASR-513 | FUNCTIONAL | SHOULD | Wave 3 | DRAFT

Statement

The ecosystem shall support counterfactual or necessity-oriented validation where the ecosystem claims that particular conditions, weaknesses, controls, or behaviors are required for an outcome.

Status

Independent validation technique. Follows the layered-validation framework in #97. CI branch UID: ASR-513.

Requirements

• ASR-513

---

Tracks Ground Control requirement ASR-513

[Brad-Edwards]

🛠️ Picked up by /implement - driver codex, branch 261-counterfactual-necessity-validation, 2026-07-28T17:48:58.154Z.

[Brad-Edwards]

gc workflow phase recorded: preflight (issue #261). Posted by the MCP server to enforce ordering between workflow steps (issue #794 MVP-2). Do not edit or delete — used by downstream tools to gate phase prerequisites.

[Brad-Edwards]

gc workflow phase recorded: plan (issue #261). Posted by the MCP server to enforce ordering between workflow steps (issue #794 MVP-2). Do not edit or delete — used by downstream tools to gate phase prerequisites.

Delivery contract

Implement issue #261 as the requirement-free workflow contract recorded by Ground Control while using ASR-513 as the repository-policy and CI UID. The delivered capability will support a bounded binary but-for necessity comparison only: one revisioned claim, immutable baseline and intervention-world identities, independently admitted proposition-truth evidence, verified intervention, declared comparability, and verified cleanup. It will not add a causal graph language, general causal engine, runtime route, database, plugin, command executor, or universal proof claim.

Implementation plan

1. Extend the canonical behavioral-relation authority in contracts/concept-authority/behavioral-relations-v1.json with one finite bounded-but-for-necessity relation and a counterfactual-validation claim surface, then advance the exact catalog coordinate from rev3 to rev4. Update the mirrored conformance fixture and every live structured claim binding, generated-artifact producer, current formal/reference document, test helper, and policy harness that binds the old coordinate. Historical issue preflight records remain historical. The published JSON Schema shape is not changing; generated/published schema parity will still be verified, and no schema-publication ledger update is required unless that check reveals a schema-content change.
2. Add implementations/python/packages/raes_conformance/necessity_validation.py as a small frozen internal seam. It will reuse BehavioralClaimBindingModel, validate_behavioral_claim_binding(), PropositionTruthResultModel, Diagnostic, and Severity; bind exact candidate/outcome/world/run/digest/criterion/matching-policy identities; require declared capabilities; and distinguish supported, refuted, inconclusive, and unsupported from execution status. A supported result will require baseline true, counterfactual false, verified intervention evidence, satisfied matching policy with no undeclared differences, admitted per-world evidence, and independently verified cleanup with no residue. Unknown/unsupported truth, baseline non-vacuity failure, no-op/partial/collateral intervention, missing evidence, mismatched identities, or failed cleanup will fail closed without exposing raw payloads or exception text.
3. Drive the implementation with implementations/python/tests/test_necessity_validation.py first. Observe focused failures before production code, then cover the positive join and digest, both-worlds-true refutation, baseline-false and indeterminate outcomes, wrong relation/subject/outcome/world identities, unsupported capability, unverified intervention, unmatched dimensions, missing evidence, cleanup failure/residue, and stable redacted diagnostics. Extend implementations/python/tests/test_behavioral_relations.py and policy tests for the new relation, claim surface, exact rev4 binding discipline, and rejection of stale coordinates.
4. Add docs/research/behavioral-validation/traceability-matrix-asr-513.md and update the current behavioral-relation reference prose to map the issue statement to the relation, comparator, tests, evidence boundary, and explicit nonclaims. The architecture-preflight note remains the durable design boundary; no new ADR is needed because ADR-022/068/072/079/081/084 already own the affected concepts.

Cross-cutting design checks

• Security: No untrusted dispatch, environment binding, filesystem discovery, executable path, shell command, secret value, raw evidence, or host/native object enters the comparator. Closed frozen inputs are supplied by trusted code; exact revision/digest/ref joins and capability checks fail closed; public diagnostics contain stable codes and coarse messages only.
• Maintainability: Reuse the existing behavioral catalog/validator, proposition truth contract, diagnostics, experiment/run references, and conformance package patterns. Add no second relation registry, truth algebra, evidence store, exception hierarchy, or logger.
• Extensibility: Keep criterion and matching-policy identities explicit so a future quantitative/statistical criterion can add a separate governed adapter without changing this binary criterion or reworking claim/run/evidence authority.
• Whole-repository view: Update the concept authority, its fixture mirror, all live claim bindings and policy harnesses, conformance implementation/tests, current formal/reference documentation, corpus/package checks, and repository-policy/governance gates atomically. Preserve module-boundary, source-size, authority-surface, and generated-schema rules.

Verification

Run the focused necessity and behavioral-relation tests during red/green/refactor, then proportionate neighboring conformance, experiment-contract, proposition-truth, and behavioral-policy tests. Before completion, with RAES_REQUIREMENT_UID=ASR-513, run:

• implementations/python/.venv/bin/python tools/check_repo_policy.py
• implementations/python/.venv/bin/python tools/check_requirement_governance.py
• implementations/python/.venv/bin/python tools/verify_all.py

The Ground Control completion gate will additionally run the configured full verification and policy commands. Do not edit CHANGELOG.md, add a changelog fragment, or change the package version. Use a consumer-visible Conventional Commit feat: title for publication.

[Brad-Edwards]

gc workflow phase recorded: preflight (issue #261). Posted by the MCP server to enforce ordering between workflow steps (issue #794 MVP-2). Do not edit or delete — used by downstream tools to gate phase prerequisites.

[Brad-Edwards]

gc workflow phase recorded: plan (issue #261). Posted by the MCP server to enforce ordering between workflow steps (issue #794 MVP-2). Do not edit or delete — used by downstream tools to gate phase prerequisites.

Plan correction after requirement-bound preflight

Issue 261 now correctly lists ASR-513 in its authoritative Requirements section, so the implementation is requirement-bound rather than requirement-free. The existing plan remains in force except for this stronger evidence-admission design:

1. Keep BoundedButForCase and the pure comparator, but make direct construction of comparison evidence unavailable. Add typed intervention, matching-policy, and cleanup verification records with immutable artifact identity plus digest-pinned producer and validator bindings.
2. Add a trusted raes_conformance evidence assembler that accepts validated ExperimentTaskModel, ExperimentRunModel, ExperimentEvidenceRecordModel, and PropositionTruthResultModel instances. It will call validate_experiment_run_against_task(), require the case world/run/snapshot/digest joins, resolve every truth and verification evidence reference through the exact run traceability and ExperimentEvidenceRecordModel.run_ref, and derive comparison booleans only from the typed verification records.
3. Preserve verification record, producer, and validator identities in the assembled result. Bare caller booleans or free-form evidence-reference strings must not be accepted at the comparison boundary.
4. Extend the red/green tests for direct-construction refusal, invalid run/task and truth/evidence-to-run joins, missing run traceability, wrong verification kind/scope, producer/validator identity preservation, and the existing positive/refuted/inconclusive/unsupported outcomes. Update the ASR-513 traceability matrix to name the trusted assembler and exact joins.

All prior catalog, finite-claim, security, non-goal, documentation, and repository-wide verification constraints remain unchanged.

[Brad-Edwards]

Execution obligation ISSUE-261-MECHANICAL-VERIFY-TIMEOUT — Opened

Category: workflow
Observed state: The repository completion command requires approximately 13 minutes, while gc_implement_mechanical is synchronously terminated by a fixed 300-second MCP call timeout.
Impact: The mandatory /implement mechanical verification phase cannot return a terminal success envelope, so agent reviews and publication cannot proceed under the structurally enforced workflow despite the repository gates passing.
Current obligation: Provide an asynchronous mechanical verification path or extend the gc_implement_mechanical transport/runtime boundary beyond the repository's configured 13-minute completion command.

Evidence

• Direct canonical RAES_REQUIREMENT_UID=ASR-513 implementations/python/.venv/bin/python tools/verify_all.py completed successfully in 13 minutes: policy, lint, contracts, 5,284 unit tests, 55 integration tests, 92% coverage, docs HTML and linkcheck all passed.
• gc_implement_mechanical(action=verify) timed out awaiting the MCP tools/call after 300 seconds while its Nox child remained active.
• A second gc_implement_mechanical(action=verify) invoked through a resumable orchestration cell also timed out at the underlying MCP tools/call boundary after 300 seconds.
• gc_assert_quality_gates returned ok=true for aces-sdl / ASR-513.

[Brad-Edwards]

Execution obligation ISSUE-261-MECHANICAL-VERIFY-TIMEOUT — Escalated

Category: workflow
Observed state: The required synchronous gate duration exceeds the connector's hard timeout and is canceled before completion.
Impact: The /implement workflow cannot legally advance to review, commit, push, or PR creation without a terminal mechanical verification result.
Current obligation: Provide an asynchronous mechanical verification path or extend the gc_implement_mechanical transport/runtime boundary beyond the repository's configured 13-minute completion command.

Evidence

• Canonical repository verification passed in 13 minutes.
• Two mandatory mechanical verification calls were terminated by the fixed 300-second MCP transport limit.
• No asynchronous mechanical-verification tool is available; async support is currently exposed only for review/preflight jobs.

Pause class: hard_external_dependency
Decision request: Please provide or enable an asynchronous gc_implement_mechanical verify path (or raise its MCP call timeout above 13 minutes), then ask me to continue issue 261. The implementation and direct canonical repository verification are complete and green.

This obligation remains open while the decision is pending.

[Brad-Edwards]

/ground-control authorize-wontfix ISSUE-261-MECHANICAL-VERIFY-TIMEOUT