Context. In a recent full-pipeline consumer pilot run, the pipeline reached sdd:done on a feature no user could run. Of 30 requirement IDs: 8 delivered working end-to-end, 19 partial, 3 missing. Of the spec's declared proof artifacts: 1 delivered as specified, 9 weakened (#[ignore]-gated, env-gated, CI-lane-only, or behind a build feature no shipped binary compiled), 5 missing. No R-ID, proof artifact, or gate required an executed user-visible demo, so sdd:done was reachable with none. The demoability artifact itself (the executed test-plan + the feature's CLI entrypoint) arrived in remediation — after done.
Problem. sdd:done certifies "manufactured," not "delivered." The empty-PR rule was honored per-PR while the feature as a whole stayed undemonstrable.
Fix (two parts):
- Demoability gate at
sdd:done. The tracker must carry a committed end-to-end proof artifact (test-plan-style) whose execution is a check, not prose. sdd:done is blocked until that check has a green run against the merged tree.
- Proof-delivery audit at validate time (deterministic). Grep merged diffs for
#[ignore], env-gating, and required-features; diff against the spec's declared proof artifacts. A proof declared executable but shipped gated fails the audit. In the pilot this grep would have flagged 9 of 15 proofs at validate time.
Acceptance criteria:
- A tracker cannot reach
sdd:done unless ≥1 executed end-to-end proof check is green against merged HEAD.
- sdd-validate emits a per-PR proof-delivery report: each declared proof classified
delivered-executable | gated(#[ignore]/env/feature) | missing, deterministic from the diff.
- A declared-executable proof that ships gated blocks the gate (or demotes to
needs-human naming the specific proof).
Relates to: extends #256 (closed — per-repo dev-env verify entrypoint; necessary but does not enforce an executed demo) and #257 sdd-gates. Complements the eval-agent (#332): this is the gate; the eval agent measures it during the run.
Context. In a recent full-pipeline consumer pilot run, the pipeline reached
sdd:doneon a feature no user could run. Of 30 requirement IDs: 8 delivered working end-to-end, 19 partial, 3 missing. Of the spec's declared proof artifacts: 1 delivered as specified, 9 weakened (#[ignore]-gated, env-gated, CI-lane-only, or behind a build feature no shipped binary compiled), 5 missing. No R-ID, proof artifact, or gate required an executed user-visible demo, sosdd:donewas reachable with none. The demoability artifact itself (the executed test-plan + the feature's CLI entrypoint) arrived in remediation — after done.Problem.
sdd:donecertifies "manufactured," not "delivered." The empty-PR rule was honored per-PR while the feature as a whole stayed undemonstrable.Fix (two parts):
sdd:done. The tracker must carry a committed end-to-end proof artifact (test-plan-style) whose execution is a check, not prose.sdd:doneis blocked until that check has a green run against the merged tree.#[ignore], env-gating, andrequired-features; diff against the spec's declared proof artifacts. A proof declared executable but shipped gated fails the audit. In the pilot this grep would have flagged 9 of 15 proofs at validate time.Acceptance criteria:
sdd:doneunless ≥1 executed end-to-end proof check is green against merged HEAD.delivered-executable | gated(#[ignore]/env/feature) | missing, deterministic from the diff.needs-humannaming the specific proof).Relates to: extends #256 (closed — per-repo dev-env verify entrypoint; necessary but does not enforce an executed demo) and #257 sdd-gates. Complements the eval-agent (#332): this is the gate; the eval agent measures it during the run.