Skip to content

fix: verify expansion artifacts before the paper release republishes them#17

Merged
magnaquant merged 2 commits into
mainfrom
fix/release-artifact-validation
Jul 2, 2026
Merged

fix: verify expansion artifacts before the paper release republishes them#17
magnaquant merged 2 commits into
mainfrom
fix/release-artifact-validation

Conversation

@magnaquant

Copy link
Copy Markdown
Owner

Summary

Follow-up to #16, closing the Medium fail-open defect found in review: the paper wrapper checked only the expansion manifest's source commit, then copied paper/expansion/results and figures into the build worktree without validating the recorded hashes, so a tampered expansion output would flow into the published PDFs (reproduced by editing a generated value with the manifest left unchanged).

  • New verify_expansion_artifacts in scripts/run_paper_experiments.py: every manifest-listed artifact must exist as a regular non-symlink file at a safe relative path inside the expansion tree and match its recorded SHA-256; every file on disk under results/ and figures/ must be listed (results/manifest.json is the sole exception, as it cannot record its own hash).
  • release_paper_artifacts.sh calls the verifier after the source-commit equality check and before copying; both wrappers' comments corrected (they overclaimed that only release-critical source dirt could corrupt a release).
  • Nine regression tests: valid tree accepted; tampered, missing, unexpected, symlinked, traversal, and absolute-path artifacts fail closed; empty manifest rejected; wrapper calls the verifier before the copy; the committed expansion tree validates.
  • CLAUDE.md documents the coordinated dependency-release flow (pyproject.toml and poetry.lock are release-critical fingerprinted inputs).
  • dependabot.yml: pip version updates move to monthly grouped cadence; security alerts remain handled immediately.
  • Both releases regenerated from clean source commit 34096ad, with the new verification active during the paper release (positive-path proof of the changed-source expansion-first flow).

Verification

  • Tamper repro now fails closed: editing ExpansionEngineMaxDailyBp with the manifest unchanged raises "expansion artifact digest mismatch".
  • Economic CSVs, figures, and target-tape hashes byte-identical; only manifests, source-digest macro, PDFs, checksums, and build manifests changed; pdftotext diff is one digest line per PDF.
  • Both manifests record source_commit 34096ad with a clean worktree at start; cross-release equality holds.
  • Local gates: ruff clean; 464/464 tests pass (455 prior + 9 new).
  • Note: a full end-to-end wrapper test cannot run in CI (it needs the owner-supplied price matrix and pinned Tectonic); the verifier logic is unit-tested and the changed-source flow was executed for real in this PR.

Merge notes

Merge by command-line fast-forward push only (git push origin fix/release-artifact-validation:main). Do NOT use squash or the GitHub rebase button: both rewrite SHAs and would orphan the recorded source commit 34096ad. Do not move research-audit-v1.

magnaprog added 2 commits July 1, 2026 23:54
…them

The paper wrapper checked only that the expansion manifest recorded the
expected source commit, then copied paper/expansion/results and figures into
the detached build worktree without validating the recorded hashes. A
tampered or unexpected expansion output would flow into the published PDFs
without any digest failure (reproduced: editing a generated value while
leaving the manifest unchanged produced a successful release reporting the
edited number).

- Add verify_expansion_artifacts to scripts/run_paper_experiments.py: every
  file listed in manifest[artifacts] must exist, be a regular non-symlink
  file inside the expansion tree with a safe relative path, and match its
  recorded SHA-256; every file on disk under results/ and figures/ must be
  listed (results/manifest.json is the only exception because it cannot
  record its own hash).
- Call it from release_paper_artifacts.sh after the source-commit equality
  check and before the copy; correct both wrappers' comments, which
  overclaimed that only release-critical source dirt could corrupt a release.
- Regression tests: valid tree accepted; tampered, missing, unexpected,
  symlinked, traversal, and absolute-path artifacts each fail closed; the
  wrapper calls the verifier before copying; the committed expansion tree
  validates.
- CLAUDE.md: document that dependency changes require regenerating both
  artifact releases because pyproject.toml and poetry.lock are
  release-critical fingerprinted inputs.
- dependabot.yml: move pip version updates to a monthly grouped cadence;
  security alerts remain handled immediately via the coordinated flow.
Both releases regenerated from clean source commit 34096ad via the release
wrappers (expansion first, then paper); the paper release ran with the new
expansion-artifact verification active. Every economic CSV, figure, and
target-tape hash is byte-identical to the prior release; the only content
change in either PDF is the source-tree digest hex in the reproducibility
appendix (pdftotext diff: one line per document).
@magnaquant magnaquant merged commit 0ad13ef into main Jul 2, 2026
13 checks passed
@magnaquant magnaquant deleted the fix/release-artifact-validation branch July 2, 2026 07:05
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants