Skip to content

fix: normalize line endings before checksum generation in release archives#4457

Open
rebel117 wants to merge 1 commit into
orchestration-agent:mainfrom
rebel117:fix-4445-normalize-line-endings-checksum
Open

fix: normalize line endings before checksum generation in release archives#4457
rebel117 wants to merge 1 commit into
orchestration-agent:mainfrom
rebel117:fix-4445-normalize-line-endings-checksum

Conversation

@rebel117
Copy link
Copy Markdown

Problem

Release archives built on different OS runners (Linux, macOS, Windows) produced different checksums because generated text files kept platform-specific line endings (CRLF on Windows vs LF on Linux/macOS).

This made reproducibility checks fail intermittently and made it harder to validate official packages.

Solution

Three-pronged fix:

  1. .gitattributes — Declares the LF line ending policy for all text files in the repository, ensuring consistent checkout behavior across platforms.

  2. scripts/normalize-and-checksum.sh — A build step that:

    • Extracts tar.gz / zip archives from the dist/ directory
    • Detects text files and normalizes CRLF → LF
    • Sets deterministic mtimes (2024-01-01) on all files
    • Rebuilds archives with sorted file lists and gzip -n (no timestamp header)
    • Generates checksums.sha256 with sorted entries
  3. docs/release-workflow.yml — A CI workflow example that builds on all three platforms, runs the normalization script, and verifies checksums match across runners before publishing.

Test plan

$ python3 -m pytest tests/test_release_checksum.py -v
tests/test_release_checksum.py::TestLineEndingNormalization::test_crlf_files_get_normalized PASSED
tests/test_release_checksum.py::TestLineEndingNormalization::test_lf_files_unchanged PASSED
tests/test_release_checksum.py::TestLineEndingNormalization::test_checksum_file_generated PASSED
tests/test_release_checksum.py::TestLineEndingNormalization::test_checksum_stability PASSED
tests/test_release_checksum.py::TestLineEndingNormalization::test_gitattributes_exists_and_declares_lf PASSED

All 5 tests pass. Existing test suite shows no regressions.

Closes #4445

…hives

Reproducible builds were breaking because release archives picked up
platform-specific CRLF endings when built on Windows runners. This made
checksums differ between Linux, macOS, and Windows builds.

- Add .gitattributes declaring LF line ending policy for all text files
- Add scripts/normalize-and-checksum.sh that extracts, normalizes CRLF
  to LF, sets deterministic mtimes, and rebuilds archives with sorted
  file lists and gzip -n for timestamp-free compression
- Include release workflow example in docs/ (multi-platform build with
  checksum verification across ubuntu, macos, windows runners)
- Add Makefile targets: make release, make checksum
- Add test suite covering CRLF normalization, LF passthrough, checksum
  generation, cross-run stability, and .gitattributes policy

Closes orchestration-agent#4445
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[ Bounty $3k ] [ CI ] Normalize line endings before checksum generation — release archives

1 participant