Skip to content

Commit 234a4df

Browse files
committed
feat: publish immutable release manifests
1 parent 6904673 commit 234a4df

7 files changed

Lines changed: 692 additions & 12 deletions

File tree

.agents/docs/2026-08-09-mcpp-template-runtime-graphics-aur-implementation-plan.md

Lines changed: 8 additions & 7 deletions
Original file line numberDiff line numberDiff line change
@@ -274,18 +274,19 @@
274274
- Add: `scripts/release/generate_manifest.py`
275275
- Add: `tests/scripts/test_release_manifest.py`
276276
- Modify: `.github/workflows/release.yml`
277-
- Modify: `docs/10-distribution.md`
277+
- Modify: `docs/09-release.md`
278+
- Modify: `docs/zh/09-release.md`
278279

279280
**Interface:**
280281

281282
- Release asset `mcpp-release.json` schema 1 with version, tag, release commit, and exact name/SHA256 for Linux x86_64/aarch64 plus every shipped platform asset.
282283

283-
- [ ] RED: fixtures reject duplicate platform/arch rows, missing sidecars, mismatched hashes, draft/prerelease input, wrong tag/version, and non-deterministic ordering.
284-
- [ ] Implement deterministic manifest generation from downloaded release artifacts and sidecars, recomputing every SHA256.
285-
- [ ] Wire the release workflow so the manifest is uploaded only after all required release assets exist and validation passes.
286-
- [ ] Add a release gate that downloads the uploaded manifest and compares it to the final GitHub release inventory.
287-
- [ ] GREEN: run manifest tests and a local fixture generation twice with byte-identical output.
288-
- [ ] Commit immutable release desired-state manifest.
284+
- [x] RED: fixtures reject duplicate platform/arch rows, missing sidecars, mismatched hashes, draft/prerelease input, wrong tag/version, and non-deterministic ordering.
285+
- [x] Implement deterministic manifest generation from downloaded release artifacts and sidecars, recomputing every SHA256.
286+
- [x] Wire the release workflow so the manifest is uploaded only after all required release assets exist and validation passes.
287+
- [x] Add a release gate that downloads the uploaded manifest and compares it to the final GitHub release inventory.
288+
- [x] GREEN: run manifest tests and a local fixture generation twice with byte-identical output.
289+
- [x] Commit immutable release desired-state manifest.
289290

290291
## Task 9: Replace Combined AUR Publishing with an mcpp-bin Reconciler
291292

.agents/docs/2026-08-09-mcpp-template-runtime-graphics-aur-validation.md

Lines changed: 18 additions & 0 deletions
Original file line numberDiff line numberDiff line change
@@ -228,6 +228,24 @@ Task 7 focused local evidence on the latest self-hosted source binary
228228
- host xlings config aggregate remains `f218aadf3792ee815c8535ce0ca0bb53f634fecbdbc5d0d47db442052d786d1b`.
229229
- Linux semantics are locally exercised. Mach-O and PE flag spelling has pure unit coverage on Linux; native macOS/Windows compilation and behavior remain latest-HEAD PR CI gates, not inferred passes.
230230

231+
### 5.7 Immutable release desired state (Task 8)
232+
233+
| Gate | RED evidence | Production change | GREEN evidence |
234+
|---|---|---|---|
235+
| manifest contract | The initial 11-case suite produced 12 failing assertions because no generator existed; fixtures cover duplicate normalized platform/arch, missing sidecars, payload/sidecar hash disagreement, draft/prerelease releases, wrong tag/version, missing required assets, duplicate release names, and invalid commit identity | Added schema-1 generation from the GitHub release JSON plus downloaded payloads; four current platforms are mandatory, every additional versioned platform payload is included, every digest is recomputed, and output is sorted deterministically | `test_release_manifest.py` passes **12/12**; randomized release-API asset order produces byte-identical output and every manifest hash equals the independently recomputed fixture payload hash |
236+
| future platform closure | Review fixture added a valid FreeBSD/riscv64 versioned payload; the first implementation stayed green while silently omitting that row | Primary-platform discovery is provider-neutral and version-exact rather than a fixed four-platform allowlist; the four present release targets remain a required floor | The focused future-platform test changed RED-to-GREEN and asserts the complete row and SHA256 occur in the manifest |
237+
| immutable publication | Previously each platform uploaded independently and downstream jobs inferred completeness from timing and filenames | New `release-manifest` job waits for all four uploaders, validates non-draft/non-prerelease identity, uploads only when absent, refuses a byte-different existing manifest, polls public API visibility, then refetches all public assets and regenerates/compares the manifest | Ruby/Psych parses `release.yml` and confirms six jobs, `publish-ecosystem` depends on `release-manifest`, Python bytecode compilation passes, and `git diff --check` is clean; live GitHub upload/refetch remains a PR/release CI gate |
238+
| maintainer contract | Release docs had no machine-readable completeness boundary or reproducible local audit command | English and Chinese release docs define schema, row scope, immutability/rerun behavior, public refetch command, and checklist gates | Documentation examples invoke the same checked-in generator and explicitly recompute payload hashes rather than trusting manifest or sidecar values |
239+
240+
Task 8 focused local evidence:
241+
242+
- RED: missing generator yielded 12 assertion failures across 11 initial tests; the later additional-platform review row independently failed because FreeBSD/riscv64 was omitted.
243+
- GREEN: `python3 tests/scripts/test_release_manifest.py` reports **12 tests OK** in 0.309s; `python3 -m py_compile` succeeds for generator and tests.
244+
- Determinism: the complete fixture shuffles API inventory with seed 398, invokes the CLI twice to separate output files, and compares exact bytes.
245+
- Public-inventory replay: downloaded the eight real versioned payload/sidecar assets from stable `v2026.8.8.4`, resolved tag commit `55a39d90fe98b5475fc394ac8487fe6804b2b84f`, and generated twice byte-identically. The four-row manifest SHA256 is `6614ba2db65c8c28cb5a9d3466bd6cf3007634221da76d25216785061c647edb`; every real sidecar and recomputed payload digest agreed.
246+
- Workflow structure: Ruby/Psych parses `.github/workflows/release.yml`; all six jobs are present and ecosystem publication is gated by the immutable manifest job.
247+
- Public GitHub upload/refetch and eventual-consistency polling cannot be truthfully claimed from local fixtures; they remain explicit latest-HEAD release CI evidence.
248+
231249
## 6. Pull request and CI
232250

233251
Not published yet. This section will record PR URL, local/remote HEAD, review state, all latest-head job IDs and terminal conclusions.

.github/workflows/release.yml

Lines changed: 103 additions & 3 deletions
Original file line numberDiff line numberDiff line change
@@ -725,16 +725,116 @@ jobs:
725725
dist/mcpp-windows-x86_64.zip
726726
dist/mcpp-windows-x86_64.zip.sha256
727727
728+
# Seal the complete, non-draft release inventory only after every platform
729+
# uploader has finished. The manifest is desired state for downstream
730+
# reconcilers: a rerun may reproduce it byte-for-byte, but may never replace
731+
# it with different bytes for the same tag.
732+
release-manifest:
733+
name: validate + seal release manifest
734+
needs: [build-release, build-linux-aarch64, build-macos, build-windows]
735+
runs-on: ubuntu-24.04
736+
permissions:
737+
contents: write
738+
timeout-minutes: 20
739+
env:
740+
GH_TOKEN: ${{ secrets.GITHUB_TOKEN }}
741+
steps:
742+
- uses: actions/checkout@v4
743+
with:
744+
fetch-depth: 0
745+
746+
- name: Resolve immutable release identity
747+
id: resolve
748+
run: |
749+
if [ "${{ github.event_name }}" = "push" ]; then
750+
TAG="${{ github.ref_name }}"
751+
elif [ -n "${{ github.event.inputs.tag }}" ]; then
752+
TAG="${{ github.event.inputs.tag }}"
753+
else
754+
VERSION=$(awk -F '"' '/^version[[:space:]]*=/{print $2; exit}' mcpp.toml)
755+
test -n "$VERSION" || { echo 'failed to read [package].version from mcpp.toml'; exit 1; }
756+
TAG="v$VERSION"
757+
fi
758+
git fetch --force origin "refs/tags/$TAG:refs/tags/$TAG"
759+
COMMIT=$(git rev-list -n 1 "refs/tags/$TAG")
760+
test -n "$COMMIT"
761+
echo "tag=$TAG" >> "$GITHUB_OUTPUT"
762+
echo "version=${TAG#v}" >> "$GITHUB_OUTPUT"
763+
echo "commit=$COMMIT" >> "$GITHUB_OUTPUT"
764+
765+
- name: Generate, upload once, and refetch manifest
766+
run: |
767+
TAG="${{ steps.resolve.outputs.tag }}"
768+
VERSION="${{ steps.resolve.outputs.version }}"
769+
COMMIT="${{ steps.resolve.outputs.commit }}"
770+
AUDIT_ROOT=$(mktemp -d)
771+
mkdir -p "$AUDIT_ROOT/assets" "$AUDIT_ROOT/publish"
772+
773+
gh api "repos/${GITHUB_REPOSITORY}/releases/tags/$TAG" \
774+
> "$AUDIT_ROOT/release.json"
775+
gh release download "$TAG" --repo "$GITHUB_REPOSITORY" \
776+
--dir "$AUDIT_ROOT/assets"
777+
python3 scripts/release/generate_manifest.py \
778+
--release-json "$AUDIT_ROOT/release.json" \
779+
--assets-dir "$AUDIT_ROOT/assets" \
780+
--version "$VERSION" \
781+
--tag "$TAG" \
782+
--commit "$COMMIT" \
783+
--output "$AUDIT_ROOT/publish/mcpp-release.json"
784+
785+
if [ -f "$AUDIT_ROOT/assets/mcpp-release.json" ]; then
786+
cmp "$AUDIT_ROOT/assets/mcpp-release.json" \
787+
"$AUDIT_ROOT/publish/mcpp-release.json"
788+
echo "Existing manifest is byte-identical; leaving it untouched."
789+
else
790+
gh release upload "$TAG" \
791+
"$AUDIT_ROOT/publish/mcpp-release.json" \
792+
--repo "$GITHUB_REPOSITORY"
793+
fi
794+
795+
# Do not trust the upload command alone. Fetch the final public
796+
# inventory into a clean directory, recompute every payload digest,
797+
# regenerate desired state, and compare the published bytes.
798+
MANIFEST_VISIBLE=false
799+
for attempt in {1..12}; do
800+
if gh api "repos/${GITHUB_REPOSITORY}/releases/tags/$TAG" \
801+
--jq '.assets[].name' | grep -Fxq mcpp-release.json; then
802+
MANIFEST_VISIBLE=true
803+
break
804+
fi
805+
echo "Waiting for mcpp-release.json API visibility ($attempt/12)"
806+
sleep 5
807+
done
808+
test "$MANIFEST_VISIBLE" = true
809+
mkdir -p "$AUDIT_ROOT/final-assets"
810+
gh api "repos/${GITHUB_REPOSITORY}/releases/tags/$TAG" \
811+
> "$AUDIT_ROOT/final-release.json"
812+
gh release download "$TAG" --repo "$GITHUB_REPOSITORY" \
813+
--dir "$AUDIT_ROOT/final-assets"
814+
test -f "$AUDIT_ROOT/final-assets/mcpp-release.json"
815+
python3 scripts/release/generate_manifest.py \
816+
--release-json "$AUDIT_ROOT/final-release.json" \
817+
--assets-dir "$AUDIT_ROOT/final-assets" \
818+
--version "$VERSION" \
819+
--tag "$TAG" \
820+
--commit "$COMMIT" \
821+
--output "$AUDIT_ROOT/final-expected.json"
822+
cmp "$AUDIT_ROOT/final-assets/mcpp-release.json" \
823+
"$AUDIT_ROOT/final-expected.json"
824+
cat "$AUDIT_ROOT/final-assets/mcpp-release.json"
825+
728826
# Publish this release into the xlings ecosystem, after ALL platform builds
729-
# have uploaded their assets:
827+
# have uploaded their assets and the immutable manifest gate has passed:
730828
# ① mirror binaries → xlings-res/mcpp (GitHub + GitCode) so XLINGS_RES
731829
# downloads resolve on every platform (incl. the CN/GitCode path);
732830
# ② open a PR against openxlings/xim-pkgindex bumping mcpp to this version
733831
# (a maintainer merges it — index git source is not on the critical path).
734-
# Best-effort / non-blocking: a failure here never fails the release.
832+
# These publication steps are required for ecosystem completeness: their
833+
# failure leaves the GitHub Release object available but keeps this workflow
834+
# red, so post-release verification cannot report a complete release.
735835
# Shared vendored scripts live in .github/tools/ (kept in sync with xlings).
736836
publish-ecosystem:
737-
needs: [build-release, build-linux-aarch64, build-macos, build-windows]
837+
needs: release-manifest
738838
runs-on: ubuntu-latest
739839
# A4 hardening: a single stuck upload once held this job >1h (6h default
740840
# ceiling). The mirror script has per-file timeouts and (post-0.0.89)

docs/09-release.md

Lines changed: 61 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -41,11 +41,48 @@ tag from `mcpp.toml`) does all of this:
4141
```
4242
build ×4 (linux x86_64 / linux aarch64 / macOS ARM64 / Windows x64)
4343
→ GitHub Release v<version> with tarballs + .sha256 sidecars
44+
→ recompute every payload hash and publish immutable mcpp-release.json
4445
→ mirror to xlings-res/mcpp on BOTH GitHub and GitCode
4546
→ open the version-bump PR against openxlings/xim-pkgindex
4647
→ workflow_run hook fires ci-fresh-install
4748
```
4849

50+
### 2.1 The immutable release manifest
51+
52+
`release-manifest` waits for all four platform upload jobs. It downloads the
53+
final non-draft, non-prerelease GitHub Release, recomputes each versioned
54+
platform payload's SHA256, verifies the corresponding `.sha256` sidecar, and
55+
publishes `mcpp-release.json` schema 1:
56+
57+
```json
58+
{
59+
"schema": 1,
60+
"version": "<version>",
61+
"tag": "v<version>",
62+
"commit": "<full-tag-commit>",
63+
"assets": [
64+
{
65+
"platform": "linux",
66+
"arch": "x86_64",
67+
"name": "mcpp-<version>-linux-x86_64.tar.gz",
68+
"sha256": "<recomputed-sha256>"
69+
}
70+
]
71+
}
72+
```
73+
74+
The array is sorted by platform, architecture, and name. It contains every
75+
versioned platform payload, including the required Linux x86_64, Linux aarch64,
76+
macOS ARM64, and Windows x86_64 assets. Versionless aliases and the source
77+
archive are intentionally not desired-state rows.
78+
79+
The workflow then downloads the public release again, regenerates the manifest,
80+
and requires a byte-for-byte match. A workflow rerun accepts an existing
81+
manifest only when it is already byte-identical; it never overwrites different
82+
bytes for the same tag. Downstream release consumers (notably the `mcpp-bin`
83+
AUR reconciler) must consume this manifest instead of guessing completeness
84+
from a moving workspace or from a subset of release assets.
85+
4986
Two steps are **not** automated:
5087

5188
- **merging the xim-pkgindex bump PR** — a maintainer does it. Until it lands,
@@ -71,6 +108,28 @@ curl -fsSL -o /tmp/p.tgz "https://github.com/xlings-res/mcpp/releases/download/$
71108
sha256sum /tmp/p.tgz # compare against pkgs/m/mcpp.lua in xim-pkgindex
72109
```
73110

111+
To repeat the release gate locally against the public GitHub inventory:
112+
113+
```bash
114+
V=<version>
115+
TAG="v$V"
116+
AUDIT=$(mktemp -d)
117+
mkdir -p "$AUDIT/assets"
118+
gh api "repos/mcpp-community/mcpp/releases/tags/$TAG" > "$AUDIT/release.json"
119+
gh release download "$TAG" -R mcpp-community/mcpp --dir "$AUDIT/assets"
120+
python3 scripts/release/generate_manifest.py \
121+
--release-json "$AUDIT/release.json" \
122+
--assets-dir "$AUDIT/assets" \
123+
--version "$V" \
124+
--tag "$TAG" \
125+
--commit "$(git rev-list -n 1 "$TAG")" \
126+
--output "$AUDIT/expected.json"
127+
cmp "$AUDIT/assets/mcpp-release.json" "$AUDIT/expected.json"
128+
```
129+
130+
This command recomputes payload hashes; it does not copy hashes out of the
131+
published manifest or blindly trust the sidecars.
132+
74133
Then a real install, in a **clean-room `XLINGS_HOME`** — never the machine's own
75134
`~/.xlings`, which can mask a broken index with cached state:
76135

@@ -163,7 +222,8 @@ would again let the index guard and the installed version drift apart.
163222
[ ] `bash .github/tools/check_version_pins.sh` passes (verifies `mcpp.toml` = `MCPP_VERSION`, and `.xlings.json` is not newer)
164223
[ ] merge to main, CI green
165224
[ ] gh workflow run release.yml --ref main
166-
[ ] release.yml green (4 builds + publish-ecosystem)
225+
[ ] release.yml green (4 builds + immutable manifest + publish-ecosystem)
226+
[ ] downloaded mcpp-release.json regenerates byte-identically from public assets
167227
[ ] mirrors serve all four platforms on BOTH hosts, sha256 recomputed
168228
[ ] merge the xim-pkgindex bump PR
169229
[ ] clean-room XLINGS_HOME: xlings install mcpp@<version> succeeds

docs/zh/09-release.md

Lines changed: 56 additions & 1 deletion
Original file line numberDiff line numberDiff line change
@@ -37,11 +37,44 @@ POSIX `sh`/dash 解析不了 —— `sh check_version_pins.sh` 会在第 95 行
3737
```
3838
四平台构建(linux x86_64 / linux aarch64 / macOS ARM64 / Windows x64)
3939
→ GitHub Release v<version>,含 tarball 与 .sha256 边车文件
40+
→ 重新计算每个载荷的哈希并发布不可变 mcpp-release.json
4041
→ 镜像到 xlings-res/mcpp 的 GitHub 与 GitCode 双端
4142
→ 向 openxlings/xim-pkgindex 开版本 bump PR
4243
→ workflow_run 钩子触发 ci-fresh-install
4344
```
4445

46+
### 2.1 不可变 release manifest
47+
48+
`release-manifest` 会等待四个平台上传 job 全部结束,然后下载最终的非 draft、
49+
非 prerelease GitHub Release,重新计算每个带版本平台载荷的 SHA256,校验对应的
50+
`.sha256` 边车文件,并发布 schema 1 的 `mcpp-release.json`
51+
52+
```json
53+
{
54+
"schema": 1,
55+
"version": "<version>",
56+
"tag": "v<version>",
57+
"commit": "<tag-所指完整-commit>",
58+
"assets": [
59+
{
60+
"platform": "linux",
61+
"arch": "x86_64",
62+
"name": "mcpp-<version>-linux-x86_64.tar.gz",
63+
"sha256": "<重新计算的-sha256>"
64+
}
65+
]
66+
}
67+
```
68+
69+
数组按平台、架构、名称排序,包含所有带版本的平台载荷;其中 Linux x86_64、
70+
Linux aarch64、macOS ARM64、Windows x86_64 四项是硬性要求。无版本别名和源码包
71+
刻意不进入 desired-state 行。
72+
73+
workflow 随后会再次下载公开 release、重新生成 manifest,并要求逐字节一致。
74+
重跑 workflow 时,已有 manifest 只有在字节完全相同时才会被接受;同一 tag 下
75+
绝不以不同内容覆盖。下游发布消费者(尤其 `mcpp-bin` AUR reconciler)必须消费
76+
该 manifest,不能从会变化的工作区或部分 release 资产猜测发布是否完整。
77+
4578
两步**没有**自动化:
4679

4780
- **合并 xim-pkgindex 的 bump PR** —— 由维护者完成。在它落地之前,发布出来的版本
@@ -66,6 +99,27 @@ curl -fsSL -o /tmp/p.tgz "https://github.com/xlings-res/mcpp/releases/download/$
6699
sha256sum /tmp/p.tgz # 与 xim-pkgindex 的 pkgs/m/mcpp.lua 对照
67100
```
68101

102+
要在本地针对 GitHub 公开资产重放 release gate:
103+
104+
```bash
105+
V=<version>
106+
TAG="v$V"
107+
AUDIT=$(mktemp -d)
108+
mkdir -p "$AUDIT/assets"
109+
gh api "repos/mcpp-community/mcpp/releases/tags/$TAG" > "$AUDIT/release.json"
110+
gh release download "$TAG" -R mcpp-community/mcpp --dir "$AUDIT/assets"
111+
python3 scripts/release/generate_manifest.py \
112+
--release-json "$AUDIT/release.json" \
113+
--assets-dir "$AUDIT/assets" \
114+
--version "$V" \
115+
--tag "$TAG" \
116+
--commit "$(git rev-list -n 1 "$TAG")" \
117+
--output "$AUDIT/expected.json"
118+
cmp "$AUDIT/assets/mcpp-release.json" "$AUDIT/expected.json"
119+
```
120+
121+
这个命令会重新计算载荷哈希,不会从已发布 manifest 抄哈希,也不会盲信边车文件。
122+
69123
然后在 **clean-room `XLINGS_HOME`** 里真装一次 —— 绝不要用本机的 `~/.xlings`
70124
它的缓存状态会把一个坏掉的索引掩盖过去:
71125

@@ -133,7 +187,8 @@ $(find "$XLINGS_HOME" -name mcpp -type f -path '*/bin/*' | head -1) --version
133187
[ ] `bash .github/tools/check_version_pins.sh` 通过(校验 `mcpp.toml` = `MCPP_VERSION`,且 `.xlings.json` 未领先)
134188
[ ] 合入 main,CI 全绿
135189
[ ] gh workflow run release.yml --ref main
136-
[ ] release.yml 全绿(4 个构建 + publish-ecosystem)
190+
[ ] release.yml 全绿(4 个构建 + 不可变 manifest + publish-ecosystem)
191+
[ ] 下载的 mcpp-release.json 能从公开资产逐字节重新生成
137192
[ ] 双端都服务四个平台,sha256 重新算过
138193
[ ] 合并 xim-pkgindex 的 bump PR
139194
[ ] clean-room XLINGS_HOME:xlings install mcpp@<version> 成功

0 commit comments

Comments
 (0)