Publish each platform VSIX independently with retries so one registry timeout cannot ship a partial release - #354
Open
Wamwea wants to merge 2 commits into
Open
Conversation
… timeout cannot ship a partial release One transient Marketplace timeout aborted the v0.31.0 publish loop after darwin-arm64, pinning darwin-x64, linux-arm64, linux-x64 and win32-x64 to v0.30.x while Open VSX served all five platforms. Both registry jobs now delegate to scripts/publish-vsixes.mjs: per-platform retry with linear backoff, every platform attempted regardless of individual failures, a pre-publish completeness check against the build matrix count, and a failure that names the platforms that never reached the registry. scripts/test-release-publish-contract.mjs executes both publish steps against a scripted registry timeout and enforces the contract ([DEPLOY-PUBLISH-COMPLETE], closes Nimblesite#348).
…te runner Adversarial review findings on the publish-completeness change: - assert each publish job's --expected equals the build matrix's vsix_target count, so a sixth platform cannot ship with a stale count - use path.delimiter and a junction-type symlink so the suite runs on a Windows checkout, matching the Windows release-gate support from Nimblesite#322 - split runPublishStep into sandbox construction and result summary Deslop flagged the suite's test-runner loop as a fifth copy of the loop carried by the four existing contract suites, so the runner now lives once in scripts/contract-suite.mjs and all five suites import it.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
TLDR
One transient registry timeout can no longer ship a partial release: both VSIX publish jobs retry each platform independently, attempt all five platforms regardless of individual failures, and fail naming the platforms that never reached the registry ([DEPLOY-PUBLISH-COMPLETE]). Closes #348.
Details
On the v0.31.0 release (run 31478952722, attempt 1), one Marketplace request timeout aborted the
set -epublish loop afterdarwin-arm64:darwin-x64failed andlinux-arm64/linux-x64/win32-x64were never attempted, while Open VSX served all five. Nothing in the run named the missing platforms.New files
scripts/publish-vsixes.mjs— one shared publisher used by both registry jobs so they cannot drift apart. Per-platform retry (3 attempts, linear backoff viaDESLOP_PUBLISH_BACKOFF_SECONDS, default 20 s), every platform attempted even when one keeps failing, and a final::error::naming the VSIXes that never reached the registry. It refuses to start when fewer VSIX artifacts exist than the build matrix produced (--expected 5) — a missing artifact upload must not become a partial release.--skip-duplicateis kept on every attempt, so retries and re-runs of a partially published tag stay idempotent. The--pre-releasehandling for hyphenated tags moved out of both YAML blocks into the script.scripts/test-release-publish-contract.mjs— contract suite for the publish steps (see the tests section below), wired intomake deployment-verify.scripts/contract-suite.mjs— the TAP-style test-runner loop that the four existing contract suites each carried a copy of (and this PR would have made a fifth — Deslop flagged the cluster, structural 1.00 / token 1.00). It now lives once;test-release-workflow-contract.mjs,test-release-version-stamping.mjs,test-verifiers.mjs, andtest-deployment-docs-contract.mjsare rewired onto it with identical output.Modified behaviour
.github/workflows/release.yml—publish-marketplaceandpublish-openvsxdelegate their loops to the shared script; both gained anactions/checkout@v7step (neither previously checked out source) and hadtimeout-minutesraised 10 → 30, because the observed Marketplace failure mode is a ~3-minute hang per request and a 10-minute budget would let the runner kill the job mid-retry, recreating the "never attempted" state.Makefile—deployment-verifyruns the new suite.docs/specs/release.md— new [DEPLOY-PUBLISH-COMPLETE] entry under Distribution channels documenting the retry, completeness, and naming guarantees.Breaking changes: none to any public API, CLI flag, or report format. Release-pipeline behaviour changes: a partial publish that previously reported success-so-far now fails the job loudly with platform names, and the publish jobs' timeout budget is 30 minutes.
How Do The Automated Tests Prove It Works?
scripts/test-release-publish-contract.mjsis black-box against the workflow itself: it extracts the two publish jobs' actualrun:blocks fromrelease.ymland executes them in a sandbox — five fake platform VSIXes underartifacts/, a stubnpxscripted to time out the way the Marketplace did, a stubazfor the token mint.marketplaceAttemptsEveryPlatformWhenOneKeepsTimingOut/openvsxAttemptsEveryPlatformWhenOneKeepsTimingOut—darwin-x64fails every attempt. Asserts all five platforms are still attempted, the job exits non-zero, an error line namesdarwin-x64as the platform missing from the registry, every recorded invocation carries--skip-duplicate, and the job does not claim "Published 5".marketplaceRetriesATransientTimeoutUntilThePlatformPublishes/openvsxRetriesATransientTimeoutUntilThePlatformPublishes— the exact v0.31.0 scenario: one timeout, then success. Assertsdarwin-x64is attempted at least twice, all five platforms publish, the job exits 0 and reportsPublished 5 VSIX(es).expectedPublishCountMatchesTheBuildMatrix— each job's--expectedequals the number ofvsix_targetentries in the build matrix, so adding a sixth platform with a stale count fails in CI, not at release time.All four behavioural tests were written first and observed failing against the pre-fix workflow with the same attempt signature as the real incident:
{"darwin-arm64":1,"darwin-x64":1,"linux-arm64":0,"linux-x64":0,"win32-x64":0}.After the runner extraction, the full deployment gate still passes: 8 release workflow contract, 5 release publish contract, 2 deployment docs contract, 7 version stamping, 33 verifier proof, and 27 action contract checks. Coverage thresholds in
coverage-thresholds.jsonare untouched — no Rust code changed.🤖 Generated with Claude Code