Improve release scripts reliability and UX by swasik · Pull Request #436 · scylladb/vector-store

swasik · 2026-05-05T13:16:17Z

Changes

This PR covers release scripts reliability and UX improvements only. Cross-compilation changes have been moved to #448.

Add unified scripts/release orchestrator: validates deps, QEMU setup, tag create/push, build, upload, summary
Semver validation regex for version argument
Colored step-counter output in all scripts (disabled in non-TTY)
GH_REPO env var for gh CLI scoping (avoids .gh-resolution file side effects from gh repo set-default)
SBOM files organized into SBOM/ directory
Temp file cleanup via EXIT trap + INT/TERM trap in run-with-release-toolchain
--init flag on docker run for proper signal forwarding
-t flag on docker run when stdout is a terminal
Quote $repo / $repo_dir in realpath and cd to handle paths with spaces
Docker push skipped for forks (only pushes from scylladb/vector-store)
Release notes Copilot prompt printed after completion
Updated docs/releasing.md documentation

Tests

Tested by creating a test release in the fork: https://github.com/swasik/vector-store/releases/tag/1.6.1-dev
Part of the script was also tested while creating the last official release: https://github.com/scylladb/vector-store/releases/tag/1.6.0

Known issues

The build-cloud-images script that is triggered when the version is created fails due to lack of authentication in GCP (RELENG-77).

Fixes: VECTOR-665

Copilot

Pull request overview

This PR introduces a top-level scripts/release flow to streamline publishing vector-store releases, while refreshing the supporting release scripts and documentation. It fits into the repo’s release tooling by combining tag creation, artifact building, GitHub release upload, and Docker publishing into a single operator-facing workflow.

Changes:

Added a new scripts/release orchestrator for dependency checks, tag push, build, and publish steps.
Updated build-release, upload-release, and run-with-release-toolchain for explicit version passing, improved terminal UX, and Docker runtime behavior.
Expanded docs/releasing.md to document the new workflow and standalone build/upload usage.

Reviewed changes

Copilot reviewed 5 out of 6 changed files in this pull request and generated 6 comments.

Show a summary per file

File	Description
`scripts/upload-release`	Adds colored step output, optional version override, GitHub release creation/upload, and Docker image/manifest publishing.
`scripts/run-with-release-toolchain`	Updates Docker invocation to use `--init` and conditionally allocate a TTY.
`scripts/release`	New end-to-end release entrypoint that validates tooling, tags the repo, builds artifacts, and uploads the release.
`scripts/build-release`	Adds colored step output, optional version override, cleanup traps, artifact summaries, and Docker/SBOM generation flow changes.
`docs/releasing.md`	Documents the new single-command release workflow and revised build/upload behavior.
`Cargo.lock`	Updates workspace package versions in the lockfile to a transient release-derived value.

Copilot

Pull request overview

Copilot reviewed 5 out of 6 changed files in this pull request and generated 2 comments.

Copilot

Pull request overview

Copilot reviewed 5 out of 6 changed files in this pull request and generated 4 comments.

Copilot

Pull request overview

Copilot reviewed 5 out of 5 changed files in this pull request and generated 11 comments.

ewienik · 2026-05-08T11:31:47Z

+Optionally, you can pass a version explicitly to override automatic detection
+from git tags:


Why do we need this override? We shouldn't allow creating same versions from different commits or build setup. Providing annotate tag before is a way to provide this. We should rather check if scylladb/vector-store has the same annotated tag as locally created.

The override exists because scripts/release orchestrator resolves the version from the annotated tag and passes it down to _build-release and _upload-release. This avoids each sub-script re-resolving it independently. The build-release script is not intended to be called manually in normal workflows — the entry point is scripts/release. Clarified this in docs/releasing.md and added underscores as the prefix of internal scripts to make it easier to spot.

The override exists because scripts/release orchestrator resolves the version from the annotated tag and passes it down to _build-release and _upload-release.

It seems not true. scripts/release has mandatory version arguments. It creates annotated tag, if it doesn't exist. If the tag exists it skip creating annotated tag, but it doesn't check if annotated tag is created in current commit. It seems error prone to me.

I think that annotated tag should be created by hand, manually. At first release script could resolve current version, resolve it, show to the user. Then version can be read by every script. It could be double check for correctness that the user want to build that version. In the future creating tag in CI should trigger all releasing process - so the version taken from annotated-tag should be de-factor a parameter to the scripts.

ewienik · 2026-05-08T11:38:23Z

+echo ""
+echo "${bold}${green}========================================"
+echo " BUILD COMPLETE: vector-store $vector_store_version"
+echo "========================================${reset}"
+echo ""


Do we really need this fancy stuff in release script?

As above - we do not need them but the summary provides immediate visibility of all produced artifacts and their paths, which is useful when debugging or verifying a release build.

ewienik · 2026-05-08T11:46:14Z

+step_header "Creating and pushing annotated tag $version"
+
+github_repo=$(git remote get-url origin | sed 's|.*github.com[:/]||;s|\.git$||')
+[[ -n $github_repo ]] || { echo "${red}error: could not determine GitHub repo from origin remote${reset}"; exit 1; }
+echo "  Releasing to: $github_repo"
+
+if git rev-parse "$version" > /dev/null 2>&1; then
+    echo "  Tag $version already exists, skipping creation."
+else
+    git tag -a "$version" -m "Release version $version"
+    echo "  ${green}Created annotated tag: $version${reset}"
+fi
+
+git push origin "$version"
+echo "  ${green}Pushed tag $version to origin ($github_repo).${reset}"


I wouldn't allow pushing anything to repository from release script. I prefer for developer to create a tag on specific commit, push manually this specific annotated tag and release script should read it only without writing anything to repo. It is less error prone in the future if someone run this script on the wrong commit. Doing this manually we can check if tag is on the correct commit without pushing it to the repository.

I'd prefer to keep tag creation and push in the script for these reasons:

The script validates semver format before creating the tag, preventing typos.

It skips creation if the tag already exists (so manual pre-creation works too).

Having it in one place reduces the chance of pushing a tag from the wrong commit — the script runs on the current HEAD which you can verify with git log.

The push is to origin (fork) — pushing to upstream for the official release is a separate manual step documented in releasing.md.

If the tag already exists on remote, the push is a no-op. If you still feel strongly, I could add a --no-tag flag to skip tag creation/push for workflows where tags are pre-created or the direct acknowledgement question.

I don't agree with providing version by hand. The clue with annotated tag is that it provides a more robust way to label version - it is in the git version history so it will stay. We shouldn't have easy way to build version not connected to the specific git commit.

I'd prefer to keep tag creation and push in the script for these reasons:

1. The script validates semver format before creating the tag, preventing typos.

You can still validate semver format of version taken from annotated tag - you can check if the tag in remote repository points to the same commit as in local repository, you can check if tag points to the HEAD, etc...

2. It skips creation if the tag already exists (so manual pre-creation works too).

The manual pre-creation should be the only way - it will restrict creation of release, but it will remove all maintenance problem with version provided by hand in the script. I think we should at first point the version in git history, then build release for this point - in other case we could end up with few releases with the same version (ie. concurrent release of the version by two developers).

3. Having it in one place reduces the chance of pushing a tag from the wrong commit — the script runs on the current HEAD which you can verify with `git log`.

And I think this could be error prone. Do you now that your HEAD points to the correct git commit? Without pre-creating annotated-tag you can mixed commits on local repo. The sync point is pushing annotated-tag to the git repo - after this operation you are sure, that only one tag exists and you can check if it points to the correct commit.

4. The push is to `origin` (fork) — pushing to `upstream` for the official release is a separate manual step documented in `releasing.md`.

It depends how you setup your local git repositry. I can have different remotes, origin could be scylldb, my fork could be named differently. Our scripts shouldn't restrict local git repository layout.

If the tag already exists on remote, the push is a no-op. If you still feel strongly, I could add a --no-tag flag to skip tag creation/push for workflows where tags are pre-created or the direct acknowledgement question.

There are problems with concurrently running release, creating tag, etc. We should limit possible combinations - one way is to restrict that we are building version taken from git history and annotated-tag ought to be created before releasing process, manually.

- Add unified release script that orchestrates tag, build, and upload - Always pass version explicitly to build-release and upload-release - Add --init to docker run for proper signal forwarding (Ctrl+C works) - Add -t flag to docker run when stdout is a terminal - Derive release target repo from origin remote - Set gh repo default before upload to support fork workflows - Add colored output to all release scripts (disabled in non-TTY) - Print Copilot prompt for generating release notes after completion - Update releasing.md documentation

ewienik · 2026-05-28T13:35:01Z

+# Override step_header for top-level release orchestrator style.
+step_header() {
+    _step=$((_step + 1))
+    echo ""
+    echo "${bold}${cyan}==> [${_step}/${total_steps}] $1${reset}"
+    echo ""
+}


Do we really need additional override?

ewienik · 2026-05-28T13:39:35Z

+    docker_args+=(-v "$user_cargo_home/registry:$docker_cargo_home/registry")
+fi
+
+[[ -t 1 ]] && docker_args+=(-t)


do we need this -t?

swasik requested a review from Copilot May 5, 2026 13:17

Copilot started reviewing on behalf of swasik May 5, 2026 13:18 View session

Copilot AI reviewed May 5, 2026

View reviewed changes

Comment thread scripts/_build-release

Comment thread scripts/upload-release

Comment thread scripts/release Outdated

Comment thread docs/releasing.md Outdated

Comment thread docs/releasing.md Outdated

Comment thread scripts/upload-release Outdated

swasik force-pushed the update-release-scripts branch from 5dee842 to f649ed4 Compare May 5, 2026 13:36

swasik requested a review from Copilot May 5, 2026 13:49

Copilot started reviewing on behalf of swasik May 5, 2026 13:50 View session

github-actions Bot added P2 area/vector labels May 5, 2026

Copilot AI reviewed May 5, 2026

View reviewed changes

Comment thread scripts/release

Comment thread scripts/release Outdated

swasik force-pushed the update-release-scripts branch 2 times, most recently from 0ff55b7 to fe99bf2 Compare May 5, 2026 14:11

swasik requested a review from Copilot May 5, 2026 14:12

Copilot started reviewing on behalf of swasik May 5, 2026 14:12 View session

Copilot AI reviewed May 5, 2026

View reviewed changes

Comment thread scripts/release Outdated

Comment thread scripts/upload-release

Comment thread scripts/build-release Outdated

Comment thread docs/releasing.md Outdated

swasik force-pushed the update-release-scripts branch 8 times, most recently from 2b1e260 to 2656177 Compare May 6, 2026 07:43

swasik requested a review from Copilot May 6, 2026 07:46

Copilot started reviewing on behalf of swasik May 6, 2026 07:47 View session

Copilot AI reviewed May 6, 2026

View reviewed changes

swasik force-pushed the update-release-scripts branch from 2656177 to 062105b Compare May 6, 2026 08:24

swasik requested a review from ewienik May 6, 2026 08:39

swasik changed the title ~~fix: improve release scripts reliability and UX~~ Improve release scripts reliability and UX May 6, 2026

ewienik requested changes May 8, 2026

View reviewed changes

swasik mentioned this pull request May 12, 2026

scripts: replace QEMU emulation with cross-compilation for arm64 builds #448

Open

swasik force-pushed the update-release-scripts branch from 062105b to eccf05c Compare May 12, 2026 14:33

swasik force-pushed the update-release-scripts branch from eccf05c to 6844854 Compare May 27, 2026 11:47

swasik requested a review from ewienik May 27, 2026 13:12

ewienik reviewed May 28, 2026

View reviewed changes

		Optionally, you can pass a version explicitly to override automatic detection
		from git tags:

Conversation

swasik commented May 5, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Changes

Tests

Known issues

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Reviewed changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull request overview

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

ewienik May 28, 2026 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Choose a reason for hiding this comment

Uh oh!

Reviewers

Assignees

Labels

swasik commented May 5, 2026 •

edited

Loading

ewienik May 28, 2026 •

edited

Loading