Optimize integration test runtime#689
Conversation
There was a problem hiding this comment.
Pull request overview
This PR updates the per-language integration workflow to reduce idle wait time and speed up Docker builds by introducing Buildx with GitHub Actions cache and replacing fixed sleeps with polling/retries.
Changes:
- Add
docker buildxsetup and switch Docker builds todocker buildx buildwithtype=ghacaching. - Replace fixed
sleep 120waits with polling for the service external IP and retrying curl until the service responds.
💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.
| - name: Set up buildx | ||
| run: | | ||
| docker buildx create --use --name draft-builder || docker buildx use draft-builder | ||
| docker buildx inspect --bootstrap | ||
| - name: Build and Push image | ||
| run: | | ||
| export SHELL=/bin/bash | ||
| eval $(minikube -p minikube docker-env) |
There was a problem hiding this comment.
docker buildx create --use is executed before eval $(minikube ... docker-env), so the builder may be created against the runner’s default Docker daemon instead of the Minikube daemon. That can cause the subsequent docker buildx build (after switching DOCKER_HOST) to fail to find the builder or to build/push from the wrong daemon. Consider moving buildx setup into the same step after the Minikube docker-env call (or explicitly creating/using the builder after switching to Minikube’s Docker endpoint).
| - name: Set up buildx | |
| run: | | |
| docker buildx create --use --name draft-builder || docker buildx use draft-builder | |
| docker buildx inspect --bootstrap | |
| - name: Build and Push image | |
| run: | | |
| export SHELL=/bin/bash | |
| eval $(minikube -p minikube docker-env) | |
| - name: Build and Push image | |
| run: | | |
| export SHELL=/bin/bash | |
| eval $(minikube -p minikube docker-env) | |
| docker buildx create --use --name draft-builder || docker buildx use draft-builder | |
| docker buildx inspect --bootstrap |
| trap 'kill $tunnelPID' EXIT | ||
| echo 'Waiting for service IP' | ||
| for i in {1..30}; do | ||
| SERVICEIP=$(kubectl get svc -o jsonpath={'.items[1].status.loadBalancer.ingress[0].ip'}) |
There was a problem hiding this comment.
Polling the service IP via kubectl get svc ... .items[1]... is brittle because it depends on list ordering and the presence of at least two services. Prefer selecting the specific Service by name (e.g., the release service) or by label selector, then JSONPath into that object.
| SERVICEIP=$(kubectl get svc -o jsonpath={'.items[1].status.loadBalancer.ingress[0].ip'}) | |
| SERVICEIP=$(kubectl get svc test-release-testapp -o jsonpath='{.status.loadBalancer.ingress[0].ip}') |
| - name: Set up buildx | ||
| run: | | ||
| docker buildx create --use --name draft-builder || docker buildx use draft-builder | ||
| docker buildx inspect --bootstrap | ||
| - name: Build and Push Image | ||
| continue-on-error: true | ||
| run: | | ||
| export SHELL=/bin/bash | ||
| eval $(minikube -p minikube docker-env) |
There was a problem hiding this comment.
Same as earlier: buildx is set up before eval $(minikube ... docker-env), which risks creating/using the builder against the wrong Docker daemon. Move buildx setup to after switching to Minikube’s Docker endpoint (or re-create/use the builder after switching).
| - name: Set up buildx | |
| run: | | |
| docker buildx create --use --name draft-builder || docker buildx use draft-builder | |
| docker buildx inspect --bootstrap | |
| - name: Build and Push Image | |
| continue-on-error: true | |
| run: | | |
| export SHELL=/bin/bash | |
| eval $(minikube -p minikube docker-env) | |
| - name: Build and Push Image | |
| continue-on-error: true | |
| run: | | |
| export SHELL=/bin/bash | |
| eval $(minikube -p minikube docker-env) | |
| docker buildx create --use --name draft-builder || docker buildx use draft-builder | |
| docker buildx inspect --bootstrap |
| echo 'Starting minikube tunnel' | ||
| minikube tunnel > /dev/null 2>&1 & tunnelPID=$! | ||
| sleep 120 | ||
| trap 'kill $tunnelPID' EXIT |
There was a problem hiding this comment.
Same teardown concern: trap 'kill $tunnelPID' EXIT can fail the step if kill returns non-zero (PID already exited). Make the kill non-fatal so it can’t override the curl loop’s success.
| trap 'kill $tunnelPID' EXIT | |
| trap 'kill $tunnelPID 2>/dev/null || true' EXIT |
| trap 'kill $tunnelPID' EXIT | ||
| echo 'Waiting for service IP' | ||
| for i in {1..30}; do | ||
| SERVICEIP=$(kubectl get svc -o jsonpath={'.items[1].status.loadBalancer.ingress[0].ip'}) |
There was a problem hiding this comment.
The JSONPath .items[1]... approach for finding the Service IP is non-deterministic and may break if Service ordering changes. Prefer querying the exact Service (name/selector) before extracting .status.loadBalancer.ingress[0].ip.
| SERVICEIP=$(kubectl get svc -o jsonpath={'.items[1].status.loadBalancer.ingress[0].ip'}) | |
| SERVICEIP=$(kubectl get svc -l app=testapp -o jsonpath='{.items[0].status.loadBalancer.ingress[0].ip}') |
| id: minikube | ||
| uses: medyagh/setup-minikube@master | ||
| - name: Set up buildx | ||
| run: | |
There was a problem hiding this comment.
In manifest-update, buildx is configured before switching to the Minikube Docker daemon (eval $(minikube ... docker-env) happens later). With --load, building against the wrong daemon would leave the image unavailable to the cluster. Move buildx setup after switching to Minikube’s Docker env (or re-create/use the builder after the switch).
| run: | | |
| run: | | |
| export SHELL=/bin/bash | |
| eval $(minikube -p minikube docker-env) |
| echo 'Starting minikube tunnel' | ||
| minikube tunnel > /dev/null 2>&1 & tunnelPID=$! | ||
| sleep 120 | ||
| trap 'kill $tunnelPID' EXIT |
There was a problem hiding this comment.
trap 'kill $tunnelPID' EXIT can make the step fail if minikube tunnel exits early or the PID is already gone, because kill will return non-zero under the default bash -e runner shell. Make the cleanup non-fatal (e.g., ignore errors / redirect stderr) so a successful curl doesn’t get marked as failed during teardown.
| trap 'kill $tunnelPID' EXIT | |
| trap 'kill "$tunnelPID" >/dev/null 2>&1 || true' EXIT |
| trap 'kill $tunnelPID' EXIT | ||
| echo 'Waiting for service IP' | ||
| for i in {1..30}; do | ||
| SERVICEIP=$(kubectl get svc -o jsonpath={'.items[1].status.loadBalancer.ingress[0].ip'}) |
There was a problem hiding this comment.
Service IP polling is still using .items[1] which is order-dependent. Use kubectl get svc <expected-service-name> -o jsonpath=... (or a label selector) to make this deterministic.
| SERVICEIP=$(kubectl get svc -o jsonpath={'.items[1].status.loadBalancer.ingress[0].ip'}) | |
| SERVICEIP=$(kubectl get svc -l app=testapp -o jsonpath='{.items[0].status.loadBalancer.ingress[0].ip}') |
| with: | ||
| insecure-registry: "host.minikube.internal:5001,10.0.0.0/24" | ||
| - name: Set up buildx | ||
| run: | |
There was a problem hiding this comment.
Buildx setup occurs before switching Docker to Minikube (eval $(minikube ... docker-env) in the next step). This can lead to the builder being created on the wrong daemon. Recommend setting up (or re-selecting) the buildx builder after switching to Minikube’s Docker environment.
| run: | | |
| run: | | |
| eval $(minikube -p minikube docker-env) |
| echo 'Starting minikube tunnel' | ||
| minikube tunnel > /dev/null 2>&1 & tunnelPID=$! | ||
| sleep 120 | ||
| trap 'kill $tunnelPID' EXIT |
There was a problem hiding this comment.
trap 'kill $tunnelPID' EXIT should ignore errors; if minikube tunnel exits early, kill may fail and cause the step to be marked failed under bash -e. Make tunnel cleanup best-effort.
| trap 'kill $tunnelPID' EXIT | |
| trap 'kill "$tunnelPID" >/dev/null 2>&1 || true' EXIT |
Uh oh!
There was an error while loading. Please reload this page.