fix(docker): ready-socket race fallback + cluster-gated gVisor --host-uds#272
Open
sumansaurabh-slice wants to merge 1 commit into
Open
fix(docker): ready-socket race fallback + cluster-gated gVisor --host-uds#272sumansaurabh-slice wants to merge 1 commit into
sumansaurabh-slice wants to merge 1 commit into
Conversation
Deploying with
|
| Status | Name | Latest Commit | Preview URL | Updated (UTC) |
|---|---|---|---|---|
| ✅ Deployment successful! View logs |
microvm | cd01a29 | Commit Preview URL Branch Preview URL |
Jun 30 2026, 02:24 PM |
Codecov Report❌ Patch coverage is
📢 Thoughts on this report? Let us know! |
…-uds Two follow-ups to the #271 push-readiness socket. 1. Race robustness (pkg/docker/client.go). waitForToolboxReady took the *first* race result including errors, so a socket-path error (e.g. an in-container process tripping maxInvalidReadyAttempts, or any listener error) won the race and failed an otherwise-healthy create. It now takes the first path that SUCCEEDS, making the health poll a true backstop for a silent OR erroring socket; the health error is surfaced when both fail. Regression test TestCreate_SocketErrorFallsBackToHealth fails pre-fix, passes post-fix. 2. gVisor --host-uds=open is now scoped to cluster nodes. #271 added it to every --with-gvisor install, including standalone hosts where the cluster-only ready socket never runs — relaxing gVisor host-UDS isolation fleet-wide with zero benefit there. install.sh runs before the node is in a cluster and cannot self-gate (SB_ENABLE_CLUSTER is set later by cluster-init/join), so the relaxation moves to ensure_gvisor_host_uds in cluster-init.sh and cluster-join.sh — the one place that knows the node is a cluster node. It appends --host-uds=open to the registered runsc runtimeArgs idempotently (no-op without gVisor or when already present; never fails the join). install.sh reverts to plain runsc, and the speculative Terraform with_gvisor_host_uds plumbing is dropped (the Terraform bootstrap always runs cluster-init/join, so cluster gVisor nodes get it automatically). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
d27485c to
cd01a29
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Two follow-ups to the #271 Docker push-readiness socket — a real race-fallback
bug, plus correctly scoping the gVisor isolation relaxation #271 introduced.
pkg/docker/client.go).waitForToolboxReadytook thefirst race result including errors, so a socket-path error — e.g. an
in-container process tripping
maxInvalidReadyAttempts, or any listenererror — won the race and failed an otherwise-healthy create. It now takes the
first path that succeeds, so the health poll is a true backstop for a
silent or erroring socket; the health error (the actual readiness
probe) is surfaced when both fail.
--host-uds=openis now scoped to cluster nodes. feat(docker): unix-socket push readiness for cluster creates #271 added it toevery
--with-gvisorinstall, including standalone / non-cluster hostswhere the cluster-only ready socket never runs (
DockerReadySocketEffective() = enabled && EnableCluster) — relaxing gVisor host-UDS isolation fleet-widewith zero benefit there.
install.shruns before the node is in acluster and can't self-gate (
SB_ENABLE_CLUSTERis set later bycluster-init.sh/cluster-join.sh), so the relaxation moves into thosescripts via
ensure_gvisor_host_uds— the one place that authoritativelyknows the host is a cluster node.
install.shreverts to plainrunsc.Why cluster-init/join, not an install.sh flag
The only signal that "this host is in a cluster" is
SB_ENABLE_CLUSTER=true,which is written by
cluster-init.sh/cluster-join.sh, afterinstall.shhas already registered the runsc runtime. The Terraform bootstrap always runs
install.sh→ thencluster-init/cluster-join, so every cluster node passesthrough one of those scripts.
ensure_gvisor_host_uds:--host-uds=opento the registeredrunscruntimeArgs,preserving
pathand any existing args (e.g.--platform=systrap);idempotent — it checks for the arg semantically (not by byte-diffing
reformatted JSON), so a re-run never bounces Docker;
warning and degrades the socket to health polling (UC-97) rather than failing
the cluster join.
Net effect: host-UDS isolation is relaxed iff (cluster mode) and
(gVisor present) — exactly where the readiness socket runs. Standalone gVisor
hosts keep runsc at the safe
--host-uds=none. No new Terraform variable isneeded (the bootstrap already runs cluster-init/join on every node).
Sandbox boot impact
Touches the create path (
waitForToolboxReady), but adds no new DB query,HTTP round-trip, file I/O, or lock acquisition.
(socket push, or socket-silent + health). Same latency as before.
errors, the create previously failed instantly; it now waits for the health
poll to resolve (success or the existing
toolboxWaitTimeout, default 30s —the same bound the health poll already used). This is the fix: a
socket-channel error must not fail a healthy sandbox.
cluster-init.sh/cluster-join.shchanges are provisioning-time only, notboot path.
Idempotency
retry/cleanup semantics; it only changes which race result is authoritative.
ensure_gvisor_host_udsis idempotent by a semantic presence check(
--host-uds=openalready inruntimeArgs→ no write, no docker restart), sore-running cluster-init/join is safe and won't kick running containers.
Failure-path consistency
N/A — no multi-step caddy+store write path changed. The only added create-path
teardown is the ready-listener
Close(), already covered by the existingdeferon every create error path.ensure_gvisor_host_udsleavesdaemon.jsonuntouched on any error.Cluster-correctness impact
ensure_gvisor_host_udslives in the cluster bootstrap scripts but is a hostdocker-config concern, not the Raft/gossip/placement path — no FSM, replication,
owner-watcher, or capacity-lease behavior changes. It runs once at init/join,
before sandboxd is (re)started into cluster mode. No split-brain / replay /
leader-change surface. Single-node (
EnableCluster=false) is unaffected:install.sh leaves runsc at
--host-uds=noneand the socket path stays off.L4 / host-port-pool changes
N/A — neither touched.
Test plan
TestCreate_SocketErrorFallsBackToHealth: spams invalid frames to tripthe listener's error path while the container is healthy-but-slow, asserts the
create still succeeds via
health. Verified it fails against the pre-fix(feat(docker): unix-socket push readiness for cluster creates #271)
client.gowithready socket: too many invalid attempts, andpasses post-fix.
pkg/dockersocket suite (incl. the new test) green on real Linux under-race(-countrepeated); deterministic, no flake.go build ./...,gofmt, fullTestCreate*run clean on darwin.ensure_gvisor_host_udslogic exercised against sixdaemon.jsonfixtures(no runsc / runsc-no-args / runsc-with-other-arg / already-has-host-uds /
no-runtimes-key / extra-top-keys) on both the jq and python3 paths —
identical, correct output; path + existing args preserved; idempotent no-op
when already present or gVisor absent.
bash -nclean on all three scripts.🤖 Generated with Claude Code