Skip to content

fix(microvm): name both causes when a command health check cannot reach the guest - #203

Merged
Shine-neko merged 1 commit into
mainfrom
fix/microvm-vsock-diagnostic
Jul 25, 2026
Merged

fix(microvm): name both causes when a command health check cannot reach the guest#203
Shine-neko merged 1 commit into
mainfrom
fix/microvm-vsock-diagnostic

Conversation

@Shine-neko

Copy link
Copy Markdown
Contributor

The vsock device backing command health checks is attached at boot, and only when the deployment already declares such a check. A command check added to a running deployment therefore cannot reach its VM at all — neither hypervisor exposes a hot-plug path for vhost-vsock.

The probe reported that failure as "cannot reach ring-agent in the guest — install it in the image", which is the wrong remedy for this cause and sends the operator debugging a perfectly good image.

The message now names both possible causes and, crucially, states the boot-time constraint: that a restart is what provisions the device. That is the actionable part, and no amount of guest-side debugging would reveal it.

On the detection. The host-side vsock socket exists only when the VM was booted with the device, so it looks like a reliable discriminator. It isn't: a crashed VMM can leave the socket behind until the reconciler reaps it, and a live VM can have its socket unlinked while keeping the device — both states the codebase already handles elsewhere. So the socket check is used only to order the two causes, putting the more likely one first, and never to assert one and hide the other. Both remedies survive either way.

Whether a VM ever restarts on its own depends on the check's on_failure: restart heals itself once the failure threshold is reached, but alert and stop never reboot it, so the check stays red until the operator restarts the deployment. Two code comments claimed the VM "eventually restarts" unconditionally; they now say what actually happens.

Docs: documented the limitation on the Cloud Hypervisor page and in the runtime comparison, which covers Firecracker too.

Tests

783 unit tests pass (4 new), cargo fmt and cargo clippy --all-targets clean. The new tests pin the contract: both causes always named, both remedies always present, and the hint only reordering them. Positions are unwrapped before comparison so a vanished phrase fails the test rather than passing on Option ordering.

The probe path itself needs a running VM and is only exercised by the e2e suites, which do not run in CI.

@Shine-neko
Shine-neko merged commit b39f86f into main Jul 25, 2026
5 checks passed
@Shine-neko
Shine-neko deleted the fix/microvm-vsock-diagnostic branch July 25, 2026 08:42
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant