Skip to content
Merged
Show file tree
Hide file tree
Changes from all commits
Commits
File filter

Filter by extension

Filter by extension

Conversations
Failed to load comments.
Loading
Jump to
Jump to file
Failed to load files.
Loading
Diff view
Diff view
3 changes: 2 additions & 1 deletion documentation/concepts/runtimes.md
Original file line number Diff line number Diff line change
Expand Up @@ -24,7 +24,8 @@ Enable just the runtimes a host actually runs: Docker-only, Podman-only, contain
| Boot time | ~1 s | ~1 s | ~1 s | ~3–5 s (cloud-init, kernel boot) | ~1 s (minimal microVM) |
| Memory overhead per workload | ~10 MB | ~10 MB | ~10 MB | ~80–150 MB (kernel + guest userland) | ~5–50 MB (minimal device model) |
| Image format | Docker image (`nginx:1.25`) | Docker/OCI image | OCI image (`nginx:1.25`) | Raw disk image on the host filesystem | Kernel (`vmlinux`) + ext4 rootfs on the host filesystem |
| Networking | Per-namespace bridge, DNS aliases | Per-namespace bridge, DNS aliases | CNI (bridge + host-local IPAM) | Per-VM /30 subnet, host-port forwarding via `socat` | Per-VM /30 subnet, host-port forwarding via `socat` (Ring-owned host TAP) |
| Networking | Per-namespace bridge, DNS aliases | Per-namespace bridge, DNS aliases | CNI (bridge + host-local IPAM) | Per-VM /30 subnet, host-port forwarding via `socat`, Ring-managed outbound NAT | Per-VM /30 subnet, host-port forwarding via `socat` (Ring-owned host TAP), Ring-managed outbound NAT |
| Host-memory admission | N/A (cgroup limits) | N/A (cgroup limits) | ✓ before create | ✓ before VM boot | ✓ before rootfs copy and VM boot |
| Crash detection | ✓ event-driven (sub-second) | ✓ reconcile-based (per scheduler tick) | ✓ reconcile-based (per scheduler tick) | ✓ reconcile-based (per scheduler tick) | ✓ reconcile-based (per scheduler tick) |
| `command` health checks | `docker exec` | `podman exec` (same API) | `Tasks.Exec` (gRPC) | In-guest `ring-agent` over AF_VSOCK | In-guest `ring-agent` over vsock (host Unix socket) |
| `kind: job` | Exit code visible | Exit code visible | Exit code visible | Clean shutdown = success (no exit code from host) | Clean shutdown (guest reboot) = success (no exit code from host) |
Expand Down
10 changes: 9 additions & 1 deletion documentation/runtimes/cloud-hypervisor.md
Original file line number Diff line number Diff line change
Expand Up @@ -196,6 +196,14 @@ If `published` is already taken on the host, Ring **refuses to start the VM** an

Both `tcp` (default) and `udp` are supported via the port's `protocol` field; `socat` uses a `UDP4-LISTEN`/`UDP4` pair for UDP.

### Outbound access

The per-VM /30 lets the guest reach the host, but reaching *external* networks (package installs, `git clone`, …) needs the host to masquerade the guest subnet. Ring installs that itself — a single idempotent rule set covering `10.42.0.0/16`, so it is set up once and re-asserted for free on later boots. The operator never touches `iptables`.

This requires `CAP_NET_ADMIN` on the Ring binary (`setcap cap_net_admin+ep $(command -v ring)`). Without it the rules are skipped: the VM still boots and serves inbound traffic, but outbound calls fail to resolve. The failure is logged at warn on the first VM start.

Note that the guest network is only allocated when the deployment publishes at least one port; a VM with no `ports:` has no network at all.

## Health checks

`tcp`, `http`, `command` all work. `tcp` and `http` probe from the host against the guest IP (no agent required). `command` goes through the in-guest `ring-agent` over AF_VSOCK port 2375, so install the agent in the guest image. If the agent isn't reachable (missing from the image, or not started yet), the `command` probe fails with an explicit message naming ring-agent rather than a bare connection error.
Expand Down Expand Up @@ -247,7 +255,7 @@ This is the canonical parity table. Other pages link here rather than restate it
| `labels:` | **Stored and usable.** Not applied to the VM (no container-label equivalent), but persisted as Ring metadata (shown in `inspect` and filterable with `ring deployment list --label key=value`), same as Docker |
| `resources.limits.cpu` | Honored as **allocation, not cap**: rounded down to whole vCPU, floor 1 (`"500m"` → 1 vCPU) |
| `resources.limits.memory` | Honored as **allocation, not cap**: VM RAM size, minimum 128 MiB |
| `resources.requests.*` | Ignored (VM is sized from `limits`); a **warning event** is recorded at create so it isn't silent |
| `resources.requests.*` | Not used to **size** the VM (that comes from `limits`); a **warning event** is recorded at create so it isn't silent. `requests.memory` is still read for host-memory admission control, where it takes precedence over `limits.memory` |
| `config.image_pull_policy` / `server` / `username` / `password` | Ignored (no image to pull); a **warning event** is recorded at create |
| `config.user` (privileged / id / group) | Ignored; a **warning event** is recorded at create |
| `kind: job` | **Supported, coarser signal.** Clean guest shutdown → `completed`. CH does not expose the workload's exit code, so Ring sees VM state only. |
Expand Down
2 changes: 2 additions & 0 deletions documentation/runtimes/firecracker.md
Original file line number Diff line number Diff line change
Expand Up @@ -55,6 +55,8 @@ ring apply -f app.yaml

What Ring does: copies the rootfs per instance, spawns a `firecracker` process, drives its REST API to set the kernel / rootfs / network / machine config, then boots. Networking is a Ring-owned TAP (a /30 subnet per VM) with `socat` host-port forwarding; outbound NAT lets guests reach external networks.

Before any of that, the deployment's memory ask is admitted against the host's available memory. A microVM reserves its whole RAM at boot, so an over-ask would otherwise die on an opaque allocation failure — and only after a full rootfs copy. The check reads `resources.requests.memory`, falling back to `resources.limits.memory`; a deployment declaring neither is not gated. Refusal is terminal (`insufficient_resources`) rather than a crash loop, since a retry won't free memory.

## Logs

The guest serial console (kernel, init, and anything the workload writes to the console) is persisted per instance and readable with the standard commands, same as every other runtime:
Expand Down
21 changes: 15 additions & 6 deletions src/runtime/cloud_hypervisor/lifecycle.rs
Original file line number Diff line number Diff line change
Expand Up @@ -639,12 +639,21 @@ impl CloudHypervisorLifecycle {
}

let net_config = match &net_alloc {
Some(n) => NetConfig {
tap: Some(n.tap_name.clone()),
ip: Some(n.host_ip.clone()),
mask: Some(n.netmask.clone()),
mac: Some(n.mac.clone()),
},
Some(n) => {
// The tap CH is about to create lets the guest reach the host;
// outbound NAT is what lets it reach the Internet (apt, git
// clone, …). Same call Firecracker makes after creating its own
// tap: idempotent, scoped to the whole guest supernet, so it is
// a no-op after the first VM on this host. Without it a CH guest
// resolved nothing, while an equivalent Firecracker guest did.
crate::hypervisor::host_nat::ensure_outbound_nat();
NetConfig {
tap: Some(n.tap_name.clone()),
ip: Some(n.host_ip.clone()),
mask: Some(n.netmask.clone()),
mac: Some(n.mac.clone()),
}
}
None => NetConfig {
tap: None,
ip: None,
Expand Down
7 changes: 7 additions & 0 deletions src/runtime/firecracker/lifecycle.rs
Original file line number Diff line number Diff line change
Expand Up @@ -393,6 +393,13 @@ impl FirecrackerLifecycle {
)));
}

// Admission control before the rootfs copy and the VM boot. A microVM
// reserves its whole memory at boot, so an over-ask dies on an opaque
// allocation failure — and here it would do so only *after* copying a
// full rootfs image. Same check Cloud Hypervisor already ran; without it
// Firecracker had no memory gate at all.
crate::hypervisor::resources::check_host_memory(deployment)?;

std::fs::create_dir_all(&self.config.socket_dir).map_err(|e| {
RuntimeError::VmStartFailed(format!(
"could not create socket_dir '{}': {}",
Expand Down