Skip to content

fix(network-vmnet): install explicit subnet route, not just interface-scoped default#1882

Open
ryan106 wants to merge 1 commit into
apple:mainfrom
ryan106:fix/vmnet-explicit-subnet-route
Open

fix(network-vmnet): install explicit subnet route, not just interface-scoped default#1882
ryan106 wants to merge 1 commit into
apple:mainfrom
ryan106:fix/vmnet-explicit-subnet-route

Conversation

@ryan106

@ryan106 ryan106 commented Jul 2, 2026

Copy link
Copy Markdown

Type of Change

  • Bug fix
  • New feature
  • Breaking change
  • Documentation update

Motivation and Context

vmnet_network_create() only results in an interface-scoped default route for the container subnet (e.g. bridge100), never an explicit non-scoped route for the subnet itself (e.g. 192.168.64.0/24). Because of that, routing to containers is entirely dependent on which global default route currently wins on the host. Any other tool that installs or changes the system's default route (a VPN client, a Tailscale exit node, etc.) can shadow it and silently break all container connectivity — even though a specific /24 route for this subnet should normally win over any /0 default route under standard longest-prefix-match rules, since no such specific route exists to begin with.

Repro (full details in #1881):

  1. Run a container, confirm it's reachable (nc -zv <container-ip> <port> succeeds).
  2. Enable a Tailscale exit node (or introduce any other competing default route).
  3. route -n get <container-ip> now resolves via the wrong interface; connectivity breaks.
  4. Disabling the exit node, or even a full container system stop && container system start, does not restore connectivity — there was never a specific subnet route to fall back to, only the interface-scoped default.
  5. Manually installing an explicit subnet route (route add -net <subnet> -netmask <mask> -interface <gateway-ip>) does restore it immediately.

This PR installs that explicit route automatically when a network starts, using route(8) via Process — the same pattern this codebase already uses for pfctl in PacketFilter.swift.

Two things this does not claim to fix, so I want to be upfront about scope:

  • It's best-effort: failures to install the route are logged, not thrown, and don't prevent the network from otherwise starting.
  • It doesn't fully eliminate exposure to a tool that actively removes a route it doesn't own, rather than just adding a competing one. Testing this against Tailscale specifically, an exit node toggle removes this route outright (not just outranks it), and that looks like a Tailscale-side behavior — tracked at Exit node overrides existing connected RFC1918 routes tailscale/tailscale#18653, which already calls out apple/container's bridge100 interface by name. This change closes the more common case (a competing default route added without deleting others) and gives container a correct baseline instead of none, but a tool like Tailscale can still remove it afterward.

Testing

  • Tested locally

  • Added/updated tests

  • Added/updated docs

  • swift build --target ContainerNetworkVmnetServer compiles cleanly.

  • I manually validated the underlying route(8) invocation this PR automates, against the exact broken scenario from [Bug]: Container subnet routing breaks when a competing default route appears (e.g. VPN/exit-node) and doesn't self-heal #1881: with a container running and connectivity broken (exit node enabled, or freshly after container system restart with no subnet route present), running

    route add -net <subnet> -netmask <mask> -interface <gateway-ip>
    

    immediately restores route -n get <container-ip> to resolve via the correct bridge interface, and connectivity to the container succeeds.

  • I was not able to do a full signed end-to-end run of the patched container-network-vmnet plugin itself — I don't have a code-signing/entitlements setup for local testing of this daemon, the same constraint noted in Add DNS forwarding groundwork #1813's testing notes. I'd appreciate it if a maintainer with that setup could confirm the automated call site behaves the same as the manual route(8) invocation I validated.

Related

…-scoped default

vmnet_network_create() only results in an interface-scoped default
route for the container subnet, never an explicit non-scoped route.
Routing to containers is therefore entirely dependent on which global
default route currently wins on the host. Any other tool that installs
its own default route (a VPN client, a Tailscale exit node, etc.) can
shadow it and silently break all container connectivity, even though
this subnet's own route should normally win under standard
longest-prefix-match rules.

Install an explicit route ourselves via route(8), matching the existing
Process-based pattern this codebase already uses for pfctl in
PacketFilter.swift. Best-effort: failures are logged, not thrown, since
this is a resilience improvement rather than a hard requirement for the
network to otherwise start successfully. It also doesn't fully
eliminate exposure to tools that actively remove routes they don't own
(see tailscale/tailscale#18653) — only to the simpler and more common
case of a competing default route being added without deleting others.

Fixes apple#1881.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: Container subnet routing breaks when a competing default route appears (e.g. VPN/exit-node) and doesn't self-heal

2 participants