P5 privileged networking and full data plane#72
Merged
Conversation
…#37) Deliver the cross-host data plane so macvz-kubelet runs as the operator's user (required because apple/container refuses to run as root) while privileged network ops execute via a root helper. Privileged network helper (#38): - pkg/network/privhelper: root daemon over a unix socket running a fixed command allowlist (pfctl/sysctl/route/ifconfig/wg/wireguard-go) for the user-run kubelet; protocol versioning, per-request arg Policy (#41), launchd install. - cmd/macvz-netd: the helper daemon (serve/install/status). - podnet.WithHelperSocket / wireguard.WithHelperSocket route privileged commands through the helper; config.privilegedHelperSocket wires it in. ClusterIP service routing (#37): - pkg/network/podnet: pf rdr rules mapping ClusterIP:port -> local micro-VM (VMIP) or remote Pod IP (via mesh), round-robin pool, in the macvz/pods anchor. - pkg/network/svcroute: Service+EndpointSlice controller computing those rules. - Cluster DNS injection (node.clusterDNS/clusterDomain -> container --dns) so micro-VMs resolve Service names. Startup ordering: bring up mesh + podNetwork BEFORE node registration so the API connection forms over final routing. Validated on two physical Macs through the helper: WireGuard mesh + handshake, podNetwork, ClusterIP routing, and e2e node-reg/pod-lifecycle/logs/exec/ port-forward all pass. See docs/MULTI_NODE_TEST_REPORT_2026-06-19.md for the run and the remaining env-specific remote-node API flap (not a MacVz bug). Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Root cause of the macvz-b NotReady flap AND a real production-safety bug: starting an apple/container micro-VM installs a `default -> <vmnet bridge>` route that steals the host's IPv4 default route, breaking ALL host outbound traffic (observed taking down an unrelated frpc tunnel + GitHub runner on the test host, and severing the remote kubelet's API connection so the node flapped NotReady). - podnet.Router removes the vmnet default route (`route -q -n delete -inet default -interface <vmnet>`) at Start and on every Attach, tolerating "not in table"/"not found"; Pod reachability uses explicit Pod/mesh routes + pf, never a host default route. - privhelper.Policy.validateRoute narrowly permits exactly this: delete (never add) the IPv4 default route, and only on the managed vmnet interface. - Report + docs upgrade this to a CRITICAL finding: do not run MacVz on a host with other networked services until this lands. Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
chimerakang
force-pushed
the
feat/p5-privileged-networking
branch
from
1a659b7 to
c064e30
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Verification