feat: host-level data path performance tuning - #2
Merged
Conversation
Add spec.host on EdgeNode to tune the VPS as a machine, independent of the proxy. Features: - spec.host.kernelMaxSocketBufferBytes (default 25MB): single knob wired to net.core.rmem_max/wmem_max and to SO_RCVBUF/SO_SNDBUF on the Envoy UDP listeners (UDP does not autotune; TCP is left to autotune). - spec.host.disableNicOffloads (default false): turn GRO/GSO off on the underlay NIC (auto-detected via netlink route lookup), applied natively over ethtool-netlink as a netdev section of the relay document. Changes: - render: drop prefer_gro, set enable_reuse_port on every listener. - agentconfig: optional netdev section in the desired-state document. - internal/netdev: native ethtool applier with an IO seam and a root-gated test. - rename the boot oneshot wg-relay.service to tunnel-boot.service (the WireGuard interface wg-relay is unchanged).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
New
spec.hostblock on the EdgeNode to tune the VPS itself (kernel and NIC), separate from the Envoy proxy config. It comes from debugging a tunnel-in-tunnel setup where throughput was poor and Envoy was dropping UDP datagrams.What you get
host.kernelMaxSocketBufferBytes(default 25MB): one value that raises the kernel socket buffer ceiling (net.core.rmem_max/wmem_max) and setsSO_RCVBUF/SO_SNDBUFon the UDP listeners. UDP needs this set explicitly; TCP keeps autotuning on its own. Useful for high-throughput or encapsulated UDP that otherwise loses packets under load.host.disableNicOffloads(default false): turns GRO/GSO off on the public NIC. Leave it off for normal traffic, where the offloads help. Turn it on for encapsulated UDP (tunnel-in-tunnel), where GRO merges packets it shouldn't and corrupts datagram boundaries, which is what was crashing Envoy. The interface is detected automatically.Also in here
prefer_grofrom the Envoy UDP listeners and setenable_reuse_porton all of them, so they spread across worker threads.wg-relay.servicetotunnel-boot.service, since it now reapplies the whole node config on boot, not just WireGuard. The WireGuard interface is still calledwg-relay.ethtool, the same way WireGuard and nftables are already handled, so nothing extra needs to be installed on the VPS.Adds one dependency,
github.com/safchain/ethtool(Apache-2.0).