Skip to content

chore(vlab): add configurable server VM mtu and use 9036 for HLAB#1556

Merged
Frostman merged 1 commit intomasterfrom
pau/vlab-server-max-mtu
Apr 13, 2026
Merged

chore(vlab): add configurable server VM mtu and use 9036 for HLAB#1556
Frostman merged 1 commit intomasterfrom
pau/vlab-server-max-mtu

Conversation

@pau-hedgehog
Copy link
Copy Markdown
Contributor

@pau-hedgehog pau-hedgehog commented Mar 13, 2026

Before this PR

There is a --interface-mtu flag on setup-vpcs that lets users manually set an MTU value. Its default is 0, meaning "don't set any MTU."

Before this PR, when setup-vpcs ran with MTU=0 (the default), two things could happen depending on whether DNS/NTP servers were configured:

Without DNS/NTP: no DHCP options object created. DHCP hands out IP addresses but advertises no MTU. Servers use their OS default (1500).

With DNS/NTP (e.g. --dns-servers 8.8.8.8): a DHCP options object is created to carry the DNS servers. The VPC API sees MTU=0 in that object and auto-fills it to 9036. DHCP now advertises 9036 jumbo MTU to all servers, including SONiC VS where jumbo frames crash the VS. There was no protection against this.

Server interfaces themselves were never configured with any MTU either. They just used the OS default (1500). So even if DHCP advertised 9036, the server interfaces weren't set up for it.

Bottom line: MTU was not actively managed. It was either not advertised, or silently wrong (9036 advertised via DHCP on VS environments that can't handle it).

What this PR changes

Auto-configure MTU based on environment

When --interface-mtu is not explicitly set (defaults to 0), the code now detects what kind of switches are running and picks the right value:

Non-VS (HLAB/production): sets MTU to 9036. Jumbo frames work here, this is the correct value.

SONiC VS with DNS/NTP configured: sets MTU to 1500. This prevents the API from auto-filling 9036 into the DHCP options object that must exist to carry the DNS/NTP servers.

SONiC VS without DNS/NTP: leaves MTU at 0. No DHCP options object is created, so the API never fills in 9036. No MTU advertised. Servers use default 1500.

Configure server interfaces

The MTU value is now passed to hhnet.sh via --mtu=<value> when configuring bonds and VLANs on server VMs. Before, server interfaces were never told what MTU to use. Now they match what DHCP advertises.

CLI validation

The --interface-mtu flag now validates that the value is between 96 and 9036 (matching the VPC API limits), and is available on vlab up in addition to setup-vpcs.

Why we cannot always leave MTU at 0 for SONiC VS

The ideal behavior for SONiC VS is "do nothing with MTU" -- don't advertise jumbo frames, let servers use their default 1500. Leaving InterfaceMTU at 0 achieves this, but only when no DHCP options object is created. The problem is that we cannot always avoid creating one.

When the user configures DNS or NTP servers, the code must create a VPCDHCPOptions object to carry those values (pkg/hhfab/testing.go:804-811):

var dhcpOpts *vpcapi.VPCDHCPOptions
if len(opts.DNSServers) > 0 || len(opts.TimeServers) > 0 || opts.InterfaceMTU > 0 {
	dhcpOpts = &vpcapi.VPCDHCPOptions{
		DNSServers:   opts.DNSServers,
		TimeServers:  opts.TimeServers,
		InterfaceMTU: opts.InterfaceMTU,
	}
}

Once that object exists, the VPC API defaulting webhook sees InterfaceMTU == 0 and sets it to 9036 (vendor/go.githedgehog.com/fabric/api/vpc/v1beta1/vpc_types.go:306-317):

if subnet.DHCP.Options != nil {
	if subnet.DHCP.Options.InterfaceMTU == 0 {
		subnet.DHCP.Options.InterfaceMTU = 9036
	}
}

There is no way to create a DHCP options object with DNS servers without also triggering the MTU default. So the code explicitly sets 1500 to fill the slot before the API can inject 9036 (pkg/hhfab/testing.go:607-608):

if len(opts.DNSServers) > 0 || len(opts.TimeServers) > 0 {
	opts.InterfaceMTU = 1500
}

When no DNS/NTP are configured, all three conditions in the if on line 805 are false, dhcpOpts stays nil, the API webhook never fires, and no MTU is advertised. Leaving InterfaceMTU at 0 is safe in this case.

Part of https://github.com/githedgehog/internal/issues/338

Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds support for configuring (and auto-selecting) the server-facing MTU used by VLAB “on-ready” networking setup, aligning HLAB defaults with a 9036 server-facing MTU while allowing overrides via CLI/env.

Changes:

  • Add interface-mtu option to hhfab vlab up on-ready flow and plumb it into setup-vpcs.
  • Auto-select MTU (9036 for non-SONiC VS) in SetupVPCs, and pass MTU through to DHCP options and server net configuration.
  • Extend hhnet.sh and server netconf command generation to support an optional --mtu= argument for VLAN/bond configs.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 3 comments.

File Description
pkg/hhfab/vlabrunner.go Adds InterfaceMTU to run opts and passes it into SetupVPCsOpts for on-ready setup.
pkg/hhfab/testing.go Adds MTU propagation to server netconf commands and introduces MTU auto-configuration logic in SetupVPCs.
pkg/hhfab/hhnet.sh Adds optional --mtu= parsing and applies MTU during bond/VLAN setup.
cmd/hhfab/main.go Adds CLI/env flag to configure server/VPC interface MTU for on-ready commands.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

You can also share your feedback on Copilot code review. Take the survey.

Comment thread cmd/hhfab/main.go Outdated
Comment thread pkg/hhfab/testing.go Outdated
Comment thread pkg/hhfab/testing.go
@pau-hedgehog pau-hedgehog added ci:+hlab Enable hybrid VLAB tests ci:+release Enable VLAB release tests labels Mar 13, 2026
@github-actions
Copy link
Copy Markdown

github-actions Bot commented Mar 13, 2026

Release Tests

  9 files   36 suites   4h 3m 12s ⏱️
 33 tests  31 ✅   2 💤 0 ❌
297 runs  111 ✅ 186 💤 0 ❌

Results for commit 8047603.

♻️ This comment has been updated with latest results.

@pau-hedgehog pau-hedgehog force-pushed the pau/vlab-server-max-mtu branch from e0ca687 to 12e8098 Compare March 14, 2026 00:58
@pau-hedgehog pau-hedgehog marked this pull request as ready for review March 14, 2026 01:07
@pau-hedgehog pau-hedgehog requested review from a team as code owners March 14, 2026 01:07
@pau-hedgehog pau-hedgehog force-pushed the pau/vlab-server-max-mtu branch from 12e8098 to 54d1d47 Compare March 23, 2026 21:21
@Frostman Frostman force-pushed the pau/vlab-server-max-mtu branch 2 times, most recently from f5a9bce to 8047603 Compare April 2, 2026 04:50
@Frostman Frostman requested review from Copilot and edipascale April 8, 2026 05:27
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

This PR introduces a configurable “server interface MTU” knob for VLAB/HLAB workflows and wires it through the on-ready setup-vpcs path, including pushing the MTU down into the server-side hhnet configuration. It also adds auto-selection logic to prefer jumbo MTU (9036) in non-SONiC-VS environments while avoiding jumbo DHCP MTU advertisement for SONiC VS.

Changes:

  • Add InterfaceMTU to VLAB run options and pass it into SetupVPCs.
  • Auto-determine InterfaceMTU inside SetupVPCs (9036 for non-VS; avoid jumbo MTU advertisement for SONiC VS when DHCP options are present).
  • Extend server netconf generation + hhnet.sh to support --mtu=<value> for bond and vlan configuration.

Reviewed changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 1 comment.

File Description
pkg/hhfab/vlabrunner.go Adds InterfaceMTU to VLAB run opts and forwards it into on-ready SetupVPCs.
pkg/hhfab/testing.go Adds MTU handling to server netconf command generation and implements MTU auto-configuration in SetupVPCs.
pkg/hhfab/hhnet.sh Adds --mtu parsing and applies MTU during bond/vlan setup.
cmd/hhfab/main.go Adds CLI flag/env var support for interface MTU and validates it via parseInterfaceMTU.

💡 Add Copilot custom instructions for smarter, more guided reviews. Learn how to get started.

Comment thread cmd/hhfab/main.go
Comment thread pkg/hhfab/testing.go
@pau-hedgehog pau-hedgehog force-pushed the pau/vlab-server-max-mtu branch from 8047603 to f331be7 Compare April 8, 2026 13:19
@pau-hedgehog pau-hedgehog removed the ci:+release Enable VLAB release tests label Apr 8, 2026
@pau-hedgehog pau-hedgehog force-pushed the pau/vlab-server-max-mtu branch from f331be7 to 12d8b31 Compare April 8, 2026 22:50
@pau-hedgehog pau-hedgehog force-pushed the pau/vlab-server-max-mtu branch from 0f209a3 to 2667511 Compare April 11, 2026 17:49
Copy link
Copy Markdown
Contributor

@edipascale edipascale left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

thanks for looking into my concerns and for the detailed explanation in the PR description!

Signed-off-by: Pau Capdevila <pau@githedgehog.com>
@pau-hedgehog pau-hedgehog force-pushed the pau/vlab-server-max-mtu branch from 2667511 to 03668f0 Compare April 13, 2026 07:09
@pau-hedgehog pau-hedgehog requested a review from Frostman April 13, 2026 10:34
@Frostman Frostman merged commit f95f8b9 into master Apr 13, 2026
26 checks passed
@Frostman Frostman deleted the pau/vlab-server-max-mtu branch April 13, 2026 15:37
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

ci:+hlab Enable hybrid VLAB tests

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants