Skip to content

tests: fix test flakiness by avoiding localhost and adding port retry#557

Open
tbutter wants to merge 1 commit into
svenstaro:mainfrom
tbutter:flakytests
Open

tests: fix test flakiness by avoiding localhost and adding port retry#557
tbutter wants to merge 1 commit into
svenstaro:mainfrom
tbutter:flakytests

Conversation

@tbutter

@tbutter tbutter commented Jul 11, 2026

Copy link
Copy Markdown

The flakiness came up during nix package upgrade NixOS/nixpkgs#513622 (comment)

AI disclosure: Used antigravity cli to find the flaky test and create a patch.

  1. Host/DNS Resolution Flakiness ( localhost vs 127.0.0.1 ): The tests previously attempted to verify port reachability and perform test requests using localhost . In sandboxed build environments (such as the Nix sandbox), hostname resolution for localhost can
    be unreliable or resolve to the IPv6 loopback address ( ::1 ) first. Because dummyhttp binds to 0.0.0.0 (IPv4-only by default), connecting via IPv6 fails or times out.
  2. Parallel Port Allocation Race Conditions: The test suite uses the port_check crate to query for a free port, which it then closes before starting the dummyhttp process. Since tests run in parallel, multiple tests could check for a free port, get the same port
    number, and cause dummyhttp to fail to bind on startup. Additionally, the startup loop would wait for the full timeout (1 second) instead of immediately failing-fast if the child process exited early.

@svenstaro

Copy link
Copy Markdown
Owner

Hey, thanks. I think a better way would be to adapt what we do over in buildbtw. That way, we can ensure a process will definitely get the port that it's locking because we can pass the lock around.

@svenstaro

Copy link
Copy Markdown
Owner

Also I'd appreciate it if you could separate out the DNS stuff to a second commit.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants