Skip to content

Deprecate lossy {mac: ip} neighbour table shape; add Windows neighbour support (#69655) - #69657

Open
ggiesen wants to merge 8 commits into
saltstack:3006.xfrom
ggiesen:fix-69655-neigh-entries
Open

Deprecate lossy {mac: ip} neighbour table shape; add Windows neighbour support (#69655)#69657
ggiesen wants to merge 8 commits into
saltstack:3006.xfrom
ggiesen:fix-69655-neigh-entries

Conversation

@ggiesen

@ggiesen ggiesen commented Jul 2, 2026

Copy link
Copy Markdown
Contributor

What does this PR do?

Starts the deprecation cycle proposed in #69655 for the neighbour table functions, and adds the missing Windows support.

network.arp, network.ip_neighs and network.ip_neighs6 gain an expand argument:

  • expand=True returns a list of {"ip": ..., "mac": ..., "dev": ..., "state": ...} entry dicts, which preserves multiple IPs per MAC (the normal case in IPv6, where every host holds at least a link-local and a global address on the same MAC) along with the interface and reachability state.
  • expand=False keeps the legacy {mac: ip} mapping.
  • Calling without expand keeps the legacy mapping and emits a DeprecationWarning via warn_until(3011, ...): per the deprecation policy the warning stays in place for 3009 and 3010, and the default flips to the new shape in 3011.

win_network (which loads as network on Windows) previously had no neighbour table support at all. This adds all three functions on top of Get-NetNeighbor, with the same expand argument and the same deprecation cycle, so network.arp behaves the same on every platform at every point of the transition. MACs are normalized to the lowercase colon-separated Unix form, states to the uppercase NUD vocabulary (REACHABLE, STALE, ...), and the permanent multicast/broadcast pseudo-neighbours Windows keeps in its cache (which Linux never stores in the neighbour table) are filtered out.

While adding the expanded shape, two parsing bugs in the existing code were fixed (they previously leaked garbage keys into the flat mapping):

  • ip neigh lines for unresolved neighbours carry no lladdr, but a flag such as router can pad them to five fields, so the parser took the state word as the MAC (an unreachable IPv6 gateway produced {"FAILED": "fe80::1"}). Inclusion is now gated on the lladdr token.
  • arp -an prints unresolved entries with an <incomplete>/(incomplete) placeholder in the MAC column, which was reported as a MAC on Linux/macOS/FreeBSD/AIX (the OpenBSD branch already skipped it). Only resolved neighbours are reported now, on every platform.

What issues does this PR fix or reference?

Fixes #69655
Fixes #60794

Note: #69621 also touches network.arp() (adds an ip neigh fallback on Linux when net-tools is absent). Whichever merges second will need a trivial rebase; the changes are logically independent.

Previous Behavior

network.arp, network.ip_neighs and network.ip_neighs6 returned {mac: ip} dicts that silently dropped every neighbour sharing a MAC with a previously parsed entry (with ip_neighs6 this loses entries for practically every host on the segment), discarded the interface and state, and could contain garbage keys such as <incomplete> or FAILED for unresolved neighbours. On Windows minions the functions did not exist.

New Behavior

Same defaults, plus a DeprecationWarning announcing the 3011 shape change; expand=True opts in to the complete list-of-entries shape now; unresolved neighbours are no longer reported; the three functions exist on Windows with cross-platform-consistent output.

Merge requirements satisfied?

  • Docs (docstrings incl. versionchanged/versionadded markers)
  • Changelog (69655.deprecated.md, 69655.fixed.md, 69655.added.md)
  • Tests written/updated

The Unix parsing changes are covered by unit tests against captured arp -an/ip neigh output (including flag-padded unresolved lines and incomplete entries, verified against live iproute2 behaviour). The Windows functions are covered by unit tests with mocked cmd.powershell output; they have not yet been exercised against a live Windows minion, which is part of why this is a draft.

Commits signed with GPG?

No

@ggiesen

ggiesen commented Jul 5, 2026

Copy link
Copy Markdown
Contributor Author

Validated the Windows side against a real Windows Server 2025 host (Get-NetNeighbor driven directly, output parsed by win_network._get_neighbors). Confirmed on real output:

  • the field names, dash-separated uppercase MACs, and CamelCase State values are as the code assumes (so the colon/lowercase MAC normalization and uppercase NUD-state normalization are correct);
  • a single neighbour serializes to a bare object rather than a one-element array, exercising the isinstance(results, dict) branch;
  • Windows keeps permanent broadcast/multicast pseudo-neighbours in the table (255.255.255.255, 224.0.0.0/4, ff00::/8) that Linux never stores.

The real host surfaced one case the mocked tests missed: a subnet-directed broadcast (e.g. 10.0.2.255) which is neither multicast nor the limited broadcast address, so the earlier filter let it through. Fixed by filtering on the link-layer address being a group (broadcast/multicast) MAC -- the low bit of the first octet -- which covers all three pseudo-neighbour kinds and keeps only real unicast hosts. Added a regression test for the subnet-broadcast case.

@ggiesen

ggiesen commented Jul 5, 2026

Copy link
Copy Markdown
Contributor Author

End-to-end confirmation on a real Windows Server 2025 host: installed the Salt 3008.2 minion, ran all three functions through salt-call --local, exercising the real path (cmd.powershell -> Get-NetNeighbor -> ConvertTo-Json -> the parser):

  • network.arp expand=True and network.ip_neighs expand=True return the real unicast IPv4 neighbours as {ip, mac, dev, state} dicts, with the broadcast/multicast/subnet-broadcast pseudo-neighbours filtered out.
  • network.ip_neighs6 expand=True returns only the real IPv6 neighbour; the permanent ff02:: multicast entries are filtered.
  • network.arp without expand emits the warn_until DeprecationWarning and returns the legacy {mac: ip} mapping.

MAC normalization (colon/lowercase), state normalization (uppercase NUD vocabulary), and the single-neighbour bare-object case all behaved as expected against the live output.

Comment thread salt/modules/win_network.py Outdated
charzl
charzl previously approved these changes Jul 10, 2026

@charzl charzl left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approve with 2 Nit suggestions.

Comment thread salt/modules/win_network.py Outdated
@ggiesen

ggiesen commented Jul 11, 2026

Copy link
Copy Markdown
Contributor Author

Good call @charzl - you're right these were straight duplicates. Both _neigh_expand_warning and _neighs_flatten are now public helpers in salt.utils.network, and salt.modules.network and salt.modules.win_network both delegate to them (the now-dead salt.utils.versions import in each module went with them). Added direct unit tests for the two helpers. Thanks for catching it.

charzl
charzl previously approved these changes Jul 11, 2026

@charzl charzl left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM, just wait for all tests pass

twangboy
twangboy previously approved these changes Jul 14, 2026
ggiesen and others added 6 commits August 7, 2026 14:37
network.arp, network.ip_neighs and network.ip_neighs6 return a flat
{mac: ip} dict, so a MAC that appears more than once in the neighbour
table keeps only one arbitrary entry. Multiple IPs per MAC is normal:
routers hold several addresses on one interface, and with IPv6 nearly
every neighbour has at least a link-local and a global address, so
ip_neighs6 drops entries for practically every host on the segment. The
flat shape also discards the interface and reachability state.

Add an expand argument to all three functions. expand=True returns a
list of {ip, mac, dev, state} entry dicts; expand=False keeps the
legacy shape. Calls without expand keep the legacy shape but emit a
DeprecationWarning via warn_until(3011): per the deprecation policy the
warning stays in place for 3009 and 3010, and the default flips to the
new shape in 3011.

Windows minions had no neighbour table support at all (win_network
loads as the network module but never implemented arp). Implement all
three functions there on top of Get-NetNeighbor, with the same expand
argument and the same deprecation cycle, so network.arp behaves
identically on every platform throughout the transition. MACs are
normalized to the lowercase colon-separated Unix form.

Fixes saltstack#69655
…tput

Review fixes on top of the expand/deprecation change:

- ip neigh lines for unresolved neighbours carry no lladdr, but a flag
  such as router can still pad them to five fields; the parser then took
  the state word as the MAC (mac='FAILED' for an unreachable IPv6
  gateway). Gate on the lladdr token instead of counting fields.
- arp -an prints unresolved entries with an <incomplete>/(incomplete)
  placeholder in the MAC column, which leaked into both return shapes on
  Linux/macOS/FreeBSD/AIX while the OpenBSD branch skipped it. Skip the
  placeholder everywhere; only resolved neighbours are reported now.
- Windows kept the permanent multicast/broadcast pseudo-neighbours
  (224.0.0.0/4, ff00::/8, 255.255.255.255) that Linux never stores in
  its neighbour table, and reported CamelCase states; filter those
  pseudo-entries and report the uppercase NUD vocabulary so consumers
  see the same data cross-platform.
- File the deprecation notice as changelog/69655.deprecated.md so
  towncrier renders it in the Deprecated section; the fixed fragment now
  covers the unresolved-entry parsing fixes.
Validated Get-NetNeighbor output on a real Windows Server 2025 host: it
also keeps permanent subnet-directed broadcast entries (e.g.
10.0.2.255), which are neither multicast nor the limited broadcast
address, so the previous filter let them through as phantom neighbours.
Every broadcast/multicast pseudo-neighbour carries a group link-layer
address (low bit of the first MAC octet set), while real hosts have a
unicast MAC, so filter on that instead. Confirmed the real IPv4/IPv6
tables and the single-entry (bare object) serialization parse correctly.
The expand-argument versionchanged markers and the Windows
neighbour-function versionadded markers pointed at 3009.0. With this PR
retargeted to 3006.x, they should reference 3006.28 (the next 3006.x
release). The deprecation still flips the default return shape in 3011,
unchanged.
The neigh_expand_warning and neighs_flatten helpers were defined
identically in both salt/modules/network.py and salt/modules/win_network.py.
Move them to salt.utils.network as shared public functions and have both
modules delegate, dropping the now-unused salt.utils.versions imports.
Add direct unit tests for the extracted helpers.
Comment thread salt/modules/win_network.py Outdated
Address review feedback: _get_neighbors ran Get-NetNeighbor through
cmd.powershell, which spins up a powershell.exe subprocess per call. Switch to
the in-process salt.utils.win_pwsh.PowerShellSession runspace (via run_json),
matching win_ip. win_network is a mixed module and cannot gate __virtual__ on
the PowerShell SDK the way win_ip does, so guard the call with HAS_PWSH_SDK and
raise a clear CommandExecutionError (instead of a NameError from instantiating
PowerShellSession) on the rare install without pythonnet; the standard Salt
onedir bundles it. Update the unit tests to mock the session path and add a
guard test.
@ggiesen

ggiesen commented Aug 12, 2026

Copy link
Copy Markdown
Contributor Author

Thanks, good call. Switched _get_neighbors to the in-process PowerShellSession (via run_json), matching win_ip. Since win_network is a mixed module and can't gate __virtual__ on the SDK the way win_ip does, I guard the call with HAS_PWSH_SDK and raise a clear error instead of the NameError you'd get from instantiating PowerShellSession without it -- the standard onedir bundles pythonnet, so that path is only for a pip-install-without-pythonnet edge case.

Re-validated on a live Windows Server 2025 box with Salt 3008.2: network.arp/ip_neighs/ip_neighs6 expand=True and the bare network.arp deprecation path all return correctly through the runspace. Unit tests updated to mock the session.

twangboy
twangboy previously approved these changes Aug 12, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

test:full Run the full test suite

Projects

None yet

4 participants