Add inline L3VNI traffic to non-mac_only MAC move tests

[bpar9]

Description of PR

Summary: All non-mac_only MAC move test cases now create both L2 and L3 traffic streams inline. L3 streams are created in get_stream_handles_dci() on vlan13 (L3VNI in Vrf101) using the same circuit_type='raw' pattern as L2 streams on vlan12. L3 traffic is inter-VLAN routed: source IP on vlan13 (80.13.0.99), destination IP is the host's L2 IP on vlan12 (e.g. 80.12.0.180), and the Ethernet destination MAC is the SAG gateway RMAC (00:11:22:33:44:55) so the switch performs L3 routing lookup. L2 and L3 streams are sent and validated together in a single step at phases 3, 6, and 9 — the test only passes when both L2 and L3 results are correct.

Files modified: test_vxlan_dci.py, vxlan_dci_input_file.yaml, and vxlan_helper.py.

Updates since last revision

Per reviewer feedback:

• Merged L2+L3 traffic into single step — L2 and L3 streams are now sent together in one _send_traffic_dci() call and validated together at phases 3, 6, and 9. Previously L3 was a separate phase (3b/6b/9b) which meant L2 pass/fail was checked independently before L3. Now all streams run simultaneously so host move correctness is validated by both L2 and L3 at once. Net: -20 lines (removed 72 lines of separate L3 phases, added 52 lines of merged logic).
• Fixed L3 stream dst_mac to use SAG gateway RMAC — L3 streams now use 00:11:22:33:44:55 (the static anycast gateway MAC from config_sag_mac()) as the Ethernet destination MAC instead of a fabricated host-style MAC. Since L3 traffic is routed (vlan13→vlan12), the dst_mac must be the router MAC so the switch performs L3 routing lookup. Added l3_rmac as a configurable field in _get_dci_mac_move_cfg() (overridable via YAML).
• Fixed L3 traffic to use inter-VLAN routing (src vlan13 → dst vlan12) — L3 stream destination IPs now use the host's L2 IP on vlan12 (same IP targeted by L2 streams) instead of a separate vlan13 IP. This matches the expected behavior: traffic enters on vlan13 and is routed via L3VNI to vlan12 where the host is learned. When the host is at dest1, L3 to dest1 passes and L3 to dest2 fails (same as L2). Removed the now-unused l3_dst_ipv4/l3_dst_ipv6 config from both YAML and Python defaults.
• Restored 6 ipv4_only/ipv6_only test methods (Solution_dci:80/81/87/88/94/95) with @pytest.mark.skip decorators removed so they can now run with L2 + L3 traffic
• Removed vlan_change_l2_to_l3 placeholder (Solution_dci:96)
• No separate test cases for L3VNI_dci:59/60/62/63 — L3 traffic is simply inline in all existing non-mac_only tests
• Moved L3VNI config to YAML input file — l3_vlan, l3_gateway_v4/v6, l3_src_ipv4/ipv6 now live in vxlan_dci_input_file.yaml under global.dci_mac_move, alongside the existing L2 config. _get_dci_mac_move_cfg() reads them via cfg.get() (with matching defaults as fallback).
• Fixed L3 IP addressing to match L2 SAG pattern — L3 IPs now use 80.13.0.x / 8000:13::x (consistent with the 80.{vlan}.0.x / 8000:{vlan}::x scheme from generate_svi_ip_sag() → generate_new_v4_ip()), instead of the incorrect 13.13.13.x / 13:13:13::x which were switch-side interface IPs.
• Fixed UnboundLocalError in verify_mac_seq() — mac_found and ip_found were never initialized before the loop in verify_mac_seq() (only assigned True inside conditional branches), causing UnboundLocalError: local variable 'ip_found' referenced before assignment at L5451 for all non-mac_only host types. Added mac_found = False / ip_found = False at function entry, matching how verify_type2_route() already initializes them. This was a pre-existing bug exposed by removing the @pytest.mark.skip decorators.
• Fixed TypeError in _log_evpn_type2_route_table() — verify_mac_seq() called _log_evpn_type2_route_table(node, item, host_type, route_type='remote') for remote-route logging, but the function signature only accepted (node, item, host_type). Added route_type='local' default parameter and used it in the log message so local vs remote routes are now distinguished. Another pre-existing bug exposed by enabling the non-mac_only code paths.

What changed:

• vxlan_dci_input_file.yaml: Added L3VNI config to global.dci_mac_move — l3_vlan: 13, gateway 80.13.0.1 / 8000:13::1, src IPs 80.13.0.99 / 8000:13::99 (dst IPs derived from host's L2 IP on vlan12 at runtime)
• _get_dci_mac_move_cfg(): Reads L3VNI config from YAML via cfg.get('l3_vlan', 13) etc., including l3_rmac (SAG gateway MAC 00:11:22:33:44:55). Defaults preserved as fallback.
• get_stream_handles_dci(): For non-mac_only host types, creates two additional raw L3 traffic streams on vlan13 (l3_src1_stream_handle, l3_src2_stream_handle) with inter-VLAN routing — src IP on vlan13, dst IP is the host's L2 IP on vlan12, dst MAC is the SAG RMAC (00:11:22:33:44:55)
• verify_mac_move_dci(): At phases 3, 6, 9 all L2 + L3 streams are sent together via _send_traffic_dci() and validated together. The test passes only when both L2 and L3 results are correct. L3 streams are auto-detected via mm_handles.get('l3_src1_stream_handle').
• verify_mac_seq() in vxlan_helper.py: Initialize mac_found = False and ip_found = False before the loop to fix UnboundLocalError on non-mac_only host types
• _log_evpn_type2_route_table() in vxlan_helper.py: Added route_type='local' parameter and dynamic log label to fix TypeError when called with route_type='remote' from verify_mac_seq()
• All non-mac_only test cases: Simple calls — verify_mac_move_dci(tc_id, move_dir, host_type) with no extra kwargs

Concrete L2/L3 stream example (orphan_to_orphan_within_dc + mac+ipv4):

Stream	src_mac	dst_mac	src_ip	dst_ip	vlan
L2 src1	02:00:00:04:12:99	02:00:00:04:12:02	80.12.0.99	80.12.0.180	12
L2 src2	02:00:00:04:12:99	02:00:00:04:12:02	80.12.0.99	80.12.0.180	12
L3 src1	02:00:00:13:12:99	00:11:22:33:44:55	80.13.0.99	80.12.0.180	13
L3 src2	02:00:00:13:12:99	00:11:22:33:44:55	80.13.0.99	80.12.0.180	13

Example log output (Phase 3, combined L2+L3):

--- PHASE 3: L2+L3 traffic BEFORE move (host at leaf0_dc1) ---
  L2 to leaf0_dc1 (host here): PASS  [expect PASS]
  L2 to leaf0_dc2 (host away): FAIL  [expect FAIL]
  L3 to leaf0_dc1 (host here): PASS  [expect PASS]
  L3 to leaf0_dc2 (host away): FAIL  [expect FAIL]
  Traffic check: PASS

Test cases with L2 + L3 traffic (20 total):

Group	Tests
orphan within DC	mac+ipv4, mac+ipv6
orphan across DC	mac+ipv4, mac+ipv6, ipv4_changes, ipv6_changes, ipv4_only, ipv6_only
MH across DC	mac+ipv4, mac+ipv6, ipv4_changes, ipv6_changes, ipv4_only, ipv6_only
MH-to-orphan across DC	mac+ipv4, mac+ipv6, ipv4_changes, ipv6_changes, ipv4_only, ipv6_only

L2 only (by design): mac_only tests (3 total) and orphan_to_pc (uses mac_only).

Type of change

• Bug fix
• Testbed and Framework(new/improvement)
• New Test case
  • Skipped for non-supported platforms
• Test case improvement

Back port request

• 202205
• 202305
• 202311
• 202405
• 202411
• 202505
• 202511

Approach

What is the motivation for this PR?

Add L3VNI traffic verification to existing MAC move test cases so that every non-mac_only test validates both L2 (vlan12) and L3 (vlan13→vlan12 inter-VLAN routed) forwarding during host mobility events. Also enables the formerly-skipped ipv4_only/ipv6_only test variants to run, and fixes pre-existing bugs in verify_mac_seq() and _log_evpn_type2_route_table() that prevented them from working.

How did you do it?

• Added L3VNI addressing config (l3_vlan, l3_gateway_v4/v6, l3_src_ipv4/ipv6, l3_rmac) to vxlan_dci_input_file.yaml under global.dci_mac_move. L3 source IPs follow the 80.{vlan}.0.x / 8000:{vlan}::x SAG pattern. L3 destination IPs are derived at runtime from the host's L2 IP on vlan12 (inter-VLAN routing). L3 dst MAC is the SAG gateway RMAC (00:11:22:33:44:55).
• Extended get_stream_handles_dci() to create L3 raw traffic streams on vlan13 for all non-mac_only host types, using the same tg_traffic_config(circuit_type='raw', ...) pattern as L2 streams. The L3 dst IP is the host's vlan12 IP (stream_info['src1']['ip_dst'] / stream_info['src2']['ip_dst']), so traffic enters vlan13 and is routed to vlan12 via L3VNI. The L3 dst MAC is the SAG RMAC so the switch performs L3 routing lookup.
• Extended verify_mac_move_dci() to send all L2+L3 streams together in a single _send_traffic_dci() call at phases 3, 6, 9. L3 stream handles are auto-detected via mm_handles.get('l3_src1_stream_handle'). After sending, both L2 and L3 stats are validated — the test passes only if both are correct. No separate traffic phases for L3.
• Removed @pytest.mark.skip from 6 ipv4_only/ipv6_only tests (Solution_dci:80/81/87/88/94/95) so they now run.
• Removed vlan_change_l2_to_l3 placeholder test (Solution_dci:96).
• Fixed verify_mac_seq() in vxlan_helper.py: added mac_found = False and ip_found = False initialization before the loop, matching the pattern in verify_type2_route(). Without this, all non-mac_only host types hit UnboundLocalError at the if not ip_found: check.
• Fixed _log_evpn_type2_route_table() in vxlan_helper.py: added route_type='local' default parameter so callers can pass route_type='remote' for remote-route logging.

How did you verify/test it?

• python3 -m py_compile passes (no syntax errors)
• flake8 --select=E9,F63,F7,F82 clean (no critical errors introduced; pre-existing F821 at L5571-5572 unrelated)
• YAML validated with yaml.safe_load()
• Reviewer ran on testbed: UnboundLocalError (ip_found), TypeError (route_type), wrong L3 dst_ip (vlan13 instead of vlan12), and wrong L3 dst_mac (fabricated instead of RMAC) were caught and fixed
• Merged L2+L3 traffic change has not yet been verified on testbed
• Full functional verification requires the 3-DC EVPN-VXLAN testbed with IXIA traffic generation

⚠️ Human review checklist

1. Combined L2+L3 traffic timing — All 4 streams (L2 src1, L2 src2, L3 src1, L3 src2) are now sent together in one _send_traffic_dci() call. Verify that running 4 simultaneous traffic items doesn't cause IXIA timing issues or unexpected packet drops, especially on the shared source port.

2. Inter-VLAN routing path — L3 streams enter on vlan13 (src IP 80.13.0.99) with dst IP on vlan12 (the host's L2 IP, e.g. 80.12.0.180) and dst MAC 00:11:22:33:44:55 (SAG RMAC). Verify that the switch has the correct routing config to forward from vlan13→vlan12 via L3VNI in Vrf101, and that the SAG gateway on vlan13 (80.13.0.1) is reachable from the traffic generator port.

3. L3 stream source MAC — L3 streams use l3_mac_src = "02:00:00:13:{sf}:99" as the Ethernet source MAC. This is a locally-administered MAC that identifies the L3 traffic source. Confirm this is acceptable or whether it should also be a known host/gateway MAC.

4. Additional uninitialized variables in verify_mac_seq() — Two runtime bugs have already been found and fixed (ip_found UnboundLocalError, route_type TypeError). Check for other uninitialized variables or signature mismatches that may only surface for specific host types or code paths. In particular, learn_type is used at L5447/L5490 inside weight-check blocks — confirm it's always assigned before use in those paths.

5. ipv4_only/ipv6_only host type handling — These 6 tests were previously skipped and are now enabled. Verify that get_stream_handles_dci() correctly handles these host types (i.e., that ipv4_only falls into the ipv4_host_types tuple and ipv6_only falls into the ipv6_host_types tuple for both L2 and L3 stream creation).

6. L3 traffic follows host location — L3 l3_src1 targets dest1's host IP; l3_src2 targets dest2's host IP. Verify this correctly tracks host mobility: when host is at dest1, l3_src1 should pass (routed to vlan12 on dest1) and l3_src2 should fail (host not at dest2).

7. YAML values vs Python defaults — L3 config now exists in both vxlan_dci_input_file.yaml and as cfg.get() defaults in _get_dci_mac_move_cfg(). The YAML values take precedence. Confirm this duplication pattern is acceptable (it matches the existing L2 config approach).

8. Not fully tested at runtime — Multiple bugs were caught by the reviewer during test runs, all in pre-existing code or newly-written L3 stream config. The latest change (merging L2+L3 into single step) has not yet been verified on testbed. A full successful test run through all 20 non-mac_only tests is needed to confirm correctness.

Any platform specific information?

Requires 3-DC EVPN-VXLAN topology with BGW spines and IXIA traffic generation (spytest framework).

Supported testbed topology if it's a new test case?

3-datacenter EVPN-VXLAN DCI topology (DC1: 4 leafs + 2 spines + 2 BGW spines, DC2: 2 leafs + 2 BGW spines, DC3: 1 leaf + 1 BGW spine).

Documentation

Test cases are documented in DCI_Solution_Testplan.xlsx.

Link to Devin session: https://cisco-demo.devinenterprise.com/sessions/61b3aeaa669d43f18dfd488a3991c980
Requested by: @bpar9

[devin-ai-integration]

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

• Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
• Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

• Disable automatic comment and CI monitoring

[vallabh78]

Can we reuse the function verify_mac_move_dci for the testcases 59, 60, 62 and 63?

[devin-ai-integration]

Good call — done. In the latest commit (60589fc), verify_l3vni_host_mobility_dci() is removed entirely. Instead, verify_mac_move_dci() now accepts optional l3_traffic_scope and l3_traffic_types keyword arguments. When provided, verify_traffic() is called at phases 3b, 6b, and 9b (after the existing point-to-point traffic checks). Existing callers are unaffected since the defaults are None.

The four L3VNI test cases now simply call:

self.verify_mac_move_dci(tc_id, move_dir=..., host_type="mac+ipv4",
                         l3_traffic_scope='cross', l3_traffic_types=['l3_v4', 'l3_v6'])

This eliminates ~200 lines of duplicated MAC move logic.

[vallabh78]

For tc 59, 60, 62,63, we dont need to create a new testcases. We can use existing testcases and enable L3 traffic items for all tc excluding mac only testcase.
for L3 traffic we can keep the same source but diffferent vlan like vlan13. That will be only the difference.

[devin-ai-integration]

@vallabh78 Done in 815bc31. Removed the 4 dedicated L3VNI test methods and their move_dir/_MAC_SUFFIX/_MOVE_BAND entries. L3VNI traffic verification is now enabled directly in the 14 existing non-mac_only MAC move test cases by passing l3_traffic_scope='cross', l3_traffic_types=['l3_v4', 'l3_v6'] to verify_mac_move_dci().

The mac_only and orphan_to_pc tests are unchanged (no L3 addresses to verify). Skipped/deferred tests are also unchanged.

[vallabh78]

check how L2 streams are created, we want to create same for L3. We dont need to pass l3_traffic_scope='cross', l3_traffic_types=['l3_v4', 'l3_v6']

@vallabh78 Done in 815bc31. Removed the 4 dedicated L3VNI test methods and their move_dir/_MAC_SUFFIX/_MOVE_BAND entries. L3VNI traffic verification is now enabled directly in the 14 existing non-mac_only MAC move test cases by passing l3_traffic_scope='cross', l3_traffic_types=['l3_v4', 'l3_v6'] to verify_mac_move_dci().

The mac_only and orphan_to_pc tests are unchanged (no L3 addresses to verify). Skipped/deferred tests are also unchanged.

@vallabh78 Done in 815bc31. Removed the 4 dedicated L3VNI test methods and their move_dir/_MAC_SUFFIX/_MOVE_BAND entries. L3VNI traffic verification is now enabled directly in the 14 existing non-mac_only MAC move test cases by passing l3_traffic_scope='cross', l3_traffic_types=['l3_v4', 'l3_v6'] to verify_mac_move_dci().

The mac_only and orphan_to_pc tests are unchanged (no L3 addresses to verify). Skipped/deferred tests are also unchanged.

[vallabh78]

check how L2 streams are created, we want to create same for L3. We dont need to pass l3_traffic_scope='cross', l3_traffic_types=['l3_v4', 'l3_v6']

@vallabh78 Done in 815bc31. Removed the 4 dedicated L3VNI test methods and their move_dir/_MAC_SUFFIX/_MOVE_BAND entries. L3VNI traffic verification is now enabled directly in the 14 existing non-mac_only MAC move test cases by passing l3_traffic_scope='cross', l3_traffic_types=['l3_v4', 'l3_v6'] to verify_mac_move_dci().
The mac_only and orphan_to_pc tests are unchanged (no L3 addresses to verify). Skipped/deferred tests are also unchanged.

@vallabh78 Done in 815bc31. Removed the 4 dedicated L3VNI test methods and their move_dir/_MAC_SUFFIX/_MOVE_BAND entries. L3VNI traffic verification is now enabled directly in the 14 existing non-mac_only MAC move test cases by passing l3_traffic_scope='cross', l3_traffic_types=['l3_v4', 'l3_v6'] to verify_mac_move_dci().
The mac_only and orphan_to_pc tests are unchanged (no L3 addresses to verify). Skipped/deferred tests are also unchanged.

[devin-ai-integration]

@vallabh78 Done in 1137561. Refactored per your feedback — L3 streams are now created inline in get_stream_handles_dci() using the same tg_traffic_config(circuit_type='raw', ...) pattern as L2 streams, just on vlan13 instead of vlan12. No l3_traffic_scope or l3_traffic_types parameters needed anywhere.

What changed:

• _get_dci_mac_move_cfg(): Added L3VNI config (vlan13, gateway 13.13.13.1 / 13:13:13::1, src/dst IPs)
• get_stream_handles_dci(): For non-mac_only host types, creates two raw L3 streams on vlan13 (l3_src1_stream_handle, l3_src2_stream_handle) — same pattern as L2 streams on vlan12
• verify_mac_move_dci(): Auto-detects L3 stream handles in mm_handles and sends/verifies at phases 3b, 6b, 9b — no extra params
• All 14 non-mac_only test cases: simplified to just verify_mac_move_dci(tc_id, move_dir, host_type)

[vallabh78]

why do we still have 59,60,62,63 testcases? Its not needed. Remove the skip marker as well. So now all tc's except the mac only will have l2 and l3 traffic items? it should be same behavior as L2

[devin-ai-integration]

@vallabh78 Done in ccb0b46. There are no separate 59/60/62/63 test cases — L3 traffic is just added inline to all existing non-mac_only tests (same behavior as L2).

Also removed:

• 6 skipped ipv4_only/ipv6_only test stubs (Solution_dci:80/81/87/88/94/95) and their @pytest.mark.skip decorators
• test_dci_mac_move_vlan_change_l2_to_l3_across_dc placeholder (Solution_dci:96)

Updated PR title/description to reflect this — no more references to L3VNI_dci:59/60/62/63 as separate test cases.

[vallabh78]

@vallabh78 Done in ccb0b46. There are no separate 59/60/62/63 test cases — L3 traffic is just added inline to all existing non-mac_only tests (same behavior as L2).

Also removed:

• 6 skipped ipv4_only/ipv6_only test stubs (Solution_dci:80/81/87/88/94/95) and their @pytest.mark.skip decorators
• test_dci_mac_move_vlan_change_l2_to_l3_across_dc placeholder (Solution_dci:96)

Updated PR title/description to reflect this — no more references to L3VNI_dci:59/60/62/63 as separate test cases.

I didnt want to remove those skipped testcases only the marker so that i can run it. Remove the 59,60,62 and 63 testcase as they are covered by adding L3 traffic in the existing tc

[devin-ai-integration]

@vallabh78 Got it — restored the 6 ipv4_only/ipv6_only test methods (Solution_dci:80/81/87/88/94/95) with the @pytest.mark.skip decorators removed so they can run. The vlan_change_l2_to_l3 placeholder (Solution_dci:96) remains removed.

Summary of current state:

• All non-mac_only tests (including the formerly-skipped ipv4_only/ipv6_only ones) now have both L2 + L3 traffic inline — same behavior as L2
• No @pytest.mark.skip markers remain in the MAC move test class
• No separate 59/60/62/63 test cases

[vallabh78]

check the input file.yaml we have L2 config there and if we can move the L3 as well

[devin-ai-integration]

@vallabh78 Done in f3edac8. Added L3VNI config to the dci_mac_move section in vxlan_dci_input_file.yaml, alongside the existing L2 config:

dci_mac_move:
    within_dc_vlan: 12
    gateway_v4: "80.12.0.1"
    ...
    # L3VNI config: same source port, different VLAN (vlan13 in Vrf101)
    l3_vlan: 13
    l3_gateway_v4: "13.13.13.1"
    l3_gateway_v6: "13:13:13::1"
    l3_src_ipv4: "13.13.13.99"
    l3_src_ipv6: "13:13:13::99"
    l3_dst_ipv4: "13.13.13.50"
    l3_dst_ipv6: "13:13:13::50"

_get_dci_mac_move_cfg() already reads these via cfg.get('l3_vlan', 13) etc., so the YAML values are used when present and defaults kick in if not.

[vallabh78]

can you check the mac move config for l2 and l3 again, for l3 that gateway ip etc doesn't seem to match. Make sure the L3 traffic stream is created with correct config.

[devin-ai-integration]

@vallabh78 Good catch — fixed in b5c2976. The L3 IPs were using the wrong addressing scheme (13.13.13.x / 13:13:13::x). They now follow the same SAG pattern as L2, which uses 80.{vlan}.0.x / 8000:{vlan}::x (generated by generate_svi_ip_sag → generate_new_v4_ip):

Before (wrong):

l3_gateway_v4: "13.13.13.1"     l3_gateway_v6: "13:13:13::1"
l3_src_ipv4:   "13.13.13.99"    l3_dst_ipv4:   "13.13.13.50"

After (correct — matches L2 pattern):

l3_gateway_v4: "80.13.0.1"      l3_gateway_v6: "8000:13::1"
l3_src_ipv4:   "80.13.0.99"     l3_dst_ipv4:   "80.13.0.50"
l3_src_ipv6:   "8000:13::99"    l3_dst_ipv6:   "8000:13::50"

Updated in both vxlan_dci_input_file.yaml and the cfg.get() defaults in _get_dci_mac_move_cfg().

[vallabh78]

2026-04-08 01:36:31,147 T0000: INFO Node leaf0_dc1: parsed 13 route(s) matching MAC/IP
FAILED2026-04-08 01:36:31,367 T0000: ERROR Exception: UnboundLocalError local variable 'ip_found' referenced before assignment
2026-04-08 01:36:31,367 T0000: INFO ========= Report(ScriptError):cisco/tortuga/solution/test_vxlan_dci.py::TestVxlanDciMacMoveTriggers::test_dci_mac_move_mh_to_orphan_across_dc_ipv6_only: Exception UnboundLocalError local variable 'ip_found' referenced before assignment in file /tests/cisco/tortuga/solution/vxlan_helper.py at line 5451 =========
2026-04-08 01:36:31,368 T0000: ERROR Exception UnboundLocalError local variable 'ip_found' referenced before assignment in file /tests/cisco/tortuga/solution/vxlan_helper.py at line 5451
2026-04-08 01:36:31,368 T0000: ERROR [0] /tests/cisco/tortuga/solution/vxlan_helper.py:5451 verify_mac_seq if not ip_found:
2026-04-08 01:36:31,368 T0000: ERROR [1] /tests/cisco/tortuga/solution/test_vxlan_dci.py:6975 verify_mac_move_dci check_mm_1 = vxlan_obj.verify_mac_seq(
2026-04-08 01:36:31,368 T0000: ERROR [2] /tests/cisco/tortuga/solution/test_vxlan_dci.py:7321 test_dci_mac_move_mh_to_orphan_across_dc_ipv6_only result = self.verify_mac_move_dci(tc_id, "mh_to_orphan_across_dc", "ipv6_only")
2026-04-08 01:36:31,368 T0000: AUDIT

[devin-ai-integration]

@vallabh78 Thanks for the test run output. I see the UnboundLocalError: local variable 'ip_found' referenced before assignment in vxlan_helper.py:5451 in verify_mac_seq. This is triggered by the ipv6_only host type — the ip_found variable is likely only assigned inside an IPv4 code path but referenced unconditionally afterwards. Since the ipv6_only tests were previously skipped (@pytest.mark.skip), this bug was never hit before.

I'm investigating the verify_mac_seq function now to fix the ip_found initialization. Will push a fix shortly.

[vallabh78]

@vallabh78 Thanks for the test run output. I see the UnboundLocalError: local variable 'ip_found' referenced before assignment in vxlan_helper.py:5451 in verify_mac_seq. This is triggered by the ipv6_only host type — the ip_found variable is likely only assigned inside an IPv4 code path but referenced unconditionally afterwards. Since the ipv6_only tests were previously skipped (@pytest.mark.skip), this bug was never hit before.

I'm investigating the verify_mac_seq function now to fix the ip_found initialization. Will push a fix shortly.

026-04-08 00:43:20,417 T0000: INFO [D5-leaf0_dc1] sonic#
2026-04-08 00:43:21,219 T0000: INFO Node leaf0_dc1: parsed 2 route(s) matching MAC/IP
2026-04-08 00:43:21,452 T0000: ERROR Exception: UnboundLocalError local variable 'ip_found' referenced before assignment
2026-04-08 00:43:21,453 T0000: INFO ========= Report(ScriptError):cisco/tortuga/solution/test_vxlan_dci.py::TestVxlanDciMacMoveTriggers::test_dci_mac_move_orphan_within_dc_mac_and_ipv4: Exception UnboundLocalError local variable 'ip_found' referenced before assignment in file /tests/cisco/tortuga/solution/vxlan_helper.py at line 5451 =========
2026-04-08 00:43:21,453 T0000: ERROR Exception UnboundLocalError local variable 'ip_found' referenced before assignment in file /tests/cisco/tortuga/solution/vxlan_helper.py at line 5451
2026-04-08 00:43:21,453 T0000: ERROR [0] /tests/cisco/tortuga/solution/vxlan_helper.py:5451 verify_mac_seq if not ip_found:
2026-04-08 00:43:21,453 T0000: ERROR [1] /tests/cisco/tortuga/solution/test_vxlan_dci.py:6975 verify_mac_move_dci check_mm_1 = vxlan_obj.verify_mac_seq(
2026-04-08 00:43:21,453 T0000: ERROR [2] /tests/cisco/tortuga/solution/test_vxlan_dci.py:7174 test_dci_mac_move_orphan_within_dc_mac_and_ipv4 result = self.verify_mac_move_dci(tc_id, "orphan_to_orphan_within_dc", "mac+ipv4")
2026-04-08 00:43:21,453 T0000: AUDIT
================== Function Hook Finalize: cisco/tortuga/solution/test_vxlan_dci.py pretest ==================

[vallabh78]

@vallabh78 Thanks for the test run output. I see the UnboundLocalError: local variable 'ip_found' referenced before assignment in vxlan_helper.py:5451 in verify_mac_seq. This is triggered by the ipv6_only host type — the ip_found variable is likely only assigned inside an IPv4 code path but referenced unconditionally afterwards. Since the ipv6_only tests were previously skipped (@pytest.mark.skip), this bug was never hit before.
I'm investigating the verify_mac_seq function now to fix the ip_found initialization. Will push a fix shortly.

026-04-08 00:43:20,417 T0000: INFO [D5-leaf0_dc1] sonic# 2026-04-08 00:43:21,219 T0000: INFO Node leaf0_dc1: parsed 2 route(s) matching MAC/IP 2026-04-08 00:43:21,452 T0000: ERROR Exception: UnboundLocalError local variable 'ip_found' referenced before assignment 2026-04-08 00:43:21,453 T0000: INFO ========= Report(ScriptError):cisco/tortuga/solution/test_vxlan_dci.py::TestVxlanDciMacMoveTriggers::test_dci_mac_move_orphan_within_dc_mac_and_ipv4: Exception UnboundLocalError local variable 'ip_found' referenced before assignment in file /tests/cisco/tortuga/solution/vxlan_helper.py at line 5451 ========= 2026-04-08 00:43:21,453 T0000: ERROR Exception UnboundLocalError local variable 'ip_found' referenced before assignment in file /tests/cisco/tortuga/solution/vxlan_helper.py at line 5451 2026-04-08 00:43:21,453 T0000: ERROR [0] /tests/cisco/tortuga/solution/vxlan_helper.py:5451 verify_mac_seq if not ip_found: 2026-04-08 00:43:21,453 T0000: ERROR [1] /tests/cisco/tortuga/solution/test_vxlan_dci.py:6975 verify_mac_move_dci check_mm_1 = vxlan_obj.verify_mac_seq( 2026-04-08 00:43:21,453 T0000: ERROR [2] /tests/cisco/tortuga/solution/test_vxlan_dci.py:7174 test_dci_mac_move_orphan_within_dc_mac_and_ipv4 result = self.verify_mac_move_dci(tc_id, "orphan_to_orphan_within_dc", "mac+ipv4") 2026-04-08 00:43:21,453 T0000: AUDIT ================== Function Hook Finalize: cisco/tortuga/solution/test_vxlan_dci.py pretest ==================

how much time?

[devin-ai-integration]

@vallabh78 I see it's also hitting mac_and_ipv4 — so ip_found is uninitialized across multiple host types in verify_mac_seq, not just ipv6_only. I'm experiencing a transient environment issue that's blocking filesystem access. Working on getting it resolved — should have the fix pushed within the next few minutes once I can read the file.

[devin-ai-integration]

@vallabh78 Fixed — pushed 128f67f.

Root cause: verify_mac_seq() never initialized mac_found and ip_found before the loop — they were only set to True inside conditional branches. So the first time if not ip_found: (L5451) ran, the variable didn't exist yet → UnboundLocalError. This affected all non-mac_only host types (mac+ipv4, ipv4_only, ipv6_only).

Fix: Added mac_found = False and ip_found = False at function entry, matching what verify_type2_route() already does at L5758-5759.

[vallabh78]

2026-04-08 02:55:03,276 T0000: INFO [D6-leaf1_dc1] sonic#
2026-04-08 02:55:04,177 T0000: INFO Node leaf1_dc1: parsed 10 route(s) matching MAC/IP
FAILED2026-04-08 02:55:04,398 T0000: ERROR Exception: TypeError _log_evpn_type2_route_table() got an unexpected keyword argument 'route_type'
2026-04-08 02:55:04,398 T0000: INFO ========= Report(ScriptError):cisco/tortuga/solution/test_vxlan_dci.py::TestVxlanDciMacMoveTriggers::test_dci_mac_move_orphan_within_dc_mac_and_ipv4: Exception TypeError _log_evpn_type2_route_table() got an unexpected keyword argument 'route_type' in file /tests/cisco/tortuga/solution/vxlan_helper.py at line 5487 =========
2026-04-08 02:55:04,399 T0000: ERROR Exception TypeError _log_evpn_type2_route_table() got an unexpected keyword argument 'route_type' in file /tests/cisco/tortuga/solution/vxlan_helper.py at line 5487
2026-04-08 02:55:04,399 T0000: ERROR [0] /tests/cisco/tortuga/solution/vxlan_helper.py:5487 verify_mac_seq _log_evpn_type2_route_table(node, item, 'mac_and_ip', route_type='remote')
2026-04-08 02:55:04,399 T0000: ERROR [1] /tests/cisco/tortuga/solution/test_vxlan_dci.py:6975 verify_mac_move_dci check_mm_1 = vxlan_obj.verify_mac_seq(
2026-04-08 02:55:04,399 T0000: ERROR [2] /tests/cisco/tortuga/solution/test_vxlan_dci.py:7174 test_dci_mac_move_orphan_within_dc_mac_and_ipv4 result = self.verify_mac_move_dci(tc_id, "orphan_to_orphan_within_dc", "mac+ipv4")
2026-

[devin-ai-integration]

@vallabh78 Fixed — pushed 2ee8066.

Root cause: _log_evpn_type2_route_table(node, item, host_type) was defined with only 3 parameters, but verify_mac_seq called it with route_type='remote' at L5443 and L5486 (for logging remote routes). Added route_type='local' default parameter to the function and used it in the log output so local vs remote routes are now distinguished.

[vallabh78]

2026-04-08 03:24:15,545 T0000: INFO Sleep for 2 sec(s)... @6884
2026-04-08 03:24:17,547 T0000: INFO ################################################################################
2026-04-08 03:24:17,547 T0000: INFO # [L3 src1] vlan=13 src_ip=80.13.0.99 dst_ip=80.13.0.50
2026-04-08 03:24:17,548 T0000: INFO ################################################################################
2026-04-08 03:24:17,548 T0000: INFO Executing: ixiangpf.traffic_config {'emulation_dst_handle': '1/2/1', 'mac_src': '02:00:00:13:12:99', 'mac_dst': '02:00:00:13:12:02', 'ip_src_addr': '80.13.0.99', 'ip_dst_addr': '80.13.0.50', 'emulation_src_handle': '1/2/7', 'mode': 'create', 'transmit_mode': 'single_burst', 'pkts_per_burst': 1000, 'rate_percent': 10, 'circuit_type': 'raw', 'frame_size': 1000, 'vlan_id': 13, 'src_dest_mesh': 'one_to_one', 'track_by': 'endpoint_pair', 'vlan': 'enable'}
2026-04-08 03:24:19,636 T0000: INFO STREAM HANDLE: "TI124-HLTAPI_TRAFFICITEM_540"
2026-04-08 03:24:19,637 T0000: ERROR Couldn't get ip prefix for handle: 1/2/7
2026-04-08 03:24:19,638 T0000: ERROR Couldn't get ip prefix for handle: 1/2/1
2026-04-08 03:24:19,638 T0000: INFO IP PREFIXES: {'emulation_src_handle': [], 'emulation_dst_handle': []}
2026-04-08 03:24:19,639 T0000: INFO Sleep for 2 sec(s)... @6904
2026-04-08 03:24:21,641 T0000: INFO ################################################################################
2026-04-08 03:24:21,641 T0000: INFO # [L3 src2] vlan=13 src_ip=80.13.0.99 dst_ip=80.13.0.50
2026-04-08 03:24:21,642 T0000: INFO ################################################################################
2026-04-08 03:24:21,643 T0000: INFO Executing: ixiangpf.traffic_config {'emulation_dst_handle': '1/2/10', 'mac_src': '02:00:00:13:12:99', 'mac_dst': '02:00:00:13:12:02', 'ip_src_addr': '80.13.0.99', 'ip_dst_addr': '80.13.0.50', 'emulation_src_handle': '1/2/7', 'mode': 'create', 'transmit_mode': 'single_burst', 'pkts_per_burst': 1000, 'rate_percent': 10, 'circuit_type': 'raw', 'frame_size': 1000, 'vlan_id': 13, 'src_dest_mesh': 'one_to_one', 'track_by': 'endpoint_pair', 'vlan': 'enable'}
2026-04-08 03:24:23,785 T0000: INFO STREAM HANDLE: "TI125-HLTAPI_TRAFFICITEM_540"
2026-04-08 03:24:23,786 T0000: ERROR Couldn't get ip prefix for handle: 1/2/7
2026-04-08 03:24:23,787 T0000: ERROR Couldn't get ip prefix for handle: 1/2/10
2026-04-08 03:24:23,787 T0000: INFO IP PREFIXES: {'emulation_src_handle': [], 'emulation_dst_handle': []}
2026-04-08 03:24:23,788 T0000: INFO Sleep for 2 sec(s)... @6928
2026-04-08 03:24:25,790 T0000: INFO Created L3 streams on vlan 13: l3_src1=TI124-HLTAPI_TRAFFICITEM_540 l3_src2=TI125-HLTAPI_TRAFFICITEM_540
2026-04-08 03:24:25,790 T0000: INFO
2026-04-08 03:24:25

Traffic item src will be in vlan 13 leaf2_dc1 but dest is in vlan 12.
expectation is when host comes up on dest1, l2 and l3 traffic will pass but traffic will fail to dest2. Same when host is moved to dest2, l2 and l3 traffic will pass and fail towards dest1.

this is sample how it should look like:
2026-03-28 17:14:40,758 T0000: INFO
2026-03-28 17:14:40,758 T0000: INFO ################################################################################
2026-03-28 17:14:40,758 T0000: INFO # TRAFFIC ITEM TI62-HLTAPI_TRAFFICITEM_540 #
2026-03-28 17:14:40,758 T0000: INFO ################################################################################
2026-03-28 17:14:40,758 T0000: INFO Received traffic: 1000
2026-03-28 17:14:40,758 T0000: INFO Sent traffic: 1000
2026-03-28 17:14:40,758 T0000: INFO 1.0
2026-03-28 17:14:40,759 T0000: INFO TRAFFIC ITEM TI62-HLTAPI_TRAFFICITEM_540 PASSED
2026-03-28 17:14:40,759 T0000: INFO Executing: ixiangpf.traffic_stats {'mode': 'traffic_item', 'streams': 'TI63-HLTAPI_TRAFFICITEM_540', 'csv_path': '/run_logs/Multi-Homing-Master_g200_v6_Mar28_38714_128_MH_set2/results_2026_03_28_09_55_17_tgen'}
2026-03-28 17:14:48,510 T0000: INFO
2026-03-28 17:14:48,511 T0000: INFO ################################################################################
2026-03-28 17:14:48,511 T0000: INFO # TRAFFIC ITEM TI63-HLTAPI_TRAFFICITEM_540 #
2026-03-28 17:14:48,511 T0000: INFO ################################################################################
2026-03-28 17:14:48,511 T0000: INFO Received traffic: 0
2026-03-28 17:14:48,511 T0000: INFO Sent traffic: 1000
2026-03-28 17:14:48,511 T0000: INFO 0.0
2026-03-28 17:14:48,511 T0000: INFO TRAFFIC ITEM TI63-HLTAPI_TRAFFICITEM_540 FAILED
2026-03-28 17:14:48,511 T0000: INFO Executing: ixiangpf.traffic_stats {'mode': 'traffic_item', 'streams': 'TI64-HLTAPI_TRAFFICITEM_540', 'csv_path': '/run_logs/Multi-Homing-Master_g200_v6_Mar28_38714_128_MH_set2/results_2026_03_28_09_55_17_tgen'}
2026-03-28 17:14:56,699 T0000: INFO
2026-03-28 17:14:56,699 T0000: INFO ################################################################################
2026-03-28 17:14:56,699 T0000: INFO # TRAFFIC ITEM TI64-HLTAPI_TRAFFICITEM_540 #
2026-03-28 17:14:56,699 T0000: INFO ################################################################################
2026-03-28 17:14:56,700 T0000: INFO Received traffic: 1000
2026-03-28 17:14:56,700 T0000: INFO Sent traffic: 1000
2026-03-28 17:14:56,700 T0000: INFO 1.0
2026-03-28 17:14:56,700 T0000: INFO TRAFFIC ITEM TI64-HLTAPI_TRAFFICITEM_540 PASSED
2026-03-28 17:14:56,700 T0000: INFO Executing: ixiangpf.traffic_stats {'mode': 'traffic_item', 'streams': 'TI65-HLTAPI_TRAFFICITEM_540', 'csv_path': '/run_logs/Multi-Homing-Master_g200_v6_Mar28_38714_128_MH_set2/results_2026_03_28_09_55_17_tgen'}
2026-03-28 17:15:05,241 T0000: INFO
2026-03-28 17:15:05,241 T0000: INFO ################################################################################
2026-03-28 17:15:05,241 T0000: INFO # TRAFFIC ITEM TI65-HLTAPI_TRAFFICITEM_540 #
2026-03-28 17:15:05,241 T0000: INFO ################################################################################
2026-03-28 17:15:05,241 T0000: INFO Received traffic: 0
2026-03-28 17:15:05,241 T0000: INFO Sent traffic: 1000
2026-03-28 17:15:05,241 T0000: INFO 0.0
2026-03-28 17:15:05,241 T0000: INFO TRAFFIC ITEM TI65-HLTAPI_TRAFFICITEM_540 FAILED
2026-03-28 17:15:05,241 T0000: INFO
2026-03-28 17:15:05,241 T0000: INFO ################################################################################
2026-03-28 17:15:05,242 T0000: INFO # l2_traffic : True, l3_traffic : True #
2026-03-28 17:15:05,242 T0000: INFO ################################################################################
2026-03-28 17:15:05,242 T0000: AUDIT [D2-spine3

[devin-ai-integration]

@vallabh78 Fixed in 355486699. L3 streams now use inter-VLAN routing: source IP on vlan13, destination IP is the host's L2 IP on vlan12.

Before (wrong): Both L3 src1 and src2 used l3_dst_ipv4 (80.13.0.50) — same vlan13 subnet as source.

After (correct): L3 dst IPs are derived from the host's L2 address on vlan12:

• l3_src1 dst_ip = stream_info['src1']['ip_dst'] (host IP at dest1, on vlan12)
• l3_src2 dst_ip = stream_info['src2']['ip_dst'] (host IP at dest2, on vlan12)

So traffic enters on vlan13 and is routed to vlan12 via L3VNI where the host is learned. When host is at dest1: L2 to dest1 passes, L2 to dest2 fails, L3 to dest1 passes, L3 to dest2 fails — same pattern as your sample output (TI62 pass, TI63 fail, TI64 pass, TI65 fail).

Also removed the now-unused l3_dst_ipv4/l3_dst_ipv6 config from both YAML and Python defaults since dst IPs are derived at runtime from the host's L2 IP.

[vallabh78]

@vallabh78 Fixed in 355486699. L3 streams now use inter-VLAN routing: source IP on vlan13, destination IP is the host's L2 IP on vlan12.

Before (wrong): Both L3 src1 and src2 used l3_dst_ipv4 (80.13.0.50) — same vlan13 subnet as source.

After (correct): L3 dst IPs are derived from the host's L2 address on vlan12:

• l3_src1 dst_ip = stream_info['src1']['ip_dst'] (host IP at dest1, on vlan12)
• l3_src2 dst_ip = stream_info['src2']['ip_dst'] (host IP at dest2, on vlan12)

So traffic enters on vlan13 and is routed to vlan12 via L3VNI where the host is learned. When host is at dest1: L2 to dest1 passes, L2 to dest2 fails, L3 to dest1 passes, L3 to dest2 fails — same pattern as your sample output (TI62 pass, TI63 fail, TI64 pass, TI65 fail).

Also removed the now-unused l3_dst_ipv4/l3_dst_ipv6 config from both YAML and Python defaults since dst IPs are derived at runtime from the host's L2 IP.

So what will be src ip, dest1_ip, dest2_ip, src-mac,dst mac look like for l2 and l3 traffic streams

[devin-ai-integration]

@vallabh78 Here are the concrete values for orphan_to_orphan_within_dc + mac+ipv4 (sf=0x12, ht1=0x02):

L2 streams (vlan 12):

Stream	src_mac	dst_mac	src_ip	dst_ip	vlan
src1 (to dest1)	02:00:00:04:12:99	02:00:00:04:12:02	80.12.0.99	80.12.0.180	12
src2 (to dest2)	02:00:00:04:12:99	02:00:00:04:12:02	80.12.0.99	80.12.0.180	12
dest1 (learn)	02:00:00:04:12:02	ff:ff:ff:ff:ff:ff	—	—	12
dest2 (learn)	02:00:00:04:12:02	ff:ff:ff:ff:ff:ff	—	—	12

L3 streams (vlan 13 → routed to vlan 12):

Stream	src_mac	dst_mac	src_ip	dst_ip	vlan
l3_src1 (to dest1)	02:00:00:13:12:99	02:00:00:13:12:02	80.13.0.99	80.12.0.180	13
l3_src2 (to dest2)	02:00:00:13:12:99	02:00:00:13:12:02	80.13.0.99	80.12.0.180	13

How it works:

• L2 src1/src2: sent from leaf2_dc1 on vlan12, forwarded via L2VNI to dest1/dest2 host
• L3 l3_src1/l3_src2: sent from leaf2_dc1 on vlan13, switch routes from vlan13→vlan12 (both in Vrf101), then forwarded to dest1/dest2 host on vlan12
• L3 dst_ip (80.12.0.180) = same host IP as L2 dst_ip — the host is on vlan12, traffic just enters on different VLANs

When host is at dest1:

• L2 src1: PASS (1000/1000), L2 src2: FAIL (0/1000)
• L3 l3_src1: PASS (1000/1000), L3 l3_src2: FAIL (0/1000)

For ipv4_changes (ht1=0x08, ht2=0x08, base+2=182, base+3=183): dest1_ip=80.12.0.182, dest2_ip=80.12.0.183 — different IPs for dest1 vs dest2, so L3 streams also get different dst_ips.

[vallabh78]

dst_mac in case of l3 should be rmac correct

[devin-ai-integration]

@vallabh78 Yes, you're right — fixed in d85b4b2. L3 stream mac_dst now uses the SAG gateway RMAC 00:11:22:33:44:55 (from config_sag_mac()) instead of the fabricated 02:00:00:13:xx:xx MAC. Since L3 traffic is routed (vlan13→vlan12), the Ethernet dst_mac must be the router MAC so the switch performs L3 routing lookup.

Added l3_rmac as a configurable field in _get_dci_mac_move_cfg() with default 00:11:22:33:44:55, so it can be overridden in YAML if needed.

Updated L3 stream table:

Stream	src_mac	dst_mac	src_ip	dst_ip	vlan
l3_src1	02:00:00:13:12:99	00:11:22:33:44:55	80.13.0.99	80.12.0.180	13
l3_src2	02:00:00:13:12:99	00:11:22:33:44:55	80.13.0.99	80.12.0.180	13

[vallabh78]

l2 and l3 needs to be together. not like this. This means only if l2 passed then host move is correct and l3 we are just running. dont create step 3b In one step run l2 and l3 to dest1 and dest2 .
2026-04-09 02:28:37,350 T0000: INFO ################################################################################
2026-04-09 02:28:37,350 T0000: INFO # ipv6_changes : traffic passed as expected when no host move #
2026-04-09 02:28:37,350 T0000: INFO ################################################################################
2026-04-09 02:28:37,351 T0000: INFO
2026-04-09 02:28:37,351 T0000: INFO --- PHASE 3b: L3VNI traffic BEFORE move (host at leaf0_dc1) ---
2026-04-09 02:28:37,351 T0000: INFO
2026-04-09 02:28:37,351 T0000: INFO ################################################################################
2026-04-09 02:28:37,351 T0000: INFO # Verify L3VNI traffic before host move (vlan13 inline streams) #
2026-04-09 02:28:37,351 T0000: INFO ################################################################################
2026-04-09 02:28:37,351 T0000: INFO Executing: ixiangpf.traffic_control {'action': 'run', 'handle': 'TI156-HLTAPI_TRAFFICITEM_540', 'max_wait_timer': 180}
2026-04-09 02:28:41,231 T0000: INFO Executing: ixiangpf.traffic_control {'action': 'run', 'handle': 'TI157-HLTAPI_TRAFFICITEM_540', 'max_wait_timer': 180}
2026-04-09 02:28:42,417 T0000: INFO Sleep for 5 sec(s)... @6576
2026-04-09 02:28:47,423 T0000: INFO Executing: ixiangpf.traffic_control {'action': 'stop', 'handle': 'TI156-HLTAPI_TRAFFICITEM_540', 'max_wait_timer': 180}
2026-04-09 02:28:48,604 T0000: INFO Executing: ixiangpf.traffic_control {'action': 'stop', 'handle': 'TI157-HLTAPI_TRAFFICITEM_540', 'max_wait_timer': 180}
2026-04-09 02:28:54,310 T0000: INFO Sleep for 5 sec(s)... @6583
2026-04-09 02:28:59,316 T0000: INFO Executing: ixiangpf.traffic_stats {'mode': 'traffic_item', 'streams': 'TI156-HLTAPI_TRAFFICITEM_540', 'csv_path': '/run_logs/DCI_L3VNI_Apr08_39033_mm_4/results_2026_04_08_18_10_25_tgen'}
2026-04-09 02:29:02,160 T0000: INFO
2026-04-09 02:29:02,160 T0000: INFO ################################################################################
2026-04-09 02:29:02,160 T0000: INFO # TRAFFIC ITEM TI156-HLTAPI_TRAFFICITEM_540 #
2026-04-09 02:29:02,160 T0000: INFO ################################################################################
2026-04-09 02:29:02,160 T0000: INFO Received traffic: 1000
2026-04-09 02:29:02,160 T0000: INFO Sent traffic: 1000
2026-04-09 02:29:02,160 T0000: INFO 1.0
2026-04-09 02:29:02,160 T0000: INFO TRAFFIC ITEM TI156-HLTAPI_TRAFFICITEM_540 PASSED
2026-04-09 02:29:02,160 T0000: INFO Executing: ixiangpf.traffic_stats {'mode': 'traffic_item', 'streams': 'TI157-HLTAPI_TRAFFICITEM_540', 'csv_path': '/run_logs/DCI_L3VNI_Apr08_39033_mm_4/results_2026_04_08_18_10_25_tgen'}
2026-04-09 02:29:04,957 T0000: INFO
2026-04-09 02:29:04,957 T0000: INFO ################################################################################
2026-04-09 02:29:04,958 T0000: INFO # TRAFFIC ITEM TI157-HLTAPI_TRAFFICITEM_540 #
2026-04-09 02:29:04,958 T0000: INFO ################################################################################
2026-04-09 02:29:04,958 T0000: INFO Received traffic: 0
2026-04-09 02:29:04,958 T0000: INFO Sent traffic: 1000
2026-04-09 02:29:04,958 T0000: INFO 0.0
2026-04-09 02:29:04,958 T0000: INFO TRAFFIC ITEM TI157-HLTAPI_TRAFFICITEM_540 FAILED
2026-04-09 02:29:04,958 T0000: INFO L3 stream to leaf0_dc1 (host here): PASS L3 stream to leaf0_dc2 (host away): FAIL
2026-04-09 02:29:04,958 T0000: INFO
2026-04-09 02:29:04,958 T0000: INFO --- PHASE 4: Move host from leaf0_dc1 to leaf0_dc2 ---
2026-04-09 02:29:04,958 T0000: INFO Executing: ixiangpf.test_control {'action': 'stop_protocol', 'handle': '/topology:1/deviceGroup:26'}
ixiangpf:info: [04/08/2026 19:27:53 info] COMPLETED: test_control : PASS
2026-04-09 02:29:04,976 T0000: INFO Sleep for 2 sec(s)... @7030
2026-04-09 02:29:06,979 T0000: INFO Executing: ixiangpf.test_control {'action': 'start_protocol', 'handle': '/topology:4/deviceGroup:24'}
ixiangpf:info: [04/08/2026 19:27:55 info] COMPLETED: test_control : PASS
2026-04-09 02:29:07,000 T0000: INFO Sleep for 5 sec(s)... @7032
2026-04-09 02:29:12,005 T0000: INFO Stopped dest1 protocol, started

[devin-ai-integration]

@vallabh78 Fixed in e784a4c. L2 and L3 streams are now sent together in one _send_traffic_dci() call and validated together — no more separate phase 3b/6b/9b.

At each traffic phase (3, 6, 9), all 4 streams (L2 src1, L2 src2, L3 src1, L3 src2) run simultaneously, and the test only passes if both L2 and L3 results are correct. If either L2 or L3 fails, the host move is reported as failed.

Example log output will now look like:

--- PHASE 3: L2+L3 traffic BEFORE move (host at leaf0_dc1) ---
  L2 to leaf0_dc1 (host here): PASS  [expect PASS]
  L2 to leaf0_dc2 (host away): FAIL  [expect FAIL]
  L3 to leaf0_dc1 (host here): PASS  [expect PASS]
  L3 to leaf0_dc2 (host away): FAIL  [expect FAIL]
  Traffic check: PASS

Net change: -20 lines (removed 72 lines of separate L3 phases, added 52 lines of merged logic).

[vallabh78]

test_dci_mac_move_orphan_within_dc_mac_and_ipv4 and test_dci_mac_move_orphan_within_dc_mac_and_ipv6 we only test L2 traffic not L3. can you check and if not add it. We can keep the same source but different vlan for L3 which is vlan 13

[devin-ai-integration]

@vallabh78 L3 traffic on vlan13 is already included for test_dci_mac_move_orphan_within_dc_mac_and_ipv4 and test_dci_mac_move_orphan_within_dc_mac_and_ipv6.

The L3 stream creation block in get_stream_handles_dci() (lines 6887-6934) runs for all non-mac_only host types regardless of move_dir. Both tests use mac+ipv4 / mac+ipv6 host types, so they enter the else branch at line 6802 which creates:

1. L2 streams on vlan12 (src1, src2)
2. L3 streams on vlan13 (l3_src1, l3_src2) — same source port, vlan13, src IP 80.13.0.99, dst IP = host's vlan12 IP, dst_mac = RMAC 00:11:22:33:44:55

Then in verify_mac_move_dci(), has_l3 is True so all 4 streams (L2+L3) are sent and validated together at phases 3, 6, 9.

The log output from your earlier run (comment #4211083234) actually confirms this — it shows "PHASE 3b: L3VNI traffic BEFORE move" for the ipv6_changes test which uses the same code path. With the latest commit (e784a4c), L3 is merged into the same phase as L2, so you should see:

--- PHASE 3: L2+L3 traffic BEFORE move (host at leaf0_dc1) ---

Could you pull the latest and re-run orphan_within_dc_mac_and_ipv4? It should show L2+L3 together in phases 3, 6, 9.