Skip to content

Add L3VNI DCI test cases: TC48, TC54, TC55, TC56#20

Open
bpar9 wants to merge 1 commit into
masterfrom
devin/1778297301-l3vni-dci-tc47-56
Open

Add L3VNI DCI test cases: TC48, TC54, TC55, TC56#20
bpar9 wants to merge 1 commit into
masterfrom
devin/1778297301-l3vni-dci-tc47-56

Conversation

@bpar9
Copy link
Copy Markdown
Owner

@bpar9 bpar9 commented May 9, 2026

Description of PR

Summary: Adds 4 new cross-DC L3VNI test cases from the DCI Solution Testplan, plus supporting helper functions in vxlan_helper.py.

Test cases added to test_vxlan_dci.py (in TestVxlanDCIBase):

  • L3VNI_dci:48 (test_base_dci_l3vni_external_intf_flap): External router interface shut/no-shut flap test. Configures VRF-lite eBGP peering (same as TC47), runs continuous traffic, flaps the ext-facing interface 3× with 15s hold, verifies BGP re-establishment and traffic recovery. Parametrized for IPv4/IPv6.
  • L3VNI_dci:54 (test_base_dci_l3vni_line_rate_performance): Runs cross-DC L3VNI traffic at 100% line rate for 1 hour. Verifies 0% loss and max latency ≤ 100µs. Includes periodic core-file health checks.
  • L3VNI_dci:55 (test_base_dci_l3vni_convergence_time): Measures convergence time after trigger events. Parametrized over: link_shut (≤50ms), bgp_clear (≤500ms), bgw_reboot (≤30s), process_kill_bgp (≤5s).
  • L3VNI_dci:56 (test_base_dci_l3vni_imix_traffic): IMIX traffic pattern (64B weight-7, 512B weight-4, 1518B weight-1) for 5 minutes with per-frame-size latency thresholds.

Helper functions added to vxlan_helper.py:

  • snapshot_interface_counters / compare_interface_counters: Capture and diff DUT interface error/drop counters
  • compute_convergence_ms: Calculate convergence time from Ixia traffic stats (prefers loss_duration_us, falls back to frames_lost/PPS)
  • get_traffic_item_stats: Detailed per-stream stats (tx/rx pkts, PPS, loss%, latency)
  • pick_cross_dc_l3_stream: Select a cross-DC L3 stream from tgen handles

Note: TC47 and TC49 (external router IPv4/IPv6 connectivity) already exist as test_base_dci_l3vni_external_connectivity.

Type of change

  • New Test case
    • Skipped for non-supported platforms

Back port request

  • 202205
  • 202305
  • 202311
  • 202405
  • 202411
  • 202505
  • 202511

Approach

What is the motivation for this PR?

Expand cross-DC L3VNI DCI test coverage to include interface flap resilience and performance/convergence benchmarking.

How did you do it?

Followed existing test patterns in TestVxlanDCIBase (e.g., test_base_dci_l3vni_external_connectivity for TC48 setup, existing traffic verification patterns for performance tests). All tests call report_result(), use st.banner()/st.log() conventions, and perform core-file checks.

How did you verify/test it?

  • Python syntax verification (py_compile) passes for both files
  • flake8 clean on the new code (existing pre-existing warnings untouched)
  • Not executed against a live testbed (requires IXIA + 3-DC topology)

Any platform specific information?

Requires IXIA (IxNetwork) traffic generator and the 3-DC EVPN-VXLAN DCI topology.

Supported testbed topology if it's a new test case?

3-datacenter EVPN-VXLAN DCI fabric: DC1 (4 leafs + 2 spines + 2 BGW spines), DC2 (2 leafs + 2 BGW spines), DC3 (1 leaf + 1 BGW spine), with external router connected to DC1/DC2 leaves.

Documentation

Refer to DCI_Solution_Testplan.xlsx for test case specifications.


Reviewer Notes — Areas to scrutinize

  1. snapshot_interface_counters parsing logic (vxlan_helper.py): The counter parsing searches for header-like strings (RX_ERR, TX_DRP) in data rows — verify this matches actual SONiC show interfaces counters output format on target devices.
  2. TC48 code duplication with TC47: The external router discovery and VRF-lite configuration is largely duplicated from test_base_dci_l3vni_external_connectivity. Consider whether a shared setup helper would be better.
  3. Convergence thresholds (TC55): link_shut ≤ 50ms is aggressive — verify this is achievable on target hardware/topology.
  4. Ixia stat field assumptions: get_traffic_item_stats assumes avg_delay/max_delay are in nanoseconds (divides by 1000 for µs). Confirm this matches the IxNetwork version in use.
  5. TC55 uses st.vtysh()/st.config() directly for trigger actions rather than vxlan_obj.config_dut() — intentional for simplicity but worth noting for consistency.

Link to Devin session: https://cisco-demo.devinenterprise.com/sessions/1849ed21f9c04ca1a02f8329683d1afd
Requested by: @bpar9

- TC48 (test_base_dci_l3vni_external_intf_flap): External router interface
  shut/no-shut flap test with VRF-lite eBGP peering validation
- TC54 (test_base_dci_l3vni_line_rate_performance): Line rate L3VNI traffic
  across DCI for 1 hour with 0-loss and latency verification
- TC55 (test_base_dci_l3vni_convergence_time): Convergence time measurement
  parametrized over link_shut/bgp_clear/bgw_reboot/process_kill_bgp triggers
- TC56 (test_base_dci_l3vni_imix_traffic): IMIX traffic pattern (64B/512B/1518B)
  with per-frame-size latency thresholds

Helper functions added to vxlan_helper.py:
- snapshot_interface_counters: Capture DUT interface error/drop counters
- compare_interface_counters: Compute counter deltas between snapshots
- compute_convergence_ms: Calculate convergence time from Ixia stats
- get_traffic_item_stats: Get detailed per-stream traffic statistics
- pick_cross_dc_l3_stream: Select cross-DC L3 stream from tgen handles
@devin-ai-integration
Copy link
Copy Markdown

🤖 Devin AI Engineer

I'll be helping with this pull request! Here's what you should know:

✅ I will automatically:

  • Address comments on this PR. Add '(aside)' to your comment to have me ignore it.
  • Look at CI failures and help fix them

Note: I can only respond to comments from users who have write access to this repository.

⚙️ Control Options:

  • Disable automatic comment and CI monitoring

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant