Adding a fixture to make sure socat is copied to all the duts and con…#23595
Closed
rraghav-cisco wants to merge 411 commits into
Closed
Adding a fixture to make sure socat is copied to all the duts and con…#23595rraghav-cisco wants to merge 411 commits into
rraghav-cisco wants to merge 411 commits into
Conversation
Collaborator
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
rraghav-cisco
force-pushed
the
copy-socat
branch
from
e0b2dc8 to
f3a336e
Compare
Collaborator
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
lizhijianrd
previously approved these changes
Apr 7, 2026
Collaborator
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
rraghav-cisco
force-pushed
the
copy-socat
branch
from
d50d571 to
f82afb5
Compare
Collaborator
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
Contributor
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
Contributor
Author
|
@lizhijianrd , pls go ahead and review this, it is good to go. |
Contributor
Author
|
azpw /run |
Collaborator
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
Contributor
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
Collaborator
|
/azp run |
|
Azure Pipelines successfully started running 1 pipeline(s). |
….yml (sonic-net#24400) <!-- Please make sure you've read and understood our contributing guidelines; https://github.com/sonic-net/SONiC/blob/gh-pages/CONTRIBUTING.md Please provide following information to help code review process a bit easier: --> ### Description of PR <!-- - Please include a summary of the change and which issue is fixed. - Please also include relevant motivation and context. Where should reviewer start? background context? - List any dependencies that are required for this change. --> Summary: Fixes sonic-net#24399 ### Type of change <!-- - Fill x for your type of change. - e.g. - [x] Bug fix --> - [ ] Bug fix - [x] Testbed and Framework(new/improvement) - [ ] New Test case - [ ] Skipped for non-supported platforms - [ ] Test case improvement ### Back port request - [ ] 202205 - [ ] 202305 - [ ] 202311 - [ ] 202405 - [ ] 202411 - [ ] 202505 - [ ] 202511 ### Approach #### What is the motivation for this PR? Sometimes we don't have a DUT reachable image-url and have to use local image file to upgrade SONiC DUT #### How did you do it? Add option in `upgrade_sonic.yml` to SCP file onto all listed DUTs and run `sudo sonic-installer install` - similar to passing image-url #### How did you verify/test it? Tested it on T2 device in lab successfully. #### Any platform specific information? N/A #### Supported testbed topology if it's a new test case? N/A ### Documentation <!-- (If it's a new feature, new test case) Did you update documentation/Wiki relevant to your implementation? Link to the wiki page? --> N/A Signed-off-by: Javier-Tan <47554099+Javier-Tan@users.noreply.github.com> Signed-off-by: Raghavendran Ramanathan <rraghav@cisco.com>
…onic-net#24379) <!-- Please make sure you've read and understood our contributing guidelines; https://github.com/sonic-net/SONiC/blob/gh-pages/CONTRIBUTING.md Please provide following information to help code review process a bit easier: --> ### Description of PR <!-- - Please include a summary of the change and which issue is fixed. - Please also include relevant motivation and context. Where should reviewer start? background context? - List any dependencies that are required for this change. --> Summary: Cisco console platform has no "Ethernet0". test_neighbor_mac.py looks for this and skips, since it is not present. In this PR, we change it to Ethernet1. ### Type of change <!-- - Fill x for your type of change. - e.g. - [x] Bug fix --> - [X] Bug fix - [ ] Testbed and Framework(new/improvement) - [ ] New Test case - [ ] Skipped for non-supported platforms - [ ] Test case improvement ### Back port request - [ ] 202205 - [ ] 202305 - [ ] 202311 - [ ] 202405 - [ ] 202411 - [ ] 202505 - [X] 202511 ### Approach #### What is the motivation for this PR? To enable test_neighbor_mac for cisco console platform. #### How did you do it? Check for the platform type and enable Ethernet1 instead of Ethernet0. #### How did you verify/test it? Ran it on our testbeds: ```====================================================================================================== PASSES ======================================================================================================= ________________________________________________________________________________________ TestNeighborMac.testNeighborMac[0] _________________________________________________________________________________________ ________________________________________________________________________________________ TestNeighborMac.testNeighborMac[1] _________________________________________________________________________________________ ------------------------------------------------------------- generated xml file: /run_logs/c0lo/neighbor/2026-05-04-18-15-08/arp/test_neighbor_mac.xml ------------------------------------------------------------- INFO:root:Can not get Allure report URL. Please check logs ============================================================================================== short test summary info ============================================================================================== PASSED arp/test_neighbor_mac.py::TestNeighborMac::testNeighborMac[0] PASSED arp/test_neighbor_mac.py::TestNeighborMac::testNeighborMac[1] ===================================================================================== 2 passed, 1 warning in 224.45s (0:03:44) ====================================================================================== sonic@arctos_cicd_all_202603:/data/tests$ ``` #### Any platform specific information? Specific to cisco console platform. Signed-off-by: Raghavendran Ramanathan <rraghav@cisco.com>
…d of exact match for platform name. (sonic-net#24368) <!-- Please make sure you've read and understood our contributing guidelines; https://github.com/sonic-net/SONiC/blob/gh-pages/CONTRIBUTING.md Please provide following information to help code review process a bit easier: --> ### Description of PR <!-- - Please include a summary of the change and which issue is fixed. - Please also include relevant motivation and context. Where should reviewer start? background context? - List any dependencies that are required for this change. --> Summary: There is a timeout increase for cisco console server in console_loopback and console_link_wiring tests. Unfortunately the if condition for platform fails sometimes - the platform name contains a revision tag at the end, that might not be same for all devices. To fix this, in this PR, the platform check is changed to "startswith" function instead of exact match. ### Type of change <!-- - Fill x for your type of change. - e.g. - [x] Bug fix --> - [X] Bug fix - [ ] Testbed and Framework(new/improvement) - [ ] New Test case - [ ] Skipped for non-supported platforms - [ ] Test case improvement ### Back port request - [ ] 202205 - [ ] 202305 - [ ] 202311 - [ ] 202405 - [ ] 202411 - [ ] 202505 - [x] 202511 ### Approach #### What is the motivation for this PR? The platform check failing due to different revision number for platform. #### How did you do it? Changed the exact match check to "startswith" check. #### How did you verify/test it? Ran it in our testbeds: | console/test_console_availability.py | 0 | 0 | 0 | 4 | 4 | |--------------------------------------|----|---|---|---|----| | console/test_console_driver.py | 1 | 0 | 0 | 0 | 1 | | console/test_console_link_wiring.py | 47 | 1 | 0 | 0 | 48 | | console/test_console_loopback.py | 8 | 0 | 0 | 0 | 8 | | console/test_console_reversessh.py | 7 | 0 | 0 | 0 | 7 | | console/test_console_stress.py | 0 | 0 | 0 | 8 | 8 | #### Any platform specific information? Specific to cisco console. Signed-off-by: Raghavendran Ramanathan <rraghav@cisco.com>
…ep with wait_until; sync from internal (sonic-net#24195) <!-- Please make sure you've read and understood our contributing guidelines; https://github.com/sonic-net/SONiC/blob/gh-pages/CONTRIBUTING.md Please provide following information to help code review process a bit easier: --> ### Description of PR <!-- - Please include a summary of the change and which issue is fixed. - Please also include relevant motivation and context. Where should reviewer start? background context? - List any dependencies that are required for this change. --> Summary: 1. Commit 650b820 - Sync container_upgrade/ from internal 2. Commit 26eb7ff - wait_until replaces time.sleep and fix both k8s and original containers co-exist Fixes # (issue) ### Type of change <!-- - Fill x for your type of change. - e.g. - [x] Bug fix --> - [x] Bug fix - [ ] Testbed and Framework(new/improvement) - [ ] New Test case - [ ] Skipped for non-supported platforms - [ ] Test case improvement ### Back port request - [ ] 202205 - [ ] 202305 - [ ] 202311 - [ ] 202405 - [ ] 202411 - [ ] 202505 - [ ] 202511 ### Approach #### What is the motivation for this PR? In `pull_run_dockers()`, the condition `name in existing_systemd_services` doesn't match for k8s containers because the container names from `container_name_mapping` (e.g., `k8s_telemetry_ds`) do not match `existing_systemd_services = ["telemetry"]`. This means the `telemetry` systemd service is never stopped when deploying k8s replacements, resulting in both running simultaneously. Replace `time.sleep` with `wait_until` when checking `is_container_running`, polls every 5 seconds for up to 30 seconds with a 5-second initial delay #### How did you do it? #### How did you verify/test it? #### Any platform specific information? #### Supported testbed topology if it's a new test case? ### Documentation <!-- (If it's a new feature, new test case) Did you update documentation/Wiki relevant to your implementation? Link to the wiki page? --> --------- Signed-off-by: Mai Bui <maibui@microsoft.com> Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Signed-off-by: Raghavendran Ramanathan <rraghav@cisco.com>
…c-net#23962) <!-- Please make sure you've read and understood our contributing guidelines; https://github.com/sonic-net/SONiC/blob/gh-pages/CONTRIBUTING.md Please provide following information to help code review process a bit easier: --> ### Description of PR <!-- - Please include a summary of the change and which issue is fixed. - Please also include relevant motivation and context. Where should reviewer start? background context? - List any dependencies that are required for this change. --> 1. Currently the testQosSaiHeadroomPoolSize will skip if the chosen port for the qos test is not 120000m plus 400 or 800G. This is not needed on disaggregated T2 skus. 2. The test_qos_sai.py tests validate and replace chosen src and dest ports with ports that exist, but this selection was flawed. It could replace ports with those found on different asics, and it could replace ports with those that did not have the same qos profiles. This PR fixes both problems. Summary: Fixes sonic-net#24001 ### Type of change <!-- - Fill x for your type of change. - e.g. - [x] Bug fix --> - [x] Bug fix - [ ] Testbed and Framework(new/improvement) - [ ] New Test case - [ ] Skipped for non-supported platforms - [ ] Test case improvement ### Back port request - [ ] 202205 - [ ] 202305 - [ ] 202311 - [ ] 202405 - [ ] 202411 - [ ] 202505 - [x] 202511: sonic-net#23963 manual backport ### Approach #### What is the motivation for this PR? Fix the issues mentioned above. #### How did you do it? **Problem 1:** I changed the testQosSaiHeadroomPoolSize logic so it only reduces the src ports and skips for non 120000m profiles on modular chassis. **Problem 2:** 1. I fixed `updateTestPortIdIp` so that any replaced src ports should be from the src asic (same with dest) 2. I fixed `updateTestPortIdIp` so that any replaced src ports must use the same portSpeedCableLength profile as the chosen src port for the test. This is to solve flakiness of these tests where the original src port was invalid, but the new src port is of a different profile and so has different pg headroom values. The qos params would not apply to the new src port and the test would fail. #### How did you verify/test it? Ran testQosSaiHeadroomPoolSize[single_asic], testQosSaiHeadroomPoolSize[single_dut_multi_asic] on q3d and j2c+ skus to verify. #### Any platform specific information? #### Supported testbed topology if it's a new test case? ### Documentation <!-- (If it's a new feature, new test case) Did you update documentation/Wiki relevant to your implementation? Link to the wiki page? --> Signed-off-by: Peter <peterbailey@arista.com> Signed-off-by: Raghavendran Ramanathan <rraghav@cisco.com>
<!-- Please make sure you've read and understood our contributing guidelines; https://github.com/sonic-net/SONiC/blob/gh-pages/CONTRIBUTING.md Please provide following information to help code review process a bit easier: --> ### Description of PR <!-- - Please include a summary of the change and which issue is fixed. - Please also include relevant motivation and context. Where should reviewer start? background context? - List any dependencies that are required for this change. --> Summary: Fixes # (issue) In the HA testbed, 8 DPU are included as part of the DUT list. However in generate_golden_config_db, it is expecting exactly two DUT corresponding to two the NPU / switch: https://github.com/sonic-net/sonic-mgmt/blob/60946b31e0c731108416bfb8603301f6ac6eafbf/ansible/library/generate_golden_config_db.py#L561 This golden config_db is used for NPU side, so the DPU listed in the topo file can be filtered out. If they are not golden config is not applied to the DUT since there are more DUT than expected when the DPU are included. ### Type of change <!-- - Fill x for your type of change. - e.g. - [x] Bug fix --> - [ ] Bug fix - [ ] Testbed and Framework(new/improvement) - [ ] New Test case - [ ] Skipped for non-supported platforms - [ ] Test case improvement ### Back port request - [ ] 202205 - [ ] 202305 - [ ] 202311 - [ ] 202405 - [ ] 202411 - [ ] 202505 - [ ] 202511 ### Approach #### What is the motivation for this PR? Fix dut_list so that golden config is applied as expected. #### How did you do it? Filter out the DPU when populating the duts_list. #### How did you verify/test it? Ran the HA testsuite on HA testbed with this fix, golden config is applied successfully. #### Any platform specific information? #### Supported testbed topology if it's a new test case? ### Documentation <!-- (If it's a new feature, new test case) Did you update documentation/Wiki relevant to your implementation? Link to the wiki page? --> Signed-off-by: dypet <dypeters@cisco.com> Signed-off-by: Raghavendran Ramanathan <rraghav@cisco.com>
<!-- Please make sure you've read and understood our contributing guidelines; https://github.com/sonic-net/SONiC/blob/gh-pages/CONTRIBUTING.md Please provide following information to help code review process a bit easier: --> ### Description of PR <!-- - Please include a summary of the change and which issue is fixed. - Please also include relevant motivation and context. Where should reviewer start? background context? - List any dependencies that are required for this change. --> Enable more route tests for SONiC VPP to increase the t1-lag-vpp coverage: - route/test_route_flow_counter.py - route/test_route_map_check.py The rest `route/` tests have already been enabled for t1-lag-vpp Summary: Fixes # (issue) sonic-net/sonic-buildimage#25772 ### Type of change <!-- - Fill x for your type of change. - e.g. - [x] Bug fix --> - [ ] Bug fix - [ ] Testbed and Framework(new/improvement) - [x] New Test case - [x] Skipped for non-supported platforms - [ ] Test case improvement ### Back port request - [ ] 202205 - [ ] 202305 - [ ] 202311 - [ ] 202405 - [ ] 202411 - [ ] 202505 - [ ] 202511 ### Approach #### What is the motivation for this PR? We want to enable all data plane testing on VPP KVM testbed. #### How did you do it? Enable the skipped route tests and ensure they can pass. #### How did you verify/test it? All enabled route tests have passed <img width="773" height="858" alt="image" src="https://github.com/user-attachments/assets/21c33dd6-e882-400d-858f-da5819aeb209" /> <img width="2705" height="770" alt="image" src="https://github.com/user-attachments/assets/a4858a97-bb82-4009-8a3c-89d966de873a" /> <img width="2180" height="946" alt="image" src="https://github.com/user-attachments/assets/b3e3d7a8-b5d2-4e50-8c94-90c6b5f1e199" /> #### Any platform specific information? #### Supported testbed topology if it's a new test case? t1-lag-vpp ### Documentation <!-- (If it's a new feature, new test case) Did you update documentation/Wiki relevant to your implementation? Link to the wiki page? --> Signed-off-by: Chenyang Wang <chenyangw233@gmail.com> Signed-off-by: Raghavendran Ramanathan <rraghav@cisco.com>
Add golden_config_db generation support for UT2 (single-node T2) topology, including multi-ASIC awareness for ZMQ and zebra_nexthop config updates. Signed-off-by: Raghavendran Ramanathan <rraghav@cisco.com>
### Description of PR
Summary:
Fix `test_pfcwd_interval_config_updates` failure on T2 topologies due to
incorrect DUT selection.
Fixes: MIGSOFTWAR-37212
**Root Cause**: On T2 topologies, the testbed has both supervisor nodes
and linecard nodes. The test was using `duthost` directly, which could
target a supervisor node. Supervisors have no frontend ASICs with
Ethernet interfaces or PFC_WD configuration, causing:
1. `enum_rand_one_frontend_asic_index` returns `None` (no frontend
ASICs)
2. Namespace prefix fixtures resolve to empty strings (wrong namespace)
3. `show pfcwd config` returns no Ethernet interfaces
4. `get_detection_restoration_times()` loop finds no interfaces, falls
through
5. Function returns `None` instead of `(detection_time,
restoration_time)`
6. Caller fails with `TypeError: cannot unpack non-iterable NoneType
object`
**Fix**:
1. Use `rand_one_dut_front_end_hostname` instead of `duthost` to ensure
the test targets a linecard (data-plane node) on T2, or the regular DUT
on non-T2 topologies
2. Fix `pytest_assert(True, ...)` -> `pytest_assert(False, ...)` to
properly fail when no interfaces are found (was a no-op before)
### Type of change
- [x] Bug fix
- [ ] Testbed and Framework(new/improvement)
- [ ] New Test case
- [ ] Skipped for non-supported platforms
- [ ] Test case improvement
### Back port request
- [ ] 202205
- [ ] 202305
- [ ] 202311
- [ ] 202405
- [ ] 202411
- [ ] 202505
- [x] 202511
### Approach
#### What is the motivation for this PR?
`test_pfcwd_interval_config_updates` fails on T2 topologies (e.g.,
T2-TITAN) with `TypeError: cannot unpack non-iterable NoneType object`
because the test targets a supervisor node that has no PFC_WD
configuration.
#### How did you do it?
- Changed fixtures and test function to use
`duthosts[rand_one_dut_front_end_hostname]` instead of `duthost`
- This fixture automatically selects a linecard on T2 topologies or the
regular DUT on other topologies
- Fixed the fallback assertion in `get_detection_restoration_times()` to
properly fail instead of silently returning None
- Applied flake8 formatting fixes throughout the file
- Added module-level documentation explaining the T2 topology handling
- Slight refactoring to make code more DRY (avoiding multiple
assignments to _duthost_ )
#### How did you verify/test it?
- Verified fix logic against error trace in MIGSOFTWAR-37212
- Confirmed `rand_one_dut_front_end_hostname` behavior in conftest.py
- Verified flake8 compliance
- Tested on a T2 testbed
#### Any platform specific information?
Affects T2 topologies where supervisor and linecard nodes are separate.
The fix is backward-compatible with all other topologies.
#### Supported testbed topology if it's a new test case?
N/A - bug fix for existing test. Test remains marked
`@pytest.mark.topology('any')`.
### Documentation
Added module-level docstring explaining T2 topology handling and inline
comment for the assertion fix.
---------
Signed-off-by: Dan Caugherty <dcaugher@cisco.com>
Signed-off-by: Raghavendran Ramanathan <rraghav@cisco.com>
…) (sonic-net#24317) ### Description of PR Summary: Follow-up to PR sonic-net#24212. Addresses two items from issue sonic-net#24236: 1. **Narrow port scope** — fixture now reads `hdrm_pool_size.src_port_ids` and `dst_port_id` from `dutQosConfig` and only applies ACL/LLDP changes to those ports, instead of all `dutInterfaces`. This reduces fanout setup/teardown time significantly on testbeds with many interfaces. 2. **Partial SONiC fanout support (LLDP container stop)** — fixture now dispatches by fanout OS: - EOS fanout: existing path unchanged (egress MAC ACL + `no lldp transmit/receive`) - SONiC fanout: stop LLDP container on the fanout to suppress fanout-self originated LLDP Full SONiC VM-noise filtering is deferred (see issue sonic-net#24236) because Broadcom SONiC does not support egress ACL, requiring a different design. Fixes # (issue) — partial fix for sonic-net#24236. ### Type of change - [x] Bug fix ### Back port request - [ ] 202205 - [ ] 202305 - [ ] 202311 - [ ] 202405 - [x] 202411 - [x] 202505 - [x] 202511 ### Approach #### What is the motivation for this PR? Two follow-up items left over from PR sonic-net#24212: 1. The original implementation applied ACL/LLDP changes to **all** `dutInterfaces` ports (could be 32+), which made setup/teardown unnecessarily slow. 2. SONiC fanouts (a significant portion of fleet fanouts) were silently skipped, so testbeds with SONiC fanouts received no protection. #### How did you do it? **Port scope optimization**: Added `dutQosConfig` to fixture parameters. The fixture now constructs `protected_port_ids` from `hdrm_pool_size.src_port_ids` and `dst_port_id`. Falls back to all `dutInterfaces` if those are unavailable. **SONiC fanout LLDP-stop**: Refactored the per-port loop to dispatch by `fanout.get_fanout_os()`: - `_apply_eos_filter()` retains the existing EOS logic - `_apply_sonic_filter()` runs `docker stop lldp` once per SONiC fanout **Why not full SONiC ACL coverage in this PR**: Broadcom SONiC does not support egress ACL (verified in `tests/acl/test_acl.py:660-664`). The EOS approach (egress MAC ACL on DUT-facing port) cannot be directly replicated. Applying ingress ACL on the DUT-facing port would filter the wrong direction (DUT->fanout, blocking PFC test traffic). Properly applying ingress ACL on VM-facing ports requires multi-tier topology discovery, which is out of scope here and tracked as a follow-up in sonic-net#24236. **Other improvements**: - Restricted `device_conn` iteration to `src_dut.hostname` to avoid touching unrelated DUTs in multi-DUT topologies - Added rc check on `docker stop lldp` so we do not record success on actual failure #### How did you verify/test it? - `py_compile` passes - `flake8 --max-line-length=120` clean (full file scan, no narrow grep) - EOS path is logic-preserving from the merged PR sonic-net#24212; relies on existing validation - SONiC LLDP-stop path is a single-command operation with rc check; safe by construction End-to-end SONiC fanout validation is pending and tracked in issue sonic-net#24236. #### Any platform specific information? - Affects testbeds running `testQosSaiHeadroomPoolSize` / `testQosSaiHeadroomPoolWatermark` on t1-lag with non-SONiC neighbors - EOS fanout (Broadcom and Arista): full coverage (unchanged from PR sonic-net#24212) - SONiC fanout: partial coverage (LLDP container stopped, VM-noise filtering tracked separately) #### Supported testbed topology if it's a new test case? N/A (existing test improvement). ### Documentation N/A --------- Signed-off-by: Xu Chen <xuchen3@microsoft.com> Signed-off-by: Raghavendran Ramanathan <rraghav@cisco.com>
…onic-net#24354) ### Description of PR Adds the **EgressDrop** probe — the 4th MMU threshold probe, completing coverage of all four buffer threshold dimensions: | Probe | Layer | What it measures | |---|---|---| | ``testQosPfcXoffProbe`` | lossless | PFC XOFF threshold | | ``testQosIngressDropProbe`` | lossless | ingress drop threshold | | ``testQosHeadroomPoolProbe`` | lossless | headroom pool size | | **``testQosEgressDropProbe``** ← this PR | lossy | egress lossy queue drop threshold | Summary: Closes # (none — feature add; companion to MMU probing series PR12 and PR13.ci sonic-net#24319) ### Type of change - [ ] Bug fix - [ ] Testbed and Framework(new/improvement) - [x] New Test case - [ ] Skipped for non-supported platforms - [ ] Test case improvement ### Back port request - [ ] 202205 - [ ] 202305 - [ ] 202311 - [ ] 202405 - [ ] 202411 - [ ] 202505 - [ ] 202511 (Master only — feature add for the probe family that itself only exists on master.) ### Approach #### What is the motivation for this PR? The MMU probing framework (merged via PR12 sonic-net#24024 and earlier) covers 3 of the 4 buffer threshold dimensions. EgressDrop closes the 4th — exactly equivalent in scope to legacy ``testQosSaiLossyQueue`` but using the probing approach: instead of relying on a static ``pkts_num_trig_egr_drp`` value from ``qos.yml`` (which is per-platform/per-SKU/per-cable-length and brittle), the probe **measures the actual threshold at runtime** via a 3-phase upper-bound / lower-bound / threshold-range search. #### How did you do it? **Production code** (3 files, +835 lines): - ``tests/saitests/probe/egress_drop_probing.py`` — PTF test case using **1 src → 1 dst** pattern (the lossy queue overflows on the egress port via cable-length-aware buffer accounting) - ``tests/saitests/probe/egress_drop_probing_executor.py`` — physical executor; polls ``EGRESS_DROP`` counter to detect threshold crossing - ``tests/saitests/probe/sim_egress_drop_probing_executor.py`` — mock executor (5 deterministic scenarios) for UT/IT **Pytest entry** (1 file, +133 lines): - ``tests/qos/test_qos_probe.py`` — ``testQosEgressDropProbe[lossy_queue_1]``, exact parametrize parity with legacy ``testQosSaiLossyQueue`` **Test infrastructure** (4 files, +1523 lines): - 27 unit tests - 23 integration tests - helpers in ``mock/it/probe_test_helper.py`` - module docstring update in ``probe/__init__.py`` **Key design decisions** (surfaced by internal r1→r5 code review): 1. **``queue`` field semantics** — EgressDrop's testParams use ``queue`` (not ``pg``) because it operates at queue level, not PG. Sibling probes keep ``pg`` because they're PG-semantic. 2. **Fail-loud on missing pool size** — ``get_pool_size()`` raises ``RuntimeError`` instead of silently falling back to ``ingress_lossless_pool`` (which would be wrong for egress). 3. **Probe runs even when expected value missing** — uses ``.get()`` for ``pkts_num_trig_egr_drp`` so the probe always proceeds and reports the measured threshold (no skip — the whole point of probing is that we don't need the expected value). #### How did you verify/test it? **Mock UT/IT (this PR runs them via the path-filter trigger from PR sonic-net#24319):** ``` pytest tests/saitests/mock/ut/test_egress_drop_probing.py → 28 PASSED (UT) pytest tests/saitests/mock/it/test_egress_drop_probing.py → 23 PASSED (IT) ``` **Physical** (TK5 lab — Arista 7060CX3 t0-116, ``tbtk5-t0-7260-2``): - ``run_0429`` (baseline, before this PR's review fixes): ``testQosEgressDropProbe[lossy_queue_1]`` PASS - ``run_0430`` (this PR's branch ``9d5717b19c``): 6 PASSED / 44 SKIPPED in 1:53:06 — same 6 PASS as baseline, same SKIP pattern matching legacy ``testQosSaiLossyQueue`` (single-asic + single-dut-multi-asic + multi-dut-{shortlink,longlink}-cross combinations) **Internal code review**: r1 → r2 → r3 → r4 → r5 chain CLOSED. 9 ✅ Resolved + 2 💬 Deferred = 11/11 findings addressed. Deferred items tracked for separate hygiene PR (family-wide ``counter_constants.py`` extraction; sim executor default value cleanup). #### Any platform specific information? - Same support matrix as legacy ``testQosSaiLossyQueue``: ``{TH3, TH4, TH5, TD3, TD5, SPC1/2/3}`` × ``t0`` topologies (t1/dualtor have no lossy queue config in current ``qos.yml``). - Platform-specific cable-length adaptation handled by the existing buffer occupancy controller (no new platform code needed). #### Supported testbed topology if it's a new test case? ``t0`` family only (matches legacy ``testQosSaiLossyQueue``). The skip rules for ``multi-dut`` and ``dualtor`` topologies are inherited from the parent fixture and produce identical SKIP patterns to the legacy test. ### Documentation Updated ``tests/saitests/probe/__init__.py`` docstring to list EgressDrop as the 4th probing test case. Internal design notes are in commit messages and in the PR conversation. ### DCO Signed-off-by: Xu Chen <xuchen3@microsoft.com> --------- Signed-off-by: Xu Chen <xuchen3@microsoft.com> Signed-off-by: Raghavendran Ramanathan <rraghav@cisco.com>
#### Why I did it SONiC with FIPS enabled should validate that the MACsec FIPS POST (Pre-Operational Self-Test) sequence has completed successfully. Currently there is no sonic-mgmt test that checks the `show macsec --post-status` CLI output. This test provides a quick health signal for MACsec/FIPS readiness without depending on traffic or control-plane tests. #### How I did it Added a new test file `tests/macsec/test_fips_post_status.py` that: 1. Checks image-level FIPS flag via `sonic-installer get-fips` 2. If FIPS is not enabled, logs and returns without failing (no-op on non-FIPS images) 3. Cross-checks runtime FIPS flags (`/proc/cmdline` for `sonic_fips=1`, `/etc/fips/fips_enable`) 4. Runs `show macsec --post-status` and verifies all reported modules have `Post_state : pass` The test overrides the autouse MACsec fixtures (`macsec_mtu_adjustment`, `load_macsec_info`) with no-ops since it only inspects DUT-level POST state and does not need MACsec neighbor/control-plane setup. #### How to verify it - Run on a MACsec-capable DUT with FIPS enabled and `--enable_macsec`: ``` pytest tests/macsec/test_fips_post_status.py --enable_macsec ``` - Run on a non-FIPS image: test exits early without failure #### Which release branch to backport - 202511 #### Description of PR - New test case #### Supported testbed topology - t0, t2, t0-sonic (same as the rest of the MACsec test suite) - No PTF traffic or neighbor emulation required Signed-off-by: Rajshekhar Biradar <rajshekhar@nexthop.ai> Signed-off-by: Raghavendran Ramanathan <rraghav@cisco.com>
…sonic-net#23754) <!-- Please make sure you've read and understood our contributing guidelines; https://github.com/sonic-net/SONiC/blob/gh-pages/CONTRIBUTING.md Please provide following information to help code review process a bit easier: --> ### Description of PR <!-- - Please include a summary of the change and which issue is fixed. - Please also include relevant motivation and context. Where should reviewer start? background context? - List any dependencies that are required for this change. --> Summary: Fixes sonic-net#23753 ### Type of change <!-- - Fill x for your type of change. - e.g. - [x] Bug fix --> - [ ] Bug fix - [ ] Testbed and Framework(new/improvement) - [ ] New Test case - [ ] Skipped for non-supported platforms - [x] Test case improvement ### Back port request - [ ] 202205 - [ ] 202305 - [ ] 202311 - [ ] 202405 - [ ] 202411 - [ ] 202505 - [x] 202511 ### Approach #### What is the motivation for this PR? `test_nhop_group_interface_flap` shuts down fanout switch ports during the test but only restores them inside the `try` block. If the test fails or hits an exception before reaching the `no_shutdown` calls, the fanout ports remain down, causing subsequent tests to fail due to missing links. #### How did you do it? Added `fanout.no_shutdown()` calls for all `src_port` entries in the `finally` block, ensuring fanout ports are unconditionally restored during cleanup regardless of test outcome. #### How did you verify/test it? Verified that fanout ports are correctly restored after both test pass and test failure scenarios. #### Any platform specific information? N/A #### Supported testbed topology if it's a new test case? N/A — this is a fix to an existing test case. ### Documentation <!-- (If it's a new feature, new test case) Did you update documentation/Wiki relevant to your implementation? Link to the wiki page? --> --------- Signed-off-by: arista-hpandya <hpandya@arista.com> Signed-off-by: Raghavendran Ramanathan <rraghav@cisco.com>
<!-- Please make sure you've read and understood our contributing guidelines; https://github.com/sonic-net/SONiC/blob/gh-pages/CONTRIBUTING.md Please provide following information to help code review process a bit easier: --> ### Description of PR <!-- - Please include a summary of the change and which issue is fixed. - Please also include relevant motivation and context. Where should reviewer start? background context? - List any dependencies that are required for this change. --> Summary: BGP t0/t1 enhancement to run nightly regression Fixes # (issue) ### Type of change <!-- - Fill x for your type of change. - e.g. - [x] Bug fix --> - [ ] Bug fix - [ ] Testbed and Framework(new/improvement) - [ ] New Test case - [ ] Skipped for non-supported platforms - [ ] Test case improvement ### Back port request - [ ] 202205 - [ ] 202305 - [ ] 202311 - [ ] 202405 - [ ] 202411 - [ ] 202505 - [ ] 202511 ### Approach #### What is the motivation for this PR? BGP t0/t1 enhancement to run nightly regression #### How did you do it? Removes the ports from porchannel or vlan if they are part of them, and make them routed port before start of the test #### How did you verify/test it? Tested with t0/t1 like config from minigraph.xml #### Any platform specific information? #### Supported testbed topology if it's a new test case? t0/t1 ### Documentation <!-- (If it's a new feature, new test case) Did you update documentation/Wiki relevant to your implementation? Link to the wiki page? --> ### Output Output ---------------------------------------------------------------------------------- live log setup ---------------------------------------------------------------------------------- INFO root:init.py:53 Completeness level not set during test execution. Setting to default level: 1 INFO root:init.py:151 Test has no defined levels. Continue without test completeness checks INFO tests.conftest:conftest.py:411 Inventory file: ['../ansible/snappi-sonic'] INFO tests.common.fixtures.ptfhost_utils:ptfhost_utils.py:326 Skip running icmp_responder at session level, it is only for dualtor testbed with active-active mux ports. INFO tests.common.helpers.dut_utils:dut_utils.py:623 dut sonic-s6100-dut1 belongs to groups ['snappi-sonic', 'sonic', 'sonic_dell64_40', 'fanout'] INFO root:dut_utils.py:648 skip empty var file /root/sonic-mgmt/tests/common/helpers/../../../ansible/group_vars/all/corefile_uploader.yml INFO root:dut_utils.py:648 skip empty var file /root/sonic-mgmt/tests/common/helpers/../../../ansible/group_vars/all/env.yml INFO paramiko.transport:transport.py:1938 Connected (version 2.0, client OpenSSH_8.4p1) INFO paramiko.transport:transport.py:1938 Authentication (password) successful! INFO tests.common.plugins.sanity_check:init.py:448 Skip sanity check according to command line argument INFO tests.conftest:conftest.py:2949 Dumping Disk and Memory Space information before test on sonic-s6100-dut1 INFO tests.conftest:conftest.py:2953 Collecting core dumps before test on sonic-s6100-dut1 INFO tests.conftest:conftest.py:2962 Collecting running config before test on sonic-s6100-dut1 INFO tests.conftest:conftest.py:3248 Skipping temporarily_disable_route_check fixture INFO root:conftest.py:1709 Using DUTs ['sonic-s6100-dut1'] in testbed 'vms-snappi-sonic' INFO tests.conftest:conftest.py:706 Randomly select dut sonic-s6100-dut1 for testing INFO root:init.py:69 -------------------- fixture snappi_api_serv_ip setup starts -------------------- INFO root:init.py:76 -------------------- fixture snappi_api_serv_ip setup ends -------------------- INFO root:init.py:69 -------------------- fixture snappi_api_serv_port setup starts -------------------- INFO root:init.py:76 -------------------- fixture snappi_api_serv_port setup ends -------------------- INFO root:init.py:81 -------------------- fixture snappi_api setup starts -------------------- WARNING snappi:snappi.py:106 Version check is disabled INFO root:init.py:85 -------------------- fixture snappi_api setup ends -------------------- INFO root:init.py:69 -------------------- fixture get_snappi_ports setup starts -------------------- INFO tests.common.snappi_tests.snappi_fixtures:snappi_fixtures.py:1622 Selecting snappi port fixture: get_snappi_ports_single_dut INFO tests.conftest:conftest.py:742 Randomly select dut sonic-s6100-dut1 for testing INFO root:init.py:69 -------------------- fixture get_snappi_ports_single_dut setup starts -------------------- INFO root:init.py:76 -------------------- fixture get_snappi_ports_single_dut setup ends -------------------- INFO root:init.py:76 -------------------- fixture get_snappi_ports setup ends -------------------- INFO root:init.py:69 -------------------- fixture tgen_ports setup starts -------------------- WARNING tests.common.snappi_tests.snappi_fixtures:snappi_fixtures.py:623 Removing port Ethernet68 from portchannel PortChannel2 for snappi testing WARNING tests.common.snappi_tests.snappi_fixtures:snappi_fixtures.py:623 Removing port Ethernet72 from portchannel PortChannel3 for snappi testing WARNING tests.common.snappi_tests.snappi_fixtures:snappi_fixtures.py:623 Removing port Ethernet76 from portchannel PortChannel4 for snappi testing INFO tests.common.snappi_tests.snappi_fixtures:snappi_fixtures.py:733 Pre-configuring IPv4: sonic-s6100-dut1 port Ethernet68 -> 20.1.1.0/31 INFO tests.common.snappi_tests.snappi_fixtures:snappi_fixtures.py:733 Pre-configuring IPv6: sonic-s6100-dut1 port Ethernet68 -> 3000:1::1/126 INFO tests.common.snappi_tests.snappi_fixtures:snappi_fixtures.py:733 Pre-configuring IPv4: sonic-s6100-dut1 port Ethernet72 -> 20.1.1.2/31 INFO tests.common.snappi_tests.snappi_fixtures:snappi_fixtures.py:733 Pre-configuring IPv6: sonic-s6100-dut1 port Ethernet72 -> 3000:1::5/126 INFO tests.common.snappi_tests.snappi_fixtures:snappi_fixtures.py:733 Pre-configuring IPv4: sonic-s6100-dut1 port Ethernet76 -> 20.1.1.4/31 INFO tests.common.snappi_tests.snappi_fixtures:snappi_fixtures.py:733 Pre-configuring IPv6: sonic-s6100-dut1 port Ethernet76 -> 3000:1::9/126 INFO root:init.py:76 -------------------- fixture tgen_ports setup ends -------------------- INFO root:init.py:77 Log analyzer is disabled INFO tests.common.plugins.memory_utilization:init.py:130 Memory utilization monitoring is disabled INFO root:init.py:81 -------------------- fixture setup_bgp_testbed setup starts -------------------- INFO tests.common.snappi_tests.snappi_fixtures:snappi_fixtures.py:641 Backing up the initial config to /etc/sonic/bgp_backup.json INFO root:init.py:85 -------------------- fixture setup_bgp_testbed setup ends -------------------- INFO root:conftest.py:3766 skip setup dualtor mux cables on non-dualtor testbed ---------------------------------------------------------------------------------- live log call ----------------------------------------------------------------------------------- INFO tests.snappi_tests.bgp.files.bgp_convergence_helper:bgp_convergence_helper.py:214 Removing configured IP and IPv6 Address from Ethernet68 INFO tests.snappi_tests.bgp.files.bgp_convergence_helper:bgp_convergence_helper.py:214 Removing configured IP and IPv6 Address from Ethernet72 INFO tests.snappi_tests.bgp.files.bgp_convergence_helper:bgp_convergence_helper.py:214 Removing configured IP and IPv6 Address from Ethernet76 INFO tests.snappi_tests.bgp.files.bgp_convergence_helper:bgp_convergence_helper.py:227 Configuring Ethernet68 to PortChannel1 with IPs 20.1.1.0,3000:1::1 INFO tests.snappi_tests.bgp.files.bgp_convergence_helper:bgp_convergence_helper.py:227 Configuring Ethernet72 to PortChannel2 with IPs 20.1.1.2,3000:1::5 INFO tests.snappi_tests.bgp.files.bgp_convergence_helper:bgp_convergence_helper.py:227 Configuring Ethernet76 to PortChannel3 with IPs 20.1.1.4,3000:1::9 INFO tests.snappi_tests.bgp.files.bgp_convergence_helper:bgp_convergence_helper.py:254 Configuring BGP v4 Neighbor 20.1.1.3 INFO tests.snappi_tests.bgp.files.bgp_convergence_helper:bgp_convergence_helper.py:254 Configuring BGP v4 Neighbor 20.1.1.5 WARNING ixnetwork_restpy.connection:connection.py:351 Verification of certificates is disabled INFO ixnetwork_restpy.connection:connection.py:348 Determining the platform and rest_port using the 10.36.77.53 address... INFO ixnetwork_restpy.connection:connection.py:348 Connection established to http://10.36.77.53:11009 on windows INFO ixnetwork_restpy.connection:connection.py:348 Using IxNetwork api server version 10.80.2412.6 INFO ixnetwork_restpy.connection:connection.py:348 User info IxNetwork/WIN-11RK5TNKNAN/8010 INFO snappi_ixnetwork.snappi_api:snappi_api.py:1516 snappi-1.40.0 INFO snappi_ixnetwork.snappi_api:snappi_api.py:1516 snappi_ixnetwork-1.42.1 INFO snappi_ixnetwork.snappi_api:snappi_api.py:1516 ixnetwork_restpy-1.9.0 INFO snappi_ixnetwork.snappi_api:snappi_api.py:1516 Config validation 0.007s INFO snappi_ixnetwork.snappi_api:snappi_api.py:1516 Ports configuration 0.054s INFO snappi_ixnetwork.snappi_api:snappi_api.py:1516 Captures configuration 0.027s INFO snappi_ixnetwork.snappi_api:snappi_api.py:1516 Location hosts ready [10.36.78.53] 0.014s INFO snappi_ixnetwork.snappi_api:snappi_api.py:1516 Speed conversion is not require for (port.name, speed) : [('Test_Port_1', 'novusHundredGigNonFanOut'), ('Test_Port_2', 'novusHundredGigNonFanOut'), ('Test_Port_3', 'novusHundredGigNonFanOut')] INFO snappi_ixnetwork.snappi_api:snappi_api.py:1516 Aggregation mode speed change 0.370s INFO snappi_ixnetwork.snappi_api:snappi_api.py:1516 Location configuration 0.454s INFO snappi_ixnetwork.snappi_api:snappi_api.py:1516 Layer1 configuration 0.052s INFO snappi_ixnetwork.snappi_api:snappi_api.py:1516 Lag Configuration 1.557s INFO snappi_ixnetwork.snappi_api:snappi_api.py:1516 Lag Ethernet Configuration 0.170s INFO snappi_ixnetwork.snappi_api:snappi_api.py:1516 Lag Protocol Configuration 0.893s INFO snappi_ixnetwork.snappi_api:snappi_api.py:1516 Convert device config : 0.117s INFO snappi_ixnetwork.snappi_api:snappi_api.py:1516 Create IxNetwork device config : 0.001s INFO snappi_ixnetwork.snappi_api:snappi_api.py:1516 Push IxNetwork device config : 1.091s INFO snappi_ixnetwork.snappi_api:snappi_api.py:1516 Devices configuration 1.217s INFO snappi_ixnetwork.snappi_api:snappi_api.py:1516 Flows configuration 0.589s INFO snappi_ixnetwork.snappi_api:snappi_api.py:1516 Start interfaces 0.512s INFO snappi_ixnetwork.snappi_api:snappi_api.py:1516 IxNet - One or more destination MACs or VPNs are invalid or unreachable and the packets configured to be sent to them were not created INFO snappi_ixnetwork.snappi_api:snappi_api.py:1516 IxNet - The Traffic Item was modified. Please perform a Traffic Generate to update the associated traffic Flow Groups INFO tests.snappi_tests.bgp.files.bgp_convergence_helper:bgp_convergence_helper.py:566 Starting all protocols ... INFO tests.common.utilities:utilities.py:121 Pause 30 seconds, reason: For Protocols To start INFO tests.snappi_tests.bgp.files.bgp_convergence_helper:bgp_convergence_helper.py:572 |---- Network_Group2 Route Withdraw Iteration : 1 ----| INFO tests.snappi_tests.bgp.files.bgp_convergence_helper:bgp_convergence_helper.py:575 Starting Traffic INFO snappi_ixnetwork.snappi_api:snappi_api.py:1516 Flows generate/apply 6.809s INFO snappi_ixnetwork.snappi_api:snappi_api.py:1516 Flows clear statistics 13.617s INFO snappi_ixnetwork.snappi_api:snappi_api.py:1516 Captures start 0.000s INFO snappi_ixnetwork.snappi_api:snappi_api.py:1516 Flows start 2.527s INFO snappi_ixnetwork.snappi_api:snappi_api.py:1516 IxNet - If ports in a lag are down, please enable Transmit Ignore Link Status port property in order to successfully start traffic or clear statistics. INFO snappi_ixnetwork.snappi_api:snappi_api.py:1516 IxNet - The Rate Monitoring Jitter Window Size was decreased to support the incoming frame rate INFO tests.common.utilities:utilities.py:121 Pause 30 seconds, reason: For Traffic To start INFO tests.snappi_tests.bgp.files.bgp_convergence_helper:bgp_convergence_helper.py:585 Withdrawing Routes from Network_Group2 INFO snappi_ixnetwork.snappi_api:snappi_api.py:1516 Setting route state 0.626s INFO tests.common.utilities:utilities.py:121 Pause 30 seconds, reason: For routes to be withdrawn INFO tests.snappi_tests.bgp.files.bgp_convergence_helper:bgp_convergence_helper.py:598 Traffic has converged after route withdraw INFO tests.snappi_tests.bgp.files.bgp_convergence_helper:bgp_convergence_helper.py:605 CP/DP Convergence Time (ms): 486.021 INFO snappi_ixnetwork.snappi_api:snappi_api.py:1516 Setting route state 0.621s INFO tests.snappi_tests.bgp.files.bgp_convergence_helper:bgp_convergence_helper.py:615 Readvertise Network_Group2 routes back at the end of iteration 1 INFO tests.snappi_tests.bgp.files.bgp_convergence_helper:bgp_convergence_helper.py:617 Stopping Traffic INFO snappi_ixnetwork.snappi_api:snappi_api.py:1516 Flows stop 5.313s INFO tests.common.utilities:utilities.py:121 Pause 30 seconds, reason: For Traffic To Stop INFO tests.snappi_tests.bgp.files.bgp_convergence_helper:bgp_convergence_helper.py:566 Starting all protocols ... INFO tests.common.utilities:utilities.py:121 Pause 30 seconds, reason: For Protocols To start INFO tests.snappi_tests.bgp.files.bgp_convergence_helper:bgp_convergence_helper.py:572 |---- Network_Group3 Route Withdraw Iteration : 1 ----| INFO tests.snappi_tests.bgp.files.bgp_convergence_helper:bgp_convergence_helper.py:575 Starting Traffic INFO snappi_ixnetwork.snappi_api:snappi_api.py:1516 Flows clear statistics 9.873s INFO snappi_ixnetwork.snappi_api:snappi_api.py:1516 Captures start 0.000s INFO snappi_ixnetwork.snappi_api:snappi_api.py:1516 Flows start 3.026s INFO tests.common.utilities:utilities.py:121 Pause 30 seconds, reason: For Traffic To start INFO tests.snappi_tests.bgp.files.bgp_convergence_helper:bgp_convergence_helper.py:585 Withdrawing Routes from Network_Group3 INFO snappi_ixnetwork.snappi_api:snappi_api.py:1516 Setting route state 0.615s INFO tests.common.utilities:utilities.py:121 Pause 30 seconds, reason: For routes to be withdrawn INFO tests.snappi_tests.bgp.files.bgp_convergence_helper:bgp_convergence_helper.py:598 Traffic has converged after route withdraw INFO tests.snappi_tests.bgp.files.bgp_convergence_helper:bgp_convergence_helper.py:605 CP/DP Convergence Time (ms): 503.167 INFO snappi_ixnetwork.snappi_api:snappi_api.py:1516 Setting route state 0.612s INFO tests.snappi_tests.bgp.files.bgp_convergence_helper:bgp_convergence_helper.py:615 Readvertise Network_Group3 routes back at the end of iteration 1 INFO tests.snappi_tests.bgp.files.bgp_convergence_helper:bgp_convergence_helper.py:617 Stopping Traffic INFO snappi_ixnetwork.snappi_api:snappi_api.py:1516 Flows stop 5.307s INFO tests.common.utilities:utilities.py:121 Pause 30 seconds, reason: For Traffic To Stop INFO tests.snappi_tests.bgp.files.bgp_convergence_helper:bgp_convergence_helper.py:636 +-------------------------------+--------------+-----------------+--------------+----------------+---------------------------------------------------+ | Event Name | Route Type | No. of Routes | Iterations | Frames Delta | Avg Control to Data Plane Convergence Time (ms) | |-------------------------------+--------------+-----------------+--------------+----------------+---------------------------------------------------| | Network_Group2 route withdraw | IPv4 | 1000 | 1 | 32 | 486 | | Network_Group3 route withdraw | IPv4 | 1000 | 1 | 32 | 503 | +-------------------------------+--------------+-----------------+--------------+----------------+---------------------------------------------------+ -------------------------------------------------------------------------------- live log logreport -------------------------------------------------------------------------------- INFO tests.snappi_tests.bgp.conftest:conftest.py:33 Test tests/snappi_tests/bgp/test_bgp_remote_link_failover.py::test_bgp_convergence_for_remote_link_failover[IPv4-1000-1-2] finished with result: passed -------------------------------------------------------------------------------- live log teardown --------------------------------------------------------------------------------- INFO root:init.py:93 -------------------- fixture setup_bgp_testbed teardown starts -------------------- INFO tests.common.snappi_tests.snappi_fixtures:snappi_fixtures.py:643 Restoring the initial config as part of teardown INFO root:init.py:102 -------------------- fixture setup_bgp_testbed teardown ends -------------------- INFO root:init.py:93 -------------------- fixture snappi_api teardown starts -------------------- INFO root:init.py:102 -------------------- fixture snappi_api teardown ends -------------------- INFO tests.conftest:conftest.py:3250 Skipping temporarily_disable_route_check fixture INFO tests.conftest:conftest.py:3031 Dumping Disk and Memory Space information after test on sonic-s6100-dut1 INFO tests.conftest:conftest.py:3035 Collecting core dumps after test on sonic-s6100-dut1 INFO tests.conftest:conftest.py:3046 Collecting running config after test on sonic-s6100-dut1 WARNING tests.conftest:conftest.py:3187 Core dump or config check failed for test_bgp_remote_link_failover.py, results: {"core_dump_check": {"failed": false, "new_core_dumps": {"sonic-s6100-dut1": []}}, "config_db_check": {"failed": true, "pre_only_config": {"sonic-s6100-dut1": {"null": {"LOOPBACK_INTERFACE": {"Loopback0": {}, "Loopback0|2.2.2.2/32": {}}}}}, "cur_only_config": {"sonic-s6100-dut1": {"null": {"PORTCHANNEL_MEMBER": {"PortChannel2|Ethernet68": {}, "PortChannel3|Ethernet72": {}, "PortChannel4|Ethernet76": {}}, "VLAN_MEMBER": {"Vlan1709|Ethernet112": {"tagging_mode": "untagged"}, "Vlan1709|Ethernet124": {"tagging_mode": "tagged"}}}}}, "inconsistent_config": {"sonic-s6100-dut1": {"null": {"VLAN": {"pre_value": {"Vlan100": {"vlanid": "100"}}, "cur_value": {"Vlan100": {"vlanid": "100"}, "Vlan1709": {"vlanid": "1709"}}}, "DEVICE_METADATA": {"pre_value": {"localhost": {"buffer_model": "traditional", "default_bgp_status": "up", "default_pfcwd_status": "disable", "docker_routing_config_mode": "split", "frr_mgmt_framework_config": "true", "hostname": "sonic", "hwsku": "Accton-AS7726-32X", "mac": "90:3c:b3:94:ad:7e", "platform": "x86_64-accton_as7726_32x-r0", "synchronous_mode": "enable", "type": "LeafRouter"}}, "cur_value": {"localhost": {"bgp_asn": "65100", "buffer_model": "traditional", "default_bgp_status": "up", "default_pfcwd_status": "disable", "docker_routing_config_mode": "split", "frr_mgmt_framework_config": "true", "hostname": "sonic", "hwsku": "Accton-AS7726-32X", "mac": "90:3c:b3:94:ad:7e", "platform": "x86_64-accton_as7726_32x-r0", "synchronous_mode": "enable", "type": "ToRRouter"}}}, "INTERFACE": {"pre_value": {"Ethernet0": {}, "Ethernet4": {}, "Ethernet64": {}, "Ethernet68": {}, "Ethernet72": {}, "Ethernet76": {}, "Ethernet0|10.1.1.1/24": {}, "Ethernet4|10.2.1.1/24": {}, "Ethernet64|2001::1/64": {}, "Ethernet64|21.1.1.1/24": {}, "Ethernet68|2002::1/64": {}, "Ethernet68|22.1.1.1/24": {}, "Ethernet72|2003::1/64": {}, "Ethernet72|23.1.1.1/24": {}, "Ethernet76|2004::1/64": {}, "Ethernet76|24.1.1.1/24": {}}, "cur_value": {"Ethernet0": {}, "Ethernet4": {}, "Ethernet0|10.1.1.1/24": {}, "Ethernet4|10.2.1.1/24": {}}}, "PORTCHANNEL": {"pre_value": {"PortChannel2": {"admin_status": "up", "fast_rate": "false", "lacp_key": "auto", "min_links": "1", "mix_speed": "false", "mtu": "9100"}}, "cur_value": {"PortChannel1": {"admin_status": "up", "fast_rate": "false", "lacp_key": "auto", "min_links": "1", "mix_speed": "false", "mtu": "9100"}, "PortChannel2": {"admin_status": "up", "fast_rate": "false", "lacp_key": "auto", "min_links": "1", "mix_speed": "false", "mtu": "9100"}, "PortChannel3": {"admin_status": "up", "fast_rate": "false", "lacp_key": "auto", "min_links": "1", "mix_speed": "false", "mtu": "9100"}, "PortChannel4": {"admin_status": "up", "fast_rate": "false", "lacp_key": "auto", "min_links": "1", "mix_speed": "false", "mtu": "9100"}}}, "PORTCHANNEL_INTERFACE": {"pre_value": {"PortChannel1": {}, "PortChannel1|2000:1::1/126": {}}, "cur_value": {"PortChannel2": {}, "PortChannel3": {}, "PortChannel4": {}, "PortChannel2|22.1.1.1/24": {}, "PortChannel2|3002::2/126": {}, "PortChannel3|23.1.1.1/24": {}, "PortChannel3|3003::2/126": {}, "PortChannel4|24.1.1.1/24": {}, "PortChannel4|3004::2/126": {}}}}}}}} tests/snappi_tests/bgp/test_bgp_remote_link_failover.py::test_bgp_convergence_for_remote_link_failover[IPv4-1000-1-2] ✓ 100% ██████████ ================================================================================= warnings summary ================================================================================= ../../../opt/venv/lib/python3.12/site-packages/_pytest/config/init.py:852 /opt/venv/lib/python3.12/site-packages/_pytest/config/init.py:852: PytestAssertRewriteWarning: Module already imported so cannot be rewritten; tests.common.plugins.ptfadapter self.import_plugin(import_spec) ../../../opt/venv/lib/python3.12/site-packages/_pytest/config/init.py:852 /opt/venv/lib/python3.12/site-packages/_pytest/config/init.py:852: PytestAssertRewriteWarning: Module already imported so cannot be rewritten; tests.common.plugins.ansible_fixtures self.import_plugin(import_spec) ../../../opt/venv/lib/python3.12/site-packages/_pytest/config/init.py:852 /opt/venv/lib/python3.12/site-packages/_pytest/config/init.py:852: PytestAssertRewriteWarning: Module already imported so cannot be rewritten; tests.common.plugins.loganalyzer self.import_plugin(import_spec) ../../../opt/venv/lib/python3.12/site-packages/_pytest/config/init.py:852 /opt/venv/lib/python3.12/site-packages/_pytest/config/init.py:852: PytestAssertRewriteWarning: Module already imported so cannot be rewritten; tests.common.plugins.sanity_check self.import_plugin(import_spec) ../../../opt/venv/lib/python3.12/site-packages/_pytest/config/init.py:852 /opt/venv/lib/python3.12/site-packages/_pytest/config/init.py:852: PytestAssertRewriteWarning: Module already imported so cannot be rewritten; tests.common.plugins.test_completeness self.import_plugin(import_spec) ../../../opt/venv/lib/python3.12/site-packages/_pytest/config/init.py:852 /opt/venv/lib/python3.12/site-packages/_pytest/config/init.py:852: PytestAssertRewriteWarning: Module already imported so cannot be rewritten; tests.common.dualtor self.import_plugin(import_spec) ../../../opt/venv/lib/python3.12/site-packages/_pytest/config/init.py:852 /opt/venv/lib/python3.12/site-packages/_pytest/config/init.py:852: PytestAssertRewriteWarning: Module already imported so cannot be rewritten; tests.common.fixtures.duthost_utils self.import_plugin(import_spec) :488 :488: DeprecationWarning: Type google._upb._message.MessageMapContainer uses PyType_Spec with a metaclass that has custom tp_new. This is deprecated and will no longer be allowed in Python 3.14. :488 :488: DeprecationWarning: Type google._upb._message.ScalarMapContainer uses PyType_Spec with a metaclass that has custom tp_new. This is deprecated and will no longer be allowed in Python 3.14. ../../../opt/venv/lib/python3.12/site-packages/google/protobuf/internal/well_known_types.py:93 /opt/venv/lib/python3.12/site-packages/google/protobuf/internal/well_known_types.py:93: DeprecationWarning: datetime.datetime.utcfromtimestamp() is deprecated and scheduled for removal in a future version. Use timezone-aware objects to represent datetimes in UTC: datetime.datetime.fromtimestamp(timestamp, datetime.UTC). _EPOCH_DATETIME_NAIVE = datetime.datetime.utcfromtimestamp(0) ptf_runner.py:3 /root/sonic-mgmt/tests/ptf_runner.py:3: DeprecationWarning: 'pipes' is deprecated and slated for removal in Python 3.13 import pipes tests/snappi_tests/bgp/test_bgp_remote_link_failover.py::test_bgp_convergence_for_remote_link_failover[IPv4-1000-1-2] /opt/venv/lib/python3.12/site-packages/pytest_ansible/host_manager/v213.py:13: DeprecationWarning: Host management is deprecated and will be removed in a future release class HostManagerV213(BaseHostManager): tests/snappi_tests/bgp/test_bgp_remote_link_failover.py::test_bgp_convergence_for_remote_link_failover[IPv4-1000-1-2] tests/snappi_tests/bgp/test_bgp_remote_link_failover.py::test_bgp_convergence_for_remote_link_failover[IPv4-1000-1-2] tests/snappi_tests/bgp/test_bgp_remote_link_failover.py::test_bgp_convergence_for_remote_link_failover[IPv4-1000-1-2] tests/snappi_tests/bgp/test_bgp_remote_link_failover.py::test_bgp_convergence_for_remote_link_failover[IPv4-1000-1-2] tests/snappi_tests/bgp/test_bgp_remote_link_failover.py::test_bgp_convergence_for_remote_link_failover[IPv4-1000-1-2] /opt/venv/lib/python3.12/site-packages/ansible/plugins/loader.py:1485: UserWarning: AnsibleCollectionFinder has already been configured warnings.warn('AnsibleCollectionFinder has already been configured') tests/snappi_tests/bgp/test_bgp_remote_link_failover.py: 45 warnings /usr/lib/python3.12/multiprocessing/popen_fork.py:66: DeprecationWarning: This process (pid=236032) is multi-threaded, use of fork() may lead to deadlocks in the child. self.pid = os.fork() tests/snappi_tests/bgp/test_bgp_remote_link_failover.py::test_bgp_convergence_for_remote_link_failover[IPv4-1000-1-2] /opt/venv/lib/python3.12/site-packages/snappi_ixnetwork/snappi_api.py:1047: DeprecationWarning: pkg_resources is deprecated as an API. See https://setuptools.pypa.io/en/latest/pkg_resources.html import pkg_resources tests/snappi_tests/bgp/test_bgp_remote_link_failover.py: 12 warnings /opt/venv/lib/python3.12/site-packages/ixnetwork_restpy/assistants/statistics/row.py:22: DeprecationWarning: datetime.datetime.utcnow() is deprecated and scheduled for removal in a future version. Use timezone-aware objects to represent datetimes in UTC: datetime.datetime.now(datetime.UTC). self._sample_time = datetime.datetime.utcnow() -- Docs: https://docs.pytest.org/en/stable/how-to/capture-warnings.html ------------------------------------------------------------------------------ live log sessionfinish ------------------------------------------------------------------------------ INFO root:init.py:67 Can not get Allure report URL. Please check logs Results (398.60s (0:06:38)): 1 passed INFO:root:Can not get Allure report URL. Please check logs --------- Signed-off-by: selldinesh <dinesh.sellappan@keysight.com> Signed-off-by: Raghavendran Ramanathan <rraghav@cisco.com>
) SKU: x86_64-arista_7280r4_32qf_32df This is the sonic-mgmt change that goes along with the sonic-buildimage change: sonic-net/sonic-buildimage#27149 ### Description of PR For SKU `x86_64-arista_7280r4_32qf_32df` increment the QSFP-DD port names based on their number of system lanes instead of number of line lanes. ### Type of change - [ ] Bug fix - [x] Testbed and Framework(new/improvement) - [ ] New Test case - [ ] Skipped for non-supported platforms - [ ] Test case improvement ### Back port request - [ ] 202205 - [ ] 202305 - [ ] 202311 - [ ] 202405 - [ ] 202411 - [ ] 202505 - [x] 202511 ### Approach #### What is the motivation for this PR? MSFT requested we increment the port names based on the number of system-side lanes instead of number of line-side lanes. #### How did you do it? Changed the `get_port_alias_to_name_map` function in port_utils.py for this SKU to increment by 4 for all ports on `x86_64-arista_7280r4_32qf_32df` SKU #### How did you verify/test it? Ran sonic-mgmt against this change and saw no related regressions. #### Any platform specific information? No #### Supported testbed topology if it's a new test case? N/A ### Documentation N/A Signed-off-by: Nathan Wolfe <nwolfe@arista.com> Signed-off-by: Raghavendran Ramanathan <rraghav@cisco.com>
<!-- Please make sure you've read and understood our contributing guidelines; https://github.com/sonic-net/SONiC/blob/gh-pages/CONTRIBUTING.md Please provide following information to help code review process a bit easier: --> ### Description of PR <!-- - Please include a summary of the change and which issue is fixed. - Please also include relevant motivation and context. Where should reviewer start? background context? - List any dependencies that are required for this change. --> Summary: On mellanox devices, both amber and red are displayed as red in the show system-health summary output. If not handle well, test might fail due to color inconsistent ### Type of change <!-- - Fill x for your type of change. - e.g. - [x] Bug fix --> - [x] Bug fix - [ ] Testbed and Framework(new/improvement) - [ ] New Test case - [ ] Skipped for non-supported platforms - [ ] Test case improvement ### Back port request - [ ] 202205 - [ ] 202305 - [ ] 202311 - [ ] 202405 - [ ] 202411 - [ ] 202505 - [x] 202511 ### Approach #### What is the motivation for this PR? solve the inconsistent of led color #### How did you do it? replace led color amber with red on mellanox device, treat them as same color as CLI output #### How did you verify/test it? run test on single setup #### Any platform specific information? Issue is seen on Mellanox, but test case updates should not effect other platforms. #### Supported testbed topology if it's a new test case? ### Documentation <!-- (If it's a new feature, new test case) Did you update documentation/Wiki relevant to your implementation? Link to the wiki page? --> Signed-off-by: Wenjun Wang <wenjwang@nvidia.com> Signed-off-by: Raghavendran Ramanathan <rraghav@cisco.com>
…nic-net#24433) ### Description of PR Summary: Skip warm reboot on Broadcom TH5 devices in `test_qos_dscp_mapping`, as TH5 only supports cold reboot for upgrades. Uses `duthost.get_asic_name()` for th5 device check. ### Type of change - [ x ] Bug fix - [ ] Testbed and Framework(new/improvement) - [ ] New Test case - [ ] Skipped for non-supported platforms - [] Test case improvement ### Back port request - [ ] 202205 - [ ] 202305 - [ ] 202311 - [ ] 202405 - [ ] 202411 - [ ] 202505 - [ x ] 202511 ### Approach #### What is the motivation for this PR? TH5 devices (e.g., Arista-7060X6-16PE-384C-B-O128S2) don't support warm reboot, they use cold reboot for upgrades. The DSCP queue mapping test currently attempts warm reboot on all non-smartswitch, non-dualtor t0 topologies, which causes failures on TH5 platforms. #### How did you do it? Added a TH5 check using `duthost.get_asic_name() == 'th5'` before the warm reboot block in `test_dscp_to_queue_mapping`. #### How did you verify/test it? - Verified `duthost.get_asic_name()` returns `'th5'` on Arista-7060X6-16PE-384C-B-O128S2 from test logs. - Confirmed the warm reboot step is skipped on TH5 and the test passes with DSCP queue mapping validation in both uniform and pipe modes. #### Any platform specific information? Affects Broadcom TH5 platforms (Arista-7060X6 variants). No impact on other platforms, warm reboot continues to run on all other supported devices. #### Supported testbed topology if it's a new test case? N/A ### Documentation N/A Signed-off-by: Priyansh Tratiya <ptratiya@microsoft.com> Signed-off-by: Raghavendran Ramanathan <rraghav@cisco.com>
…et#24391) ### Description of PR Summary: Fix flaky IPv6 failures in `test_pfcwd_function.py` where `ip neigh flush all` on the PTF host fails with `"Failed to send flush request: No such file or directory"` (rc=1), causing test_pfcwd_actions, test_pfcwd_multi_port, and test_pfcwd_mmu_change to abort during setup. The `remove_ip.sh` script that runs immediately prior can leave the PTF in a state where the neighbor table references deleted interfaces. When `ip neigh flush all` attempts to flush entries for those interfaces, iproute2 returns a non-zero exit code. Since the flush is a best-effort cleanup step (the subsequent ping/arping will correctly re-establish needed ARP/NDP entries regardless), the command should tolerate errors. ### Type of change - [x] Bug fix ### Back port request - [ ] 202205 - [ ] 202305 - [ ] 202311 - [ ] 202405 - [ ] 202411 - [ ] 202505 - [x] 202511 ### Approach #### What is the motivation for this PR? `test_pfcwd_function` IPv6 tests are intermittently failing on the 7060X6 T0 testbed (`vms91-t0-7060x6-moby-512-3`) because `ip neigh flush all` on the PTF returns rc=1 with `"Failed to send flush request: No such file or directory"`. This is a non-critical cleanup command that should not cause test failure. The actual PFCwd functionality is not impacted, all IPv4 tests pass, and the DUT is healthy throughout. #### How did you do it? Added `module_ignore_errors=True` to the `ip neigh flush all` and `ip -6 neigh flush all` commands on the PTF host in the `resolve_arp` method. These commands are purely preparatory cleanup; the immediately following ping and arping/ping6 commands are what actually establish the required neighbor entries. #### How did you verify/test it? Re-ran `test_pfcwd_function.py` on the same testbed (`vms91-t0-7060x6-moby-512-3`, 7060X6, internal-202511 image). All 10 test items passed (8 passed, 2 skipped as expected for non-cisco-8000 platform). #### Any platform specific information? #### Supported testbed topology if it's a new test case? N/A ### Documentation N/A Signed-off-by: Priyansh Tratiya <ptratiya@microsoft.com> Signed-off-by: Raghavendran Ramanathan <rraghav@cisco.com>
…onic-net#24417) ### Description of PR Summary: Two small fixes to `test_reporting/junit_xml_parser.py` so the script works on Windows and accepts JUnit XML reports with arbitrary file names. Fixes # (n/a) ### Type of change - [x] Bug fix - [ ] Testbed and Framework(new/improvement) - [ ] New Test case - [ ] Skipped for non-supported platforms - [ ] Test case improvement ### Back port request - [ ] 202205 - [ ] 202305 - [ ] 202311 - [ ] 202405 - [ ] 202411 - [ ] 202505 - [ ] 202511 ### Approach #### What is the motivation for this PR? 1. Directory-mode XML discovery used overly narrow globs (`tr.xml`, `*test*.xml`). Reports with other naming conventions (e.g. `tr-<host>-<timestamp>.xml`) were silently dropped, producing `provided directory ... does not contain any valid XML files`. Filtering JUnit reports by filename is a poor heuristic — the right discriminator is the XML content. 2. The CSV report timestamp used `%H:%M:%S`, which contains `:`. That character is illegal in Windows filenames and caused `OSError: [Errno 22] Invalid argument: 'report_06-May-2026-09:01:32.969895.csv'` when running the script on Windows. #### How did you do it? 1. Replaced the three filename-pattern globs with a single recursive `**/*.xml` glob. Non-JUnit XMLs are still rejected by `validate_junit_xml_file` via the existing try/except in `validate_junit_xml_archive` (logged as `could not parse <file>: ... - skipping` in non-strict mode), so we filter by content instead of by filename. 2. Changed the CSV timestamp format from `%d-%b-%Y-%H:%M:%S.%f` to `%d-%b-%Y-%H-%M-%S.%f` so the resulting filename is valid on Windows (and still unambiguous on Linux). #### How did you verify/test it? * Reproduced the original `OSError` on Windows; confirmed the new format produces `report_06-May-2026-09-01-32.969895.csv` with no error. * Reproduced the "does not contain any valid XML files" symptom by placing `tr-W6950-128OC-2026-04-27-22-17-31.xml` in a directory and running `python junit_xml_parser.py --directory --validate-only <dir>`. After the fix, the file is discovered and validated successfully. #### Any platform specific information? The CSV-filename fix is specifically for Windows; on Linux the previous format also worked. The new format is portable. #### Supported testbed topology if it's a new test case? n/a — this is a fix to a reporting helper script, not a test case. ### Documentation n/a Signed-off-by: Xin Wang <xiwang5@microsoft.com> Signed-off-by: Raghavendran Ramanathan <rraghav@cisco.com>
…nic-net#24299) Why I did it Add a test to verify that files under /etc/sonic/credentials/ are excluded from techsupport bundles. This complements sonic-net/sonic-utilities#4505 which adds the exclusion to generate_dump. How I did it Added test_credentials_dir_removed_from_show_techsupport to tests/show_techsupport/test_techsupport_no_secret.py. The fixture creates a test file with a non-standard extension (.dat) under /etc/sonic/credentials/ so it is not caught by the existing *.key/*.pem exclusion - making it a true test of the directory-level exclusion. After techsupport runs, the test checks that no files remain under etc/sonic/credentials/ in the extracted dump. Signed-off-by: Raghavendran Ramanathan <rraghav@cisco.com>
Signed-off-by: Raghavendran Ramanathan <rraghav@cisco.com>
Signed-off-by: Raghavendran Ramanathan <rraghav@cisco.com>
|
12 tasks
Contributor
Author
|
Closed this PR, since it is now messed up. and raised #24525. |
Collaborator
|
/azp run |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
20 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
…sole fanouts.
Description of PR
Sometimes the console script execute the "socat" command in dut or console-fanout. But the socat executable is not copied to the hosts. This fixture is to copy the socat from tests/console/socat to the DUTs and console-fanouts.
Summary:
Fixes issues where socat is needed, but not yet available in the fanouts.
Type of change
Back port request
Approach
What is the motivation for this PR?
The tests execute socat without copyting the required executable.
How did you do it?
Added a new fixture to the console/conftest.py folder.
How did you verify/test it?
Ran a couple of tests: