sonic-yang-mgmt: uses clause with leaf-list, choice not honored by bradh352 · Pull Request #21907 · sonic-net/sonic-buildimage

bradh352 · 2025-03-04T02:44:53Z

Why I did it

When a uses clause imports a grouping, it was only processing leaf entries and ignoring leaf-list and choice clauses. That means that for instance in bgp route maps, route_map_in and route_map_out validations would fail.

Honoring the uses refine clause is now also honored which is depended upon in sonic-utilities.

This now precompiles the uses clause and integrates it as if the uses clause was not part of the schema as multiple end users were having to do this additional processing.

This fixes that behavior and adds test cases to ensure it doesn't regress in the future.

This is the proper fix, replacing #21078 that just worked around it.

Removal of uses logic in sonic-utilities here: sonic-net/sonic-utilities#3814

Fixes #22382

Work item tracking

Microsoft ADO (number only):

How I did it

Added leaf-list lookup.

How to verify it

See test cases pass

Which release branch to backport (provide reason below if selected)

202405
202411

Tested branch (Please provide the tested image version)

master as of 20250304

Description for the changelog

sonic-yang-mgmt: uses clause with leaf-list, choice not honored

Link to config_db schema for YANG module changes

N/A

A picture of a cute animal (not mandatory but encouraged)

Signed-off-by: Brad House (@bradh352)

mssonicbld · 2025-03-04T02:46:39Z

/azp run Azure.sonic-buildimage

azure-pipelines · 2025-03-04T02:46:50Z

Azure Pipelines successfully started running 1 pipeline(s).

mssonicbld · 2025-03-05T01:03:57Z

/azp run Azure.sonic-buildimage

azure-pipelines · 2025-03-05T01:04:08Z

Azure Pipelines successfully started running 1 pipeline(s).

mssonicbld · 2025-03-05T01:04:51Z

/azp run Azure.sonic-buildimage

azure-pipelines · 2025-03-05T01:05:01Z

Azure Pipelines successfully started running 1 pipeline(s).

mssonicbld · 2025-03-05T01:40:07Z

/azp run Azure.sonic-buildimage

azure-pipelines · 2025-03-05T01:40:18Z

Azure Pipelines successfully started running 1 pipeline(s).

mssonicbld · 2025-03-05T01:47:15Z

/azp run Azure.sonic-buildimage

azure-pipelines · 2025-03-05T01:47:28Z

Azure Pipelines successfully started running 1 pipeline(s).

mssonicbld · 2025-03-05T01:55:48Z

/azp run Azure.sonic-buildimage

azure-pipelines · 2025-03-05T01:56:00Z

Azure Pipelines successfully started running 1 pipeline(s).

mssonicbld · 2025-03-05T02:08:57Z

/azp run Azure.sonic-buildimage

azure-pipelines · 2025-03-05T02:09:08Z

Azure Pipelines successfully started running 1 pipeline(s).

mssonicbld · 2025-03-05T02:11:59Z

/azp run Azure.sonic-buildimage

azure-pipelines · 2025-03-05T02:12:08Z

Azure Pipelines successfully started running 1 pipeline(s).

bradh352 · 2025-03-05T02:15:58Z

@ganglyu This is the proper fix for #21078 that resolves the underlying issue instead of attempting to work around it. Please review and hopefully approve.

bradh352 · 2025-03-05T02:20:14Z

@wen587, @venkatmahalingam you approved the PR that this is replacing. Can you review and approve this instead?

mssonicbld · 2025-03-05T10:50:28Z

/azp run Azure.sonic-buildimage

azure-pipelines · 2025-03-05T10:50:40Z

Azure Pipelines successfully started running 1 pipeline(s).

bradh352 · 2025-04-03T11:37:55Z

@adyeung just wanted to see if we could move this forward. Thanks.

zhangyanzhao · 2025-06-12T16:10:01Z

@qiluo-msft @dgsudharsan can you please help to take a look at this issue? Thanks.

zjswhhh · 2025-06-18T16:02:53Z

Hi @vaibhavhd - can you find a reviewer for this one?

Copilot

Pull Request Overview

This PR fixes handling of YANG uses clauses by including leaf-list and choice entries and honoring refine, and adds corresponding test cases and sample config updates for BGP route-map validation.

Extend grouping preprocessing to capture leaf-list and choice nodes.
Introduce _compileUsesClause to inline uses clauses into the schema.
Add tests and sample config entries for valid BGP neighbor/peergroup route-map references.

Reviewed Changes

Copilot reviewed 4 out of 4 changed files in this pull request and generated 2 comments.

File	Description
src/sonic-yang-models/tests/yang_model_tests/tests_config/bgp.json	Add valid BGP_NEIGHBOR_AF_ROUTE_MAP_VALID and BGP_PEERGROUP_AF_ROUTE_MAP_VALID configs
src/sonic-yang-models/tests/yang_model_tests/tests/bgp.json	Define new test cases for valid route-map references
src/sonic-yang-models/tests/files/sample_config_db.json	Populate `route_map_in` / `route_map_out` entries in sample DB
src/sonic-yang-mgmt/sonic_yang_ext.py	Add `_compileUsesClause*` methods, update grouping preprocessor

Comments suppressed due to low confidence (2)

src/sonic-yang-mgmt/sonic_yang_ext.py:167

This new helper lacks a docstring. Consider adding a brief description of its purpose, parameters (refine, model), and effects to aid future maintainers.

    def _compileUsesClauseRefineApply(self, refine, model):

src/sonic-yang-mgmt/sonic_yang_ext.py:217

The new compileUsesClause* methods introduce significant logic but lack direct unit tests. Consider adding focused tests for _compileUsesClauseModel (e.g., grouping with leaf-list, choice, and refine) to ensure correct behavior and prevent regressions.

    def _compileUsesClauseModel(self, module, model):

mssonicbld · 2025-06-19T11:46:30Z

/azp run Azure.sonic-buildimage

azure-pipelines · 2025-06-19T11:46:43Z

Azure Pipelines successfully started running 1 pipeline(s).

bradh352 · 2025-06-22T19:18:38Z

another spurious test failure ... rebased to get a clean build

mssonicbld · 2025-06-22T19:18:49Z

/azp run Azure.sonic-buildimage

azure-pipelines · 2025-06-22T19:19:01Z

Azure Pipelines successfully started running 1 pipeline(s).

@bradh352

When a uses clause imports a grouping, it was only processing leaf entries and ignoring leaf-list and choice clauses. That means that for instance in bgp route maps, route_map_in and route_map_out validations would fail. Honoring the uses `refine` clause is now also honored which is depended upon in sonic-utilities. This now precompiles the uses clause and integrates it as if the uses clause was not part of the schema as multiple end users were having to do this additional processing. This fixes that behavior and adds test cases to ensure it doesn't regress in the future. Signed-off-by: Brad House (@bradh352)

mssonicbld · 2025-06-23T12:57:11Z

/azp run Azure.sonic-buildimage

azure-pipelines · 2025-06-23T12:57:24Z

Azure Pipelines successfully started running 1 pipeline(s).

bradh352 · 2025-06-23T12:57:55Z

and the last build failed due to mellanox build failure related to some nfs mount failure on the build system ... then also a test timeout. Rebasing again to force another rebuild. None of the failure are related to the content in this PR.

bradh352 · 2025-06-23T22:11:40Z

All checks passed. @qiluo-msft can you merge?

yejianquan · 2025-06-24T09:12:55Z

Hi @qiluo-msft , could you help to suggest whether this change should goto 202505 branch?

qiluo-msft · 2025-06-24T23:04:05Z

I consider this as a new feature for use case like BGP_NEIGHBOR_AF. No need to backport.

hsw0 · 2025-08-03T01:15:25Z

I consider this as a new feature for use case like BGP_NEIGHBOR_AF. No need to backport.

@qiluo-msft It fixed #21078, which affects 202505 and older versions. Please reconsider backporting.

Verma-Anukul · 2025-10-14T12:38:16Z

@qiluo-msft @bradh352 @r12f

Can you please get it merged in 202505 also

mssonicbld · 2025-10-17T18:30:28Z

Cherry-pick PR to msft-202412: Azure/sonic-buildimage-msft#1731

bradh352 requested a review from qiluo-msft as a code owner March 4, 2025 02:44

bradh352 marked this pull request as draft March 4, 2025 02:44

bradh352 force-pushed the bradh352/uses-leaflist branch from 5e93791 to 0a069b5 Compare March 5, 2025 01:04

bradh352 force-pushed the bradh352/uses-leaflist branch from f7775f7 to 1c8718c Compare March 5, 2025 02:11

bradh352 changed the title ~~yang: add route_map_in/route_map_out bgp valid tests to trigger leaflist bug under uses~~ sonic-yang-mgmt: uses clause with leaf-list not honored Mar 5, 2025

bradh352 marked this pull request as ready for review March 5, 2025 02:16

bradh352 mentioned this pull request Mar 5, 2025

[libyang bug] workaround leaf-list via uses bug in BGP route-map #21078

Closed

1 task

bradh352 mentioned this pull request Mar 5, 2025

libyang: bug when uses clause references a leaf-list #21525

Closed

bradh352 force-pushed the bradh352/uses-leaflist branch from 1c8718c to 363be19 Compare March 5, 2025 10:48

bradh352 force-pushed the bradh352/uses-leaflist branch from 363be19 to 15ef7ab Compare March 6, 2025 01:05

bradh352 mentioned this pull request Apr 19, 2025

Bug: sonic-yang-mgmt uses clause evaluation ignores leaf-list and refine #22382

Closed

bhouse-nexthop mentioned this pull request May 20, 2025

Generic Configuration Updater (GCU) performance enhancements sonic-net/sonic-utilities#3831

Merged

bhouse-nexthop mentioned this pull request Jun 12, 2025

libyang: upgrade to v3 step 1 -- import libyang3 / libyang3-py3, remove stale versions #21679

Merged

bradh352 mentioned this pull request Jun 12, 2025

Enhancement: Upgrade to libyang3 #22385

Open

Copilot AI reviewed Jun 19, 2025

View reviewed changes

Comment thread src/sonic-yang-mgmt/sonic_yang_ext.py Outdated

Comment thread src/sonic-yang-mgmt/sonic_yang_ext.py Outdated

qiluo-msft reviewed Jun 19, 2025

View reviewed changes

Comment thread src/sonic-yang-mgmt/sonic_yang_ext.py Outdated

qiluo-msft reviewed Jun 19, 2025

View reviewed changes

Comment thread src/sonic-yang-mgmt/sonic_yang_ext.py Outdated

qiluo-msft approved these changes Jun 19, 2025

View reviewed changes

bradh352 added 2 commits June 23, 2025 08:56

implement review comments

aae8d09

mssonicbld mentioned this pull request Oct 17, 2025

[action] [PR:21907] sonic-yang-mgmt: uses clause with leaf-list, choice not honored Azure/sonic-buildimage-msft#1731

Merged

2 tasks

mssonicbld mentioned this pull request Nov 9, 2025

[action] [PR:3831] Generic Configuration Updater (GCU) performance enhancements Azure/sonic-utilities.msft#254

Merged

Conversation

bradh352 commented Mar 4, 2025 • edited Loading Uh oh! There was an error while loading. Please reload this page.

Uh oh!

Why I did it

Work item tracking

How I did it

How to verify it

Which release branch to backport (provide reason below if selected)

Tested branch (Please provide the tested image version)

Description for the changelog

Link to config_db schema for YANG module changes

A picture of a cute animal (not mandatory but encouraged)

Uh oh!

mssonicbld commented Mar 4, 2025

Uh oh!

azure-pipelines Bot commented Mar 4, 2025

Uh oh!

mssonicbld commented Mar 5, 2025

Uh oh!

azure-pipelines Bot commented Mar 5, 2025

Uh oh!

mssonicbld commented Mar 5, 2025

Uh oh!

azure-pipelines Bot commented Mar 5, 2025

Uh oh!

mssonicbld commented Mar 5, 2025

Uh oh!

azure-pipelines Bot commented Mar 5, 2025

Uh oh!

mssonicbld commented Mar 5, 2025

Uh oh!

azure-pipelines Bot commented Mar 5, 2025

Uh oh!

mssonicbld commented Mar 5, 2025

Uh oh!

azure-pipelines Bot commented Mar 5, 2025

Uh oh!

mssonicbld commented Mar 5, 2025

Uh oh!

azure-pipelines Bot commented Mar 5, 2025

Uh oh!

mssonicbld commented Mar 5, 2025

Uh oh!

azure-pipelines Bot commented Mar 5, 2025

Uh oh!

bradh352 commented Mar 5, 2025

Uh oh!

bradh352 commented Mar 5, 2025

Uh oh!

mssonicbld commented Mar 5, 2025

Uh oh!

azure-pipelines Bot commented Mar 5, 2025

Uh oh!

bradh352 commented Apr 3, 2025

Uh oh!

zhangyanzhao commented Jun 12, 2025

Uh oh!

zjswhhh commented Jun 18, 2025

Uh oh!

Copilot AI left a comment

Choose a reason for hiding this comment

Pull Request Overview

Reviewed Changes

Uh oh!

Uh oh!

Uh oh!

Uh oh!

Uh oh!

mssonicbld commented Jun 19, 2025

Uh oh!

azure-pipelines Bot commented Jun 19, 2025

Uh oh!

bradh352 commented Jun 22, 2025

Uh oh!

mssonicbld commented Jun 22, 2025

Uh oh!

azure-pipelines Bot commented Jun 22, 2025

Uh oh!

mssonicbld commented Jun 23, 2025

Uh oh!

bradh352 commented Mar 4, 2025 •

edited

Loading