[generic_config_updater] Cache loadData() calls to reduce redundant YANG parsing (#4476) [202511] - #4815
Merged
Conversation
|
Azure Pipelines: There may be pipelines that require an authorized user to comment /azp run to run. |
Collaborator
|
/azp run |
|
Azure Pipelines: Successfully started running 1 pipeline(s). |
…ANG parsing (sonic-net#4476) The two caches in this PR target different layers: 1. _currently_loaded_hash in SonicYangCfg.loadData() — skips re-parsing when the same config (by content hash) is loaded consecutively. This helps when multiple validators call loadData() with identical config within a single move validation. 2. _validate_config_cache in ConfigWrapper.validate_config_db() — caches the validation result for a given config hash, so if the same config state is validated again later, it returns the cached pass/fail without calling loadData() at all. Per-operation analysis Operation Helps? Why REMOVE (individual) ❌ No Each DFS step removes one item → unique config at each step. Neither cache hits because every state is different. ADD⚠️ Marginal Typically 1 move → few loadData calls total. Cache might save 1 call if FullConfigMoveValidator and NoDependencyMoveValidator validate the same state. REPLACE (scalar)⚠️ Marginal Same as ADD — few moves, small absolute savings. REMOVE (batched via sonic-net#4478) ✅ Yes sonic-net#4478 collapses N individual REMOVEs into 1 bulk REPLACE move. That single move still triggers multiple validator calls with the same config. Cache deduplicates those, reducing loads/move from ~10.6x to ~7.7 --------- Signed-off-by: Rithvick Reddy Munagala <rimunagala@microsoft.com> (cherry picked from commit 5d54e44) Signed-off-by: rimunagala <rimunagala@microsoft.com>
rimunagala
marked this pull request as ready for review
September 11, 2026 18:18
rimunagala
force-pushed
the
gcu-202511-06-pr4476
branch
from
September 11, 2026 18:18
e33eb62 to
773aa7f
Compare
|
Azure Pipelines: There may be pipelines that require an authorized user to comment /azp run to run. |
Collaborator
|
/azp run |
|
Azure Pipelines: Successfully started running 1 pipeline(s). |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why I did it
Cherry-pick of #4476. Every move validation re-parses the entire config through libyang. On
202511this shows up as the exact invariantloadData calls == 2 × moves, measured on real hardware (38 = 2×19, 20 = 2×10, 114 = 2×57). At 512 ports that is 1020 parses; upstream reported ~1027.How I did it
Cherry-pick plus one manual conflict resolution — please review this hunk specifically.
#4476 adds md5-based caching inside the
if reload_config:block introduced by #3831. On this branch that condition was widened toif reload_config or sy.root is None:(see PR 1). The resolution keeps the widened condition and takes #4476's caching body:The optimisation is fully preserved — the hash check still skips redundant loads. Only the entry condition differs from master, and only because
202511lacks #4118.How to verify it
loadDatacount collapses to 2, independent of scale (512-port case: 1020 → 2). Confirmed on hardware in all four leaf-list scenarios. Full suite green: 460 passed, 81 subtests.Backport notes
Stack-level verification
Measured on the assembled stack, in a container built from the genuine
202511sonic_yang_mgmtwheel (libyang 1.0.73, SWIGimport yang as ly):202511: 434 passed, 80 subtests)flake8 --diff, pre-commit hook semantics (4.0.1,--max-line-length=120)Not run:
sonic-mgmt. I looked into runningtests/generic_config_updater/test_apply_patch_perf.pyand it skips on every LAG topology - its fixture builds the port list fromPORTminusPORTCHANNEL_MEMBER, so on a T1 it finds 0 usable ports and bails withNeed at least 2 admin-up ports, have 0. Across the last 45 days of nightly runs, every202511execution of it landed on at1-*-lagbed and skipped, so no202511baseline for that test exists. It does run cleanly on t0 / m0 / mx / dualtor. Happy to book one of those beds and run it before merge - just ask.