Skip to content

[generic_config_updater] Cache loadData() calls to reduce redundant YANG parsing (#4476) [202511] - #4815

Merged
vmittal-msft merged 1 commit into
sonic-net:202511from
rimunagala:gcu-202511-06-pr4476
Sep 11, 2026
Merged

[generic_config_updater] Cache loadData() calls to reduce redundant YANG parsing (#4476) [202511]#4815
vmittal-msft merged 1 commit into
sonic-net:202511from
rimunagala:gcu-202511-06-pr4476

Conversation

@rimunagala

@rimunagala rimunagala commented Aug 27, 2026

Copy link
Copy Markdown
Contributor

PR 6 of 8 in a 202511 GCU backport stack. Draft until the PRs ahead of it merge.

Merge order is mandatory - this cherry-pick assumes the earlier ones. Because the commits are sequential, this PR currently also shows the commits belonging to the PRs ahead of it. Once those merge I will rebase and it will reduce to its own single commit. Please review the last commit only for now.

Stack starts at #4810. Tracking issue: #4823 - full stack, merge order and evidence.

Why I did it

Cherry-pick of #4476. Every move validation re-parses the entire config through libyang. On 202511 this shows up as the exact invariant loadData calls == 2 × moves, measured on real hardware (38 = 2×19, 20 = 2×10, 114 = 2×57). At 512 ports that is 1020 parses; upstream reported ~1027.

How I did it

Cherry-pick plus one manual conflict resolution — please review this hunk specifically.

#4476 adds md5-based caching inside the if reload_config: block introduced by #3831. On this branch that condition was widened to if reload_config or sy.root is None: (see PR 1). The resolution keeps the widened condition and takes #4476's caching body:

if reload_config or sy.root is None:
    <#4476 md5 hash-caching body>

The optimisation is fully preserved — the hash check still skips redundant loads. Only the entry condition differs from master, and only because 202511 lacks #4118.

How to verify it

loadData count collapses to 2, independent of scale (512-port case: 1020 → 2). Confirmed on hardware in all four leaf-list scenarios. Full suite green: 460 passed, 81 subtests.

Backport notes

  • Manual conflict as described above. This is the one hunk in the stack that intentionally differs from master.

Stack-level verification

Measured on the assembled stack, in a container built from the genuine 202511 sonic_yang_mgmt wheel (libyang 1.0.73, SWIG import yang as ly):

Check Result
Full GCU unit suite 460 passed, 81 subtests, 0 failed (clean 202511: 434 passed, 80 subtests)
flake8 --diff, pre-commit hook semantics (4.0.1, --max-line-length=120) 0 issues - this commit individually, and the stack as a whole
Diff coverage, this PR 100%
Diff coverage, whole stack 93.0% (698 lines measured, 649 covered) - pipeline gate is 80%

Not run: sonic-mgmt. I looked into running tests/generic_config_updater/test_apply_patch_perf.py and it skips on every LAG topology - its fixture builds the port list from PORT minus PORTCHANNEL_MEMBER, so on a T1 it finds 0 usable ports and bails with Need at least 2 admin-up ports, have 0. Across the last 45 days of nightly runs, every 202511 execution of it landed on a t1-*-lag bed and skipped, so no 202511 baseline for that test exists. It does run cleanly on t0 / m0 / mx / dualtor. Happy to book one of those beds and run it before merge - just ask.

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

@mssonicbld

Copy link
Copy Markdown
Collaborator

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).

@rimunagala rimunagala added the Tested for 202511 branch Verified/tested against the 202511 release branch label Aug 28, 2026

@vaibhavhd vaibhavhd left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved for 202511 backport stack tracked in #4823.

…ANG parsing (sonic-net#4476)

The two caches in this PR target different layers:

1. _currently_loaded_hash in SonicYangCfg.loadData() — skips re-parsing when the same config (by content hash) is loaded consecutively. This helps when multiple validators call loadData() with identical config within a single move validation.

2. _validate_config_cache in ConfigWrapper.validate_config_db() — caches the validation result for a given config hash, so if the same config state is validated again later, it returns the cached pass/fail without calling loadData() at all.

Per-operation analysis

Operation	Helps?	Why
REMOVE (individual)	❌ No	Each DFS step removes one item → unique config at each step. Neither cache hits because every state is different.
ADD	⚠️ Marginal	Typically 1 move → few loadData calls total. Cache might save 1 call if FullConfigMoveValidator and NoDependencyMoveValidator validate the same state.
REPLACE (scalar)	⚠️ Marginal	Same as ADD — few moves, small absolute savings.
REMOVE (batched via sonic-net#4478)	✅ Yes	sonic-net#4478 collapses N individual REMOVEs into 1 bulk REPLACE move. That single move still triggers multiple validator calls with the same config. Cache deduplicates those, reducing loads/move from ~10.6x to ~7.7

---------

Signed-off-by: Rithvick Reddy Munagala <rimunagala@microsoft.com>

(cherry picked from commit 5d54e44)
Signed-off-by: rimunagala <rimunagala@microsoft.com>
@rimunagala
rimunagala marked this pull request as ready for review September 11, 2026 18:18
@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
There may be pipelines that require an authorized user to comment /azp run to run.

@mssonicbld

Copy link
Copy Markdown
Collaborator

/azp run

@azure-pipelines

Copy link
Copy Markdown
Azure Pipelines:
Successfully started running 1 pipeline(s).

@vmittal-msft
vmittal-msft merged commit 6c5a4fc into sonic-net:202511 Sep 11, 2026
9 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Tested for 202511 branch Verified/tested against the 202511 release branch

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants