Carry green query values forward from the previous cache file [-10.6% incr-unchanged] - #35
Draft
xmakro wants to merge 2 commits into
Draft
Carry green query values forward from the previous cache file [-10.6% incr-unchanged]#35xmakro wants to merge 2 commits into
xmakro wants to merge 2 commits into
Conversation
xmakro
force-pushed
the
perf/cache-carry-forward
branch
from
July 29, 2026 11:40
4925561 to
c1e2fca
Compare
xmakro
changed the base branch from
perf/base-0728
to
perf/carry-unchanged-nodes
July 30, 2026 14:40
xmakro
force-pushed
the
perf/cache-carry-forward
branch
from
July 30, 2026 14:40
c1e2fca to
8178ee7
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Saving the incremental query cache used to require every value it contains to be in memory: values of green nodes that were never used during the session were loaded from the previous cache file by the promotion pass, only to be re-encoded, byte for byte equivalent, into the new file. On a warm rebuild this decode and re-encode round trip for the entire cache dominates the cost of saving.
Instead, copy the previous cache file's data region into the new file verbatim and reference the values of green nodes at their old positions. Copying the region unchanged keeps every internal absolute reference valid: type and predicate shorthands, symbol offsets and allocation data all stay at their offsets. The lookup tables around the region are adjusted instead: source file indices preserve all previous assignments and new files append after them, the allocation index keeps the previous entries and offsets this session's entries, expansions are keyed by their stable hash and merge, and each carried region keeps its syntax context table. After eight carried generations the save falls back to a full re-encode, which compacts away dead data left behind by red nodes and stale tables.
The change is two commits on top of the dep graph carry (rust-lang#160214). The first gives each data region its own syntax context table, since encoded context ids are only meaningful within the region written by their session. The second is the carry itself. An earlier version had a third commit re-tagging disk-cached values with their dep node, because the session-local index tag would go stale for values whose bytes outlive the session that wrote them. With the dep graph carry keeping every green node at its index, the index tag stays valid for as long as the value lives, and that commit is gone; re-pointing green entries into the carried region became an identity copy for the same reason.
The promotion pass is skipped when carrying, since the bytes it would re-encode are provably already in the new file, and still runs on saves that cannot carry, such as the periodic compaction session. Carried values are never decoded, so the verification that loading performs would not run for them: when carrying, the save decodes and verifies the same subset of not-loaded values that promotion would have, and drops them again.
The new file is written to a staging path and swapped in when complete, following the dep graph's staging file precedent, so the previous file can stay mapped while its data region is copied, on every platform including Windows. The header format version is bumped so caches written by earlier compilers are discarded cleanly.
Measured on the six primary rustc-perf crates (regex, syn, serde, cargo, ripgrep, hyper; instructions:u geomean, optimized stage2 builds of both sides): incr-unchanged -10.6%, incr-patched -1.4%, full and incr-full flat as a control, 25 cells improved / 3 regressed with the largest regression about +0.35%. rustc-perf forces
-Zincremental-verify-ichon every benchmarked invocation, and under that flag the verification sweep decodes every carried value, so rust-timer would currently report about -2.3% incr-unchanged instead. These numbers predate the rebase onto the dep graph carry and need re-measuring on the new base.tests/incremental passes in full (178 tests). A 16-generation churn and deletion stress, in the four mode combinations of
-Zincremental-verify-ichand-Zthreads=8and including an error session, replays warnings byte-identically through carried spans and hygiene, and its cache size trace shows the periodic compaction working.