Skip to content

Carry green query values forward from the previous cache file [-10.6% incr-unchanged] - #35

Draft
xmakro wants to merge 2 commits into
perf/carry-unchanged-nodesfrom
perf/cache-carry-forward
Draft

Carry green query values forward from the previous cache file [-10.6% incr-unchanged]#35
xmakro wants to merge 2 commits into
perf/carry-unchanged-nodesfrom
perf/cache-carry-forward

Conversation

@xmakro

@xmakro xmakro commented Jul 8, 2026

Copy link
Copy Markdown
Owner

Saving the incremental query cache used to require every value it contains to be in memory: values of green nodes that were never used during the session were loaded from the previous cache file by the promotion pass, only to be re-encoded, byte for byte equivalent, into the new file. On a warm rebuild this decode and re-encode round trip for the entire cache dominates the cost of saving.

Instead, copy the previous cache file's data region into the new file verbatim and reference the values of green nodes at their old positions. Copying the region unchanged keeps every internal absolute reference valid: type and predicate shorthands, symbol offsets and allocation data all stay at their offsets. The lookup tables around the region are adjusted instead: source file indices preserve all previous assignments and new files append after them, the allocation index keeps the previous entries and offsets this session's entries, expansions are keyed by their stable hash and merge, and each carried region keeps its syntax context table. After eight carried generations the save falls back to a full re-encode, which compacts away dead data left behind by red nodes and stale tables.

The change is two commits on top of the dep graph carry (rust-lang#160214). The first gives each data region its own syntax context table, since encoded context ids are only meaningful within the region written by their session. The second is the carry itself. An earlier version had a third commit re-tagging disk-cached values with their dep node, because the session-local index tag would go stale for values whose bytes outlive the session that wrote them. With the dep graph carry keeping every green node at its index, the index tag stays valid for as long as the value lives, and that commit is gone; re-pointing green entries into the carried region became an identity copy for the same reason.

The promotion pass is skipped when carrying, since the bytes it would re-encode are provably already in the new file, and still runs on saves that cannot carry, such as the periodic compaction session. Carried values are never decoded, so the verification that loading performs would not run for them: when carrying, the save decodes and verifies the same subset of not-loaded values that promotion would have, and drops them again.

The new file is written to a staging path and swapped in when complete, following the dep graph's staging file precedent, so the previous file can stay mapped while its data region is copied, on every platform including Windows. The header format version is bumped so caches written by earlier compilers are discarded cleanly.

Measured on the six primary rustc-perf crates (regex, syn, serde, cargo, ripgrep, hyper; instructions:u geomean, optimized stage2 builds of both sides): incr-unchanged -10.6%, incr-patched -1.4%, full and incr-full flat as a control, 25 cells improved / 3 regressed with the largest regression about +0.35%. rustc-perf forces -Zincremental-verify-ich on every benchmarked invocation, and under that flag the verification sweep decodes every carried value, so rust-timer would currently report about -2.3% incr-unchanged instead. These numbers predate the rebase onto the dep graph carry and need re-measuring on the new base.

tests/incremental passes in full (178 tests). A 16-generation churn and deletion stress, in the four mode combinations of -Zincremental-verify-ich and -Zthreads=8 and including an error session, replays warnings byte-identically through carried spans and hygiene, and its cache size trace shows the periodic compaction working.

@xmakro xmakro changed the title Carry green query values forward from the previous cache file Carry green query values forward from the previous cache file [-14.9% incr-unchanged, -4.3% avg] Jul 26, 2026
@xmakro xmakro changed the title Carry green query values forward from the previous cache file [-14.9% incr-unchanged, -4.3% avg] Carry green query values forward from the previous cache file [-16.3% incr-unchanged, -5.0% avg] Jul 26, 2026
@xmakro
xmakro force-pushed the perf/cache-carry-forward branch from 4925561 to c1e2fca Compare July 29, 2026 11:40
@xmakro xmakro changed the title Carry green query values forward from the previous cache file [-16.3% incr-unchanged, -5.0% avg] Carry green query values forward from the previous cache file [-10.6% incr-unchanged] Jul 29, 2026
@xmakro
xmakro changed the base branch from perf/base-159 to perf/base-0728 July 29, 2026 11:40
@xmakro
xmakro changed the base branch from perf/base-0728 to perf/carry-unchanged-nodes July 30, 2026 14:40
@xmakro
xmakro force-pushed the perf/cache-carry-forward branch from c1e2fca to 8178ee7 Compare July 30, 2026 14:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant