Repository navigation
[pull] master from microsoft:master - #144
Merged
Merged
Conversation
When a packfile in the shared object cache is corrupt or truncated (e.g. from a
past disk-full event), 'git multi-pack-index write/verify' fails with "could not
load pack N". The existing self-heal only deletes and rewrites the
multi-pack-index (MIDX), which does not fix the underlying pack: the rewrite
re-scans the same bad pack and keeps failing. The corruption then recurs
indefinitely.
PackfileMaintenanceStep now routes write/verify failures through a recovery path
that, when git reports a pack-load failure:
- Detection (always runs, even with recovery disabled): verifies each pack in
the object cache with 'git verify-pack' and reports every unreadable pack via
telemetry (Operation=FoundCorruptPack). The "could not load pack N" ordinal is
an internal MIDX position, not a filename, so per-pack verification is how we
find the actual bad file.
- Removal (gated, see kill switch below): deletes each corrupt pack's files
(.pack/.idx/.keep/.rev; Operation=DeletedCorruptPack), then deletes and
regenerates the MIDX from the packs that remain (fast path, no full repack).
Missing objects are re-fetched on demand.
- Corrupt prefetch pack (special case): prefetch packs are incremental and
ordered by timestamp, so a corrupt one invalidates every later prefetch pack
too - leaving a hole would let the newest surviving timestamp advance past it
so a later prefetch never backfills the gap. Recovery removes the corrupt
prefetch pack and every later prefetch pack
(Operation=DeletedHealthyPrefetchPack for the healthy ones removed purely due
to ordering), then requests a prefetch (via a callback GitMaintenanceScheduler
wires to a PrefetchStep, only when using a cache server) to re-download them
and rebuild the commit-graph.
Kill switch: the destructive pack removal is gated by a new git config,
gvfs.enable-packfile-recovery (default true). When false, GVFS still detects and
reports corrupt packs (Operation=FoundCorruptPack, then
CorruptPackRecoverySkipped) but deletes nothing and does not request a prefetch;
the non-destructive MIDX rewrite still runs, so behavior degrades to today's.
This gives a field kill switch without a redeploy if the destructive path ever
misbehaves.
This is stacked on the git-output bounding change: recovery runs additional git
commands (verify-pack, MIDX rewrites) against the corrupt repo, so it relies on
that change to keep a noisy stderr from OOM-ing the mount mid-recovery.
Review follow-ups:
- prefetchRestoreNeeded is now set only after a corrupt prefetch pack is
actually removed (RemovePackFileSet returns whether the .pack file was
deleted), instead of as soon as one is detected. If deletion is blocked,
the restore no longer runs while the corrupt pack is still present.
- DetectAndRemoveCorruptPacks now remembers, for the lifetime of a single
maintenance run, that it already reported corrupt packs with recovery
disabled, and skips the redundant per-pack verify-pack rescan on later
MIDX failures in that same run.
- DetectAndRemoveCorruptPacks now parses the corrupt pack's filename directly
out of the write/verify failure's stderr when git includes it (e.g.
"packfile pack-1234.pack does not match index" / "wrong index v2 file size
in pack-1234.idx"), and verifies only that candidate instead of every pack
in the object cache. This only helps when git actually names the file,
which it does for the verify-triggered failures this code mostly handles
(not for the rarer write-path "could not load pack N", which is genuinely
an unresolvable internal ordinal - confirmed by reading git's midx-write.c).
Falls back to verifying every pack whenever no candidate can be parsed, or
the parsed candidate turns out to be healthy, so detection is never less
thorough than before.
Tests:
- A verify failure reporting a pack-load error removes the corrupt pack and
rewrites the MIDX from the remaining good packs (recovery enabled).
- With recovery disabled, the same failure still verifies each pack and reports
the corrupt one but deletes nothing.
- A corrupt prefetch pack removes it and every later prefetch pack, keeps the
earlier healthy one, and requests a prefetch.
- A verify failure that names the corrupt pack directly verifies only that
pack (fast path).
- A verify failure that names a pack which turns out to be healthy falls back
to verifying every pack (fallback path).
Assisted-by: Claude Sonnet 5
Signed-off-by: Tyrie Vella <tyrielv@gmail.com>
…overy Auto-recover corrupt packfiles in packfile maintenance
The upgrade test job selected the last-known-good installer with `(Get-ChildItem gvfs-lkg\SetupGVFS*.exe).FullName`. Releases now publish both an x64 and an arm64 installer, so the glob matches two files and `.FullName` returns an array. `Start-Process -FilePath` then fails with "Cannot convert 'System.Object[]' to the type 'System.String'". Select the x64 installer explicitly. The x64 installer has no architecture suffix; the arm64 one is named `SetupGVFS.<version>-arm64.exe`. These tests run on an x64 runner and download the x64 "new" installer, so the x64 LKG installer is the correct match. Apply the same guard to the "new" installer selection and throw a clear error if no x64 installer is present. Assisted-by: Claude Opus 4.8 Signed-off-by: Tyrie Vella <tyrielv@gmail.com>
Address self-review feedback on the multi-installer fix. Replace the `-arm64` denylist plus `Select-Object -First 1` with a shared `Select-X64Installer` helper that positively matches the x64 asset by its suffix-less name (`SetupGVFS.<version>.exe`) and requires exactly one match. The denylist would still pass a future non-x64 asset (for example a `-x86` or `-arm` installer) and `-First 1` would then pick an arbitrary file. The positive allowlist matches the documented x64 naming contract and fails loudly when the directory holds an unexpected number of installers. The helper also removes the duplicated filter across the LKG and new installer selection, and its error message reports the directory and the files found. Note in a comment that arm64 upgrade is not exercised here because the runner is x64. Assisted-by: Claude Opus 4.8 Signed-off-by: Tyrie Vella <tyrielv@gmail.com>
Fix upgrade tests when the LKG release has multiple installers
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to subscribe to this conversation on GitHub.
Already have an account?
Sign in.
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
See Commits and Changes for more details.
Created by
pull[bot] (v2.0.0-alpha.4)
Can you help keep this open source service alive? 💖 Please sponsor : )