feat(model-host): sbproxy models prune to reclaim unreferenced weight blobs#716
Merged
Conversation
… blobs The weight store is already content-addressed (files live in blobs/sha256 keyed by digest, and a duplicate partial is dropped when the blob already exists, so two models sharing a shard store it once) and the cache-budget sweep already deletes unreferenced blobs. But an operator had no way to reclaim orphans on demand: blobs left by an interrupted pull, or shards a removed model no longer keeps alive, only got collected when a byte budget forced an eviction pass. Add `sbproxy models prune`. It runs under the same collection lock as the budget sweep, so it never races a concurrent pull's promote, and reclaims exactly the blobs no cached artifact references. `--dry-run` reports the reclaimable bytes and orphan count without deleting anything. A cold or already-clean cache is a no-op. `--cache-dir` overrides the target, else the configured cache directory (with `-f`) then the platform default, the same resolution `models list` uses. This is the standalone-prune slice of WOR-1862. The store is verified already content-addressed with cross-model dedup; the hf-hub 1.0 / hf-xet transport migration is tracked separately (hf-hub 1.0 does not bundle Xet; Xet is the separate hf-xet crate, so that scope needs its own evaluation).
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What
sbproxy models prune— reclaim content-addressed weight blobs that no cached artifact references (orphans from an interrupted pull, or shards a removed model no longer keeps alive).--dry-runreports reclaimable bytes + orphan count and deletes nothing.--cache-diroverrides the target, else the configured cache dir (with-f) then the platform default — same resolutionmodels listuses.models.prune,schema_version: 1) and text output.Live-validated: cold cache →
0 B; planted orphan → dry-run reports4096 B / 1 bloband leaves it; real prune reclaims it; a referenced blob survives.Context — what WOR-1862 actually needs
Investigating the ticket surfaced two things worth recording:
promote_blobstores each file atblobs/sha256/<digest>and drops a duplicate partial when the blob already exists, so two serve entries sharing a base model already store its shards once. Resumable downloads (partials/+ resume metadata) also already exist. That scope bullet is largely satisfied by existing code.blocking,rustls-tls,socks; Xet lives in a separatehf-xet1.5.3 crate with its own CAS/chunking model. So "hf-hub 1.0 with the Xet feature" (the ticket premise) is inaccurate, and swapping our working resumable-HTTP transport for hf-hub alone adds little. The real Xet migration is a larger, uncertain effort that deserves its own evaluation.So this PR ships the clean, self-contained, offline part of 1862 (the
prunecommand, satisfying the "prune on a cold cache is a no-op and on a dirty cache reclaims exactly the unreferenced bytes" acceptance criterion). The transport migration is left for a focused follow-up; findings are on the ticket.Tests
Unit: cold no-op, dry-run non-destructive, exact reclamation, referenced blob survives, second prune is a no-op. Full gate green: nextest 6985/6985, clippy
-D warnings, rustdoc, doctest, llms-full regen.