Allow HF checkpoint cache dir to be overridden via HF_HOME#20904
Allow HF checkpoint cache dir to be overridden via HF_HOME#20904chenweng-quic wants to merge 1 commit into
Conversation
The converted checkpoint cache was always placed under Path.home(), which is host-local. Honoring the HF_HOME environment variable lets the cache live on a shared disk that multiple hosts can reuse, avoiding redundant downloads and conversions per host.
🔗 Helpful Links🧪 See artifacts and rendered test results at hud.pytorch.org/pr/pytorch/executorch/20904
Note: Links to docs will display an error until the docs builds have been completed. ❗ 1 Active SEVsThere are 1 currently active SEVs. If your PR is affected, please view them below: ✅ You can merge normally! (1 Unrelated Failure)As of commit 697e3ea with merge base a6d812a ( BROKEN TRUNK - The following job failed but were present on the merge base:👉 Rebase onto the `viable/strict` branch to avoid these failures
This comment was automatically generated by Dr. CI and updates every 15 minutes. |
meta-cla
Bot
added
the
CLA Signed
This PR needs a
|
1 participant
User description
Summary
The converted Meta/HF checkpoint cache in
hf_download.pywas always placed underPath.home(), which is host-local. This change honors theHF_HOMEenvironment variable when building the cache path, so the cache can live on a shared disk that multiple hosts reuse — avoiding redundant downloads and conversions per host.Changes
examples/models/llama/hf_download.py: useHF_HOME(falling back toPath.home()) as the base for themeta_checkpointscache directory.Test plan
HF_HOMEunset: cache path resolves to~/.cache/meta_checkpoints(unchanged behavior).HF_HOME=/shared/disk: cache path resolves to/shared/disk/.cache/meta_checkpoints, shareable across hosts.