Optional xet-backed HF downloads (LLAMA_XET) - #1
Open
assafvayner wants to merge 1 commit into
Open
Conversation
Adds an opt-in path that reconstructs xet-backed HuggingFace files through
the xet-core C API (xet_capi) instead of a direct HTTP GET. Gated behind the
LLAMA_XET CMake option (OFF by default), so the normal build is unchanged.
- CMakeLists.txt / common/CMakeLists.txt: add LLAMA_XET option; when ON,
add_subdirectory the xet_capi crate from XET_CORE_DIR, define LLAMA_USE_XET,
and link xet_capi.
- common/hf-cache.{h,cpp}: parse the tree API's top-level size and xetHash;
when a file is xet-backed, record its xet hash and build the read-token URL
(<endpoint>/api/models/<repo>/xet-read-token/<commit>).
- common/download.{h,cpp}: carry xet hash / token URL / sha256 / size on the
download task; add a per-file xet download (guarded by LLAMA_USE_XET) using
the xet_session.hpp C++ wrapper. On any error it warns and falls back to the
existing HTTP download. The CAS endpoint comes from the token response.
The xet C API and CMake glue live on the xet-core assaf/c-api branch
(huggingface/xet-core#881).
Assisted-by: Claude Code (Opus 4.8)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Adds an opt-in path that downloads xet-backed HuggingFace files through the xet-core C API (
xet_capi) instead of a direct HTTP GET. Gated behind theLLAMA_XETCMake option (OFF by default), so the normal build and behavior are unchanged.This is an experiment to evaluate presenting a native xet client integration to the llama.cpp maintainers; it is not intended for upstream as-is.
Changes
CMakeLists.txt,common/CMakeLists.txt): newLLAMA_XEToption. When ON,add_subdirectorythexet_capicrate fromXET_CORE_DIR, defineLLAMA_USE_XET, and linkxet_capi(with platform native libs handled by the crate's CMake glue).common/hf-cache.{h,cpp}): parse the tree API's top-levelsizeandxetHash; for xet-backed files, record the xet hash and build the read-token URL (<endpoint>/api/models/<repo>/xet-read-token/<commit>).common/download.{h,cpp}): carry xet hash / token URL / sha256 / size on the download task; add a per-file xet download (guarded byLLAMA_USE_XET) via thexet_session.hppC++ RAII wrapper. On any error it warns and falls back to the existing HTTP download. The CAS endpoint comes from the token response'scasUrl.Dependencies
The xet C API crate and its CMake glue live on the xet-core
assaf/c-apibranch: huggingface/xet-core#881. Build with-DLLAMA_XET=ON -DXET_CORE_DIR=/path/to/xet-core.Verification
End-to-end against a xet-backed repo (
bartowski/SmolLM2-135M-Instruct-GGUF): the model is reconstructed via the xet protocol and written into the standard HF cache layout (blobs/snapshots/refs); withLLAMA_XET=OFFthe build is byte-for-byte the existing HTTP path.