feat(refit): add canonical S3 revision receiver

[nv-hwoo]

Summary

Add the receiver-side reconstruction and SGLang installation path for canonical S3 revisions. The receiver seeds one private mutable checkpoint from the launch Hugging Face snapshot, applies exact-base XOR deltas in place, verifies the complete target identity, and installs the prepared checkpoint through SGLang’s native model loader.

This is a parallel effort along the current refit API work. A separate follow-up PRs will merge this into the standard refit API.

Changes

• Add receiver configuration and lifecycle state.
• Resolve revision metadata from the immutable catalog.
• Seed a private mutable checkpoint from the launch snapshot.
• Parse safetensors headers to locate tensor byte ranges.
• Download and validate the canonical delta index.
• Download S3 buckets concurrently.
• Apply XOR deltas directly to mmap-backed tensor regions.
• Verify per-tensor checksums and the complete target digest.
• Persist exact installed version, digest, format, and file identity.
• Install prepared weights through SGLang’s DefaultModelLoader.
• Distinguish failures before model mutation from failures after mutation begins.
• Mark receivers poisoned when an installation may have partially modified model state.
• Expose preparation, installation, status, and timing results to the framework integration.

Receive Flow

1. Resolve metadata-only revision 0 from the catalog.
2. Seed one private mutable checkpoint from the launch Hugging Face snapshot.
3. Verify its canonical format and target digest.
4. Resolve the requested target revision.
5. Require its base version and digest to equal the installed identity.
6. Download and validate the delta index.
7. Apply changed buckets directly to mmap-backed safetensors.
8. Verify tensor coverage and the complete reconstructed target identity.
9. Install the prepared checkpoint through SGLang’s native loader.
10. Report the exact installed version, digest, and receiver state.

Correctness

• The receiver accepts only an exact base_version + base_digest match.
• Canonical tensor coverage must equal the local checkpoint tensor set.
• Clean and dirty tensor metadata must preserve shape, dtype, and byte size.
• Bucket ordinals and tensor assignments must be unique and complete.
• Every reconstructed dirty tensor is checksum-verified.
• The final snapshot digest must equal the catalog target digest.
• Unexpected changes to local checkpoint files invalidate reuse.
• The checkpoint is marked poisoned before in-place mutation and cleared only after successful reconstruction.
• Loader setup failures remain recoverable; failures after possible model writes poison the receiver.

SGLang Integration

The receiver uses SGLang’s native DefaultModelLoader to install the prepared safetensors checkpoint. It does not alter the runner’s configured model path or load format, and it classifies failures according to whether model mutation may have begun.

Validation

• Receiver, checkpoint reconstruction, and SGLang integration tests: 15 passed
• Full focused refit stack: 55 passed
• pre-commit run --from-ref origin/main --to-ref hwoo/mx-delta-phase3-prs
• cargo fmt
• cargo clippy
• cargo check
• Multi-node Miles validation exercised canonical publication, S3 reconstruction, exact-version installation, and resumed generation using the complete stack.