Support for Multi-GPU and --ssd-streaming#568

Draft

iSevenDays wants to merge 57 commits into

antirez:mainfrom

iSevenDays:mgpu-ssd-merge-main

Commits on May 21, 2026

Commits on May 27, 2026

chore: ignore claude-dev-team workspace and symlinks

cchuter
and
claude
committed
docs: multi-GPU pipeline-parallel v0 design

cchuter
and
claude
committed
docs: address rounds 3-4 of codex spec review

cchuter
and
claude
committed
feat(mgpu): monotonic-contiguous layer-placement packer (wave 1) (#1)

cchuter
and
claude
authored
feat(mgpu): device-aware CUDA plumbing (wave 1) (#2)

cchuter
and
claude
authored
feat(mgpu): per-device selective model cache mechanism (wave 1) (#3)

cchuter
and
claude
authored
feat(mgpu): engine + placement scaffolding (wave 2) (#4)

cchuter
and
claude
authored

Commits on May 28, 2026

Commits on Jun 2, 2026

Commits on Jun 8, 2026

Commits on Jun 9, 2026

Commits on Jun 17, 2026

fix(cuda): allow Q4_K routed experts on prefill (n_tokens > 1)

cchuter
and
claude
committed

Commits on Jul 16, 2026

feat(ssd): port CUDA SSD streaming into mgpu build (single-tier)
ds4-mgpu-ssd
committed
feat(ssd): wire single-tier SSD streaming prefill producer + init

ds4-mgpu-ssd
and
claude
committed
Fix cuda prefill failed: implement SSD streaming stubs and port expert staging

ds4-mgpu-ssd
and
claude
committed
feat(ssd): per-tier SSD weight-streaming across multiple GPUs

ds4-mgpu-ssd
and
claude
committed
Merge antirez/main into feat/mgpu-ssd: 3-way union of 169 upstream commits + per-device SSD streaming

ds4-mgpu-ssd
and
claude
committed
Fix SSD streaming batch prefill: use ssd_current() for sort_expert_count

ds4-mgpu-ssd
and
claude
committed

Commits on Jul 17, 2026

Add Quick Start section to README

ds4-mgpu-ssd
and
claude
committed
Fix CUDA sticky errors bricking server: fail-fast so systemd restarts

ds4-mgpu-ssd
and
claude
committed
Fix multi-GPU prefill illegal-memory-access at the batch embed step

ds4-mgpu-ssd
and
claude
committed