Support for Multi-GPU and --ssd-streaming

[iSevenDays]

Correctness Tests.

1. Single-GPU SSD (CUDA_VISIBLE_DEVICES=0, --ctx 131072): "What is 8+8?" -> content="16"
2. 2-GPU SSD (--gpu-vram 45,45, --ctx 131072): "What is 8+8?" -> content="16", nvidia-smi confirms both GPUs active (28.5GB / 28.0GB)

Test 1 — Single-GPU SSD:

Server (background):

cd /root/ds4-mgpu-ssd && CUDA_VISIBLE_DEVICES=0 ./ds4-server --cuda --ssd-streaming --ssd-streaming-cache-experts 16GB --ctx 131072 -m /root/antirez/ds4/gguf/DeepSeek-V4-Flash-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-chat-v2-imatrix.gguf --host 0.0.0.0 --port 8012 &

(I actually used v4 flash official q2 quant, so you can just ln (link) it directly)

Request (separate call, once "listening on" appeared in logs):

curl -s http://localhost:8012/v1/chat/completions -H "Content-Type: application/json" -d '{"model":"ds4","messages":[{"role":"user","content":"What is 8+8? Just the number."}],"max_tokens":48}'

Test 2 — 2-GPU SSD:

Server (background):

cd /root/ds4-mgpu-ssd && ./ds4-server --cuda --ssd-streaming --ssd-streaming-cache-experts 16GB --gpu-vram 45,45 --ctx 131072 -m /root/antirez/ds4/gguf/DeepSeek-V4-Flash-IQ2XXS-w2Q2K-AProjQ8-SExpQ8-OutQ8-chat-v2-imatrix.gguf --host 0.0.0.0 --port 8013 &

Request:

curl -s http://localhost:8013/v1/chat/completions -H "Content-Type: application/json" -d '{"model":"ds4","messages":[{"role":"user","content":"What is 8+8? Just the number."}],"max_tokens":48}'

Both returned "content": "16".

context window of 131k is supported on just a single 4090d (48Gb VRAM)

Quickstart:

./download_model.sh q2-imatrix
make cuda CUDA_ARCH=sm_89
./ds4-server --cuda --ssd-streaming --ssd-streaming-cache-experts 16GB --ctx 131072 --gpu-vram 45,45

curl -s -w "\nTTFT: %{time_starttransfer}s\nTotal: %{time_total}s\n" http://localhost:8013/v1/chat/completions -H "Content-Type: application/json" -d @/tmp/bench_short_req.json

Info: thanks user @cchuter for initial multi-gpu support. This MR resolves issues with main branch, and adds ssd-streaming support.