int8 (comfy_kitchen) models fail on non-default CUDA compute_device: libcudart.so loaded by POSIX name on Windows, then cudaErrorIllegalAddress

[dmmdea]

Symptom

With a comfy_kitchen int8-quantized transformer (LTX-2.5 int8-convrot) loaded through UNETLoaderDisTorch2MultiGPU with compute_device set to any non-default CUDA device (e.g. cuda:1), the first int8 linear in sampling fails on Windows:

1. \FileNotFoundError: Could not find module 'libcudart.so'\ — \p2p_registry._get_libcudart()\ hard-codes the POSIX library name, which can never resolve on Windows; the uncaught exception propagates through \wrap_for_dlpack_with_device_guard\ and kills sampling.
2. With the library name fixed locally (platform-aware candidates), the same call then raises \ orch.AcceleratorError: CUDA error: an illegal memory access was encountered\.

\compute_device: cuda:0\ works (historical default); bf16 models on cuda:1/cuda:2 work (no int8 dispatch). The breakage is specifically int8 dispatch x non-default device.

Stack (captured via a ctypes CDLL interception shim)

\
comfy_kitchen\backends\eager\quantization.py:1228 op_int8_linear
comfy_kitchen\backends\cuda_init.py:1937 int8_linear -> wrap_for_dlpack(x_2d)
ComfyUI-MultiGPU_init.py wrap_for_dlpack_with_device_guard -> p2p_registry -> CDLL('libcudart.so')
\\

What our local patch attempt shows (important)

We patched \p2p_registry.py\ locally: platform-aware cudart names (\cudart64_13.dll\ etc. on Windows) + treating P2P as unavailable on Windows/WDDM (fail-closed, since \cudaDeviceCanAccessPeer\ over-reports there). That cleanly fixes (1) — but (2) persists, and notably no \[MultiGPU DLPack] CPU-staging\ log line ever appears: the export guard's cross-device branch never triggers, so the faulting dereference happens on tensors the guard sees as same-device. Root cause looks like the int8 kernel launch relying on the process-default current device rather than the tensor's device — which the non-default \compute_device\ routing exposes. A cross-file to comfy_kitchen may be warranted; happy to test candidate fixes on this rig.

Environment

Windows 11 - ComfyUI 0.34.0 (12d52794) - ComfyUI-MultiGPU b51c99a (upstream HEAD at report time) - comfy_kitchen 0.2.31 - torch cu130 - 3x Blackwell (2x RTX 5060 Ti + RTX 5070 Ti) - launched with --disable-dynamic-vram per #191 guidance.

Repro

Any int8-convrot diffusion model via UNETLoaderDisTorch2MultiGPU with compute_device set to a non-default CUDA device; first SamplerCustomAdvanced execution fails at the first quantized linear (in our graphs, the LTX text-embeddings connector).

[ylzbj1-stack]

Independent confirmation on a different model + GPU pair, plus a deeper root cause we hit on the same stack.

Our setup: MiniMax H3 video DiT (34 GB int8_convrot) on 2x RTX 3080 20G, Windows, torch 2.10.0+cu130, DisTorch2 via UNETLoaderDisTorch2MultiGPU.

Same crash class, independently reproduced:

1. libcudart.so FileNotFoundError on Windows in p2p_registry._get_libcudart() - identical to this report. Our fix is the same approach: resolve the versioned DLL from torch/lib (cudart64_13.dll on cu130). Patch: multigpu_p2p-registry_windows-cudart.patch

2. After fixing that, we still hit repeated access violation segfaults through three paths (VAE eviction -> unpatch_model, second-prompt partially_load, and even fresh block streaming). faulthandler traces all point at dangling pointers into aimdo/vbar-managed quantized (int8_convrot) weights after any device move; the vbar ledger itself started returning negative garbage (vbar_free_memory = -8809040871136690176k). Python-level guards cannot fix a corrupted kernel-side ledger.

What we shipped: a static pipeline split custom node - blocks are placed across GPUs once and never move (hybrid residency: resident blocks + CPU-streamed blocks for long sequences), with activation-space LoRA so quantized weights keep the int8 fast path. On H3 88-frame jobs it beats dynamic sharding (10.91 vs 11.15 s/it) with zero crashes in 10+ runs, and reaches 15-second single segments (360 frames) reliably.

Repo (MIT, patches + full pitfall list): https://github.com/ylzbj1-stack/ComfyUI-StaticPipeline

Hope the datapoint and the Windows patch help - happy to provide faulthandler traces if useful.