Does the distorch2 plugin support int8 models?

[sansin-sms]

Today, while trying the INT8 version of the Minimax‑H3 model, I found that neither the UNETLoaderDisTorch2MultiGPU nor the CLIPLoaderDisTorch2MultiGPU node can properly distribute the VRAM usage of the INT8 model to cuda:1 — cuda:1 remains idle all the time. Is this because the plugin does not support VRAM allocation for INT8 models? And do you have any plans to upgrade the plugin in the future?

[ipechman]

[Bug] MiniMax H3 mixed INT8/INT4 checkpoint ignores DisTorch2 cuda:1 donor allocation on Windows

Summary

I am trying to run a mixed-quantization MiniMax H3 checkpoint across two NVIDIA GPUs using UNETLoaderDisTorch2MultiGPU.

DisTorch2 accepts and prints the requested allocation:

#cuda:0;12.0;cuda:1

However, cuda:1 never receives the requested model weights. During generation, GPU 0 remains fully utilized while GPU 1 stays at approximately CUDA-context-only memory usage.

The generation itself does not crash. The problem appears to be a silent failure or fallback where the model uses cuda:0 plus system RAM instead of placing approximately 12 GiB of UNET weights on cuda:1.

Relevant console output

[INFO] Found quantization metadata version 1
[INFO] Detected mixed precision quantization
[INFO] Using mixed precision operations
[INFO] Native ops: float8_e5m2, float8_e4m3fn, int8_tensorwise,
convrot_w4a4, asym_w4a8_int8
[INFO] model weight dtype torch.bfloat16, manual cast: torch.bfloat16
[INFO] model_type FLOW_AV

[MultiGPU DisTorch V2] Full allocation string: #cuda:0;12.0;cuda:1

[INFO] Applying MiniMax H3 Memory Efficient Sage Attention Patch
to all transformer blocks

[INFO] Requested to load MiniMaxH3
[INFO] 0 models unloaded.
[INFO] Model MiniMaxH3 prepared for dynamic VRAM loading.
19995MB Staged. 208 patches attached.

The allocation string is printed, but I do not see any later message confirming that a meaningful portion of the model was actually placed on cuda:1.

Actual GPU usage

During active sampling:

GPU 0:
Memory: 12486 MiB / 16376 MiB
Utilization: 100%
Power: approximately 245–257 W

GPU 1:
Memory: 390 MiB / 15356 MiB
Utilization: 0%
Power: approximately 16 W

These values remained essentially unchanged throughout the monitoring interval.

The ComfyUI Python process appears in the process list for both GPUs, but cuda:1 never rises beyond approximately 390 MiB.

I understand that a donor GPU may have little or no compute utilization because computation is still performed on cuda:0. The unexpected part is that the donor GPU also has no meaningful VRAM allocation. If 12 GiB of weights were placed on cuda:1, I would expect its VRAM usage to increase substantially.