omni: INT4 per-block zero-point GEMM + calling adapter (torchao asymmetric INT4) - #658
Open
JWLHS wants to merge 4 commits into
Open
omni: INT4 per-block zero-point GEMM + calling adapter (torchao asymmetric INT4)#658JWLHS wants to merge 4 commits into
JWLHS wants to merge 4 commits into
Conversation
…tric INT4) onednn_int4_gemm_preconverted gains an optional zp_u8 [G, N]. When provided it applies per-block zero points inside oneDNN, w = (q - zp) * scale — native support for INT4 models quantized on the torchao backend (Int4PlainInt32Tensor: int32 qdata byte view + per-block zp/scale, asymmetric). The per-block zero point centers biased weight distributions per block, lowering quantization error vs symmetric INT4.
- adapters/int4_gemm.py: int4_gemm wrapper over onednn_int4_gemm_preconverted with optional zp_u8 + capability check - config flag OMNIXPU_INT4_GEMM + patch component registration
- zp 存在：探测 preconverted 是否接受 zp_u8；不支持则警告+返回 None （调用方回退自身 python/torchao 路径），不报错 - 无 zp（wa4）：3 参调用，兼容不支持 zp 的老 kernel - apply() 报告 preconverted 可用性与 zp 支持
…correctness tests (a16 part)
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
This PR ports the INT4 per-block zero-point GEMM (torchao asymmetric INT4 format) and its ComfyUI calling adapter:
onednn_int4_gemm_preconverted: per-block zero points (w = (q - zp) * scaleinside oneDNN), fp16/bf16 activations × u4 weights, group sizes 32/64/128/256, K up to 12288 without NaN.int4_gemmadapter with runtime capability probing (falls back cleanly when the kernel lacks per-block zp support).tests/test_int4_preconverted_correctness.py(per-block zp vs manualw=(q-zp)*scale, fp16/bf16/f32, K=12288, CPU-tensor rejection).Verification:
onednn_int4_gemm_preconverted6/6 pass on the same wheel.Note: the W4A8 s8u4 GEMM previously bundled here is split out and stays in #629; this PR only carries the verified INT4 (a16) path.