Skip to content

feat: allow online FP8 quantization with W4A16 dense FFN - #1454

Merged
ZhangZhiPku merged 4 commits into
alibaba:develop/pro5000-optfrom
HongminTan:develop/pro5000-opt-qwen
Sep 23, 2026
Merged

ZhangZhiPku merged 4 commits into
alibaba:develop/pro5000-optfrom
HongminTan:develop/pro5000-opt-qwen

Conversation

@HongminTan

Copy link
Copy Markdown
Collaborator

No description provided.

Copilot AI lite review requested due to automatic review settings September 22, 2026 09:35

Copilot AI left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Copilot was unable to review this pull request because the user who requested the review has reached their quota limit.

Comment thread rtp_llm/model_loader/w4a16_weight.py
Comment thread rtp_llm/models_py/kernels/cuda/w4a16_sm120.py Outdated
Comment thread rtp_llm/model_loader/weight_module.py
Comment thread rtp_llm/models_py/modules/hybrid/w4a16_dense_mlp.py
Comment thread rtp_llm/config/model_config.py
@rtp-llm-review-bot

rtp-llm-review-bot commented Sep 22, 2026

Copy link
Copy Markdown
Collaborator

PR #1454 评审:未发现阻塞项,等待人工审核

  • 标题:feat: allow online FP8 quantization with W4A16 dense FFN(@HongminTan,open)
  • 评审版本:ade358e46ca1

无阻塞项

非阻塞发现

已确认修复

  • [P2] support 白名单包含无量化权重类的 "FP8" 方法,启用时会静默退化为 BF16 — 当前 head 第 39–43 行白名单已移除 "FP8" 和 "FP8_PER_CHANNEL_QUARK",仅保留 FP8_DYNAMIC_PER_TENSOR/FP8_PER_BLOCK/FP8_PER_CHANNEL_COMPRESSED,发现所指的 "FP8" 方法放行路径已被删除。

自动代码评审 · head ade358e46ca1 · 未发现阻塞项,等待人工审核

Comment thread rtp_llm/model_loader/w4a16_weight.py
Comment thread rtp_llm/models_py/kernels/cuda/w4a16_sm120.py
Comment thread rtp_llm/cpp/normal_engine/speculative/MtpBatchStreamProcessor.cc
Comment thread rtp_llm/model_loader/weight_module.py
Comment thread rtp_llm/cpp/models/PyWrappedModel.cc
Comment thread rtp_llm/models_py/kernels/cuda/w4a16_sm120.py
Comment thread rtp_llm/models_py/kernels/cuda/w4a16_sm120.py
@ZhangZhiPku
ZhangZhiPku merged commit d3c8d1a into alibaba:develop/pro5000-opt Sep 23, 2026
3 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

4 participants