[Klaud Cold] qwen3.8next-fp8-h100-sglang-agentic-mtp: Qwen3.8-Flash-Next FP8 SGLang AgentX on H100 / H100 上 Qwen3.8-Flash-Next FP8 SGLang AgentX 配方 - #2756
Conversation
…AgentX on H100 Add the Qwen3.8-Flash-Next AgentX recipe on H100, served by SGLang with native NEXTN MTP. H100 is Hopper, so FP8: NVFP4 needs SM100 tensor cores. The SGLang cookbook does not list H100, so this is the H200 arm adjusted for the smaller part rather than a verified command. TP8 with EP8 instead of the cookbook's TP4/EP4, because 172.8 GiB at TP4 is ~43 GiB per rank of an 80 GB card and leaves too little for the 256k-capped agentic traces; TP8 halves that. Memory fraction 0.75 rather than 0.85, matching the Qwen3.5 H100 sibling. The SSM state is float32, as Hopper's flashinfer verify kernel requires and unlike the bfloat16 the Blackwell arms must use. No launcher change: the H100 model-path gate is inside the multinode branch, so the single-node path leaves MODEL_PATH unset and the bench script downloads into the mounted HuggingFace cache. 新增 H100 上的 Qwen3.8-Flash-Next AgentX 配方，由 SGLang 以原生 NEXTN MTP 提供 服务。H100 属 Hopper 架构，故使用 FP8（NVFP4 需要 SM100 张量核心）。 SGLang cookbook 未列出 H100，因此本配方是按较小硬件调整后的 H200 分支，而非官方 验证命令：采用 TP8/EP8 而非 cookbook 的 TP4/EP4——172.8 GiB 在 TP4 下每卡约 43 GiB，对 80 GB 显存而言留给 256k 智能体轨迹的空间过少，TP8 可将其减半；显存 占用取 0.75 而非 0.85，与 Qwen3.5 H100 同类配方一致。SSM 状态为 float32，这是 Hopper 上 flashinfer 验证内核的要求，与 Blackwell 分支必须使用的 bfloat16 相反。 无需改动 launcher：H100 的权重路径分支位于多节点条件内，单节点路径下 MODEL_PATH 保持未设置，基准脚本会下载到已挂载的 HuggingFace 缓存。 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…link Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase For PR verification, add the PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs 感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 如需进行 PR 验证，请为此 PR 添加 PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档 |
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
There was a problem hiding this comment.
Beyond the inline finding on the stale comment block, I also checked --reasoning-parser auto at line 194 — it's the only script in the repo using auto instead of a model-specific parser name, but nothing in the diff suggests this is a copy-paste error rather than an intentional choice for this model, so I'm not flagging it as a bug.
Extended reasoning...
The confirmed finding (stale comment referencing the Qwen3.5 curve/version instead of the actual Qwen3.8 values used in this script) will be posted inline and is sufficient reason to hold off on approval. I additionally checked the --reasoning-parser auto usage at line 194, which is unique across the 143 reasoning-parser references in the repo (every other sibling script passes a model-specific parser name like qwen3, deepseek_v4, minimax_m3, etc.). I could not find a qwen3.8next sibling script (e.g. an H200 arm) in this repo to compare against, so I could not confirm whether auto is a deliberate, valid choice for this specific model or an oversight; there is no vendored SGLang source in this checkout to verify the parser's behavior. I'm surfacing this as an examined-but-inconclusive item rather than a new finding, since I lack a comparison point to say it's wrong.
| # quality (golden_al_distribution/README.md). 3.39 is the Qwen3.5 MTP curve at | ||
| # num_speculative_tokens=3, thinking_on (golden_al_distribution/qwen3.5_mtp.yaml) | ||
| # -- the same value the GB300 Qwen3.5 AgentX srt-slurm recipes pin. |
There was a problem hiding this comment.
🟡 Stale copy-pasted comment block from the Qwen3.5 H100 sibling still claims the pinned golden AL is "3.39 ... the Qwen3.5 MTP curve" from golden_al_distribution/qwen3.5_mtp.yaml, and that the image is pinned for v0.5.16 vs the sibling's v0.5.12, even though this script actually exports SGLANG_SIMULATE_ACC_LEN=2.32 from golden_al_distribution/qwen3.8next_mtp.yaml a few lines below and uses a differently-named image tag (qwen38flashnext) with no version pin claim.
Extended reasoning...
A reviewer or future engineer auditing/reusing this recipe reads the outer comment, believes the pinned acceptance length is 3.39 for a Qwen3.5-style curve, and either miscopies that stale value into a new arm or is confused when the actual exported value (2.32) disagrees with the documentation, since the code correctly uses 2.32 but the surrounding prose was never updated after copy-paste from the qwen3.5_fp8_h100_mtp.sh template.
Verification: nit. The candidate is factually correct and the contradiction is on the changed lines. The outer comment (diff lines 114-122) states "3.39 is the Qwen3.5 MTP curve at num_speculative_tokens=3, thinking_on (golden_al_distribution/qwen3.5_mtp.yaml)" and claims the image is pinned because "SGLANG_SIMULATE_ACC_TOKEN_MODE landed in SGLang v0.5.16 ... rather than the non-MTP agentic sibling's…
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=33039903416 |
Same fix as PRs #2753 and #2752. validate_perf_changelog.py requires the appended suffix to start with "\n- config-keys:" unless the base file already ends in a blank line, and merge_with_reuse.sh refuses the merge without it. Historical bytes were already exact; this only inserts the separator. 与 PR #2753、#2752 相同的修复：追加条目必须与历史之间空一行，否则 merge_with_reuse.sh 会拒绝合并。历史字节本就完全一致，此处仅插入该分隔空行。 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
/reuse-sweep-run 33039903416 |
|
/reuse-sweep-run |
# Conflicts: # perf-changelog.yaml
|
/reuse-sweep-run 33039903416 |
1 participant
Summary / 摘要
Qwen3.8-Flash-Next AgentX recipe on H100, served by SGLang with native NEXTN MTP. Fifth arm of the one-PR-per-chip set, mirroring the H200 arm (#2753).
H100 上的 Qwen3.8-Flash-Next AgentX 配方，由 SGLang 提供服务，采用原生 NEXTN MTP。这是按芯片划分系列的第五个分支，对应 H200 分支（#2753）。
Config key / 配置项
qwen3.8next-fp8-h100-sglang-agentic-mtp—Qwen/Qwen3.8-Flash-Next-FP8,lmsysorg/sglang:qwen38flashnext,cluster:h100-dgxc, TP8/EP8, conc[1, 4, 8, 12, 16], no KV offloading.Not a verified cookbook command / 并非 cookbook 验证命令
The SGLang cookbook offers B300, GB300, B200, H200, MI350X and MI355X — not H100. So this is the H200 arm adjusted for the smaller part, and it is stated as such rather than presented as verified:
qwen3.5-fp8-h100-sglang-agentic-mtpsibling.--mem-fraction-static 0.75, not 0.85. 80 GB HBM3 has far less slack than H200's 141 GB HBM3e; 0.75 is what the Qwen3.5 H100 sibling uses.gated_delta_rule_mtpverify kernel asserts float32; the Blackwell arms must use bfloat16 because SGLang rejects anything else on SM100+. The two are genuinely opposite and the comments say so.SGLang cookbook 提供 B300、GB300、B200、H200、MI350X 与 MI355X，不含 H100。因此本配方是按较小硬件调整后的 H200 分支，并如实标注而非声称已验证：TP8/EP8 而非 TP4/EP4（TP4 下每卡约 43 GiB，80 GB 卡上留给 256k 轨迹的空间不足；TP8/EP8 亦与现有 Qwen3.5 H100 同类配方一致）；显存占用 0.75 而非 0.85；使用 FP8（Hopper 无 SM100 张量核心）；SSM 状态为 float32（Hopper 的 flashinfer 验证内核要求 float32，而 Blackwell 上 SGLang 只接受 bfloat16，两者确实相反，注释中已写明）。
Acceptance length / 接受长度
SGLANG_SIMULATE_ACC_LEN=2.32—golden_al_distribution/qwen3.8next_mtp.yamlthinking_on[3], the committed curve from #2749. Three speculative tokens per verification step is the MTP=3 cell.SGLANG_SIMULATE_ACC_LEN=2.32，取自 #2749 已提交的golden_al_distribution/qwen3.8next_mtp.yaml中thinking_on[3]。Files / 改动文件
benchmarks/single_node/agentic/qwen3.8next_fp8_h100_mtp.sh(new — the H100 launcher builds the script name without a framework infix, so no_sglangin the filename)configs/nvidia-master.yaml— new entry after the Qwen3.5 FP8 H100 siblingperf-changelog.yaml— appended entryNo launcher change needed: the H100 model-path gate lives inside the multinode branch, so the single-node path leaves
MODEL_PATHunset and the bench script downloads into the mounted HuggingFace cache.无需改动 launcher：H100 的权重路径分支位于多节点条件内，单节点路径下
MODEL_PATH保持未设置，基准脚本会下载到已挂载的 HuggingFace 缓存。Set: B200 (#2751), B300 (#2752), H200 (#2753), MI355X (#2754), H100 (this).
系列：B200（#2751）、B300（#2752）、H200（#2753）、MI355X（#2754）、H100（本 PR）。
🤖 Generated with Claude Code
Note
Low Risk
Benchmark and config-only changes with no production serving or auth paths; main risk is mis-tuned H100 memory/TP settings affecting benchmark validity rather than user-facing systems.
Overview
Adds the day-zero AgentX coding benchmark arm for Qwen/Qwen3.8-Flash-Next-FP8 on H100 via SGLang with NEXTN MTP, completing the per-chip Qwen3.8-Flash-Next set.
A new launcher script
qwen3.8next_fp8_h100_mtp.shmirrors the H200 recipe but tunes for 80 GB Hopper: TP8/EP8,--mem-fraction-static 0.75, flashinfer linear-attention backends,--mamba-ssm-dtype float32(Hopper verify path), 256k-capped trace corpus, optional HiCache when KV offload is requested, the usual multi_tokenizercached_tokens_detailspatch, and throughput runs pinned toSGLANG_SIMULATE_ACC_LEN=2.32(eval-only keeps real verification).configs/nvidia-master.yamlregistersqwen3.8next-fp8-h100-sglang-agentic-mtp(lmsysorg/sglang:qwen38flashnext,cluster:h100-dgxc, agentic-coding at TP8/EP8, conc 1/4/8/12/16, KV offloading none only).perf-changelog.yamldocuments the addition.Reviewed by Cursor Bugbot for commit 859d657. Bugbot is set up for automated code reviews on this repo. Configure here.