[Klaud Cold] qwen3.8next-fp4-b300-sglang-agentic-mtp: Qwen3.8-Flash-Next NVFP4 SGLang AgentX on B300 / B300 上 Qwen3.8-Flash-Next NVFP4 SGLang AgentX 配方 - #2752
Conversation
|
Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase For PR verification, add the PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs 感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 如需进行 PR 验证，请为此 PR 添加 PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档 |
…link Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
| - "Add the day-zero Qwen3.8-Flash-Next NVFP4 AgentX recipe on B300 with SGLang native NEXTN MTP at TP4 and concurrency 1/4/8/12/16." | ||
| - "Serve RadixArk/Qwen3.8-Flash-Next-NVFP4 with modelopt_fp4 quantization, the trtllm_mha attention backend, and the flashinfer_trtllm MoE runner, following the Qwen3.5 NVFP4 B300 sibling." | ||
| - "Pin throughput runs to an interim acceptance length of 3.24 measured at three speculative tokens; eval-only runs keep real target verification." | ||
| pr-link: TBD |
There was a problem hiding this comment.
🟡 New changelog entry uses pr-link: TBD instead of a real PR URL
Extended reasoning...
Every other of the 781 existing perf-changelog.yaml entries cites a real https://github.com/.../pull/NNNN URL; this new entry is the only one with a placeholder 'TBD', so anyone auditing performance history for this recipe cannot trace the change back to its PR, and if merged as-is the placeholder becomes a permanent, un-updatable artifact in the byte-sensitive append-only file.
Verification: nit. perf-changelog.yaml:6501 sets pr-link: TBD. This is the single exception among 782 pr-link fields in the file — every other entry cites a real https://github.com/SemiAnalysisAI/InferenceX/pull/NNNN URL (e.g. lines 6453, 6468, 6477, 6492 all point at real PRs). The new entry (6493-6501) for qwen3.8next-fp4-b300-sglang-agentic-mtp is the only placeholder, so the recipe's changelog entry…
|
|
||
|
|
||
| # Qwen3.8-Flash-Next NVFP4 AgentX on B300 via SGLang with native NEXTN MTP. | ||
| # Day-zero recipe; mirrors the B200 arm, with B300's larger HBM leaving more | ||
| # room per rank at the same TP4 layout. | ||
| qwen3.8next-fp4-b300-sglang-agentic-mtp: | ||
| image: lmsysorg/sglang:qwen38flashnext | ||
| model: RadixArk/Qwen3.8-Flash-Next-NVFP4 | ||
| model-prefix: qwen3.8next | ||
| runner: cluster:b300-nv | ||
| precision: fp4 | ||
| framework: sglang | ||
| multinode: false | ||
| scenarios: | ||
| agentic-coding: | ||
| - dram-utilization: 0.8 | ||
| search-space: | ||
| - { tp: 4, ep: 1, spec-decoding: mtp, kv-offloading: none, conc-list: [1, 4, 8, 12, 16] } |
There was a problem hiding this comment.
🟡 New block inserted with double blank line before it but no blank line after, before the pre-existing '# Controlled AgentX power A/B' comment (nit: formatting)
Extended reasoning...
The pre-existing comment block describing qwen3.5-fp8-b300-sglang-agentic-power-ab (previously separated from the prior entry by exactly one blank line) now immediately follows the new qwen3.8next-fp4-b300-sglang-agentic-mtp entry's last search-space line with no blank line, making the comment read as if it documents the new qwen3.8next entry rather than the power-ab entry it actually precedes, confusing future readers/editors of this config file.
Verification: nit (cosmetic only). The described formatting is real: configs/nvidia-master.yaml lines 7033-7034 show a double blank line before the new qwen3.8next-fp4-b300-sglang-agentic-mtp: block, and line 7050 (- { tp: 4, ep: 1, ... conc-list: [1, 4, 8, 12, 16] }) is immediately followed at line 7051 by the pre-existing # Controlled AgentX power A/B: comment with no intervening blank line. That…
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=33035415388 |
…AgentX on B300 Add the Qwen3.8-Flash-Next AgentX recipe on B300, served by SGLang with native NEXTN MTP. Mirrors the B200 arm: RadixArk/Qwen3.8-Flash-Next-NVFP4 at TP4 with modelopt_fp4 quantization, following the Qwen3.5 NVFP4 B300 sibling. Throughput runs pin an interim acceptance length of 3.24 at three speculative tokens; eval-only runs keep real target verification. 新增 B300 上的 Qwen3.8-Flash-Next AgentX 配方，由 SGLang 以原生 NEXTN MTP 提供 服务。与 B200 分支一致：使用 RadixArk/Qwen3.8-Flash-Next-NVFP4，TP4， modelopt_fp4 量化，参照 Qwen3.5 NVFP4 B300 同类配方。吞吐运行按 3 个投机 token 锁定临时接受长度 3.24，仅评测运行仍使用真实目标验证。 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…link Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Same correction as the B200 arm. The SGLang cookbook playground's verified single-node command for Qwen3.8-Flash-Next on B300 is --tp 1: 126 GiB of NVFP4 weights fit on one 288 GB part. It takes the hybrid GDN linear-attention backends rather than a single --attention-backend, reads quantization from the checkpoint, and drops the trtllm_mha / flashinfer_trtllm MoE flags inherited from the Qwen3.5 recipe. --max-running-requests stays explicit, since NEXTN silently resets it to 48 when unset. Acceptance length moves from the interim 3.24 to the committed golden thinking_on value of 2.32. 与 B200 分支相同的修正。SGLang cookbook playground 给出的 Qwen3.8-Flash-Next B300 单节点验证命令为 --tp 1：126 GiB 的 NVFP4 权重可放入单张 288 GB 卡。该命令 改用混合 GDN 线性注意力的独立后端，量化直接从权重读取，并去掉了沿袭自 Qwen3.5 配方的 trtllm_mha 与 flashinfer_trtllm MoE 参数。--max-running-requests 保持显式 设置，因为 NEXTN 在其未设置时会静默重置为 48。接受长度由临时值 3.24 改为已提交 黄金曲线的 thinking_on 取值 2.32。 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
e9ed5de to
5a369bc
Compare
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=33036503421 |
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 5a369bc. Configure here.
| --speculative-num-steps 3 | ||
| --speculative-eagle-topk 1 | ||
| --speculative-num-draft-tokens 4 | ||
| --reasoning-parser auto |
There was a problem hiding this comment.
Missing agentic tool-call parser
Medium Severity
The serve command sets --reasoning-parser auto and omits --tool-call-parser. This model's in-tree speedbench collector and every Qwen3.5 AgentX SGLang sibling pass --reasoning-parser qwen3 and --tool-call-parser qwen3_coder as the official serving parsers. Without qwen3_coder, tool calls stay in free text, which breaks live-assistant replay and any tool-using eval.
Reviewed by Cursor Bugbot for commit 5a369bc. Configure here.
flashinfer's gated_delta_rule_mtp asserts initial_state.dtype == torch.float32 and aborts CUDA graph capture on the bfloat16 SSM state the cookbook command specifies. Confirmed on the H200 arm, which died at "Capture target verify CUDA graph"; this arm runs the same GDN backend, the same kernel and the same NEXTN, so it would fail identically once it got that far. flashinfer 的 gated_delta_rule_mtp 断言 initial_state 必须为 float32，遇到 cookbook 命令指定的 bfloat16 SSM 状态会在 CUDA graph 捕获阶段中止。该问题已在 H200 分支确认；本分支使用相同的 GDN 后端、相同内核与相同的 NEXTN，运行到同一阶段 必然同样失败。 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=33036805661 |
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=33038522522 |
My previous commit copied the H200 float32 fix here, and SGLang refused to start at all: ValueError: --linear-attn-decode-backend flashinfer on SM100+ requires --mamba-ssm-dtype bfloat16, got 'float32' The two architectures want opposite things. On SM100+ the flashinfer linear attention decode backend requires a bfloat16 state and validates it in server_args before launch. On Hopper the same backend's MTP verify kernel asserts a float32 state during CUDA graph capture. So the cookbook's bfloat16 is correct here and the H200 arm is the exception, not the other way round. Reverted, with the asymmetry written down so neither arm gets 'fixed' to match the other again. 上一个提交把 H200 的 float32 改动照搬到本分支，导致 SGLang 直接拒绝启动：SM100+ 上使用 flashinfer 线性注意力解码后端时必须搭配 bfloat16。两种架构的要求正好相反： SM100+ 在 server_args 中校验并要求 bfloat16；而 Hopper 上同一后端的 MTP 验证内核 在 CUDA graph 捕获阶段要求 float32。因此 cookbook 的 bfloat16 在此处是正确的， H200 才是例外。现已回退，并把这一不对称写入注释，避免两侧再被互相“对齐”。 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=33039186365 |
|
/reuse-sweep-run |
Same fix as PR #2753. validate_perf_changelog.py requires the appended suffix to start with "\n- config-keys:" unless the base file already ends in a blank line, and merge_with_reuse.sh could not resolve the perf-changelog conflict without it: ERROR: new changelog entries must be separated from history by one empty line and appended at the end Historical bytes were already exact; this only inserts the separator. 与 PR #2753 相同的修复。validate_perf_changelog.py 要求追加部分以 "\n- config-keys:" 开头（除非基础文件本身以空行结尾），缺少该空行时 merge_with_reuse.sh 无法解决 perf-changelog 冲突。历史字节本就完全一致，此处仅插入 分隔空行。 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
/reuse-sweep-run |
# Conflicts: # perf-changelog.yaml
* qwen3.8next-fp4-b300-sglang-agentic-mtp: day-zero Qwen3.8-Flash-Next AgentX on B300 Add the Qwen3.8-Flash-Next AgentX recipe on B300, served by SGLang with native NEXTN MTP. Mirrors the B200 arm: RadixArk/Qwen3.8-Flash-Next-NVFP4 at TP4 with modelopt_fp4 quantization, following the Qwen3.5 NVFP4 B300 sibling. Throughput runs pin an interim acceptance length of 3.24 at three speculative tokens; eval-only runs keep real target verification. 新增 B300 上的 Qwen3.8-Flash-Next AgentX 配方，由 SGLang 以原生 NEXTN MTP 提供 服务。与 B200 分支一致：使用 RadixArk/Qwen3.8-Flash-Next-NVFP4，TP4， modelopt_fp4 量化，参照 Qwen3.5 NVFP4 B300 同类配方。吞吐运行按 3 个投机 token 锁定临时接受长度 3.24，仅评测运行仍使用真实目标验证。 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Fill perf-changelog pr-link for #2752 / 补全 #2752 的 perf-changelog pr-link Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Use the cookbook's verified TP1 command and the committed golden AL Same correction as the B200 arm. The SGLang cookbook playground's verified single-node command for Qwen3.8-Flash-Next on B300 is --tp 1: 126 GiB of NVFP4 weights fit on one 288 GB part. It takes the hybrid GDN linear-attention backends rather than a single --attention-backend, reads quantization from the checkpoint, and drops the trtllm_mha / flashinfer_trtllm MoE flags inherited from the Qwen3.5 recipe. --max-running-requests stays explicit, since NEXTN silently resets it to 48 when unset. Acceptance length moves from the interim 3.24 to the committed golden thinking_on value of 2.32. 与 B200 分支相同的修正。SGLang cookbook playground 给出的 Qwen3.8-Flash-Next B300 单节点验证命令为 --tp 1：126 GiB 的 NVFP4 权重可放入单张 288 GB 卡。该命令 改用混合 GDN 线性注意力的独立后端，量化直接从权重读取，并去掉了沿袭自 Qwen3.5 配方的 trtllm_mha 与 flashinfer_trtllm MoE 参数。--max-running-requests 保持显式 设置，因为 NEXTN 在其未设置时会静默重置为 48。接受长度由临时值 3.24 改为已提交 黄金曲线的 thinking_on 取值 2.32。 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Use a float32 Mamba SSM state so the NEXTN verify kernel accepts it flashinfer's gated_delta_rule_mtp asserts initial_state.dtype == torch.float32 and aborts CUDA graph capture on the bfloat16 SSM state the cookbook command specifies. Confirmed on the H200 arm, which died at "Capture target verify CUDA graph"; this arm runs the same GDN backend, the same kernel and the same NEXTN, so it would fail identically once it got that far. flashinfer 的 gated_delta_rule_mtp 断言 initial_state 必须为 float32，遇到 cookbook 命令指定的 bfloat16 SSM 状态会在 CUDA graph 捕获阶段中止。该问题已在 H200 分支确认；本分支使用相同的 GDN 后端、相同内核与相同的 NEXTN，运行到同一阶段 必然同样失败。 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Keep the bfloat16 SSM state on Blackwell My previous commit copied the H200 float32 fix here, and SGLang refused to start at all: ValueError: --linear-attn-decode-backend flashinfer on SM100+ requires --mamba-ssm-dtype bfloat16, got 'float32' The two architectures want opposite things. On SM100+ the flashinfer linear attention decode backend requires a bfloat16 state and validates it in server_args before launch. On Hopper the same backend's MTP verify kernel asserts a float32 state during CUDA graph capture. So the cookbook's bfloat16 is correct here and the H200 arm is the exception, not the other way round. Reverted, with the asymmetry written down so neither arm gets 'fixed' to match the other again. 上一个提交把 H200 的 float32 改动照搬到本分支，导致 SGLang 直接拒绝启动：SM100+ 上使用 flashinfer 线性注意力解码后端时必须搭配 bfloat16。两种架构的要求正好相反： SM100+ 在 server_args 中校验并要求 bfloat16；而 Hopper 上同一后端的 MTP 验证内核 在 CUDA graph 捕获阶段要求 float32。因此 cookbook 的 bfloat16 在此处是正确的， H200 才是例外。现已回退，并把这一不对称写入注释，避免两侧再被互相“对齐”。 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * chore: prepare PR 2752 ingest recovery * fix: recover PR 2752 ingest --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
Same fix as PRs #2753 and #2752. validate_perf_changelog.py requires the appended suffix to start with "\n- config-keys:" unless the base file already ends in a blank line, and merge_with_reuse.sh refuses the merge without it. Historical bytes were already exact; this only inserts the separator. 与 PR #2753、#2752 相同的修复：追加条目必须与历史之间空一行，否则 merge_with_reuse.sh 会拒绝合并。历史字节本就完全一致，此处仅插入该分隔空行。 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…ext FP8 SGLang AgentX on H100 / H100 上 Qwen3.8-Flash-Next FP8 SGLang AgentX 配方 (#2756) * qwen3.8next-fp8-h100-sglang-agentic-mtp: day-zero Qwen3.8-Flash-Next AgentX on H100 Add the Qwen3.8-Flash-Next AgentX recipe on H100, served by SGLang with native NEXTN MTP. H100 is Hopper, so FP8: NVFP4 needs SM100 tensor cores. The SGLang cookbook does not list H100, so this is the H200 arm adjusted for the smaller part rather than a verified command. TP8 with EP8 instead of the cookbook's TP4/EP4, because 172.8 GiB at TP4 is ~43 GiB per rank of an 80 GB card and leaves too little for the 256k-capped agentic traces; TP8 halves that. Memory fraction 0.75 rather than 0.85, matching the Qwen3.5 H100 sibling. The SSM state is float32, as Hopper's flashinfer verify kernel requires and unlike the bfloat16 the Blackwell arms must use. No launcher change: the H100 model-path gate is inside the multinode branch, so the single-node path leaves MODEL_PATH unset and the bench script downloads into the mounted HuggingFace cache. 新增 H100 上的 Qwen3.8-Flash-Next AgentX 配方，由 SGLang 以原生 NEXTN MTP 提供 服务。H100 属 Hopper 架构，故使用 FP8（NVFP4 需要 SM100 张量核心）。 SGLang cookbook 未列出 H100，因此本配方是按较小硬件调整后的 H200 分支，而非官方 验证命令：采用 TP8/EP8 而非 cookbook 的 TP4/EP4——172.8 GiB 在 TP4 下每卡约 43 GiB，对 80 GB 显存而言留给 256k 智能体轨迹的空间过少，TP8 可将其减半；显存 占用取 0.75 而非 0.85，与 Qwen3.5 H100 同类配方一致。SSM 状态为 float32，这是 Hopper 上 flashinfer 验证内核的要求，与 Blackwell 分支必须使用的 bfloat16 相反。 无需改动 launcher：H100 的权重路径分支位于多节点条件内，单节点路径下 MODEL_PATH 保持未设置，基准脚本会下载到已挂载的 HuggingFace 缓存。 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Fill perf-changelog pr-link for #2756 / 补全 #2756 的 perf-changelog pr-link Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> * Separate the appended changelog entry from history with a blank line Same fix as PRs #2753 and #2752. validate_perf_changelog.py requires the appended suffix to start with "\n- config-keys:" unless the base file already ends in a blank line, and merge_with_reuse.sh refuses the merge without it. Historical bytes were already exact; this only inserts the separator. 与 PR #2753、#2752 相同的修复：追加条目必须与历史之间空一行，否则 merge_with_reuse.sh 会拒绝合并。历史字节本就完全一致，此处仅插入该分隔空行。 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com> --------- Co-authored-by: Claude Opus 5 (1M context) <noreply@anthropic.com>
1 participant
Summary / 摘要
Qwen3.8-Flash-Next AgentX recipe on B300, served by SGLang with native NEXTN MTP. Mirrors the B200 arm (#2751) at the same TP4 layout. SGLang is this model's plan-of-record engine per
MODELS.md.B300 上的 Qwen3.8-Flash-Next AgentX 配方，由 SGLang 提供服务，采用原生 NEXTN MTP，与 B200 分支（#2751）保持一致的 TP4 布局。按
MODELS.md，SGLang 是该模型的 PoR 引擎。Config key / 配置项
qwen3.8next-fp4-b300-sglang-agentic-mtp—RadixArk/Qwen3.8-Flash-Next-NVFP4,lmsysorg/sglang:qwen38flashnext,cluster:b300-nv, TP4/EP1, conc[1, 4, 8, 12, 16], no KV offloading.Recipe decisions / 配方要点
Checkpoint.
RadixArk/Qwen3.8-Flash-Next-NVFP4, 126 GiB,quantization_config.quant_method = modelopt— so--quantization modelopt_fp4is the right flag, the same one the Qwen3.5 NVFP4 B300 sibling uses. The checkpoint'signorelist keepsmtp.*unquantized, so NEXTN needs no external drafter.Image.
lmsysorg/sglang:qwen38flashnext, the model bring-up tag published 2026-08-26 (verified on Docker Hub).TP4/EP1. B300's larger HBM leaves more room per rank than B200 at the same layout. EP1 keeps the day-zero arm off the DeepEP/a2a path.
Serve flags are carried from
qwen3.5_fp4_b300_sglang_mtp.shunchanged:trtllm_mhaattention,flashinfer_trtllmMoE runner,fp8_e4m3KV cache, bf16 Mamba SSM, NEXTN with 3 steps / eagle-topk 1 / 4 draft tokens.No launcher change needed:
runners/launch_b300-nv.shalready resolvesagentic/<prefix>_<precision>_b300_<framework>_mtp.sh.权重：
RadixArk/Qwen3.8-Flash-Next-NVFP4，126 GiB，quant_method = modelopt，故使用--quantization modelopt_fp4，与 Qwen3.5 NVFP4 B300 同类配方一致；其ignore列表保留mtp.*不量化，因此 NEXTN 无需外部草稿模型。镜像：
lmsysorg/sglang:qwen38flashnext（2026-08-26 发布的模型适配标签，已在 Docker Hub 核实）。TP4/EP1：B300 显存更大，同布局下每卡余量优于 B200；EP1 让首发分支避开 DeepEP/a2a 路径。
服务参数沿用
qwen3.5_fp4_b300_sglang_mtp.sh未作改动。无需改动 launcher：
runners/launch_b300-nv.sh已能解析该脚本名。Acceptance length / 接受长度
SGLANG_SIMULATE_ACC_LEN=3.24. Three speculative tokens per verification step is the MTP=3 cell, measured thinking=off in speedbench-al run 33031708148 (MTP 1–6 → AL 1.87 / 2.61 / 3.24 / 3.74 / 4.18 / 4.40). This is an interim value — thethinking=oncollection is still running andgolden_al_distribution/qwen3.8next_mtp.yamlis not committed yet. Refresh this constant when that curve lands.SGLANG_SIMULATE_ACC_LEN=3.24，取自 speedbench-al 运行 33031708148 的 thinking=off 实测 MTP=3 值。这是临时值：thinking=on采集仍在进行，黄金曲线尚未提交，待其合入后需刷新。Files / 改动文件
benchmarks/single_node/agentic/qwen3.8next_fp4_b300_sglang_mtp.sh(new)configs/nvidia-master.yaml— new entry after the Qwen3.5 NVFP4 B300 siblingperf-changelog.yaml— appended entryPart of a four-PR set, one per chip: B200 (#2751), B300 (this), H200, MI355X.
本 PR 属于按芯片划分的四个 PR 之一：B200（#2751）、B300（本 PR）、H200、MI355X。
🤖 Generated with Claude Code
Note
Low Risk
Benchmark and YAML/changelog additions only; no changes to core inference or auth paths.
Overview
Adds a day-zero AgentX benchmark path for RadixArk/Qwen3.8-Flash-Next-NVFP4 on B300 via SGLang with native NEXTN MTP.
New script
qwen3.8next_fp4_b300_sglang_mtp.shlaunches the server with cookbook-style TP1 (126 GiB NVFP4 fits one GPU), flashinfer linear-attention prefill/decode, bfloat16 Mamba SSM (Blackwell requirement), NEXTN (3 steps / 4 draft tokens), optional HiCache sizing that accounts for draft KV, GSM8K via lm-eval, and throughput runs pinned toSGLANG_SIMULATE_ACC_LEN=2.32(thinking-on golden AL); eval-only keeps real verification.Registers
qwen3.8next-fp4-b300-sglang-agentic-mtpinnvidia-master.yaml(lmsysorg/sglang:qwen38flashnext,cluster:b300-nv, TP1/EP1, conc 1–16, no KV offload) and documents it inperf-changelog.yaml.Reviewed by Cursor Bugbot for commit ddc5026. Bugbot is set up for automated code reviews on this repo. Configure here.