[AMD][MI35X] Bump Qwen3.5 MXFP4 MI355X SGLang single-node image to v0.5.18

[yichiche]

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=33291791635
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=33291791635

Motivation

qwen3.5-fp4-mi355x-sglang and qwen3.5-fp4-mi355x-sglang-mtp are still pinned to lmsysorg/sglang-rocm:v0.5.16-rocm720-mi35x-20260730, which is roughly three weeks behind the current SGLang ROCm nightly. This bumps both single-node MI355X Qwen3.5 MXFP4 arms to the latest v0.5.17 mi35x image so the benchmark matrix reflects current upstream SGLang + aiter performance on MI355X.

Modifications

Bump the image: value for both config keys in configs/amd-master.yaml from lmsysorg/sglang-rocm:v0.5.16-rocm720-mi35x-20260730 to lmsysorg/sglang-rocm:v0.5.17-rocm720-mi35x-20260817. No other config fields change — model, runner, TP/EP search space, and concurrency ranges are untouched.

Append the corresponding trigger entry to perf-changelog.yaml (append-only, at the end of the file) covering both config keys.

Benchmark scripts under benchmarks/single_node/fixed_seq_len/ are unchanged; this is a plain image bump.

Accuracy Tests

No accuracy-affecting logic changes in this repo — the change is limited to the container tag and the changelog trigger. Model correctness is covered by the upstream SGLang release validation for the v0.5.17 ROCm image.

Benchmarking

Repo validation was run locally:

• python -m pytest utils/matrix_logic/ -q → 231 passed.
• python utils/matrix_logic/generate_sweep_configs.py full-sweep --config-files configs/amd-master.yaml --model-prefix qwen3.5 --precision fp4 --runner-type mi355x emits the new tag on both arms: 10 generated configs with spec-decoding: none and 9 with spec-decoding: mtp, all referencing lmsysorg/sglang-rocm:v0.5.17-rocm720-mi35x-20260817.

End-to-end MI355X throughput/latency numbers will come from the sweep triggered on this PR (full-sweep-fail-fast).

---

Note

Medium Risk
Benchmark recipe changes (container, KV cache dtype, and multi-GPU collective path) alter served behavior and break like-for-like perf trends, though scope is limited to two MI355X Qwen3.5 MXFP4 fixed-seq configs.

Overview
Updates qwen3.5-fp4-mi355x-sglang and qwen3.5-fp4-mi355x-sglang-mtp to lmsysorg/sglang-rocm:v0.5.18-rocm720-mi35x-20260829 in configs/amd-master.yaml, with a matching perf-changelog.yaml entry for the fixed-seq-len arms.

The paired launch scripts qwen3.5_fp4_mi355x.sh and qwen3.5_fp4_mi355x_mtp.sh no longer use --enable-aiter-allreduce-fusion. They set ROCM_QUICK_REDUCE_QUANTIZATION=INT8 for ROCm quick all-reduce and add --kv-cache-dtype fp8_e4m3, matching the published SGLang MXFP4-on-MI355X cookbook. Throughput on this arm is not directly comparable to the prior series that used AITER fused all-reduce (#1680).

^{Reviewed by Cursor Bugbot for commit 93ee88e. Bugbot is set up for automated code reviews on this repo. Configure here.}

[github-actions]

Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase As a PR reviewer and CODEOWNER, I have reviewed this and have.

For PR verification, add the full-sweep-fail-fast label (strongly recommended) to this PR — the benchmark sweep only runs on labeled PRs. Use full-sweep-enabled only if you need matrix jobs to keep running past a failure.

PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs

---

感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 As a PR reviewer and CODEOWNER, I have reviewed this and have。

如需进行 PR 验证，请为此 PR 添加 full-sweep-fail-fast 标签（强烈推荐）— 基准测试 sweep 仅在带有标签的 PR 上运行。仅当需要矩阵任务在失败后继续运行时才使用 full-sweep-enabled。

PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档

[github-actions]

Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase As a PR reviewer and CODEOWNER, I have reviewed this and have.

For PR verification, add the full-sweep-fail-fast label (strongly recommended) to this PR — the benchmark sweep only runs on labeled PRs. Use full-sweep-enabled only if you need matrix jobs to keep running past a failure.

PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs

---

感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 As a PR reviewer and CODEOWNER, I have reviewed this and have。

如需进行 PR 验证，请为此 PR 添加 full-sweep-fail-fast 标签（强烈推荐）— 基准测试 sweep 仅在带有标签的 PR 上运行。仅当需要矩阵任务在失败后继续运行时才使用 full-sweep-enabled。

PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档

[github-actions]

Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase As a PR reviewer and CODEOWNER, I have reviewed this and have.

For PR verification, add the full-sweep-fail-fast label (strongly recommended) to this PR — the benchmark sweep only runs on labeled PRs. Use full-sweep-enabled only if you need matrix jobs to keep running past a failure.

PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs

---

感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 As a PR reviewer and CODEOWNER, I have reviewed this and have。

如需进行 PR 验证，请为此 PR 添加 full-sweep-fail-fast 标签（强烈推荐）— 基准测试 sweep 仅在带有标签的 PR 上运行。仅当需要矩阵任务在失败后继续运行时才使用 full-sweep-enabled。

PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档

[github-actions]

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=32212675501
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=32212675501

[yichiche]

Cookbook: sgl-project/sglang#35445

[github-actions]

see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=33291791635
see unofficial run visualizer at https://inferencex.semianalysis.com/evaluation?unofficialRun=33291791635

[yichiche]

/reuse-sweep-run

[chunfangamd]

As a PR reviewer and CODEOWNER, I have reviewed this and have:

• Verified that as of the moment of typing this, this is the latest version of PR_REVIEW_CHECKLIST.md
• Verified that the general code quality meets the InferenceX standard and does not make the code quality any worse.
• Verified that this PR has passed PR validation. Please link to GitHub Action workflow that shows this. https://github.com/SemiAnalysisAI/InferenceX/actions/runs/33291791635
• Verified that this PR passes evals. Please link to GitHub Action workflow that shows this. https://github.com/SemiAnalysisAI/InferenceX/actions/runs/33291791635
• Verified that speculative decoding PRs uses chat templates to align the AL distribution to real world
• For agentic workloads: verified that speculative-decoding configs (EAGLE / MTP / draft models) run with simulated synthetic acceptance, with the acceptance-length value taken from the committed golden AL curve in golden_al_distribution/ for that model, thinking mode, and draft length. A submission may choose any supported draft length, but it may not substitute a different acceptance target.
• Verified against the current MODELS.md that this PR does not submit a deprecated model, scenario, or model-scenario combination.
• Verified that the model architecture isn't changed with benchmark hacks like using --hf-overrides to skipping indexer for every x layers on models that don't natively support this. As a general rule, we won't accept optimizations that reduces the number of model architecture FLOPs. Anything that makes that same computation run faster is fair game; FLOPs at lower precisions is fine, given that the config passes private evals. As an general north star princple, we should only use optimizations which is used in production by customers that care about accuracy
• If an company claims that they support vLLM/SGLang as first class LLM inference engines on their hardware, I have verified that the respective vLLM submission made using upstream https://hub.docker.com/u/vllm docker repo, upstream SGLang https://hub.docker.com/u/lmsysorg docker repo. The only exceptions are for new hardware, such as MI455X UALoE72, Vera Rubin NVL72, Rubin NVL8, etc., and for new model architectures where there is an actual reason why vLLM/SGLang does not fundamentally support them yet as supported by vLLM/SGLang community maintainers
• If an company claims that they support vLLM/SGLang as first class upstream in-tree LLM inference engines on their hardware, I have have verified that the respective vLLM/SGLang submission has been made before additional frameworks (TRT-LLM, ATOM, etc.). The only exceptions are for new hardware, such as MI455X UALoE72, Vera Rubin NVL72, Rubin NVL8, etc., and for new model architectures where there is an actual reason why vLLM/SGLang does not fundamentally support them yet.
• Verified that every single-node vLLM/SGLang recipe in this PR is documented in the official vLLM recipes and/or the SGLang cookbook:
  • I linked the corresponding upstream PR in the vLLM recipe repo or SGLang repo and verified that it is MERGED before this InferenceX PR merges. An opened, draft, or closed-without-merge upstream PR does not satisfy this requirement. If the matching recipe was already published, I linked the published recipe/cookbook page in the additional detail section below.
• Verified that this PR does not patch the inference engine or serving stack — the pinned image must run as shipped. This covers .patch files / git apply / patch, inline patches embedded in benchmark scripts (e.g. a python3/sed heredoc that rewrites installed engine sources before serving), in-place edits of site-packages, monkey-patching, overwriting container files, and installing forked/rebuilt engine wheels on top of the pinned image. The only exception is a patch covered by a filled-out waiver at docs/waiver/<PR_NUMBER>.md — named after the PR that introduces the patch and filed in that same PR, stating what is patched, why the unmodified upstream image cannot run this benchmark, the upstream PR/issue link, and the removal plan — which I have linked below in the additional detail section.
• If this PR uses append-only: true, verified that it only adds generated points or recipe variants inside a selected existing config/scenario and existing same-image visual curve: every previously generated point remains present with the same recipe, no prior point is removed or rerun, and every benchmark-affecting change in the complete diff can affect only the corresponding newly appended points (never an existing point), regardless of which file contains it.
• If any of the above criteria cannot reasonably be satisfied, I have provided additional reasoning below.

Additional detail section:

• insert any additional info here

1. MERGED SGLang recipe PR for ROCM_QUICK_REDUCE_QUANTIZATION=INT8 and removal of --enable-aiter-allreduce-fusion: sgl-project/sglang#31737
2. MERGED SGLang recipe PR for --kv-cache-dtype fp8_e4m3: sgl-project/sglang#35445

Signed: @chunfangamd

[Klaud-Cold]

✅✅✅ Verdict: PASS ✅✅✅

✅ Check 0 (CODEOWNER): PASS — @chunfangamd is a named owner of configs/amd-master.yaml; the other changed paths carry only the catch-all, which a recognized CODEOWNER satisfies.
✅ Check 1 (sweep on in-PR commit): PASS — commit e054e0e (in this PR) has all 19 executed single-node 8k1k / and all 4 eval / check-runs green in run 33291791635, the run linked in the sign-off.
✅ Check 2 (evals real): PASS — GSM8K em_strict 0.967–0.971 (n_eff 1319) across both the spec-none and MTP arms, on lmsysorg/sglang-rocm:v0.5.18-rocm720-mi35x-20260829, the exact image this PR pins.
✅ Check 3 (recipe): PASS — published cookbook page backed by MERGED sgl-project/sglang#31737 (2026-07-20) and #35445 (2026-08-19); all major args match: model amd/Qwen3.5-397B-A17B-MXFP4, --attention-backend aiter, --page-size 16, --disable-radix-cache, --kv-cache-dtype fp8_e4m3, ROCM_QUICK_REDUCE_QUANTIZATION=INT8 with --enable-aiter-allreduce-fusion dropped, spec steps 3 / topk 1 / draft-tokens 4 with MTP concurrency capped at 128 (EAGLE is SGLang's resolved alias of the cookbook's NEXTN). TP2 sweep points and --max-running-requests $CONC are InferenceX sweep tuning — informational only.
✅ Check 4 (reuse command): PASS — /reuse-sweep-run posted by @yichiche (COLLABORATOR).
✅ Check 5 (latest template): PASS — every item of the current docs/PR_REVIEW_CHECKLIST.md template is present and checked.
✅ Check 6 (upstream image / ordering): PASS — upstream lmsysorg/sglang-rocm:v0.5.18-rocm720-mi35x-20260829 on established MI355X; both entries stay framework: sglang, so no ordering question.
✅ Check 7 (deprecations): PASS — qwen3.5 Single-turn 8k1k fp4 is an active combination per MODELS.md as of 2026-08-30.
✅ Check 8 (no architecture hacks): PASS — no --hf-overrides or config edits; fp8 KV cache and INT8-quantized collectives are lower-precision same-FLOPs changes, with evals passing.
✅ Check 9 (spec-decode chat template): PASS — the MTP benchmark client passes --use-chat-template.
✅ Check 10 (no engine patches): PASS — only server flags/env change; the pinned image runs as shipped.
➖ Check 11 (agentic golden AL): N/A — fixed-seq-len only, no agentic spec-decode changes; the non-agentic MTP arm correctly runs real acceptance.
➖ Check 12 (append-only): N/A — the new perf-changelog entry does not set append-only: true.

Note: the branch tip moved to 8b98e48 after this sign-off (a merge of main touching only CollectiveX files; none of this PR's four recipe files changed). Per gate policy a fresh sign-off is needed for the new commit.

[yichiche]

@Oseltamivir Can you help merge this PR? Verdict: pass, with huge performance gains, especially in MTP mode for Qwen. Would be more than appreciated.

[cursor]

Cursor Bugbot has reviewed your changes using default effort and found 2 potential issues.

^{❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.}

^{Reviewed by Cursor Bugbot for commit 52570c9. Configure here.}

[cquil11]

/stage-results 33291791635

[github-actions]

@cquil11 staging run 33291791635. Existing staged runs will be preserved; a completion link will be posted here.

[cquil11]

/stage-results 33291791635

[github-actions]

@cquil11 staged run 33291791635: https://inferencemax-app-git-staging-semianalysisai.vercel.app/inference?i_dates=2026-08-30~r33291791635

This run remains available across future /stage-results requests. Staging the same run ID again updates its staged data. Staging workflow

[cquil11]

/reuse-sweep-run 33291791635