[Klaud Cold] minimaxm3-fp4-mi355x-atom-agentic-mtp: MiniMax-M3 MXFP4 ATOM AgentX on MI355X with the Inferact EAGLE3 drafter / MI355X 上采用 Inferact EAGLE3 草稿模型的 MiniMax-M3 MXFP4 ATOM AgentX 配方 - #2734
Conversation
…M AgentX on MI355X Add the MiniMax-M3 MXFP4 agentic-coding recipe on MI355X served by ATOM, following the upstream ATOM recipe serve shape (https://github.com/ROCm/ATOM/blob/main/recipes/Qwen3.5.md): TP4, FP8 KV cache, native MTP with three draft tokens, and ATOM defaults everywhere else. MiniMax-M3 ships native MTP modules, so no external drafter is loaded. Throughput runs pin acceptance to the committed golden curve (AL 2.83); eval-only runs keep real target verification. Image is the latest rocm/atom-dev nightly. 新增 MI355X 上由 ATOM 服务的 MiniMax-M3 MXFP4 智能体编码配方，服务参数遵循 上游 ATOM recipe（https://github.com/ROCm/ATOM/blob/main/recipes/Qwen3.5.md） 的形态：TP4、FP8 KV 缓存、原生 MTP 3 个草稿 token，其余参数一律保持 ATOM 默认值。MiniMax-M3 自带原生 MTP 模块，因此不加载外部草稿模型。吞吐运行将接受 长度锁定到已提交的黄金曲线（AL 2.83），仅评测运行仍使用真实目标验证。镜像使用 最新的 rocm/atom-dev nightly。 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
…link Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
|
Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase For PR verification, add the PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs 感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 如需进行 PR 验证，请为此 PR 添加 PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档 |
2 similar comments
|
Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase For PR verification, add the PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs 感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 如需进行 PR 验证，请为此 PR 添加 PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档 |
|
Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase For PR verification, add the PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs 感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 如需进行 PR 验证，请为此 PR 添加 PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档 |
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit caf7a90. Configure here.
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
| - "Add a day-zero MiniMax-M3 MXFP4 AgentX recipe on MI355X with ATOM native MTP at TP4 and concurrency 1/4/8/12/16." | ||
| - "Follow the upstream ATOM recipe serve shape: FP8 KV cache, three MTP draft tokens, and ATOM defaults for every other server knob." | ||
| - "Pin throughput runs to the committed golden acceptance length of 2.83 at three draft tokens; eval-only runs keep real target verification." | ||
| pr-link: TBD |
There was a problem hiding this comment.
🔴 New entry's pr-link is 'TBD', which is not in the canonical placeholder set PR_LINK_PLACEHOLDERS = {"XXX", ".../pull/XXX"} checked by utils/validate_perf_changelog.py's validate_added_pr_link/compare_entries (used by utils/prepare_perf_changelog_merge.py at merge time). docs/configuration-procedures.md line 254 explicitly requires replacing 'TBD' with the real PR URL 'immediately after creating the PR'; this PR is already open (per the PR metadata) yet still carries the stale 'TBD'.
Extended reasoning...
When a maintainer runs the merge-preparation flow (prepare_perf_changelog_merge.py -> canonicalize_appended_links -> compare_entries -> validate_added_pr_link), the link 'TBD' matches neither the expected 'https://github.com/.../pull/' nor any entry in PR_LINK_PLACEHOLDERS, so it raises ChangelogValidationError("new PR entry must use ... or an XXX placeholder; found 'TBD'"), blocking automated merge/canonicalization until someone manually edits the changelog line. Fix: use the documented 'XXX' placeholder (or the real PR URL) instead of 'TBD'.
Verification: normal — the appended perf-changelog entry ends with pr-link: TBD, which the merge-preparation tooling rejects. PR_LINK_PLACEHOLDERS (utils/validate_perf_changelog.py:24-27) = {"XXX", ".../pull/XXX"} does not contain "TBD". At merge time utils/merge_with_reuse.sh:174,187 runs prepare_perf_changelog_merge.py --pr-number <PR>, which calls canonicalize_appended_links; at utils/prepare_p
| NUM_SPEC_TOKENS=3 | ||
| # golden_al_distribution/minimaxm3_eagle3.yaml: minimax-m3.thinking_on[3]. | ||
| # AgentX pins every submission for a model to one golden acceptance curve, so | ||
| # the native-MTP arm targets the same acceptance length as the EAGLE3 arm. | ||
| SPEC_DECODE_AL=2.83 |
There was a problem hiding this comment.
🔴 New ATOM native-MTP recipe pins SPEC_DECODE_AL=2.83 from golden_al_distribution/minimaxm3_eagle3.yaml (the non-GQA curve), but every other MiniMax-M3 EAGLE3/MTP sibling script (minimaxm3_fp4_b300_mtp.sh, minimaxm3_fp4_b200_mtp.sh, minimaxm3_fp4_b200_trt_mtp.sh, minimaxm3_fp4_b300_trt_mtp.sh, minimaxm3_fp8_h100_mtp.sh, minimaxm3_fp8_h200_mtp.sh, minimaxm3_fp4_mi355x_mtp.sh) explicitly uses golden_al_distribution/minimaxm3_eagle3_gqa.yaml (2.78) and calls out 2.83 by name as belonging to a different draft head ('that head is not what this script runs').
Extended reasoning...
Because AgentX's stated methodology requires every submission for a given model to be pinned to the same golden acceptance-length curve for cross-engine comparability, this recipe's throughput numbers are pinned to a different (higher) synthetic acceptance length than every other MiniMax-M3 recipe in the repo, making its reported throughput non-comparable/inflated relative to siblings and violating the very consistency rule the script's own comment cites. The fix is to pin SPEC_DECODE_AL=2.78 from minimaxm3_eagle3_gqa.yaml, matching the established convention for this model.
Verification: normal. The new script pins throughput to a different golden acceptance-length curve than every other MiniMax-M3 sibling, breaking the cross-submission comparability its own comment invokes and inflating reported throughput. benchmarks/single_node/agentic/minimaxm3_fp4_mi355x_atom_mtp.sh:47-50: # golden_al_distribution/minimaxm3_eagle3.yaml: minimax-m3.thinking_on[3]. # AgentX pins every submi
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=32919251035 |
Swap --method mtp for the retired single-turn 8k1k MI355X ATOM recipe's speculative setup: --method eagle3 with Inferact/MiniMax-M3-EAGLE3 and three draft tokens. Port that recipe's remaining server flags too (ptpc_fp8 online quant exclusions, ATOM_FORCE_ATTN_TRITON, AITER_QUICK_REDUCE_QUANTIZATION, block size 128, memory fraction 0.8, max-num-batched-tokens 32768), keeping prefix caching enabled and leaving max-model-len at the model default so agentic traces are not clipped to the 8k1k scenario value. 将投机解码从原生 MTP 换成已退役的单轮 8k1k MI355X ATOM 配方所用方案： --method eagle3，草稿模型 Inferact/MiniMax-M3-EAGLE3，3 个草稿 token。同时移植 该配方的其余服务参数（ptpc_fp8 在线量化排除项、ATOM_FORCE_ATTN_TRITON、 AITER_QUICK_REDUCE_QUANTIZATION、block size 128、显存占用 0.8、 max-num-batched-tokens 32768），并保持前缀缓存启用、max-model-len 使用模型默认值， 以免智能体长上下文轨迹被裁剪到 8k1k 场景的长度。 Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
functionstackx
changed the title
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=32924358550 |
1 participant
Summary / 摘要
Day-zero MiniMax-M3 MXFP4 AgentX recipe on MI355X served by ATOM, with EAGLE3 speculative decoding against the Inferact drafter. The server flags are a port of the retired single-turn 8k1k MI355X ATOM recipe; the serve shape follows the upstream ROCm/ATOM Qwen3.5 recipe — TP4, FP8 KV cache, three draft tokens, ATOM defaults for everything it does not name.
在 MI355X 上新增由 ATOM 服务的 MiniMax-M3 MXFP4 AgentX 配方，采用 EAGLE3 投机解码，草稿模型为 Inferact 版本。服务参数移植自已退役的单轮 8k1k MI355X ATOM 配方；服务形态遵循上游 ROCm/ATOM Qwen3.5 recipe：TP4、FP8 KV 缓存、3 个草稿 token，其余未指明的参数保持 ATOM 默认。
Config key / 配置项
minimaxm3-fp4-mi355x-atom-agentic-mtp—amd/MiniMax-M3-MXFP4,rocm/atom-dev:nightly_202608251555(latest ATOM nightly, verified on Docker Hub),cluster:mi355x-amds, TP4, conc[1, 4, 8, 12, 16], no KV offloading.Speculative decoding / 投机解码
--method eagle3 --draft-model Inferact/MiniMax-M3-EAGLE3 --num-speculative-tokens 3— the drafter and draft length the retired 8k1k MI355X ATOM recipe used, and MiniMax-M3's plan-of-record draft perMODELS.md. The checkpoint's native MTP modules are not used. Throughput runs pin--spec-decode-acceptance-length 2.83fromgolden_al_distribution/minimaxm3_eagle3.yaml(thinking_on[3]); eval-only runs drop the pin and use real target verification.--method eagle3 --draft-model Inferact/MiniMax-M3-EAGLE3 --num-speculative-tokens 3，即已退役 8k1k MI355X ATOM 配方所用的草稿模型与草稿长度，也是MODELS.md中 MiniMax-M3 的 PoR 草稿方案；不使用 checkpoint 自带的原生 MTP 模块。吞吐运行按golden_al_distribution/minimaxm3_eagle3.yaml的thinking_on[3]锁定--spec-decode-acceptance-length 2.83，仅评测运行取消锁定并使用真实目标验证。Ported from the retired 8k1k MI355X ATOM recipe / 移植自已退役的 8k1k MI355X ATOM 配方
ptpc_fp8online_quant_configwith the MiniMax-M3 exclusion list,ATOM_FORCE_ATTN_TRITON=1,AITER_QUICK_REDUCE_QUANTIZATION=INT4,--block-size 128,--gpu-memory-utilization 0.8,--max-num-batched-tokens 32768,--kv_cache_dtype fp8,--trust-remote-code.Two deliberate departures / 两处刻意不同：
Prefix caching stays ON.
--no-enable_prefix_cachingis not carried over — trace replay is exactly the workload prefix caching pays for.--max-model-lenleft at the model default, so agentic long-context traces are not clipped to the 8k1k scenario's 32768.No
--hf-overridesindex-cache block (the vLLM MI355X arm sets one; this recipe deliberately does not).保持前缀缓存启用：不沿用
--no-enable_prefix_caching——轨迹回放正是前缀缓存最有价值的负载。--max-model-len使用模型默认值，避免长上下文智能体轨迹被裁剪到 8k1k 场景的 32768。不添加
--hf-overrides索引缓存参数（vLLM MI355X 分支有设置，本配方刻意不设）。TP4 is both what the upstream recipe prescribes and what this model wants: MiniMax-M3 has four KV heads, so one KV head per rank keeps the AITER sparse-attention fast path. No launcher change needed —
runners/launch_mi355x-amds.shalready resolves theagentic/ATOM_mtpscript name and routesamd/MiniMax-M3*weights to the NFS HF cache.Files / 改动文件
benchmarks/single_node/agentic/minimaxm3_fp4_mi355x_atom_mtp.sh(new)configs/amd-master.yaml— new entry after the vLLM MI355X siblingperf-changelog.yaml— appended entryNote / 说明
Sibling PR #2733 builds the same config key from the current InferenceX ATOM AgentX siblings instead (explicit cudagraph capture sizes,
--max-num-batched-tokens 16384, noATOM_FORCE_ATTN_TRITON). The two are mutually exclusive candidates — merge whichever sweeps green, not both.同类 PR #2733 以当前 InferenceX 的 ATOM AgentX 配方为蓝本实现同一配置项（显式 cudagraph 尺寸、
--max-num-batched-tokens 16384、不设ATOM_FORCE_ATTN_TRITON）。两者互斥，只应合并全量 sweep 通过的那一个。🤖 Generated with Claude Code
Note
Low Risk
Changes are limited to benchmark scripts, cluster config, and changelog; no production inference or auth paths are modified.
Overview
Adds a day-zero AgentX recipe for
amd/MiniMax-M3-MXFP4on MI355X served by ATOM (minimaxm3-fp4-mi355x-atom-agentic-mtp), with a new launcher script that runs agentic trace replay against an ATOM OpenAI server.The server setup ports the retired single-turn 8k1k MI355X ATOM recipe: Inferact EAGLE3 drafter (
Inferact/MiniMax-M3-EAGLE3), three speculative tokens,ptpc_fp8online quant exclusions, Triton attention env, block size 128, and 0.8 GPU memory fraction. It also follows the upstream ATOM serve shape (TP4, FP8 KV cache, three draft tokens). Prefix caching stays enabled and max sequence length is not clipped to the 8k1k 32768 cap.Throughput runs pin synthetic spec-decode acceptance length 2.83; eval-only runs drop the pin. The config registers TP4 / EP1 / no KV offload with concurrency 1, 4, 8, 12, 16 on
cluster:mi355x-amds, andperf-changelog.yamldocuments the entry.Reviewed by Cursor Bugbot for commit b2eb156. Bugbot is set up for automated code reviews on this repo. Configure here.