feat(config): add GLM-5.2 GB300 AgentX concurrency-1 disaggregated point / 添加 GLM-5.2 GB300 AgentX 并发度 1 分离式配置 - #2720
Conversation
新增 GLM-5.2 GB300 AgentX 并发度 1 的预填充与解码分离配置，并移除已替代的聚合配置。
|
Thanks for the contribution! Please reach out to respective companies' CODEOWNER to fill in the latest PR_REVIEW_CHECKLIST.md before pinging core maintainer on Slack for review. In order for the signoff PR check bot to trigger, you must follow the PR_REVIEW_CHECKLIST.md template correctly, including the phrase For PR verification, add the PR authors are responsible for ensuring that after merging, all GitHub Action jobs fully pass. A lot of the time, failures are just flakes and simply re-running the failed jobs will fix it. See GitHub's docs on re-running failed jobs 感谢你的贡献！请联系相应公司的 CODEOWNER 填写最新的 PR_REVIEW_CHECKLIST.md，然后再在 Slack 上联系核心维护者进行审阅。为了触发 signoff PR 检查机器人，你必须正确遵循 PR_REVIEW_CHECKLIST.md 模板，包括保留英文语句 如需进行 PR 验证，请为此 PR 添加 PR 作者有责任确保合并后所有 GitHub Action 任务完全通过。 很多时候失败只是偶发抖动（flake），重新运行失败的任务即可解决。参见 GitHub 关于重新运行失败任务的文档 |
在性能变更日志中补充 PR #2720 链接。
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
| - "Move the GLM-5.2 concurrency-1 AgentX point from an aggregated TP8 topology to a disaggregated 1-prefill/1-decode topology with TP4 attention-DP prefill and TP8 decode." | ||
| - "Use MTP depth 5 with the committed golden acceptance target, tensor-parallel MoE on decode, and the CUTEDSL top-k path." | ||
| - "Align recipe KV offloading with the master config and keep the benchmark client colocated with the frontend orchestrator." | ||
| - "Retire the superseded aggregated configuration key while retaining the existing disaggregated topology points." | ||
| pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/XXX |
There was a problem hiding this comment.
🟡 The new perf-changelog.yaml entry appended by this PR ends with pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/XXX instead of the actual PR number (2720). Every other entry in the file uses a real PR number, so this leaves a broken/dead link at the end of an append-only, byte-sensitive changelog. Please replace XXX with 2720 before merge.
Extended reasoning...
The last entry in perf-changelog.yaml (lines 6396-6400, appended by this PR) ends with:
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/XXXXXX is a literal, unfilled placeholder rather than an actual PR number. This is PR #2720, and the correct value is simply 2720.
Where this comes from: every prior entry in the file uses a real numeric PR link, e.g. the immediately preceding entry ends with pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/2650 (line 6390), and other nearby entries reference 2685 and 2707. The new entry added by this PR is the only one with XXX instead of a number, which strongly suggests the author copy-pasted the changelog template and forgot to substitute in the real PR number before committing.
Why nothing catches this automatically: the changelog is append-only and treated as byte-sensitive (the PR checklist explicitly calls out appending a new entry without editing historical ones), but there's no schema validation or CI check enforcing that pr-link resolves to a real, existing PR — a string like .../pull/XXX is syntactically valid YAML and a syntactically valid URL, so nothing fails at parse time or link-check time (XXX would 404 if fetched, but no such link check appears to run in CI for this file).
Concrete proof / step-by-step:
- Open
perf-changelog.yamland go to line 6400 (the last line of the file, added in this diff). - Read the value:
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/XXX. - Compare to the entry directly above it (line 6390):
pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/2650— a real, resolvable PR number. - Navigate to
https://github.com/SemiAnalysisAI/InferenceX/pull/XXX— this 404s, whereas.../pull/2650resolves to a real PR. - This PR is feat(config): add GLM-5.2 GB300 AgentX concurrency-1 disaggregated point / 添加 GLM-5.2 GB300 AgentX 并发度 1 分离式配置 #2720 (per the PR metadata), so the correct fix is to replace
XXXwith2720.
Impact: this doesn't affect any runtime or benchmark behavior — it's purely a traceability/documentation defect in the changelog. But because the file is append-only and treated as immutable history once merged, an incorrect placeholder value can't be silently patched later without an explicit follow-up commit calling out the correction, so it's worth fixing now while the entry is still open in this PR.
Fix: change line 6400 from pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/XXX to pr-link: https://github.com/SemiAnalysisAI/InferenceX/pull/2720.
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=32758589490 |
将 GLM-5.2 扫描临时限定为并发度 1，并同步 main 分支的变更。
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=32790767048 |
将 GLM-5.2 并发度 1 配置添加到现有曲线，并保留所有已发布点。
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Want higher recall? High effort reviews run extra passes and find more bugs. A team admin can switch effort levels in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 5e7d158. Configure here.
| AIPERF_HTTP_X_DYNAMO_SESSION_ID_FROM_CORRELATION_ID: 'true' | ||
| AIPERF_USE_DYNAMO_CONV_AWARE_ROUTING: '0' | ||
| OPENAI_API_KEY: EMPTY | ||
| KV_OFFLOADING: dram |
There was a problem hiding this comment.
DRAM offload env vars missing
High Severity
The new recipe sets KV_OFFLOADING to dram in benchmark.env, but omits KV_OFFLOAD_BACKEND and TOTAL_CPU_DRAM_GB. agentic_srt.sh sources benchmark_lib.sh, which exits immediately when DRAM offload is enabled without those variables, so the new concurrency-1 point fails before replay starts. Sibling GLM-5.2 srt-slurm recipes keep KV_OFFLOADING as none and do not take this path.
Reviewed by Cursor Bugbot for commit 5e7d158. Configure here.
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=32894023218 |
恢复 GLM-5.2 配置的完整七点扫描覆盖。
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=33004185970 |
1 similar comment
|
see unofficial run visualizer at https://inferencex.semianalysis.com/inference?unofficialRun=33004185970 |
1 participant
Description
Add a GLM-5.2 GB300 AgentX concurrency-1 point to the existing disaggregated configuration using a 1-prefill/1-decode topology. The aggregated concurrency-1 point and all six existing disaggregated points remain unchanged.
The new recipe uses TP4 attention-DP prefill and TP8 decode, MTP depth 5 with the committed golden acceptance target, tensor-parallel MoE on decode, CUTEDSL top-k, and DRAM KV offloading. The regular changelog entry selects the complete seven-point disaggregated configuration and its applicable existing eval leaves for the final full sweep.
中文说明
将 GLM-5.2 GB300 AgentX 并发度 1 的新点添加到现有分离式配置中，采用 1 个预填充工作进程和 1 个解码工作进程的拓扑。原有聚合式并发度 1 点以及现有 6 个分离式点均保持不变。
新配置使用 TP4 注意力数据并行预填充和 TP8 解码、MTP 深度 5 及已提交的黄金接受长度目标、解码侧张量并行 MoE、CUTEDSL top-k 和 DRAM KV 缓存卸载。常规性能变更日志条目会在最终完整扫描中选择全部 7 个分离式点及其适用的现有评测任务。
Related Issue
N/A
Type of Change
Checklist
perf-changelog.yamland have not edited historical entriesOWNER/MEMBER/COLLABORATOR) has commented/reuse-sweep-runon this PR. Do this only once there is a final full sweep that is all green with evals passing, since after this comment the sweep label will no longer automatically kick off new sweeps. Remove and re-add the label to force one.