[WIP][AMD][AgentX] Add Qwen3.8 FP8 MI355X two-node vLLM - #2724
Conversation
新增固定序列与 AgentX 的双节点 TP8/PP2 CI 路径，并验证 Slurm 节点上的模型快照一致性。 Co-authored-by: Cursor <cursoragent@cursor.com>
Run the shared agentic script via Bash so multinode sweeps do not depend on its executable bit. Co-authored-by: Cursor <cursoragent@cursor.com>
Preserve the append-only changelog while making the branch eligible for current CI workflows. Co-authored-by: Cursor <cursoragent@cursor.com>
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
Replace temporary PR placeholders so the changelog and support matrix reference the review that introduces the configuration. Co-authored-by: Cursor <cursoragent@cursor.com>
haic0
added
agentx-fast
|
Dispatched trusted sweep run for approved external revision 已为获批的外部提交 |
Remove the fixed 8k/1k matrix and client path so CI selects only the validated multi-node AgentX configuration. Co-authored-by: Cursor <cursoragent@cursor.com>
|
Dispatched trusted sweep run for approved external revision 已为获批的外部提交 |
haic0
changed the title
Make Qwen3.8 MODEL_PATH overridable via QWEN38_MODEL_PATH, add cluster staging verification/docs, restore the missing EVALS matrix bucket, and guard empty eval-config outputs so agentic-only sweeps skip eval jobs cleanly. Co-authored-by: Cursor <cursoragent@cursor.com>
CI hardening update (b0f0729)Applied follow-up fixes from the failed e2e run analysis:
Blocker before re-running trusted sweepThe last run failed because the model directory was missing on assigned nodes: Please stage identical snapshots on all MI355X Slurm nodes, then verify: bash benchmarks/multi_node/qwen3.8_vllm_multi_nodes/verify_model_staging.sh --local
bash benchmarks/multi_node/qwen3.8_vllm_multi_nodes/verify_model_staging.sh --nodes NODE1,NODE2After staging is confirmed, remove and re-add |
![cursor[bot]](https://avatars.githubusercontent.com/in/1210556?s=60&v=4){kind=small}
Resolve perf-changelog.yaml append-only conflict by keeping main entries and placing the Qwen3.8 AgentX changelog block at the end. Co-authored-by: Cursor <cursoragent@cursor.com>
|
Dispatched trusted sweep run for approved external revision 已为获批的外部提交 |
There was a problem hiding this comment.
Cursor Bugbot has reviewed your changes using default effort and found 1 potential issue.
❌ Bugbot Autofix is OFF. To automatically fix reported issues with cloud agents, enable autofix in the Cursor dashboard.
Reviewed by Cursor Bugbot for commit 4b94621. Configure here.
| echo "ERROR: Qwen3.8 snapshots differ between ${nodes[0]} and ${nodes[1]}" >&2 | ||
| echo " ${nodes[0]}: ${manifests[0]}" >&2 | ||
| echo " ${nodes[1]}: ${manifests[1]}" >&2 | ||
| exit 1 |
There was a problem hiding this comment.
Staging check always mismatches
Medium Severity
run_manifest prints a labeled status line and the digest on stdout, and --nodes captures that whole stream for comparison. Different host labels make the captured strings unequal even when digests match, so the recommended two-node staging check always reports a mismatch.
Reviewed by Cursor Bugbot for commit 4b94621. Configure here.
Source qwen3.8_env.sh before the generic multinode MODEL_PATH default and always export the staged snapshot root from QWEN38_MODEL_PATH. Co-authored-by: Cursor <cursoragent@cursor.com>
|
Dispatched trusted sweep run for approved external revision 已为获批的外部提交 |
1 participant
Summary
Test plan
mainmulti_node.agenticNote
Medium Risk
Introduces a new multi-node Slurm/Ray/vLLM orchestration path and cluster prerequisites (staged snapshots, Docker image,
eno0networking); CI workflow matrix filtering changes could affect when eval jobs run.Overview
Adds Qwen3.8-2.4T-A95B FP8 agentic-coding benchmarks on two MI355X nodes via a new
vllm-multi-nodespath: Slurm exclusive allocation, Docker on each node, Ray cluster (16 GPUs), and vLLM serve with TP8/PP2 and thevllm/vllm-openai-rocm:qwen38image.The new runtime under
benchmarks/multi_node/qwen3.8_vllm_multi_nodes/handles submission, per-node control, AgentX client wiring throughagentic_srt.sh, and pre-flight checks that matching Hugging Face snapshots exist on both nodes (snapshot_manifest.py/verify_model_staging.sh).launch_mi355x-amds.shsourcesqwen3.8_env.shfor staged model paths under/modelsinstead of/it-share/data, and routesvllm-multi-nodesinto the multi-node agentic recipe layout.CI/config:
amd-master.yamldefinesqwen3.8-fp8-mi355x-vllm-multi-nodes-agentic(conc 1);perf-changelog.yamldocuments it;MODELS.mdrecords Qwen3.8 as added 2026-08-14. E2E workflow adds aneval-configmatrix slice for non-agenticrun-evaljobs and tightens several eval jobifguards so empty config strings do not spawn matrix jobs.Reviewed by Cursor Bugbot for commit bcfcbe7. Bugbot is set up for automated code reviews on this repo. Configure here.