[WIP][AMD][AgentX] Kimi-K3 vLLM 1P1D DSpark disagg on MI355X - #2675
Closed
YukioZzz wants to merge 2 commits into
Closed
[WIP][AMD][AgentX] Kimi-K3 vLLM 1P1D DSpark disagg on MI355X#2675YukioZzz wants to merge 2 commits into
YukioZzz wants to merge 2 commits into
Conversation
YukioZzz
requested review from
1am9trash,
billishyahao,
chunfangamd,
seungrokj and
yctseng0211
as code owners
August 19, 2026 07:53
![claude[bot]](https://avatars.githubusercontent.com/in/1236702?s=60&v=4){kind=small}
…nalysisAI#2602) Add the agentic trace-replay recipe, models_vllm.yaml serve body, container patches, and harness wiring for Kimi-K3 MXFP4 1P1D disaggregated serving. MoRIIO #51052 hybrid-transfer support lands in the following commit.
YukioZzz
force-pushed
the
yichaozhu/k3-rocm-agentx-disagg
branch
from
August 19, 2026 10:00
f427482 to
3fd103e
Compare
Apply vLLM #51052 MoRIIO hybrid-state transfer for Kimi-K3 1P1D: unified diff patch, wait_all image prep, Kimi-K3 MoRI queue sizing, MultiConnector DRAM tier, and write-mode batch barrier. Drops the drafter-invariant kv_cache_utils hunk because draft engines run prefill init and layer counts match the target.
YukioZzz
force-pushed
the
yichaozhu/k3-rocm-agentx-disagg
branch
from
August 19, 2026 11:04
3fd103e to
4e824dd
Compare
Collaborator
|
lots of patches in this PR, please upstream the patches before reopenning per the AgentX guidelines to avoid flooding the mi355 queue |
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
2 participants
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
[WIP] Adds the Kimi-K3 MXFP4 1P1D disaggregated AgentX arm on MI355X (
cluster:mi355x-amds):kimik3-fp4-mi355x-vllm-disagg-agentic-mtpnum_speculative_tokens=2, synthetic AL 2.51)SimpleCPUOffloadConnector@ conc 8/12/16 (dram-utilization: 0.50)vllm/vllm-openai-rocm:nightly-ac7509e2b1db40fec2f03dde1ed4e9dfdc2338c9vllm/vllm-router:nightly-20260809-d2ba586Harness changes are scoped to what is required to run this arm:
k3_moriio_51052.patchapplied in-container for Kimi-K3 hybrid MLA/KDA over MoRIIOIOEngine.wait_allfor MoRIIO write mode (READ fallback if build fails on any node)apply_k3_container_patches.sh(gfx950 AITER / Triton MLA / KV-pool fixes)trace_replay.sh, MoRIIO + SimpleCPU offload wiring inserver_vllm.sh, log collection + CI artifact uploadSupersedes the debug-heavy commit history on the prior branch; one squashed commit on current
main.Validation (End-to-End Tests, branch pre-squash)
All runs: 1P1D TP8, DSpark MTP, MoRIIO write path, 3600 s agentic trace replay on MI355X.
Test plan
yichaozhu/k3-rocm-agentx-disaggwith the squashed commit (kimik3-fp4-mi355x-vllm-disagg-agentic-mtpmaster config)/reuse-sweep-runafter full green sweep with evals passingRelated
YukioZzz/yichaozhu/k3-rocm-agentx-disagg