Refuse an out-of-regime anchor, and price sparse attention as selection plus indexing

[araina-amd]

Description

An anchor is only a calibration if the measurement describes the deployment it is applied to. This branch makes that true for the serving engine, the GPU part, the attention-kernel family, the quantizer, and speculative decode; then it prices DeepSeek-V4, GLM-5.2, and MiniMax-M3 from the attention they actually run, including under attention-DP, instead of a top-k floor or a dense charge that is the wrong shape for the work.

The rest of the branch is the closed-loop replay that those projections are scored against. A conversation that never pauses, a prefix charged once per sharer, a cache that evicts the head before the tail, a disaggregated prefill pool on the wrong clock, and an agentic session replayed as independent requests are not small biases: they are the difference between a finding about the topology and a stalled loop.

Type of change

• Bug fix (non-breaking change which fixes an issue)
• New feature (non-breaking change which adds functionality)

Changes

Regime: do not call a measurement a match when it is not

• Hardware part is regime-defining. Nothing in an anchor recorded which GPU produced it, so GLM-5.2's MXFP4 Quark timings (gfx950) were pricing an MI325X deployment. The curve read 4.3× the measured throughput and a twentieth of the measured TTFT, the worst in the matrix, while reporting itself calibrated. Unrecorded parts stay unknown rather than wrong; the store stamps the part at build time for the pairs whose part can be argued for.
• Serving engine is a regime axis. Every engine's anchor for a checkpoint hashed to one signature, so mori-sglang and sglang could share a measurement. The two report very different KV pools because one shards the MLA latent and the other replicates it.
• Attention backends compare by kernel family, not engine spelling. AITER's MLA / unified / DeepSeek-V4 flags were treated as distinct from the family the projector models, so the axis refused a matching harvest and the run reported benchmark mode while calibrating nothing.
• A quantizer's name is not a dtype. vLLM and SGLang record quantization_config.quant_method as the toolchain, so an MXFP4 anchor comes back as quark. Quark emits fp8 and fp4 alike, and every GLM-5.2-MXFP4 anchor on MI355X was refused on a weight-dtype mismatch against the checkpoint it was measured on. Toolchain names now read as an unknown weight dtype, which regime_distance skips, rather than a wrong one.
• Coverage is legible. A prefill probe too short to fit a context curve is refused (decode stays calibrated; prefill and TTFT fall back). A decode sweep that does not reach the asked batch is warned, not signed: the old warning claimed under-cost / over-throughput, which holds on the closed-form path and reverses under trace replay.

Harvest: measure the deployment it is labelled with

• A speculative harvest now actually speculates. Flags were accepted into the regime signature but never passed to the engine; AnchorStore then rebuilt the regime from meta and dropped the speculative keys; acceptance was unpinned on dummy weights, so the measured rate was the non-speculative one filed under a speculative label.
• Speculative methods reach each engine by name. Collapsing every method to MTP sent EAGLE3 and DSpark harvests looking for draft layers inside checkpoints that have none, and the EAGLE3 draft was never passed on.
• ATOM is driveable. Its image carries neither vLLM nor SGLang, so its load generator is the vLLM legacy client ATOM vendors.
• Every client draws a fresh seed. All three default to a fixed one, so each repeat of a probe resent the same prompts, the prefix cache served them, and the prefill probe priced only the uncached tail instead of the whole prompt.
• --trust-remote-code given through --server-args reaches the client too, which loads the same tokenizer (Kimi-K3). When the median ITL sits well above mean TPOT the stream is coalescing tokens, and the decode step is read from mean TPOT instead.

Attention: selection and indexing are not one multiplier

• DeepSeek-V4 is priced from the per-layer compression schedule the checkpoint ships. A compressed layer attends over a local window plus a pool, so the term that keeps growing with context is the indexer, not the attention.
• GLM-5.2 and MiniMax-M3 select the same way on every layer, so they had no schedule and fell through to a top-k floored at 0.15 — nine to twenty-seven times the selection it names on the agentic corpus, and nothing for the indexing it stands in for. Selection falls like topk/context; indexing grows with context as dense attention does. Priced apart, GLM-5.2 is 0.032 of dense at 131k falling to 0.021 at 365k. sparse_indexer_n_heads was the missing input (it sizes no cache, so it was never recorded). sparse_indexer_cost_scale prices the kernel: fused aiter on gfx950, unfused Torch on gfx942.
• Attention-DP decode is priced from the attention it runs. Under attention-DP a rank runs every head of its own sequences, so dense decode attention over the whole context turns compute-bound, and that compute is what selection or compression removes. Charged dense, DeepSeek-V4-Pro on ATOM with attention-DP projected an ITL several times what was measured. DPA decode is now costed like prefill: from the compression schedule, else the uniform selection, else the configured scale, plus the unfused indexer's excess. INFERASIM_DPA_DECODE_DENSE=1 restores the dense charge for ablation.
• Anchors are transported by the model's whole decode step, at the width the target verifies at, not by a bare forward pass. A measured step carries its fixed costs, and a compute-only ratio grew them with batch too. Within that shape a resident sequence adds its compressed pools and window, not a dense read of its context.
• The fitted KV stream scales with the target's KV width, so fp8 and bf16 KV no longer project alike. A single-context anchor carries its KV delta from the analytic step. A prefill chunk is simulated at the context already resident, not at its own length. An anchor's attention-DP size is read from its server args when the meta does not record it.
• Configs that said bases now inherit. The loader only read extends, so every field the base was there to supply was a dataclass default.
• Index keys sit beside the KV cache and are not divided by tensor parallelism; they keep the stack dtype when KV is quantized. Decode context parallelism is the axis that actually shrinks an MLA cache.

Replay: a client that thinks, a pool that is shared, a cache that evicts leaves

• Think time. Closed-loop replay issued the next turn the instant the last one returned, so N conversations were N requests permanently in flight. The AgentX lanes show a 14–17s median gap. Without it the ladder is the engine's ceiling at every rung.
• A shared prefix is reserved once. Prefill already skipped the hit; the admission gate charged every request its whole context, inflating the footprint by more than an order of magnitude and putting a whole request latency in front of TTFT.
• The prefix cache and the KV pool share one allocation; each was being handed the whole of it.
• Eviction is leaf-first (LRU among leaves), which is what a radix cache does. Recency-only eviction is the cyclic-sweep pathology: every block is dropped exactly before it is reused. A split now reports the reuse it already computed.
• Admission is off the waiting queue by prefix, not by age. A closed loop holds one request per client; handing a freed slot to another client by arrival order drops the reuse it was about to collect.
• The replay reports over the window the harness reports over. duration_ms and prefill_exclusive were accepted at run_des and never handed down, so every cached/trace run ignored the clock and SGLang/ATOM replays scheduled vLLM's unified batch.
• Admission-window and warmup knobs are opt-in, on ablation against the AgentX corpus: the opening burst is the harness's one-token lane-advance, not a warm start, and still sits in the reported average.
• A split's prefill pool is fed from the simulation clock. Arrivals stamped on the decode clock were released against a prefill clock that stopped when its queue drained, so the population in flight collapsed and decode ran at minimum batch — looking like a finding about disaggregation.
• --kv-pool-tokens and --workload-resident-tokens take the pool the engine allocated and the length-biased resident mean, so a memory-model error no longer shows up as an admission error.

Replay: AgentX sessions the way the harness drives them

• Sessions, not one request per slot. Each request waits on the previous turn of its stream and on any subagents it joins, plus its own recorded think_ms from the Mooncake trace. A session takes a client slot when one frees and holds it until its whole tree drains, which is the shared-sampler client AIPerf's AgentX lanes run.
• Concurrency counts sessions, so the server admits a session's subagents up to its own sequence limit. Capping the batch at the concurrency queued a subagent behind its sibling's whole decode, which dominated tail TTFT.
• A turn whose prompt extends the request it waits on is not routed until that resolves, or both would prefill.
• The replay stops issuing at the horizon and rates over the window plus the drain, and reports total tok/s (prompt plus generated) beside output tok/s.
• New knobs:
  • --max-num-seqs caps running sequences as the engine does.
  • --des-client-idle-cap-ms mirrors the harness's system-idle guard, which pulls client timers forward when nothing is in flight.
  • INFERASIM_DES_LOCKSTEP prices a step at the combined batch of that many schedulers stepping together, as SGLang attention-DP ranks do at every MoE layer; mixed steps take a prefill batch so lockstep ranks each attend over their own chunk.
  • --uncached-prompt-latency-us adds an opt-in TTFT-only host term per prompt token the cache did not serve, capped by --uncached-prompt-latency-max-tokens; both default to 0.

Testing

New and extended tests under tests/unit/projection/ for uniform sparse attention, hybrid compressed attention, speculative harvest, serving-anchor harvest, closed-loop think time / cache–pool sharing / measurement window / prefix cache, disaggregated reporting and DES scheduling, and anchor reuse across regime axes. All 448 tests under tests/unit/projection/ pass.

The hybrid-attention TTFT test now runs at one client. Under load TTFT also holds the wait behind other prefills, which grows faster than the prompt for queueing reasons,