[fank]

Actual GPU failure

The controlled H100 probe proved unpacked repeats exact with Stream-K off, then the packed launch failed native BS shape validation (1024 !=2048). The original image changed JIT scale storage but omitted native launcher geometry. No packed output pass exists yet. The previous no-native-rebuild assumption was wrong.

Fix

• Read explicit USE_PACKED_BS kernel metadata, default false for legacy cubins.
• Validate packed [E,G,N/2] shapes while keeping all common dtype/device/contiguity checks. Require packed-K group scales, aligned N and non-TMA BS; no bypass.
• Rebuild ONLY the host launcher with Humming's stable Torch ABI recipe, one C++ translation unit; update its existing native manifest source hash without changing other artifact entries. No vLLM/CUDA-kernel rebuild.
• Probe now requires exact equality of all transformed non-scale tensors and all expanded scale bytes before launching.

Addressing audit: launcher passes bs.data_ptr unchanged when TMA-BS is off; its TMA descriptor helper returns early. JIT global/shared copy strides use N/2 bytes per group and EGN/2 storage, while the scale reader expands each byte to the original two byte values. Shape correction matches those strides, not an oversized fake view.

Evidence / limits

24 CPU repository tests pass. Modified launcher compiled successfully in CPU-only audit container capped2 CPUs/4GiB RAM, no swap, one translation unit. Python compilation passed. Actual corrected-image GPU shape acceptance/output equality/graph replay and serving memory qualification remain pending. Default feature flags remain off. No GPU job launched from this session.

Source failure log retained by serving under results/packctl; do not treat controlled tests as production-default Stream-K qualification. No equality tolerances relaxed.

[fank]

Actual H100 controlled gate PASS on corrected image sha256:3f4e61bcc73cc5bf87d73ab69d46db718016b2ff28d9b5cd77e62c87693de978 (source642dd15):

PACKED_SCALE_KERNEL_GATE_PASS {"cases":8,"experts":8,"layer":0,"tp_rank":0,"stream_k_disabled":true,"scope":"controlled_storage_kernel_only"}

• Real8-expert Vision drafter subset, w13/w2 TP2 shapes, indexed top6, rows1/8/48/256.
• All transformed non-scale tensors match exactly; all unpacked packed-scale bytes match exactly. Scale storage halves: w13[8,128,2048]->[8,128,1024], w2[8,32,4096]->[8,32,2048].
• Packed native shape checks accept the real half-width tensors. Eager comparisons and CUDA graph input-change/restoration checks passed, without relaxed tolerances.
• This is NOT production-default Stream-K bitwise parity: the unmodified default baseline itself varied533/12288 elements,max_abs0.0625; controlled stream-K-off repeats were exact.

Serving is proceeding to isolated same-corrected-image flag-off/on evaluation with default kernels. Full-model behavior, production-path performance and actual KV savings remain pending; no production adoption claimed. Raw evidence is retained in serving results/packdiag,packctl,packfix; earlier failed results are not erased by this pass.

[fank]

Full serving follow-up: same corrected image642dd15 (sha256:3f4e61bcc73cc5bf87d73ab69d46db718016b2ff28d9b5cd77e62c87693de978), TP2 H100 NVL, only HUMMING_USE_PACKED_BS toggled; default Stream-K kernels. Both suites exited0.

Metric	Off	On
Reported KV tokens	1,129,474	1,490,579 (+361,105,+31.97%)
Model load memory/GPU	78.61 GiB	76.45 GiB
Final graph memory/GPU	0.61 GiB	0.61 GiB
c1 decode tok/s	208.8	202.9 (-2.8%)
c4 aggregate tok/s	594.1	585.3 (-1.5%)
c8 aggregate tok/s	952.8	942.3 (-1.1%)
Cold1M TTFT	245.659s	245.454s
Stability completed/errors	438/0	443/0

Both: vision3/3,boundary8/8,needles2/2; full ladders completed.

Acceptance audit from raw accept.json:

• conc1 aggregate33.12% ->34.84%.
• Per domain off->on: code30.56->37.20%, EN23.77->23.97%, DE27.03->25.79%, agentic56.58->66.78%, long35.23->35.32%, vision37.80->37.62%.
• Mixed conc4 44.76->43.31% (-1.45 percentage points). Generated lengths differ and tokens within requests are correlated; these are descriptive single-pair results, not independent Bernoulli samples or evidence packing improves acceptance.

Assessment: recommend the opt-in for the separate daytime capacity-oriented rollout, accepting the observed1–3% decode cost for32% more KV capacity. Do not call that cost proven noise, claim bitwise full-output parity, or add the separate residual-buffer saving without a combined run. Production configuration and tonight's old-image restore are unchanged. Raw evidence retained by serving under results/packoff and results/packon.