Repository navigation

Support Opt-in FP8 KV Cache quantization with FA2-4 backend. - #129

Open

0z5a wants to merge 14 commits into

InferMatrixLab:mainfrom

0z5a:kv/fp8-kv-cache

Open

Support Opt-in FP8 KV Cache quantization with FA2-4 backend.#129
0z5a wants to merge 14 commits into
InferMatrixLab:mainfrom
0z5a:kv/fp8-kv-cache

Commits on Oct 15, 2026
- [KV] FP8 e4m3 KV cache with per-vector scales
  
  0z5a
  and
  claude
  committed
  f6286f5
- [Docs] FP8 KV cache usage, semantics and limits
  
  0z5a
  and
  claude
  committed
  ee375cb
Commits on Oct 16, 2026
- [KV] Fused FP8 quantize-and-store kernel for Triton
  
  0z5a
  and
  claude
  committed
  0e530b2
- [Docs] FP8 KV: fused Triton writer, current token is read in FP8
  
  0z5a
  and
  claude
  committed
  f36f2f9
Commits on Oct 21, 2026
- [Fix] Merge main into FP8 KV cache support
  0z5a
  committed
  6087e67
Commits on Nov 1, 2026
- Merge remote-tracking branch 'upstream/main' into kv/fp8-kv-cache
  0z5a
  committed
  b2803a2
- Run the merged FP8 tests on GPUs with less shared memory
  0z5a
  committed
  fbfeaad
Commits on Nov 6, 2026
- fix(attention): reject FP8 per-vector KV on FlashAttention with guidance
  0z5a
  committed
  8baa95f
- feat(attention): support FP8 per-vector KV on FA2-4
  0z5a
  committed
  d2043f5
- feat(attention): advertise FP8 KV on FA2-4
  0z5a
  committed
  529151c
- fix(engine): reject CUDA graphs for FP8 KV on FlashAttention
  0z5a
  committed
  65163d4
- fix(attention): keep FA3 paged FP8 disabled until validated
  0z5a
  committed
  9c81c31
- perf(attention): fuse FP8 KV staging into one Triton launch
  0z5a
  committed
  9fed631
- perf(attention): cache FP8 staging buffers and page-table map
  0z5a
  committed
  260404a