feat(kernels): Rust wrapper for the hd256 windowed paged prefill - #883
Merged
FeathBow merged 1 commit intoAug 15, 2026
Merged
Conversation
Collaborator
Author
|
@codex review please |
There was a problem hiding this comment.
💡 Codex Review
Here are some automated review suggestions for this pull request.
Reviewed commit: ac3f8b8a9b
ℹ️ About Codex in GitHub
Codex has been enabled to automatically review pull requests in this repo. Reviews are triggered when you
- Open a pull request for review
- Mark a draft as ready
- Comment "@codex review".
If Codex has suggestions, it will comment; otherwise it will react with 👍.
When you sign up for Codex through ChatGPT, Codex can also answer questions or update the PR, like "@codex address that feedback".
Signed-off-by: Feathbow <feathbow@gmail.com>
FeathBow
force-pushed
the
feat/kernels-windowed-paged-wrappers
branch
from
August 15, 2026 10:04
ac3f8b8 to
68f1f50
Compare
4 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
Closes #882.
The windowed C entry points landed with no callers; this wraps the prefill one in the Gemma 4 serving shape — attention read only, the pool-write prep owns the scatter — with
sm_scaleexplicit (Gemma 4 runs unscaled attention; the scale is model policy, not geometry) andwindow_leftunder its inclusive-distance contract.PrefillPagedPlan, cross-checking the plan's recordedcta_tile_qagainst the kernel's own derivation, so a plan tiled under an override cannot be silently misread.checked_paged_geometry, now serving prep writes and attention reads — and scalar narrowing on this path goes through checked conversions throughout, in the wrapper and in the plan alike, so a length pasti32fails rather than truncating into a legal-looking bound. The batch tile query also moves its positivity check into the i32 domain, where the single-request path already had it.PrefillPagedPlan, where the arithmetic that needs them lives. The GQA group size is an integer division bynum_kv_heads— in the plan's own tile arithmetic, in the C tile queries and again in the kernel, none of which guard the divisor — so every plan settles the head relation before dividing, and the wrapper is left checking only that the plan and the layout agree. And because the attention kernels compute addresses from page ids and the last-page length with no device-side bounds check, the plan records both bounds from its host inputs, at construction and on every in-place update, committed only after the update fully succeeds; the wrapper compares them against the pool geometry before every launch.The windowed decode C entry deliberately stays wrapper-less: serving decodes through this entry at one token, and the dedicated decode form (non-partitioned versus split-KV) is a measurement-gated decision for the batching stage.
Test Env
Single GPU (sm_89, x86_64), CUDA 12.9,
PEGAINFER_REQUIRE_GPU=1.Verification
Errbefore any launch, through the same checked-geometry seam the prep wrappers use.hd256_window_prefill_reject) drives the reject paths through the public surface: a page id past the pool and a last-page length pastpage_sizeboth fail asErrbefore any launch and the same shapes in range launch; a batch-plan length pasti32fails rather than truncating into a legal-looking bound; and a zero KV head count fails when the plan is built, before anything divides by it.Type of Change