Skip to content

fix(inference): keep transfer attention dense for single samples#133

Open
trungtpham wants to merge 2 commits into
NVIDIA:mainfrom
trungtpham:fix/transfer-dense-attention-sm120
Open

fix(inference): keep transfer attention dense for single samples#133
trungtpham wants to merge 2 commits into
NVIDIA:mainfrom
trungtpham:fix/transfer-dense-attention-sm120

Conversation

@trungtpham

Copy link
Copy Markdown
Contributor

Summary

  • align multi-control transfer with the standard two-way single-sample inference path
  • omit varlen metadata during inference so SM120 can select cuDNN instead of incompatible NATTEN kernels
  • retain varlen attention for grad-enabled execution and add focused routing coverage

Test plan

  • Run Python syntax checks on the modified files
  • Run focused attention unit tests in the project CUDA environment
  • Run video transfer inference on RTX PRO 6000 with the imaginaire4 v12 image
  • Confirm H20/H100/A100 transfer inference remains healthy

trungtpham and others added 2 commits July 23, 2026 17:00
Match standard two-way inference routing so SM120 transfer can use cuDNN instead of falling back to incompatible varlen NATTEN kernels.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants