[Fix] Restore backend dispatch under torch.compiler.disable - #1111
Merged
Merged
Conversation
@dispatch stacked over @torch.compiler.disable was silently discarded: functools.wraps copies _torchdynamo_orig_callable onto the dispatch wrapper, so dynamo's innermost_fn unwrapping bypasses it and backend selection never ran for the affected top-level entries: - chunk_kda (fla/ops/kda/chunk.py) — the flash_kda backend from fla-org#852 was never actually selected - fused_kda_gate (fla/ops/kda/gate.py) - chunk_gated_delta_rule (fla/ops/gated_delta_rule/chunk.py) Swap the decorator order so dispatch stays the outermost wrapper while the call remains excluded from compile graphs. Note this activates the flash_kda backend for inference-mode chunk_kda calls on systems with the flash_kda package installed, which is the intent of fla-org#852. Closes fla-org#1110
5 tasks done
zhiyuan1i
approved these changes
Aug 27, 2026
4 of 5 tasks
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
Fixes #1110. For ops decorated as
the dispatch wrapper is silently discarded at import time:
functools.wrapscopies_torchdynamo_orig_callablefrom thecompiler.disablewrapper onto the dispatch wrapper, so dynamo'sinnermost_fnunwrapping bypasses dispatch entirely and backend selection never runs. Swapping the decorator order (@torch.compiler.disableoutermost) keeps dispatch intact while the call remains excluded from compile graphs.Affected entry points, all fixed here (one line each):
chunk_kda(fla/ops/kda/chunk.py) — theflash_kdabackend from [KDA] Support FLASHKDA backend #852 was never actually selected;test_flash_kda_chunk*passed vacuously via the Triton path matching the same fp64 goldfused_kda_gate(fla/ops/kda/gate.py) — no registered backend implements it today, so no runtime change; fixed for consistencychunk_gated_delta_rule(fla/ops/gated_delta_rule/chunk.py) — activatesflash_qladispatch on systems with the package installed (the intent of [GDN] Add FlashQLA backend dispatch #998 / b4cfd53)Impact / blast radius
This is a behavior fix, not just a refactor: on systems with
flash_kdainstalled, inference-modechunk_kdacalls now genuinely route to theflash_kdabackend (it hasdefault_enable=True). That is what #852 intended, but it is an observable change for those users; anyone who needs the old behavior can setFLA_FLASH_KDA=0(orFLA_DISABLE_BACKEND_DISPATCH=1). Systems without these packages are unaffected — verifiers fall back to the default Triton path, so CI is unchanged.Test plan
tests/ops/test_kda.py— full suite passes;test_flash_kda_chunk*now genuinely exercise the flash_kda backend (verified by spying on the backend entry)tests/ops/test_gdn.py,tests/ops/test_gdn2.py,tests/ops/test_gdn_kernels.py— pass (flash_qlanot installed here; dispatch iterates and falls back). One flake:test_gdn2.py::test_chunk_varlen[cu_seqlens[0, 64, 128]-H2-K64-V64-gateFalse-torch.float32]failed once in the full-suite run and passed on isolated rerun with comfortable margins (all ratios ~1e-3); the fix does not alter the GDN code path withoutflash_qlainstalledscripts/find_dependent_tests.pyfor the three touched filesBenchmark / NCU (kernel changes only)
Neutral — decorator order only; no kernel, numerics, or scheduling change. The newly-activated
flash_kdainference path is faster than the Triton default (see #852 / FlashKDA benchmarks), not slower.Checklist
Breaking changes
None for default configurations. Behavior change for users with
flash_kda/flash_qlainstalled, as described above (restores the intended behavior of #852/#998).