Fix finalized quant remainder dtypes#629
Merged
Merged
Conversation
|
| Filename | Overview |
|---|---|
| libreyolo/quant/api.py | Updates finalized checkpoint export, load, reprepare, and dequantize dtype handling for quantized models. |
| tests/unit/test_quantize.py | Adds CPU round-trip coverage for finalized quant checkpoint dtype preservation. |
Reviews (2): Last reviewed commit: "Protect all quant-module buffers from th..." | Re-trigger Greptile
![greptile-apps[bot]](https://avatars.githubusercontent.com/in/867647?s=60&v=4){kind=small}
The export path guarded quant buffer dtypes with a hardcoded name list, which silently excluded buffers it did not know about (NVFP4's weight_block_scale, MXFP4's weight_block_exp, and anything a future recipe registers). Those two are stored as float8_e4m3fn and int8 today, so the fp32-only cast did not corrupt them in practice, but the design invited exactly that bug. The keep-exact set is now collected from the finalized quant modules' own registered buffers, so the invariant is structural: quant buffers keep their contract dtypes, only the non-quantized remainder is cast. Adds nvfp4/mxfp4 fp16-remainder round-trip tests asserting the scale and exponent dtypes survive export and reload. Claude-Session: https://claude.ai/code/session_018Bkjq2cuWtXkEB6xHcLoRH
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Found during the tier-2 kernel campaign: two dtype bugs in the finalized-checkpoint path.
The loader now casts only the non-quantized remainder before load, quant buffers keep their contract dtypes, export preserves _q_w_gscale as fp32, and re-preparing restores ordinary parameters to fp32. Adds a focused CPU round-trip test. Full suite: 56 passed, 1 skipped.
Note: stacked on kernels/tier1-4 (#623); the deferred fused-GEMM kernel itself is intentionally NOT in this PR (evidence and scratch kernel live outside the repo).
Code provenance
All changes are original, written for this PR against the existing LibreYOLO codebase. No third-party code was ported, adapted, or derived. The frozen W4 packing contract is unchanged.
Need help on this PR? Tag
/codesmithwith what you need. Autofix is disabled.