fp4 is going to open?

[MarsMeng1994]

https://arxiv.org/pdf/2501.17116
Optimizing Large Language Model Training Using FP4 Quantization

[Mr-Philo]

Hi @MarsMeng1994 , thanks for your attention! This paper's training framework is exactly MS-AMP, with the FP4 components under PR review and maybe further polish. Please stay tuned!

[AbhinavDutta]

Hi Authors, any follow up? It is disappointing to find the code for fp4 absent so long after the paper publication (especially when the paper claims that the code is available?). The main thing I wanted to check out was if the gap between STE and DGE + OCC is significant, and if so are DGE and OCC equally responsible for the improvements or one of them doing most of the heavy-lifting ?

[Mr-Philo]

Hi Authors, any follow up? It is disappointing to find the code for fp4 absent so long after the paper publication (especially when the paper claims that the code is available?). The main thing I wanted to check out was if the gap between STE and DGE + OCC is significant, and if so are DGE and OCC equally responsible for the improvements or one of them doing most of the heavy-lifting ?

Hi @AbhinavDutta , thank you very much for your attention. Our FP4 code has been fully polished and is ready for release. However, since the MS-AMP library management team requires additional process to address UT and security-related issues, and I lack the permissions to handle this matter, the code has not yet been merged into the main branch. Nevertheless, you can find all FP4-related code in PR #203 , which I believe will be helpful for your verification.