Hi, I was working on the adls labs when I found out that the linearBlockMinifloat has backward which takes arguments same as the forward pass:
|
def backward( |
|
ctx, |
|
grad_output: Tensor, |
|
width: int, |
|
exponent_width: int, |
|
exponent_bias_width: int, |
|
block_size: list[int] | int = [16], |
|
skip_first_dim: bool = False, |
|
): |
Other precision backward passes do not have this arguments. Further, torch when calculating backprop wont pass these arguments, hence causing Missing 3 positional arguments error (Block size has default). This is either a bug, or have not been implemented yet, this precision hence cannot be used for QAT (only backward is broken).
Hi, I was working on the adls labs when I found out that the
linearBlockMinifloathas backward which takes arguments same as the forward pass:mase/src/chop/nn/quantizers/block_minifloat.py
Lines 84 to 92 in 488d49e
Other precision backward passes do not have this arguments. Further, torch when calculating backprop wont pass these arguments, hence causing
Missing 3 positional argumentserror (Block size has default). This is either a bug, or have not been implemented yet, this precision hence cannot be used for QAT (only backward is broken).