Fix scale_by_rprop returning a one-step-stale update#1725
Open
winklemad wants to merge 2 commits into
Open
Conversation
`scale_by_rprop` computes the correct current update in `prev_updates` (`step_size * sign(g)`, zeroed where the gradient sign flips) but then emits `state.prev_updates` -- the *previous* step's update -- instead. The freshly computed `prev_updates` is passed into the final `tree.map` (as `g`) but the lambda body ignores it and returns `prev_g` (`state.prev_updates`). As a result every Rprop update lags one iteration behind and the first step emits all zeros (no progress). The generic convergence tests miss it because the optimizer still converges, just more slowly. Emit `prev_updates` directly. Its value now matches the reference Rprop sequence (verified against PyTorch's `Rprop`). `state.prev_updates` is still stored for the next step's sign computation, which is correct and unchanged. Added a per-step regression test asserting the emitted updates.
The scale_by_rprop fix makes the first step emit a real update instead of zeros, so the optax.rprop docstring example now converges slightly faster. Update its expected objective-function values accordingly. Signed-off-by: winklemad <winklemad@gmail.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No linked issue — found by reading the code.
The bug
scale_by_rpropcomputes the correct current update inprev_updates(step_size * sign(g), zeroed where the gradient sign flips), but the emittedupdatesis built fromstate.prev_updates— the previous step's update — instead of the value just computed:The freshly-computed
prev_updatesis passed in asgbut the lambda never uses it — it returnsprev_g(state.prev_updates). So every Rprop update lags one step behind, and the first step emits all zeros:The generic convergence tests (
alias_test.py) miss it because the optimizer still converges, just more slowly — they never assert per-step values.Fix
Emit
prev_updates(the current step's update) directly.state.prev_updatesis still stored for the next step's sign computation — that use is correct and unchanged. With the fix the emitted sequence matches the reference Rprop exactly.Testing
Added
test_scale_by_rprop_step_values, asserting the per-step emitted updates against the hand-computed Rprop sequence. It fails before the change and passes after;optax/_src/transform_test.pypasses (24 tests) andoptax.rpropstill converges.ruffis clean and the changed lines arepyink-formatted.