Fix projection_box crash on scalar bounds with multi-leaf pytrees - #1724
Open
winklemad wants to merge 1 commit into
Open
Fix projection_box crash on scalar bounds with multi-leaf pytrees#1724winklemad wants to merge 1 commit into
winklemad wants to merge 1 commit into
Conversation
`projection_box` passes `lower`/`upper` straight into `jax.tree.map(jnp.clip, tree, lower, upper)`, which requires them to be pytrees with the same structure as `tree`. A scalar bound therefore crashes on any multi-leaf pytree with `ValueError: Expected dict, got 0`, even though the docstring documents `lower`/`upper` as "a scalar or tree". This also breaks `projection_hypercube`, whose default `scale=1` is a scalar, so `projection_hypercube(params)` raises on the common case of a multi-leaf parameter tree. Broadcast a scalar bound to the tree structure before clipping (a bound that already matches the tree is passed through unchanged), mirroring how the sibling `projection_linf_ball` already broadcasts its scalar `scale`. Added tests covering scalar bounds on a multi-leaf tree for both `projection_box` and `projection_hypercube`.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
No linked issue — found by reading the code.
The bug
projection_boxforwardslower/upperdirectly intojax.tree.map(jnp.clip, tree, lower, upper), which requires them to be pytrees with the same structure astree. So a scalar bound crashes on any multi-leaf pytree, even though the docstring documents them as "a scalar or tree with the same structure astree":projection_hypercube's defaultscale=1is a scalar, soprojection_hypercube(params)fails on the common case of a multi-leaf parameter tree (single-leaf inputs happen to work, which is why the existing tests never caught it).Smoking gun
The sibling
projection_linf_ballalready works around this by broadcasting its scalarscaleto the tree before callingprojection_box:so
projection_linf_ball(params, 1.0)succeeds on the exact multi-leaf tree thatprojection_hypercube(params)crashes on.Fix
Broadcast a scalar bound to the tree structure inside
projection_box(a bound that already matches the tree's structure is passed through unchanged, so both a scalar and a matching tree are accepted). This fixesprojection_boxfor scalar bounds and, transitively,projection_hypercube.Testing
Added sub-tests for scalar bounds on a multi-leaf tree to both
test_projection_boxandtest_projection_hypercube. They fail before the change (ValueError) and pass after; the fulloptax/projections/_projections_test.pysuite passes (39 tests / 43 sub-tests).ruffis clean and the changed lines arepyink-formatted.