-
Notifications
You must be signed in to change notification settings - Fork 29k
[SPARK-54750][SQL] Fix ROUND returning NULL for Decimal values with l… #53529
New issue
Have a question about this project? Sign up for a free GitHub account to open an issue and contact its maintainers and the community.
By clicking “Sign up for GitHub”, you agree to our terms of service and privacy statement. We’ll occasionally send you account related emails.
Already on GitHub? Sign in to your account
[SPARK-54750][SQL] Fix ROUND returning NULL for Decimal values with l… #53529
Conversation
…ow runtime precision ### What changes were proposed in this pull request? Fixed a bug in `RoundBase` where `ROUND` function incorrectly returns `NULL` for certain Decimal values. The issue was caused by using the input decimal's runtime precision instead of the target type's precision when calling `toPrecision()`. For example, `ROUND(PERCENTILE_APPROX(2150 / 1000.0, 0.95), 3)` incorrectly returned `NULL` instead of `2.15`. ### Why are the changes needed? The fix changes `decimal.toPrecision(decimal.precision, s, mode)` to `decimal.toPrecision(p, s, mode)` in both `nullSafeEval` and `doGenCode` methods, where `p` is the target DecimalType's precision. ### Does this PR introduce _any_ user-facing change? Yes, this fixes a bug where ROUND could return NULL for valid decimal inputs. ### How was this patch tested? Added regression test in `ApproximatePercentileQuerySuite`.
64fa14e to
b030ced
Compare
|
Hi @yaooqinn @gengliangwang |
yaooqinn
left a comment
There was a problem hiding this comment.
Choose a reason for hiding this comment
The reason will be displayed to describe this comment to others. Learn more.
LGTM
|
@qindongliang, can you make the GA green? I speculate it is due to your GitHub settings. |
Hi @yaooqinn, Thank you for triggering the CI. I investigated the failure and found the root cause: The build itself succeeded (all layers are cached from Apache's registry), but the push This happens because:
My questions:
|
|
LGTM |
|
I'm not familiar with this area either, maybe you can check https://github.com/qindongliang?tab=packages for what is missing |
|
I found the issue - I needed to grant my home repo write access to the packages in GitHub Container Registry settings. The image push is now successful. CI is currently building and should turn green soon if everything goes smoothly. ✅ Reference: https://github.com/orgs/community/discussions/32184 |
### What changes were proposed in this pull request? Fixed a bug in `RoundBase` where `ROUND` function incorrectly returns `NULL` for certain Decimal values. The issue was caused by using the input decimal's runtime precision instead of the target type's precision when calling `toPrecision()`. For example, `ROUND(PERCENTILE_APPROX(2150 / 1000.0, 0.95), 3)` incorrectly returned `NULL` instead of `2.15`. ### Why are the changes needed? The fix changes `decimal.toPrecision(decimal.precision, s, mode)` to `decimal.toPrecision(p, s, mode)` in both `nullSafeEval` and `doGenCode` methods, where `p` is the target DecimalType's precision. ### Does this PR introduce _any_ user-facing change? Yes, this fixes a bug where ROUND could return NULL for valid decimal inputs. ### How was this patch tested? Added regression test in `ApproximatePercentileQuerySuite`. ### Was this patch authored or co-authored using generative AI tooling? No. Closes #53529 from qindongliang/SPARK-54750-fix-round-precision. Lead-authored-by: qindongliang <[email protected]> Co-authored-by: Kent Yao <[email protected]> Signed-off-by: Kent Yao <[email protected]> (cherry picked from commit 1e6d743) Signed-off-by: Kent Yao <[email protected]>
### What changes were proposed in this pull request? Fixed a bug in `RoundBase` where `ROUND` function incorrectly returns `NULL` for certain Decimal values. The issue was caused by using the input decimal's runtime precision instead of the target type's precision when calling `toPrecision()`. For example, `ROUND(PERCENTILE_APPROX(2150 / 1000.0, 0.95), 3)` incorrectly returned `NULL` instead of `2.15`. ### Why are the changes needed? The fix changes `decimal.toPrecision(decimal.precision, s, mode)` to `decimal.toPrecision(p, s, mode)` in both `nullSafeEval` and `doGenCode` methods, where `p` is the target DecimalType's precision. ### Does this PR introduce _any_ user-facing change? Yes, this fixes a bug where ROUND could return NULL for valid decimal inputs. ### How was this patch tested? Added regression test in `ApproximatePercentileQuerySuite`. ### Was this patch authored or co-authored using generative AI tooling? No. Closes #53529 from qindongliang/SPARK-54750-fix-round-precision. Lead-authored-by: qindongliang <[email protected]> Co-authored-by: Kent Yao <[email protected]> Signed-off-by: Kent Yao <[email protected]> (cherry picked from commit 1e6d743) Signed-off-by: Kent Yao <[email protected]>
### What changes were proposed in this pull request? Fixed a bug in `RoundBase` where `ROUND` function incorrectly returns `NULL` for certain Decimal values. The issue was caused by using the input decimal's runtime precision instead of the target type's precision when calling `toPrecision()`. For example, `ROUND(PERCENTILE_APPROX(2150 / 1000.0, 0.95), 3)` incorrectly returned `NULL` instead of `2.15`. ### Why are the changes needed? The fix changes `decimal.toPrecision(decimal.precision, s, mode)` to `decimal.toPrecision(p, s, mode)` in both `nullSafeEval` and `doGenCode` methods, where `p` is the target DecimalType's precision. ### Does this PR introduce _any_ user-facing change? Yes, this fixes a bug where ROUND could return NULL for valid decimal inputs. ### How was this patch tested? Added regression test in `ApproximatePercentileQuerySuite`. ### Was this patch authored or co-authored using generative AI tooling? No. Closes #53529 from qindongliang/SPARK-54750-fix-round-precision. Lead-authored-by: qindongliang <[email protected]> Co-authored-by: Kent Yao <[email protected]> Signed-off-by: Kent Yao <[email protected]> (cherry picked from commit 1e6d743) Signed-off-by: Kent Yao <[email protected]>
|
Merged to master and '4.1.1', '4.0.2', '3.5.8' |

What changes were proposed in this pull request?
Fixed a bug in
RoundBasewhereROUNDfunction incorrectly returnsNULLfor certain Decimal values. The issue was caused by using the input decimal's runtime precision instead of the target type's precision when callingtoPrecision().For example,
ROUND(PERCENTILE_APPROX(2150 / 1000.0, 0.95), 3)incorrectly returnedNULLinstead of2.15.Why are the changes needed?
The fix changes
decimal.toPrecision(decimal.precision, s, mode)todecimal.toPrecision(p, s, mode)in bothnullSafeEvalanddoGenCodemethods, wherepis the target DecimalType's precision.Does this PR introduce any user-facing change?
Yes, this fixes a bug where ROUND could return NULL for valid decimal inputs.
How was this patch tested?
Added regression test in
ApproximatePercentileQuerySuite.Was this patch authored or co-authored using generative AI tooling?
No.