Skip to content

Conversation

@qindongliang
Copy link
Contributor

What changes were proposed in this pull request?

Fixed a bug in RoundBase where ROUND function incorrectly returns NULL for certain Decimal values. The issue was caused by using the input decimal's runtime precision instead of the target type's precision when calling toPrecision().

For example, ROUND(PERCENTILE_APPROX(2150 / 1000.0, 0.95), 3) incorrectly returned NULL instead of 2.15.

Why are the changes needed?

The fix changes decimal.toPrecision(decimal.precision, s, mode) to decimal.toPrecision(p, s, mode) in both nullSafeEval and doGenCode methods, where p is the target DecimalType's precision.

Does this PR introduce any user-facing change?

Yes, this fixes a bug where ROUND could return NULL for valid decimal inputs.

How was this patch tested?

Added regression test in ApproximatePercentileQuerySuite.

Was this patch authored or co-authored using generative AI tooling?

No.

@github-actions github-actions bot added the SQL label Dec 18, 2025
@qindongliang
Copy link
Contributor Author

…ow runtime precision

### What changes were proposed in this pull request?
Fixed a bug in `RoundBase` where `ROUND` function incorrectly returns `NULL` for certain Decimal values. The issue was caused by using the input decimal's runtime precision instead of the target type's precision when calling `toPrecision()`.
For example, `ROUND(PERCENTILE_APPROX(2150 / 1000.0, 0.95), 3)` incorrectly returned `NULL` instead of `2.15`.
### Why are the changes needed?
The fix changes `decimal.toPrecision(decimal.precision, s, mode)` to `decimal.toPrecision(p, s, mode)` in both `nullSafeEval` and `doGenCode` methods, where `p` is the target DecimalType's precision.
### Does this PR introduce _any_ user-facing change?
Yes, this fixes a bug where ROUND could return NULL for valid decimal inputs.
### How was this patch tested?
Added regression test in `ApproximatePercentileQuerySuite`.
@qindongliang qindongliang force-pushed the SPARK-54750-fix-round-precision branch from 64fa14e to b030ced Compare December 18, 2025 15:09
@qindongliang
Copy link
Contributor Author

Hi @yaooqinn @gengliangwang
The CI failure in "Base image build" is expected for external contributors due to GitHub's security restrictions on GHCR package writes from forked PRs.
Could you please help trigger the official CI when convenient? Thanks!

Copy link
Member

@yaooqinn yaooqinn left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@yaooqinn
Copy link
Member

@qindongliang, can you make the GA green? I speculate it is due to your GitHub settings.

@qindongliang
Copy link
Contributor Author

@qindongliang, can you make the GA green? I speculate it is due to your GitHub settings.

Hi @yaooqinn,

Thank you for triggering the CI. I investigated the failure and found the root cause:

The build itself succeeded (all layers are cached from Apache's registry), but the push
to ghcr.io/qindongliang/apache-spark-ci-image-docs:... failed with permission_denied: write_package.

This happens because:

  1. The workflow uses ${{ github.repository_owner }} to generate the image URL,
    which becomes ghcr.io/qindongliang/... in my fork
  2. GitHub doesn't allow Actions to automatically create new packages in the Container Registry
  3. I've already set "Read and write permissions" for workflows in my repo settings,
    but it still doesn't work for creating new packages

My questions:

  1. Is there a recommended way to configure fork repos to allow pushing to ghcr.io?
    Should I manually create placeholder packages first?
  2. Or could you run the CI directly from the apache/spark side where the packages
    already exist?

Any guidance would be appreciated. Thank you!
image

@qlong
Copy link

qlong commented Dec 19, 2025

LGTM

@yaooqinn
Copy link
Member

I'm not familiar with this area either, maybe you can check https://github.com/qindongliang?tab=packages for what is missing

@qindongliang
Copy link
Contributor Author

Thanks @yaooqinn! @qlong

I found the issue - I needed to grant my home repo write access to the packages in GitHub Container Registry settings.

The image push is now successful. CI is currently building and should turn green soon if everything goes smoothly. ✅

Reference: https://github.com/orgs/community/discussions/32184

@yaooqinn yaooqinn closed this in 1e6d743 Dec 19, 2025
yaooqinn added a commit that referenced this pull request Dec 19, 2025
### What changes were proposed in this pull request?

Fixed a bug in `RoundBase` where `ROUND` function incorrectly returns `NULL` for certain Decimal values. The issue was caused by using the input decimal's runtime precision instead of the target type's precision when calling `toPrecision()`.

For example, `ROUND(PERCENTILE_APPROX(2150 / 1000.0, 0.95), 3)` incorrectly returned `NULL` instead of `2.15`.

### Why are the changes needed?

The fix changes `decimal.toPrecision(decimal.precision, s, mode)` to `decimal.toPrecision(p, s, mode)` in both `nullSafeEval` and `doGenCode` methods, where `p` is the target DecimalType's precision.

### Does this PR introduce _any_ user-facing change?

Yes, this fixes a bug where ROUND could return NULL for valid decimal inputs.

### How was this patch tested?

Added regression test in `ApproximatePercentileQuerySuite`.

### Was this patch authored or co-authored using generative AI tooling?

No.

Closes #53529 from qindongliang/SPARK-54750-fix-round-precision.

Lead-authored-by: qindongliang <[email protected]>
Co-authored-by: Kent Yao <[email protected]>
Signed-off-by: Kent Yao <[email protected]>
(cherry picked from commit 1e6d743)
Signed-off-by: Kent Yao <[email protected]>
yaooqinn added a commit that referenced this pull request Dec 19, 2025
### What changes were proposed in this pull request?

Fixed a bug in `RoundBase` where `ROUND` function incorrectly returns `NULL` for certain Decimal values. The issue was caused by using the input decimal's runtime precision instead of the target type's precision when calling `toPrecision()`.

For example, `ROUND(PERCENTILE_APPROX(2150 / 1000.0, 0.95), 3)` incorrectly returned `NULL` instead of `2.15`.

### Why are the changes needed?

The fix changes `decimal.toPrecision(decimal.precision, s, mode)` to `decimal.toPrecision(p, s, mode)` in both `nullSafeEval` and `doGenCode` methods, where `p` is the target DecimalType's precision.

### Does this PR introduce _any_ user-facing change?

Yes, this fixes a bug where ROUND could return NULL for valid decimal inputs.

### How was this patch tested?

Added regression test in `ApproximatePercentileQuerySuite`.

### Was this patch authored or co-authored using generative AI tooling?

No.

Closes #53529 from qindongliang/SPARK-54750-fix-round-precision.

Lead-authored-by: qindongliang <[email protected]>
Co-authored-by: Kent Yao <[email protected]>
Signed-off-by: Kent Yao <[email protected]>
(cherry picked from commit 1e6d743)
Signed-off-by: Kent Yao <[email protected]>
yaooqinn added a commit that referenced this pull request Dec 19, 2025
### What changes were proposed in this pull request?

Fixed a bug in `RoundBase` where `ROUND` function incorrectly returns `NULL` for certain Decimal values. The issue was caused by using the input decimal's runtime precision instead of the target type's precision when calling `toPrecision()`.

For example, `ROUND(PERCENTILE_APPROX(2150 / 1000.0, 0.95), 3)` incorrectly returned `NULL` instead of `2.15`.

### Why are the changes needed?

The fix changes `decimal.toPrecision(decimal.precision, s, mode)` to `decimal.toPrecision(p, s, mode)` in both `nullSafeEval` and `doGenCode` methods, where `p` is the target DecimalType's precision.

### Does this PR introduce _any_ user-facing change?

Yes, this fixes a bug where ROUND could return NULL for valid decimal inputs.

### How was this patch tested?

Added regression test in `ApproximatePercentileQuerySuite`.

### Was this patch authored or co-authored using generative AI tooling?

No.

Closes #53529 from qindongliang/SPARK-54750-fix-round-precision.

Lead-authored-by: qindongliang <[email protected]>
Co-authored-by: Kent Yao <[email protected]>
Signed-off-by: Kent Yao <[email protected]>
(cherry picked from commit 1e6d743)
Signed-off-by: Kent Yao <[email protected]>
@yaooqinn
Copy link
Member

Merged to master and '4.1.1', '4.0.2', '3.5.8'

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants