Skip to content

fix(spark): return NaN and -Infinity from pow where Spark does - #26382

Open
abp002 wants to merge 1 commit into
apache:mainfrom
abp002:spark-pow-strictmath
Open

abp002 wants to merge 1 commit into
apache:mainfrom
abp002:spark-pow-strictmath

Conversation

@abp002

@abp002 abp002 commented Oct 15, 2026

Copy link
Copy Markdown
Contributor

Which issue does this PR close?

Rationale for this change

datafusion-spark returns 1 for pow(1, NaN) and pow(±1, ±Infinity) where Spark returns NaN, and Infinity for pow(-0.0, -1.0) where Spark returns -Infinity.

What changes are included in this PR?

The special case for a zero base is removed, since powf already gives Infinity for 0.0 and -Infinity for -0.0 with a negative odd exponent. A NaN exponent, or a base of 1 or -1 with an infinite exponent, now returns NaN.

What is the testing strategy for this PR?

New and updated cases in spark/math/pow.slt, for both the scalar and the array path.

Are there any user-facing changes?

pow and power return NaN or -Infinity in these cases, as Spark does.

@github-actions github-actions Bot added sqllogictest SQL Logic Tests (.slt) spark Changes to datafusion-spark crate labels Oct 15, 2026
@abp002

abp002 commented Oct 18, 2026

Copy link
Copy Markdown
Contributor Author

Hi @comphead, could you approve the CI run when you have a moment? Thanks!

@abp002
abp002 force-pushed the spark-pow-strictmath branch from c60cd36 to 81b37da Compare October 24, 2026 20:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

spark Changes to datafusion-spark crate sqllogictest SQL Logic Tests (.slt)

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug] pow differs from Spark for -0.0, NaN and infinite exponents

1 participant