Repository navigation
[EPIC] A Collection of Wrong Results Bugs - Q4 2026 #26672
Description
This ticket lists all the open wrong results results bugs; Please add any new issues you find in comments.
Specifically where DataFusion produces a result but that result is incorrect.
Rationale / Call To Action
As a foundational piece of technology, it is important that users can count on DataFusion's results to be correct. For the most part I think we do a very good job and we have many tests
As part of my ongoing quest to figure out how to prioritize all the proposals / PRs to DataFusion, I think correctness bugs are quite high on the list so I wish to track them here.
I have also been needled several times in discussions with various professors from TUM (who shall remain nameless) that they don't think DataFusion takes correctness seriously. While I disagree with this sentiment, I do think it is worth time focusing on correctness.
The List ™️
Correlated subqueries and decorrelation
- Incorrect results for NOT IN subqueries with nulls #8241
- Returns the wrong grouped row when an outer
GROUP BYconsumes a subquery containingORDER BY ... OFFSET. #23534 - Wrong results: correlated
EXISTS/INsubqueries over a groupless aggregate get the wrong outer rows #24960 - Wrong results: correlated
NOT IN (subquery)ignores NULLs in the subquery #25347 - Wrong results: correlated
NOT INloses the correlation when it is an equality on theINvalue column #25480 - Wrong results: LATERAL with SELECT DISTINCT and a non-equality correlated filter returns duplicate rows #25808
- Wrong results: correlated aggregate subquery with a lossy
CASTin its filter returns one row per inner value #26014 - Wrong results: a window function in an
INsubquery sees every row when the correlated filter repeats theINpredicate #26032 - Wrong results: Correlated subquery confuses same-named columns of two of its relations #26234
- Wrong results: correlated scalar subquery with nested aggregates #26508
- Multi-key null-aware join silently defaults to one NOT IN value key #26569
Sort order and equivalence / ordering properties
- Sort options for DataType::Dictionary is incorrect #4649
- Incorrect join ordering properties can cause required sorts to be removed #25052
- Incorrect ORDER BY results with mismatched Parquet timestamp units #25817
- Wrong ORDER BY results after an outer join when the nullable side equates a column to
coalesce/CASE#26036 - Incorrect results: ORDER BY b, a over a subquery with ORDER BY b LIMIT n is not sorted by (b, a) when SortPreservingMergeExec has a fetch #26162
- Incorrect ORDER BY results after sort pushdown through custom execution plans #26656
TopK, LIMIT and dynamic filters
- TopK dynamic filter returns wrong rows when one sort key contains another (DynamicFilterPhysicalExpr remaps the inner column first) #26029
- Recursive CTE: MIN/MAX aggregate dynamic filter survives reset_state, later iterations return wrong results #26054
- enable_window_topn returns wrong results when a projection under the filter drops a column (wrong ranking function chosen) #26416
- Wrong results: UNION ALL ... ORDER BY ... LIMIT when a sort key is constant per arm (partitioned TopK shares a threshold over a shortened key) #26454
- Wrong result:
count(*) OVER ()sees only the LIMIT rows when ORDER BY ... LIMIT sits on top of another window (target_partitions > 1) #26617 - Wrong results:
ORDER BY <double>, ... LIMIT ndrops-0.0rows when 0.0 arrives in an earlier batch (TopK dynamic filter) #26618
Logical optimizer rewrites (filter pushdown, unwrap cast, volatile expressions)
- Incorrect result when use volatile function alias in having clause #7976
- Unsafe / incorrect
unwrap_castoptimizations for narrowing casts (e.g. Decimal) #23095 - Projection schema can go stale after SimplifyExpressions; OptimizeProjections silently normalizes it #24284
- Narrowing
try_castis incorrectly unwrapped from comparisons, dropping NULL-on-overflow semantics #24702 - BETWEEN evaluates its operand two times, so a volatile operand returns wrong rows #25457
- Wrong results: COALESCE on a volatile operand evaluates it two times, and fails when the output is declared non-nullable #25477
- Wrong results: eliminate_cross_join drops an equi-join key that spans two cross-joined relations #26058
- Wrong results: with
pushdown_filters = true, a filter on a volatile subquery column is evaluated again in the Parquet scan #26267 - Wrong results: a filter on a constant column keeps the row of an aggregate without GROUP BY #26545
-
push_down_filtersubstitutes the wrong expression whencolandCAST(col AS ...)share a schema name (HAVING and correlated IN give type errors) #26619
Aggregation, GROUP BY, constraints and set operations
- Group by duplicate GROUPING SETS returns wrong result #5672
- Some aggregates silently ignore
IGNORE NULLSandORDER BYon arguments #9924 - INTERSECT ALL returns wrong number of records from RHS #12955
- EXCEPT ALL returns the wrong number of records #12956
- Incorrect results of aggregation with grouping sets with single target partition #16965
- Incorrect query results for GROUP BY with UNIQUE constraint #21507
- Incorrect results with unique constraints #23634
- Incorrect results: GROUP BY column dropped based on a nullable UNIQUE constraint #23819
- Incorrect results: GROUP BY returns two rows for the NULL group with a nullable UNIQUE constraint #23820
- Wrong results: EmitTo::First(n) collision-list compaction in GroupValuesColumn can overwrite a not-yet-visited entry's list #25233
-
nth_valueandapprox_percentile_contsilently ignoreDISTINCT#26497 - sum() over DECIMAL silently wraps on overflow instead of erroring #26648
Joins
- NestedLoopJoinExec memory-limited fallback returns an empty result (wrong count = 0) instead of spilling or erroring #25701
- Sort-merge join ignores a join filter that references no columns (wrong FULL JOIN results) #26002
Parquet pruning, statistics and file reading
- Incorrect Schema Adaption for CSV #4918
- Parquet Statistics Pruning Ignores ColumnOrder, resulting in potentially incorrect statistics #8342
- Incorrect results: Bloom filters on
UInt8,Int8,UInt16andInt16columns always return false negatives #9779 - Incorrect statistics read for struct array in parquet #10609
- Pruning of floating point Parquet columns is incorrect when
NaNis present #15812 - Wrong results: filter marked as pushed down at plan time can be silently dropped per-file when schema evolution makes the runtime pushdown checker reject the adapted predicate #25268
- Incorrect Parquet pruning for timestamp-to-time casts across midnight #26451
- Plain
SELECTof a Parquet float column returns wrong values: NaN rows are replaced by the statistics constant when min == max #26490 - Parquet bloom filter pruning drops rows holding the other signed zero:
WHERE c = 0.0misses-0.0rows (and vice versa) #26491
Type coercion, casts and scalar function semantics
- [dbolympics] Comparing a
Durationto anIntervalwith a days field returns wrong results (e.g.ts - ts <= INTERVAL '3 days') #26671 - Inconsistent and incorrect
structfield coercion #14396 -
date_partreturning wrong results due to overflows #14738 - Incorrect type coercion for IS SIMILAR TO #20342
- concat_ws on array arguments silently stringifies the arrays #21703
-
CASEexpressions drop field metadata, silently stripping Arrow extension types and breaking metadata-aware UDFs #24860 - Recursive CTE silently casts recursive term to the static term's type, producing wrong results or a non-terminating query #25893
- Comparing a timestamp cast to a time zone against a literal returns wrong results #25941
- Spark
negativereturns wrong results for decimal columns #25942 - Struct union type coercion takes field nullability from the wrong field when field order differs #26296
-
EXTRACT EPOCHinconsistently extracts fractional seconds #26673 -
EXTRACT(NANOSECOND)onDurationandIntervalcounts whole seconds twice #26678 -
date_partdeclares a non-nullable result for non-null input, but can return NULL #26685
Writing files (round trip)
-
COPY … PARTITIONED BYwrites partition values containing=unescaped (p0=x=y); DuckDB and Polars cannot read the dataset or silently drop the partition column #26492 - CSV
ESCAPEoption is used when reading but ignored when writing:INSERT/COPYfiles cannot be read back with the same options (rows merged, backslashes dropped) #26514
SQL planning
Chore
Already Fixed in 56.0.0
- Scalar subquery returning no rows is incorrectly marked non-nullable #24513
- When performing set comparison subqueries (such as > ANY / > ALL / = ANY, etc.) on the same table (without using aliases), silently returns wrong results (0 rows or all rows). #25258
- Correlated
EXISTSsubquery withOFFSETreturns wrong results #25283 - Wrong results: correlated NOT IN with a non-equality correlation returns no rows (null-aware LeftAnti join ignores the residual filter) #25336
- Wrong results: constant NOT IN (subquery) in a WHERE clause ignores NULLs in the subquery #25340
- Wrong results: nullable constant
NOT IN (subquery)returns every row when the subquery column isNOT NULL#25473 - Wrong results:
NOT IN (subquery)ignores NULLs from a nullable outer expression overNOT NULLcolumns #25474 - Wrong results: a correlated filter under the nullable side of an outer join is pulled above the join #25507
- Wrong results: a correlated filter under an aggregate with a grouping set is pulled above the aggregate #25519
- Wrong results: a correlated filter under a window function is pulled above the window #25792
- Wrong results: a correlated filter under the right side of an ASOF JOIN is pulled above the join #25834
- Integer interval inference can incorrectly remove ORDER BY #25232
-
ORDER BY date_trunc(...)can return incorrectly ordered results for named timezones #25353 -
ORDER BY CAST(ts AS TIME)can return incorrectly ordered results across midnight #25506 - Parquet scan claims equivalences from a pruning-only predicate, giving wrong ORDER BY and LIMIT results #25779
- Wrong results:
SortPreservingMergeExeckeeps orderings of its input that do not hold after the merge #26646 -
LimitPushdowncan return incorrect rows for fetchedSortPreservingMergeExecover ordered Parquet scans #24272 - Wrong results: TopK dynamic filter + pushdown_filters re-reads already-delivered row groups (duplicate rows displace the true top-k) #24352
- EnsureRequirements can silently drop fetch during distribution reoptimization #24807
-
enable_join_dynamic_filter_pushdownsilently drops rows: the pushed filter is remapped by column NAME and lands on the wrong same-named column #25244 - Join dynamic filter pushdown through
ProjectionExeccan map duplicate output aliases to the wrong expression #25263 - Filter pushdown picks the wrong column when a join output has duplicated column names, causing wrong results with TopK dynamic filters #25296
- ORDER BY ... LIMIT returns wrong rows for List/Struct sort keys with NULL elements #25998
-
CoalescePartitionsExec::cardinality_effectreturnsEqualwhenfetchis set #26050 -
LEAD/LAG IGNORE NULLSreturns incorrect results acrossNULLgaps and withLIMIT#25471 - Wrong results: window aggregates over
RANGE ... N PRECEDINGdepend on batch boundaries when the ORDER BY value is NULL #25859 - Window referencing a named window with an added frame, OVER (w ROWS ...), silently drops PARTITION BY and ORDER BY #26364
- FilterPushdown physical optimizer incorrectly remaps filters through ProjectionExec with duplicate column names #21246
- Expression simplifications may produce incorrect results with NULLs #24246
- Incorrect results: leaf expression pushdown duplicates volatile expressions (random() evaluated twice) #24678
- unions_to_filter drops computed projections and returns wrong results #25071
- power and log simplifications return wrong results for base 1, 0 or negative #25137
- Wrong results: leaf expression pushdown removes a computed column that has the same name as its input column #25414
- Wrong results from
INwhen a list item is aCASEthat reads a column #26082 - Handling of Aggregates with multiple groupings sets is incorrect #17910
- Incorrect results: ORDER BY drops a sort key based on a nullable UNIQUE constraint #23818
- Streaming aggregation produces wrong result with grouping sets #24421
- percentile_cont is wrong for
Decimal32with lower interpolation precision #24576 - A fixed GROUP BY column is incorrectly used to decide when groups are complete #25018
- COUNT(... ORDER BY ...) can panic or undercount with nullable ordering keys #25055
- Wrong results: a list
unnestkeeps its input's GROUP BY key, so a later DISTINCT or GROUP BY drops rows #25844 -
corr,covar_pop,covar_sampandregr_*silently ignoreDISTINCT#26471 - Mark joins don't support null mark columns #21309
-
PiecewiseMergeJoinchanges state before draining its output coalescer: the Left/Full unmatched pass repeats forever, queued Right/Full rows can be dropped #24297 - PiecewiseMergeJoin drops rows when a stream batch spans multiple output chunks (small batch_size) #24348
- Memory-limited NestedLoopJoin silently drops unmatched rows when it spills #24745
- PiecewiseMergeJoin returns wrong results for List keys with NULL elements and fails on Struct keys #25957
- Incorrect filtering results when
collect_statistics = trueand column statistics are partially disabled #18241 - incorrect results when using NOT physical expression in
physical_expr::analyze#19264 - Parquet filter pushdown silently drops a
get_fieldpredicate when the file needs schema adaptation (wrong results) #24109 - Wrong results: dynamic row-group pruning drops row groups without slicing the carried RowSelection #24355
- Parquet scan silently drops one of two columns with the same name #24381
- CAST statistics can produce incorrect MIN/MAX results #25221
- Missing Parquet row-group statistics can produce incorrect file MIN/MAX #25226
- Parquet runtime pruning can drop NULLS FIRST TopK rows when null_count is missing #25239
- Incorrect filter statistics for overflowing integer multiplication #25794
- make_array returns wrong row count for null-array input #21841
- Projection field metadata is lost during physical plan optimization #24721
- Spark
map_from_arrays/map_from_entriesreturn wrong results for sliced lists #25419 - date_bin can return incorrectly ordered NULLs after timestamp overflow #25690
-
WITH ORDINALITYis silently dropped on table functions #24388
Activity
comphead commentedon Oct 24, 2026
Adding #26700 to test operators SLT with different batch sizes
kumarUjjawal commentedon Oct 24, 2026
This hasn't been resolved for a long time. I will take a look at this, I see @alamb you have reviewed the last PR which got closed due to inactivity, will see if I can borrow some of the ideas from there.
kumarUjjawal commentedon Oct 24, 2026
Seems this is already fixed on main
alamb commentedon Oct 24, 2026
Thanks @kumarUjjawal 🙏 let's just close it out
AdamGS commentedon Oct 25, 2026
This is a great list and seems fun, I'll try and pick some up.
alamb commentedon Oct 25, 2026
Thanks @AdamGS -- many of them already have PRs even -- it just needs someone to look at and help them along
kumarUjjawal commentedon Oct 26, 2026
#22906 fixes the decimal scale-narrowing bug. The logical and physical simplifiers now use shared cast-family checks, and this narrowing cast stays in the predicate instead of becoming d = 1.2300.
Verified on main at 350ca76: