What is the problem the feature request solves?
planningHelper in CometDeltaNativeScanExec.scala reuses core's listing, splitting and dynamic partition pruning, but its contract is implicit: partition filters are supplied separately while data filters come from originalPlan, and predicates used for file selection are not distinguished from predicates the reader evaluates.
Describe the potential solution
A contract that takes the original scan plus the current partition and data filters and returns the final FilePartition objects with their metadata, with the existing reader-free helper as the implementation and the serializer consuming only its result. Plan copies and filter rewrites keep the current protection against stale cached selection. Regression tests compare serialized paths, order, split starts and lengths, partition values and metadata against Spark's final partitions, including a non-DV file with a nonzero split start and dynamic partition pruning with AQE on and off.
Additional context
Raised in the #5365 review (item 4); filed as a follow-up.
What is the problem the feature request solves?
planningHelperinCometDeltaNativeScanExec.scalareuses core's listing, splitting and dynamic partition pruning, but its contract is implicit: partition filters are supplied separately while data filters come fromoriginalPlan, and predicates used for file selection are not distinguished from predicates the reader evaluates.Describe the potential solution
A contract that takes the original scan plus the current partition and data filters and returns the final
FilePartitionobjects with their metadata, with the existing reader-free helper as the implementation and the serializer consuming only its result. Plan copies and filter rewrites keep the current protection against stale cached selection. Regression tests compare serialized paths, order, split starts and lengths, partition values and metadata against Spark's final partitions, including a non-DV file with a nonzero split start and dynamic partition pruning with AQE on and off.Additional context
Raised in the #5365 review (item 4); filed as a follow-up.