Skip to content

Route Delta deletion-vector metadata preparation through the scan's instrumented reader #5948

Description

@dwsmith1983

What is the problem the feature request solves?

Deletion-vector preparation in native/core/src/execution/delta_dv.rs fetches the data file's footer through DFParquetMetadata directly. It fills the shared metadata cache correctly, but it runs before the scan creates its instrumented reader, so the preparation footer and page-index I/O bypass the final scan's scan_io_* counters while the later open reports a cache hit.

Describe the potential solution

A preparation hook in the shared scan builder, or a shared metadata helper with the same metrics owner, so preparation and normal reads share one place for metadata policy and instrumentation. The preparation accounting must land in the final scan's counters; a separate factory whose counters are discarded keeps the gap. Cold-cache and warm-cache tests verify that preparation I/O is counted, the footer is reused, and warm opens avoid storage reads, while keeping the INT96 stamping on cached metadata.

Additional context

Raised in the #5365 review (item 7); filed as a follow-up.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions