Skip to content

core: ExtensionTable drops its filter and best_effort_filter in both proto directions #1382

Description

@nielspardon

RelProtoConverter.visit(ExtensionTable) (core/src/main/java/io/substrait/relation/RelProtoConverter.java:424) never calls setFilter or setBestEffortFilter, and ProtoRelConverter.newExtensionTable (core/src/main/java/io/substrait/relation/ProtoRelConverter.java:668) never reads them. The other three read types (NamedScan, LocalFiles, VirtualTableScan) carry both filters in both directions, and the POJO models them on AbstractReadRel.

A read whose mandatory filter must remove every row therefore comes back with no filter, and a consumer returns every row with no error:

ExtensionTable rel =
    ExtensionTable.from(detail)
        .initialSchema(NamedStruct.of(List.of("id"), R.struct(R.I32)))
        .filter(sb.bool(false))
        .bestEffortFilter(sb.bool(false))
        .build();
relProtoConverter.toProto(rel).getRead().hasFilter();          // false
relProtoConverter.toProto(rel).getRead().hasBestEffortFilter(); // false

Decoding a ReadRel{extension_table, filter, best_effort_filter} gives getFilter() == Optional.empty() and getBestEffortFilter() == Optional.empty().

The RelSamples ExtensionTable sample carries no filter, so no round-trip test catches it. #1278 is the copy-on-write side of the same field.

Measured on main at ca8315b. Found while reviewing #1260.

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Type

    No type

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions