Skip to content

[FEA] Bring filter_join_indices_jit UDF features to parity with transform #23567

Description

@lamarrr

Is your feature request related to a problem? Please describe.

filter_join_indices_jit and transform both evaluate a row-wise UDF over typed column inputs,
but the filtered-join path does not expose the same UDF capabilities as transform. In
particular, it lacks a consistent model for:

  • Reporting row-level UDF evaluation errors to the caller.
  • Accepting precompiled LTO IR UDFs.
  • Passing caller-provided device data to the UDF.

As a result, a predicate that works with transform may need to be rewritten or recompiled for a
filtered join. The two paths also risk developing incompatible UDF ABIs and error semantics.

Describe the solution you'd like

Use the same UDF representation and execution conventions in filter_join_indices_jit and
transform:

  • Accept the shared CUDA or LTO IR UDF abstraction, including explicit entry-point information.
  • Accept an optional pointer to caller-owned device data and pass it to the predicate using the same
    ABI convention as transform.
  • Support fallible predicates. Aggregate per-row failures and throw cudf::evaluation_error after
    execution, consistently with transform.
  • Support LTO IR predicates by linking the predicate fragment with the filtered-join kernel through
    RTCX/nvJitLink.
  • Keep null-aware and non-null-aware predicate signatures consistent with transform where their
    semantics overlap.
  • Share reflection, ABI-type mapping, dispatcher generation, and cache-key logic instead of
    maintaining filtered-join-specific variants.

Add coverage for CUDA and LTO IR predicates with user data enabled and disabled, successful and
fallible evaluation, null-aware and non-null-aware inputs, inner/left/full join semantics, and cold
and warm JIT caches.

Describe alternatives you've considered

  • Keep a separate filtered-join UDF contract. This limits functionality and requires callers to
    maintain different versions of equivalent row-wise operations.
  • Express every predicate as an AST. AST predicates are useful, but they cannot replace arbitrary
    precompiled or user-defined device functions.

Additional context
This proposal is implemented in a prototype #23565

Activity

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Metadata

Metadata

Assignees

No one assigned

    Labels

    feature requestNew feature or requestlibcudfAffects libcudf (C++/CUDA) code.

    Projects

    • Status
      No status

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions