Skip to content

refactor: retain hash join build batches after allocation failure - #26182

Open
peterxcli wants to merge 1 commit into
apache:mainfrom
peterxcli:refactor/hash-join-retain-build-batches
Open

peterxcli wants to merge 1 commit into
apache:mainfrom
peterxcli:refactor/hash-join-retain-build-batches

Conversation

@peterxcli

Copy link
Copy Markdown
Member

Which issue does this PR close?

Part of #24768. Splits the build refactor out of #25217 and #26015 for easier review.

Rationale for this change

The sort-merge fallback needs the build rows after a hash table allocation fails.

What changes are included in this PR?

Separate hash-table construction from input collection and retain the batches when building or concatenating them fails. Also adds Clippy allowances for Expr hash-set keys.

What is the testing strategy for this PR?

Tests cover retained batches, memory accounting, and existing memory-limit behavior for ordered and unbounded inputs.

Are there any user-facing changes?

None. The fallback comes in a follow-up PR.

Include the existing Expr-key Clippy allowances required by workspace lint checks.

Split from apache#25217 and apache#26015.

Co-authored-by: Jay Zhan <jayzhan211@gmail.com>
@github-actions github-actions Bot added optimizer Optimizer rules physical-plan Changes to the physical-plan crate labels Oct 10, 2026
@codecov-commenter

Copy link
Copy Markdown

Codecov Report

❌ Patch coverage is 87.15084% with 23 lines in your changes missing coverage. Please review.
✅ Project coverage is 82.78%. Comparing base (4e6e564) to head (18ab171).

Files with missing lines Patch % Lines
...tafusion/physical-plan/src/joins/hash_join/exec.rs 87.15% 1 Missing and 22 partials ⚠️
Additional details and impacted files
@@           Coverage Diff            @@
##             main   #26182    +/-   ##
========================================
  Coverage   82.77%   82.78%            
========================================
  Files        1148     1148            
  Lines      451075   451220   +145     
  Branches   451075   451220   +145     
========================================
+ Hits       373397   373520   +123     
- Misses      54930    54938     +8     
- Partials    22748    22762    +14     

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@peterxcli
peterxcli marked this pull request as ready for review October 11, 2026 10:41
@peterxcli

Copy link
Copy Markdown
Member Author

@jayzhan211 ptal, thanks!

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

optimizer Optimizer rules physical-plan Changes to the physical-plan crate

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants