Skip to content

fix: release RemoteRun stream after peer close#25791

Merged
mergify[bot] merged 3 commits into
matrixorigin:mainfrom
aptend:fix/remoterun-stream-close-25760
Jul 17, 2026
Merged

fix: release RemoteRun stream after peer close#25791
mergify[bot] merged 3 commits into
matrixorigin:mainfrom
aptend:fix/remoterun-stream-close-25760

Conversation

@aptend

@aptend aptend commented Jul 16, 2026

Copy link
Copy Markdown
Contributor

What type of PR is this?

  • API-change
  • BUG
  • Improvement
  • Documentation
  • Feature
  • Test and CI
  • Code Refactoring

Which issue(s) this PR fixes:

Fixes #25760

What this PR does / why we need it:

  • Distinguish a closed receive path from completed local morpc Stream cleanup.
  • Poison backend reuse after a closed receive channel or nil message, while still invoking Stream.Close(true) to release local ownership.
  • Preserve the FIN/FIN_ACK Close(false) reuse path introduced by fix: reuse CN pipeline connections after clean teardown #25713.
  • Keep teardown exactly once through the existing closeOnce guard.
  • Cover closed channels, nil messages, repeated close calls, and peer close while waiting for FIN_ACK.

Testing

  • make thirdparties
  • .agents/skills/mo-dev/scripts/mo-cgo-test -v -count=1 -run ^(TestMessageSenderOnClientNegotiatedStreamTeardown|TestMessageSenderOnClientReceiveBatchReturnsStreamClosed)$ -timeout=120s ./pkg/sql/compile
  • .agents/skills/mo-dev/scripts/mo-cgo-test -race -count=10 -run ^(TestMessageSenderOnClientNegotiatedStreamTeardown|TestMessageSenderOnClientReceiveBatchReturnsStreamClosed)$ -timeout=240s ./pkg/sql/compile
  • .agents/skills/mo-dev/scripts/mo-cgo-test -count=1 -timeout=120s ./pkg/sql/compile
  • .agents/skills/mo-dev/scripts/mo-cgo-test -race -count=1 -timeout=240s ./pkg/sql/compile
  • go build ./pkg/sql/compile
  • go vet ./pkg/sql/compile
  • go test -count=1 -run ^(TestCloseStreamWithCloseConn|TestLockedStream)$ -timeout=120s ./pkg/common/morpc
  • .agents/skills/mo-dev/scripts/mo-cgo-test -count=1 -run ^TestTxnComputationWrapper_Run$ -timeout=120s ./pkg/frontend

@aptend
aptend requested review from XuPeng-SH and aunjgr as code owners July 16, 2026 04:11
@qodo-code-review

Copy link
Copy Markdown

Qodo reviews are paused for this user.

Troubleshooting steps vary by plan Learn more →

On a Teams plan?
Reviews resume once this user has a paid seat and their Git account is linked in Qodo.
Link Git account →

Using GitHub Enterprise Server, GitLab Self-Managed, or Bitbucket Data Center?
These require an Enterprise plan - Contact us
Contact us →

@XuPeng-SH XuPeng-SH left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Approved after a full RemoteRun/MORPC lifecycle review.

The change correctly separates receive-path closure from local Stream ownership release. A closed/nil receive now poisons backend reuse but still reaches exactly one sender-owned teardown through closeOnce and Stream.Close(true), which unregisters the stream and closes the unusable backend. The negotiated FIN/FIN_ACK success path remains unchanged and still uses Close(false) for reuse.

Unhappy-path closure checked:

  • Q1: acquired Stream ownership reaches Close on receive close/nil, FIN failure, cancellation, malformed response, and legacy teardown; sender close is idempotent.
  • Q2: peer-close and FIN_ACK waits terminate through channel closure/nil, context timeout, or backend shutdown; no new lock cycle was introduced across stream/backend cleanup.
  • Q3: the fix removes stale activeStreams/futures retention and adds no unbounded state.

Fresh validation on head 9026feaad3:

  • focused teardown/stream-closed tests: -race -count=30;
  • full pkg/sql/compile race suite;
  • full pkg/common/morpc race suite;
  • dependent TestTxnComputationWrapper_Run frontend regression;
  • go build and go vet for pkg/sql/compile.

@mergify mergify Bot added the queued label Jul 17, 2026
@mergify

mergify Bot commented Jul 17, 2026

Copy link
Copy Markdown
Contributor

Merge Queue Status

  • Entered queue2026-07-17 01:47 UTC · Rule: main · triggered by rule Automatic queue on approval for main
  • Checks passed · in-place
  • Merged2026-07-17 02:49 UTC · at 3145827c2966ec6264ed76cba1e2451d26d61eb3 · squash

This pull request spent 1 hour 2 minutes 21 seconds in the queue, including 1 hour 2 minutes 4 seconds running CI.

Required conditions to merge
  • #review-threads-unresolved = 0 [🛡 GitHub branch protection]
  • github-review-approved [🛡 GitHub branch protection]
  • github-review-decision = APPROVED [🛡 GitHub branch protection]
  • any of [🛡 GitHub branch protection]:
    • check-success = Matrixone Compose CI / multi cn e2e bvt test docker compose(PESSIMISTIC)
    • check-neutral = Matrixone Compose CI / multi cn e2e bvt test docker compose(PESSIMISTIC)
    • check-skipped = Matrixone Compose CI / multi cn e2e bvt test docker compose(PESSIMISTIC)
  • any of [🛡 GitHub branch protection]:
    • check-success = Matrixone Standlone CI / Multi-CN e2e BVT Test on Linux/x64(LAUNCH, PROXY)
    • check-neutral = Matrixone Standlone CI / Multi-CN e2e BVT Test on Linux/x64(LAUNCH, PROXY)
    • check-skipped = Matrixone Standlone CI / Multi-CN e2e BVT Test on Linux/x64(LAUNCH, PROXY)
  • any of [🛡 GitHub branch protection]:
    • check-success = Matrixone Standlone CI / e2e BVT Test on Linux/x64(LAUNCH, PESSIMISTIC)
    • check-neutral = Matrixone Standlone CI / e2e BVT Test on Linux/x64(LAUNCH, PESSIMISTIC)
    • check-skipped = Matrixone Standlone CI / e2e BVT Test on Linux/x64(LAUNCH, PESSIMISTIC)
  • any of [🛡 GitHub branch protection]:
    • check-success = Matrixone CI / UT Test on Ubuntu/x86
    • check-neutral = Matrixone CI / UT Test on Ubuntu/x86
    • check-skipped = Matrixone CI / UT Test on Ubuntu/x86
  • any of [🛡 GitHub branch protection]:
    • check-success = Matrixone Utils CI / Coverage
    • check-neutral = Matrixone Utils CI / Coverage
    • check-skipped = Matrixone Utils CI / Coverage
  • any of [🛡 GitHub branch protection]:
    • check-success = Matrixone CI / SCA Test on Linux/arm64
    • check-neutral = Matrixone CI / SCA Test on Linux/arm64
    • check-skipped = Matrixone CI / SCA Test on Linux/arm64

@mergify
mergify Bot merged commit d8ca645 into matrixorigin:main Jul 17, 2026
23 of 24 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

kind/bug Something isn't working size/M Denotes a PR that changes [100,499] lines

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Bug]: RemoteRun skips morpc Stream.Close after receive channel closes

3 participants