FIX: Drain requests on shutdown - #2965
Open
Roman Lutz (romanlutz) wants to merge 6 commits into
Open
Roman Lutz (romanlutz) wants to merge 6 commits into
Roman Lutz (romanlutz) wants to merge 6 commits into
Conversation
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 21bf73f6-4b26-4f5f-bb22-01fa72e49186
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 21bf73f6-4b26-4f5f-bb22-01fa72e49186
Reverse 55f0ad3 here so this branch contains only the runtime shutdown correction. Preserve the original commit in history; the notebook fix is being moved to the MCP notebook lifecycle session. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com>
Preserve upstream memory disposal after retained request drainage. Complete each cleanup phase and retain all failures without restoring the separately published MCP notebook change. Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 21bf73f6-4b26-4f5f-bb22-01fa72e49186
Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 21bf73f6-4b26-4f5f-bb22-01fa72e49186
Merge the exact main base tested by CI. Re-raise a sole request or cleanup error unchanged, retaining grouped reporting for multiple failures and completing all cleanup stages. Verify error identity and idempotent repeated shutdown without weakening the upstream lifespan regression. Co-authored-by: Copilot <223556219+Copilot@users.noreply.github.com> Co-authored-by: Copilot App <223556219+Copilot@users.noreply.github.com> Copilot-Session: 21bf73f6-4b26-4f5f-bb22-01fa72e49186
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
A disconnected HTTP request can leave its middleware-owned child task running after the caller is cancelled. Runtime shutdown previously closed resources without waiting for these retained runtime and management requests. Normal server graceful shutdown can mask this gap, so the regression exercises the real middleware cancellation path directly.
RuntimeLifecyclenow closes admission immediately and drains retained requests and any accepted configuration apply before shutting down the scheduler, services, and memory. Shutdown uses one shared, shielded task so repeated caller cancellation cannot interrupt cleanup. Each cleanup phase still runs if an earlier phase fails, and request/cleanup failures are preserved together in an exception group.This stays within runtime task-lifetime ownership; it does not change
MessageSendServicetask ownership. The MCP notebook changes were split into a separate branch and are absent from this PR's net diff.Tests and Documentation
Added regressions for disconnected runtime and management requests, shutdown admission, repeated cancellation, child failures and sibling drainage, accepted apply completion, and resource-close ordering. Extended upstream memory-disposal tests to cover failures/cancellation in individual and multiple cleanup phases. Updated the GUI documentation with shutdown behavior.
Validation on the final merged code:
uv run --frozen --no-sync pytest -n 4 --dist=loadfile -q tests\unit\backend tests\unit\setup: 2,262 passed, 5 skipped (215 warnings).uv run --frozen --no-sync pre-commit run --files pyrit\backend\services\runtime_lifecycle.py pyrit\backend\middleware\runtime.py tests\unit\backend\test_runtime_lifecycle.py doc\gui\0_gui.md: all applicable hooks passed, including Ruff and ty; hooks without matching files were skipped normally.The full repository suite, live integration tests, and E2E tests were not run for this update.
JupyText was not run: no paired notebook or executable documentation changes remain in this changeset.