fix(core): stop a surviving grandchild from hanging code execution - #793
Merged
Conversation
`UnsafeLocalCodeExecutor` relied on `spawn`'s own `timeout` option, which kills the interpreter and then leaves the promise waiting on 'close'. 'close' only fires once every stdio stream is closed, so an interpreter that forked rather than exec'd its work leaves a survivor holding those pipes and 'close' never arrives. The promise never settles, the executor's own timeout branch never runs, and the caller waits forever. No timeout value bounds that wait, which is why raising the harness budget twice (#633, #662) did not stop the Windows flake in #622: the wait is unbounded at 5s, at 30s, and at any other number. Windows is where it shows because `powershell` and `python` there are far likelier to leave a live descendant than `bash`/`python3`. Run the timer here instead and release the read ends along with the kill, so the timeout is enforced rather than merely requested. This is the treatment `LocalEnvironment.execute` already applies for the same reason; the executor never got it. Also prefer an explicit `timedOut` flag over inferring the timeout from the close signal. Windows does not report a terminating signal the way POSIX does, so a killed child can close there with a `null` signal and silently skip the timeout message. The regression test reproduces the CI symptom exactly: on the unfixed executor it fails with `Test timed out in 30000ms`, the same string seen on windows-latest. With the fix it completes in ~0.5s. Bug: #622
9 tasks
AmaadMartin
approved these changes
Aug 22, 2026
AmaadMartin
left a comment
Collaborator
There was a problem hiding this comment.
Approve. The fix runs the timeout timer in the executor and destroys the read ends on the kill, so 'close' fires even when a forked grandchild holds the pipes. I verified this mirrors LocalEnvironment.execute (core/src/environment/local_environment.ts:136-148) field for field, and the new regression test reproduces the #622 symptom and would hang on the unfixed code. No suppressions, no instanceof, and no new public types; all run-tests jobs, including windows-latest, passed at review time.
Open
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Link to Issue or Description of Change
LocalEnvironment, still open)Problem:
UnsafeLocalCodeExecutorusedspawn's owntimeoutoption, which kills the interpreter and then leaves the promise waiting on'close'.'close'only fires once every stdio stream is closed, so an interpreter that forked rather than exec'd its work leaves a survivor holding those pipes —'close'never arrives, the promise never settles, and the executor's own timeout branch never runs.No timeout value bounds that wait. That is why raising the harness budget twice (#633 5s→…, #662 →30s) never stopped the flake: the wait is unbounded at 5s, at 30s, and at any other number. Windows is where it surfaces because
powershellandpythonthere are far likelier to leave a live descendant thanbash/python3.Reproduced locally with the executor's exact spawn shape:
Evidence from CI run 32415850940 (windows-latest, sha
540fa763) — all five failures are this executor, and three return empty output rather than slow output:Solution:
Run the timer here rather than delegating to
spawn, and release the read ends along with the kill so the timeout is enforced rather than merely requested.This is exactly the treatment
LocalEnvironment.executealready applies, for the same reason, with a comment describing the same failure (core/src/environment/local_environment.ts:139-148). The executor simply never got it.Also prefers an explicit
timedOutflag over inferring the timeout from the close signal — Windows does not report a terminating signal the way POSIX does, so a killed child can close there withsignal === nulland silently skip the timeout message.Alternative considered and rejected. Keeping
spawn'stimeoutand destroying the pipes on'exit'is a smaller diff and also works ({how:'close', ms:3006}), but'exit'fires on every exit including healthy ones, and Node does not guarantee all'data'events have been delivered by then. It survived 3×20 000 lines locally without truncating — but that is a timing assumption, which is precisely how you get a Windows-only flake. Destroying only on the timeout path is correct by construction.The stack
PR 2 is stacked on this branch and should merge after it. It is deliberately separate: this one is a behaviour fix worth reviewing on its own, and the revert is only safe once this lands.
Testing Plan
Unit Tests:
Added
times out even when the script leaves a child holding the pipes open, mirroring the existingLocalEnvironmentregression test. It reproduces the CI symptom exactly — on the unfixed executor it fails withTest timed out in 30000ms, the same string seen on windows-latest:npm run test:unit— 241 files, 3643 tests passed. Also clean:build,ts:check,lint,format:check.Manual End-to-End (E2E) Tests:
Not applicable — the behaviour change is entirely inside the spawn teardown and is covered by the regression test above.
Checklist
Additional context
This is the same defect class as #782, now confirmed in a second location. Worth considering a shared spawn helper that owns teardown once, rather than open-coding it a third time. #782 is still worth fixing, but it will not fix #622 — different file, no shared code, and its proposed
detached+ process-group remedy relies on POSIX process groups that Node does not implement on Windows.