Skip to content

fix(chat): close the source stream when an adapter stops reading early - #1172

Open
abhiramshaji-cyber wants to merge 1 commit into
vercel:mainfrom
abhiramshaji-cyber:fix/stream-return-1170
Open

abhiramshaji-cyber wants to merge 1 commit into
vercel:mainfrom
abhiramshaji-cyber:fix/stream-return-1170

Conversation

@abhiramshaji-cyber

Copy link
Copy Markdown
Contributor

Summary

When an adapter stops reading a streamed reply early, the model stream behind thread.post() now gets closed instead of running on with nobody reading it.

Fixes #1170. Hi @r13i, hope you're doing well! Thank you so much for this report. The repro was spot on and made it easy to confirm, and your suggested return() fix pointed straight at the problem. I went with a slightly different shape (below), so please tell me if you'd rather see your version.

What this code is for

handleStream() in packages/chat/src/thread.ts runs when someone passes an async iterable to thread.post(), for example an AI SDK textStream. It feeds the stream through fromFullStream, withoutReasoning and takeUntilAborted, then hands it to the adapter's stream(), or to fallbackStream() (post, then edit) when the adapter has none.

Before this PR

An SMS adapter's stream() reads the reply and throws "too long" once it passes the length limit.

  1. The app calls thread.post(model()).
  2. handleStream() passes the adapter wrappedStream, a plain object whose iterator has next() and nothing else.
  3. The adapter's for await throws, so JavaScript looks for return() on that iterator to close it.
  4. There is no return(), so nothing is passed on to takeUntilAborted.
  5. takeUntilAborted stays paused at its yield, and its finally, the code that calls iterator.return() on the source, never runs.
  6. model() stays open, and in the repro from the issue cleanedUp stays false. textOnlyStream has the same gap, so a failed postMessage inside fallbackStream() leaves the source open too.

After this PR

Same adapter, same thread.post(model()).

  1. handleStream() passes the adapter tapText(textStream, ...), an async generator.
  2. The adapter's for await throws, and JavaScript calls return() on the tapText generator.
  3. That ends the for await inside tapText, which calls return() on takeUntilAborted.
  4. takeUntilAborted runs its finally and calls iterator.return() on model(). cleanedUp is true.
  5. When the adapter reads to the end, tapText passes every chunk through and fills accumulated the same way the old wrapper did.

The diagram

flowchart LR

classDef box fill:#eef1ff,stroke:#4c5bd4,color:#111827
classDef bad fill:#ffd6d6,stroke:#cc0000,color:#111827,stroke-width:3px
classDef good fill:#c8f0c8,stroke:#008800,color:#111827,stroke-width:3px

subgraph BEFORE["BEFORE: cleanedUp stays false"]
direction TB
  B1["adapter stream() leaves for await early"] --> B2{"wrappedStream has only next()"}
  B2 -->|"return() not found"| B3["takeUntilAborted stays paused at yield"]
  B3 --> B4["model() stays open"]
end

subgraph AFTER["AFTER: cleanedUp is true"]
direction TB
  A1["adapter stream() leaves for await early"] --> A2{"THE CHANGE: tapText async generator"}
  A2 -->|"return() passed on"| A3["takeUntilAborted finally runs"]
  A3 --> A4["model() gets return()"]
  A2 -->|"adapter reads to the end"| A5["accumulated text posted as before"]
end

B1 ~~~ A1

class B1 box
class A1,A5 box
class B2,B3,B4 bad
class A2,A3,A4 good

linkStyle 1 stroke:#cc0000,stroke-width:5px
linkStyle 4 stroke:#008800,stroke-width:5px

style BEFORE fill:#fff5f5,stroke:#cc0000,color:#111827
style AFTER fill:#f3fff3,stroke:#008800,color:#111827
Loading

handleStream and the functions it calls, takeUntilAborted among them
thread.post to handleStream and everything it calls, Functions view in Architect

The change

Both hand built iterators are replaced by small async generators next to withoutReasoning: tapText() for the adapter path and textOnly() for the fallback, sharing a chunkText() that returns the text of a string or markdown_text chunk. An async generator's for await calls return() on its source by itself when the loop exits early, the same way withoutReasoning already does, so the finally in takeUntilAborted now runs on every early exit. That is the whole fix; nothing else in handleStream() moved.

What changes for a user

  • A break, a thrown error, or a failed fallback post now closes the source stream, so the model request stops.
  • Normal streams are unchanged: same chunks, same accumulated text, same skipping of task_update, plan_update and reasoning in the fallback.
  • Abort still works through takeUntilAborted, and a source whose iterator has no return() still stops cleanly.
  • If an adapter reads part of the stream and then returns null, the fallback now gets a closed stream instead of the rest. No adapter in this repo does that (Slack and Teams return null before reading), and the chunks it read were already lost before this change.

One question, if you have a moment: an adapter that throws before reading a single chunk still leaves the source open, because takeUntilAborted never started and has no finally to run yet. Would you like handleStream() to close the raw stream in that case too, or is reading at least once the case you hit in practice? Any feedback or tips are very welcome!

Test plan

New source stream cleanup tests in packages/chat/src/thread.test.ts cover a throw mid loop, an early break (checking the posted text too), a failed fallback post, an abort, and a source iterator with no return(). The first three fail on main and pass here.

Checklist

  • All commits are signed and verified
  • All commits are signed off for the DCO (git commit -s)
  • pnpm validate passes
  • Changeset added (or N/A, see CONTRIBUTING.md)
  • Documentation updated (or N/A)

The wrappers that handleStream passes to adapter.stream() and to the post and edit fallback were hand built iterators with next() but no return(). Leaving the loop early never reached the finally in takeUntilAborted, so the source generator stayed open. Both wrappers are now async generators, which forward return() and throw() to the source.

Fixes vercel#1170

Signed-off-by: Abhiram Shaji <write4abhiram@gmail.com>
@abhiramshaji-cyber
abhiramshaji-cyber requested a review from a team November 16, 2026 19:41
@vercel

vercel Bot commented Nov 16, 2026

Copy link
Copy Markdown
Contributor

@abhiramshaji-cyber is attempting to deploy a commit to the Vercel Team on Vercel.

A member of the Team first needs to authorize it.

This branch has not been deployed

No deployments
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

Adapter stream(): the wrapped stream has no return(), so the source stream is never closed

1 participant