Skip to content

Discard the rest of a multipart upload whose route's pipe loses its destination after the response #3096

Description

@dawsontoth

Problem

Since #3069, the operations API discards the rest of a multipart upload once its response has finished, unless the route still holds the file part. A route holds it by reading it: piping it, iterating it, or a data or readable listener (isAbandoned). So a route that starts payload.pipe(destination) and then answers is holding the part.

If that destination later fails, or the route calls payload.unpipe(), Node removes the pipe and leaves the part paused but not destroyed. Measured on Node 24.21, after a destination write error and after a manual unpipe():

readableFlowing data listeners readable listeners destroyed close fired
false 0 0 false no

The response hook has already run, and the part's close listener is the only later way back into the discard. So nothing discards the rest of the upload, and the one-second grace is never armed. busboy's backpressure holds the socket, and a client that keeps sending stalls until the request timeout: operationsApi.network.timeout, 120 s by default, and 0 disables it.

server/DESIGN.md documents this as a known limit of #3069. Nothing in core reads a payload after answering (deployComponent awaits ingestion first). Reaching it takes a user route on the legacy Fastify (custom functions) server, or a future operation, that answers while its pipe is still running. stream.pipeline() is not affected, because it destroys the part on error, and that already re-enters the discard.

Suggested fix

After the response, while a handed-out part is still held, recheck it whenever it loses a listener. Node's pipe cleanup removes the part's data listener on unpipe and on a destination error, so the part emits removeListener for data. Right after that, the existing isAbandoned predicate returns true. pause is not a usable signal: a part already paused by backpressure emits none.

  • In releaseUnreadUpload, when the part is still held, listen for its removeListener events and call discardUnreadUpload once isAbandoned holds. Stop listening on the part's end or close.
  • Test it in unitTests/server/serverHelpers/multipartParser.test.js. Pipe a part into a slow Writable, release it while it is held, then destroy the destination with an error: the raw request must reach its end. A part that is still piped must be left alone.

From the review comment on #3069.

Metadata

Metadata

Assignees

No one assigned

    Labels

    No labels
    No labels

    Fields

    Priority

    P3

    Projects

    No projects

      Milestone

      No milestone

      Relationships

      None yet

      Development

      No branches or pull requests

      Issue actions