Skip to content

messages: remove omitempty from IsTruncated in multipart upload results - #22

Merged
ncw merged 1 commit into
rclone:masterfrom
billythekidz:fix-multipart-istruncated-tag
Oct 14, 2026
Merged

ncw merged 1 commit into
rclone:masterfrom
billythekidz:fix-multipart-istruncated-tag

Conversation

@billythekidz

Copy link
Copy Markdown

Description

This PR removes the omitempty XML tag attribute from IsTruncated in ListMultipartUploadsResult and ListMultipartUploadPartsResult in messages.go.

Fixes #21 (following up on discussion in #21 (comment)).
Relates to upstream rclone issue: rclone/rclone#10078.


Root Cause

In messages.go, the multipart results structs previously defined IsTruncated as:

type ListMultipartUploadsResult struct {
    ...
    IsTruncated bool `xml:"IsTruncated,omitempty"`
}

type ListMultipartUploadPartsResult struct {
    ...
    IsTruncated bool `xml:"IsTruncated,omitempty"`
}

Because of omitempty, when the result was not truncated (IsTruncated == false), Go's encoding/xml encoder omitted the <IsTruncated> tag entirely from the XML response.

However, according to the Amazon S3 REST API specifications:

When strict S3 clients (such as the official AWS Go SDK v1 used in Docker Distribution / OCI Registry distribution/distribution v2.8.x and v3.x) unmarshal the response, IsTruncated is modeled as a pointer: *bool.

Because <IsTruncated> was absent, resp.IsTruncated was left unpopulated (nil).

When Docker Distribution attempts to list parts during multipart chunked upload (distribution/registry/storage/driver/s3-aws/s3.go:593):

for *resp.IsTruncated { // <--- DEREFERENCES NIL POINTER
    ...
}

This resulted in an immediate crash:

panic: runtime error: invalid memory address or nil pointer dereference
[signal SIGSEGV: segmentation violation code=0x1 addr=0x0 pc=0x... ]

causing registry instances backed by rclone serve s3 / gofakes3 to enter a restart crash loop whenever an image blob upload occurred.


Changes Made

  1. messages.go:

    • Changed xml:"IsTruncated,omitempty" to xml:"IsTruncated" in ListMultipartUploadsResult.
    • Changed xml:"IsTruncated,omitempty" to xml:"IsTruncated" in ListMultipartUploadPartsResult.
    • This matches the existing behavior in ListBucketResultBase and ListBucketVersionsResult, where IsTruncated is already defined without omitempty.
  2. messages_test.go:

    • Added unit tests TestListMultipartUploadPartsResultIsTruncated and TestListMultipartUploadsResultIsTruncated to verify that <IsTruncated>false</IsTruncated> is explicitly emitted in serialized XML.

Verification & Testing

  • Unit tests pass cleanly: go test -v ./....
  • End-to-end verified with Docker Distribution (distribution:2.8.3) connected to rclone serve s3:
    • Before fix: Every multi-chunk docker push immediately panicked with nil pointer dereference at s3.go:593.
    • After fix: Multi-chunk image pushes (hello-world, alpine:latest) succeed with HTTP 202 Accepted and 201 Created; images can be pulled and cataloged without issues.

@ncw

ncw commented Oct 13, 2026

Copy link
Copy Markdown
Member

Can you fix the format error then I'll merge - thank you :-)

AWS S3 specifications require <IsTruncated> to always be present in
ListMultipartUploadsResult and ListMultipartUploadPartsResult responses,
even when false.

Previously, xml:"IsTruncated,omitempty" caused Go's xml encoder to omit
the tag when IsTruncated was false. This caused strict S3 clients (such as
the AWS Go SDK used in Docker Distribution registry v2/v3) to unmarshal
the field as a nil *bool, leading to a nil pointer dereference panic
when checking *resp.IsTruncated.

Matches existing behavior in ListBucketResultBase and ListBucketVersionsResult.

Fixes rclone#21
Ref rclone/rclone#10078
@billythekidz
billythekidz force-pushed the fix-multipart-istruncated-tag branch from 9851a3d to 46b6bfd Compare October 13, 2026 11:59
@billythekidz

Copy link
Copy Markdown
Author

@ncw I have fixed the import ordering in messages_test.go using gofmt and force-pushed the updated commit (46b6bfd). Ready for review and merge. Thank you!

@ncw ncw left a comment

Copy link
Copy Markdown
Member

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Perfect - thank you :-)

@ncw
ncw merged commit a47d252 into rclone:master Oct 14, 2026
7 checks passed
@ncw

ncw commented Oct 14, 2026

Copy link
Copy Markdown
Member

@billythekidz I tagged v0.0.11 if you want to make a PR pulling this back into rclone to fix rclone/rclone#10078.

Thank you

@billythekidz

Copy link
Copy Markdown
Author

@ncw PR submitted to rclone/rclone: rclone/rclone#10089. Thank you for tagging the release!

ncw pushed a commit to rclone/rclone that referenced this pull request Oct 15, 2026
…istribution Registry

gofakes3 prior to v0.0.11 defined the IsTruncated XML field in
ListMultipartUploadsResult and ListMultipartUploadPartsResult with
`xml:"IsTruncated,omitempty"`. When an upload result was not truncated
(IsTruncated == false), the <IsTruncated> tag was omitted from the XML
response.

According to the AWS S3 REST API specifications for ListParts and
ListMultipartUploads, IsTruncated is a required boolean element. Strict
S3 clients such as the official AWS Go SDK (used by Docker Distribution
and OCI registries) unmarshal this field as a *bool pointer. Because the
tag was omitted, the pointer remained nil, causing Docker Distribution
to panic with a nil-pointer dereference at s3.go:593 (for *resp.IsTruncated)
during multi-chunk blob uploads.

Updating github.com/rclone/gofakes3 to v0.0.11 brings in rclone/gofakes3#22,
which removes omitempty and ensures <IsTruncated>false</IsTruncated> is
always present in the response XML.

Fixes #10078
hirthwork pushed a commit to hirthwork/rclone that referenced this pull request Oct 16, 2026
…istribution Registry

gofakes3 prior to v0.0.11 defined the IsTruncated XML field in
ListMultipartUploadsResult and ListMultipartUploadPartsResult with
`xml:"IsTruncated,omitempty"`. When an upload result was not truncated
(IsTruncated == false), the <IsTruncated> tag was omitted from the XML
response.

According to the AWS S3 REST API specifications for ListParts and
ListMultipartUploads, IsTruncated is a required boolean element. Strict
S3 clients such as the official AWS Go SDK (used by Docker Distribution
and OCI registries) unmarshal this field as a *bool pointer. Because the
tag was omitted, the pointer remained nil, causing Docker Distribution
to panic with a nil-pointer dereference at s3.go:593 (for *resp.IsTruncated)
during multi-chunk blob uploads.

Updating github.com/rclone/gofakes3 to v0.0.11 brings in rclone/gofakes3#22,
which removes omitempty and ensures <IsTruncated>false</IsTruncated> is
always present in the response XML.

Fixes rclone#10078
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

messages.go: remove omitempty from IsTruncated in ListMultipartUploadPartsResult

2 participants