Repository navigation
messages: remove omitempty from IsTruncated in multipart upload results - #22
Merged
Merged
Conversation
This was referenced Oct 13, 2026
Member
|
Can you fix the format error then I'll merge - thank you :-) |
AWS S3 specifications require <IsTruncated> to always be present in ListMultipartUploadsResult and ListMultipartUploadPartsResult responses, even when false. Previously, xml:"IsTruncated,omitempty" caused Go's xml encoder to omit the tag when IsTruncated was false. This caused strict S3 clients (such as the AWS Go SDK used in Docker Distribution registry v2/v3) to unmarshal the field as a nil *bool, leading to a nil pointer dereference panic when checking *resp.IsTruncated. Matches existing behavior in ListBucketResultBase and ListBucketVersionsResult. Fixes rclone#21 Ref rclone/rclone#10078
billythekidz
force-pushed
the
fix-multipart-istruncated-tag
branch
from
October 13, 2026 11:59
9851a3d to
46b6bfd
Compare
Author
|
@ncw I have fixed the import ordering in |
Member
|
@billythekidz I tagged v0.0.11 if you want to make a PR pulling this back into rclone to fix rclone/rclone#10078. Thank you |
7 tasks done
Author
|
@ncw PR submitted to rclone/rclone: rclone/rclone#10089. Thank you for tagging the release! |
ncw
pushed a commit
to rclone/rclone
that referenced
this pull request
Oct 15, 2026
…istribution Registry gofakes3 prior to v0.0.11 defined the IsTruncated XML field in ListMultipartUploadsResult and ListMultipartUploadPartsResult with `xml:"IsTruncated,omitempty"`. When an upload result was not truncated (IsTruncated == false), the <IsTruncated> tag was omitted from the XML response. According to the AWS S3 REST API specifications for ListParts and ListMultipartUploads, IsTruncated is a required boolean element. Strict S3 clients such as the official AWS Go SDK (used by Docker Distribution and OCI registries) unmarshal this field as a *bool pointer. Because the tag was omitted, the pointer remained nil, causing Docker Distribution to panic with a nil-pointer dereference at s3.go:593 (for *resp.IsTruncated) during multi-chunk blob uploads. Updating github.com/rclone/gofakes3 to v0.0.11 brings in rclone/gofakes3#22, which removes omitempty and ensures <IsTruncated>false</IsTruncated> is always present in the response XML. Fixes #10078
hirthwork
pushed a commit
to hirthwork/rclone
that referenced
this pull request
Oct 16, 2026
…istribution Registry gofakes3 prior to v0.0.11 defined the IsTruncated XML field in ListMultipartUploadsResult and ListMultipartUploadPartsResult with `xml:"IsTruncated,omitempty"`. When an upload result was not truncated (IsTruncated == false), the <IsTruncated> tag was omitted from the XML response. According to the AWS S3 REST API specifications for ListParts and ListMultipartUploads, IsTruncated is a required boolean element. Strict S3 clients such as the official AWS Go SDK (used by Docker Distribution and OCI registries) unmarshal this field as a *bool pointer. Because the tag was omitted, the pointer remained nil, causing Docker Distribution to panic with a nil-pointer dereference at s3.go:593 (for *resp.IsTruncated) during multi-chunk blob uploads. Updating github.com/rclone/gofakes3 to v0.0.11 brings in rclone/gofakes3#22, which removes omitempty and ensures <IsTruncated>false</IsTruncated> is always present in the response XML. Fixes rclone#10078
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Description
This PR removes the
omitemptyXML tag attribute fromIsTruncatedinListMultipartUploadsResultandListMultipartUploadPartsResultinmessages.go.Fixes #21 (following up on discussion in #21 (comment)).
Relates to upstream rclone issue: rclone/rclone#10078.
Root Cause
In
messages.go, the multipart results structs previously definedIsTruncatedas:Because of
omitempty, when the result was not truncated (IsTruncated == false), Go'sencoding/xmlencoder omitted the<IsTruncated>tag entirely from the XML response.However, according to the Amazon S3 REST API specifications:
IsTruncatedas a required boolean element.IsTruncatedas a required boolean element.When strict S3 clients (such as the official AWS Go SDK v1 used in Docker Distribution / OCI Registry
distribution/distributionv2.8.x and v3.x) unmarshal the response,IsTruncatedis modeled as a pointer:*bool.Because
<IsTruncated>was absent,resp.IsTruncatedwas left unpopulated (nil).When Docker Distribution attempts to list parts during multipart chunked upload (distribution/registry/storage/driver/s3-aws/s3.go:593):
This resulted in an immediate crash:
causing registry instances backed by
rclone serve s3/gofakes3to enter a restart crash loop whenever an image blob upload occurred.Changes Made
messages.go:xml:"IsTruncated,omitempty"toxml:"IsTruncated"inListMultipartUploadsResult.xml:"IsTruncated,omitempty"toxml:"IsTruncated"inListMultipartUploadPartsResult.ListBucketResultBaseandListBucketVersionsResult, whereIsTruncatedis already defined withoutomitempty.messages_test.go:TestListMultipartUploadPartsResultIsTruncatedandTestListMultipartUploadsResultIsTruncatedto verify that<IsTruncated>false</IsTruncated>is explicitly emitted in serialized XML.Verification & Testing
go test -v ./....distribution:2.8.3) connected torclone serve s3:docker pushimmediately panicked with nil pointer dereference ats3.go:593.hello-world,alpine:latest) succeed with HTTP 202 Accepted and 201 Created; images can be pulled and cataloged without issues.