Skip to content

Fix HTTP 416 making files an exact multiple of 10 MiB permanently unreadable - #489

Closed
giggio wants to merge 1 commit into
jstaf:masterfrom
giggio:fix-416-exact-chunk-multiple
Closed

giggio wants to merge 1 commit into
jstaf:masterfrom
giggio:fix-416-exact-chunk-multiple

Conversation

@giggio

@giggio giggio commented Aug 6, 2026

Copy link
Copy Markdown

Symptom

Some files can never be read through the mount. Every open() fails with EREMOTEIO ("Remote I/O error"), and the log shows the whole file downloading successfully and then one request too many:

INF Downloading bytes 136314880-146800639/146800640. name=VID_20231129_132554_10_036.insv
INF Downloading bytes 146800640-157286399/146800640. name=VID_20231129_132554_10_036.insv
ERR Failed to fetch remote content. error="HTTP 416 - : " op=Open

It looks intermittent, because it only happens when onedriver actually takes the download path — a cache miss, or Not using cached item due to file hash mismatch. Clearing the cache makes it worse, since a miss is exactly what forces the failing path.

Cause

GetItemContentStream in fs/graph/drive_item.go runs the multipart loop floor(size/chunk) + 1 times. That equals ceil(size/chunk) for every size except an exact multiple of the chunk size, where it is one trip too many. The final request then asks for bytes=<size>-<size+chunk-1> — a range starting at EOF — which the Graph API correctly rejects with 416 Range Not Satisfiable. The error is returned as fatal, so the item is unreadable for good.

So the condition is exactly: size % 10485760 == 0 and size > 10485760. Sizes at or below one chunk take the one-shot path and are unaffected.

Fix

One line — use ceil instead of floor + 1:

for i := 0; i < int((item.Size+downloadChunkSize-1)/downloadChunkSize); i++ {

Verification

In a tree of ~88,600 files, 32 had a size that was an exact multiple of 10485760. Of those, 24 were re-fetched during a verification pass and all 24 failed; the remaining 8 were served from cache and never hit the download path. Of the ~88,570 files whose size was not an exact multiple, none failed for this reason — including 179 files of the same type and similar sizes.

After the fix, all 24 read and hash correctly, and no other file changed behaviour.

The arithmetic on its own is decisive:

size trips before last range starts at   trips after
146800640 (14 chunks) 15 146800640 = EOF → 416 14
20800304 2 10485760 ok 2
15728640 2 10485760 ok 2
10485761 2 10485760 ok 2

Notes

  • Unrelated to onedriver crashes with message "slice bounds out of range" #255. That issue is about cache/database corruption; this is a plain off-by-one in the range loop, and the two only look connected because fighting the cache is the natural (and counterproductive) reaction to the symptom.
  • I did not add a test: the loop is inline in GetItemContentStream, which needs an authenticated Graph client, so covering it would mean extracting the trip count into its own function first. Happy to do that if you'd prefer it tested.

The multipart download loop runs floor(size/chunk)+1 times. For any size
that is not an exact multiple of the chunk size that equals
ceil(size/chunk), so the extra trip is harmless and the bug is invisible.
When the size *is* an exact multiple it is one trip too many, and that
trip requests "bytes=<size>-<size+chunk-1>" - a range starting at EOF,
which the Graph API answers with HTTP 416.

GetItemContentStream returns that as a fatal error, so the file can never
be read: every open() fails with EREMOTEIO for as long as the content is
not cached, and clearing the cache only forces the failing path again.

Observed on a 146800640 byte file, exactly 14 chunks: all 14 ranges
download, then a 15th is requested and fails. In a tree of ~88600 files,
every file whose size was an exact multiple of 10485760 was unreadable,
and no other file was affected.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant