500 GB to R2: what happens when a huge upload stops, and when it starts again
- Huge files: uploaded as S3 multipart, streamed, never loaded into memory. For 500 GB that is 9,847 parts of 52 MiB, five in flight at a time. Tested end to end against Cloudflare R2.
- Interrupted: nothing is lost. The parts already on the server stay there, and a journal on disk records each one. Resume — even after quitting the app — verifies it is the same file, adopts those parts without re-reading them, and uploads only what is missing.
- Orphaned parts: incomplete uploads are shown in the bucket and you can delete them; they are billed until you do. RocketBucket never aborts one automatically, because it may belong to another client.
- Memory: about 900 MB to 1 GB at peak for the entire 500 GB run, including the resume. When every part is already on the server, the resume peaks at 49.1 MB — it reads nothing.
We uploaded a 500 GB file to Cloudflare R2 with RocketBucket, stopped it partway, and started it again. It finished — 536,870,912,000 bytes, 9,847 parts. We watched memory for the whole run, from the first part to the assembled object, and it never went above about 900 MB to 1 GB. This is what happens underneath, and what we had to fix before that sentence was true.
What 500 GB looks like to S3
S3, and everything that speaks its API, will not take a big file in one request. It takes parts: at most 10,000 of them, each between 5 MiB and 5 GiB, and then one final call that glues them into an object. RocketBucket's default part is 8 MiB, but 500 GB divided by 8 MiB is 64,000 parts, so before the upload starts it does the arithmetic the other way: 500 GB ÷ 10,000, rounded up to a whole mebibyte, is 52 MiB. That decision is permanent. A part size cannot change once the first part is on the server, and a resume must cut the file in exactly the same places or nothing lines up.
Five parts are in flight at once. The file is streamed, never loaded, so the memory a transfer needs is roughly five parts plus one buffer — about 300 MB — and the network library's own copies on top of that. The 900 MB to 1 GB we measured is that envelope.
Two things are written as the upload runs, and both exist only for the moment it stops. The object is stamped at creation with a random commit token in its metadata. And a small journal on disk gets one line per part — its number, its size, and our own SHA-256 of its bytes — written only after the server has accepted that part. We hash it ourselves instead of trusting the server's ETag because under SSE-KMS the ETag is not the part's MD5, and there would be nothing to compare against later.
What happens when it stops
An upload stops for two reasons: the connection drops, or you press Stop. The second one has a confession attached. Until this week, pressing Stop during a single large file did nothing. The cancel flag was checked between files, and a 9,847-part file is one file. It is fixed — cancellation now reaches into the part uploads and takes effect within the parts already in flight — and it was found the way most of this was found, by running a real 10 GB resume and pressing the button.
Either way, the same rule applies: keep everything. The journal is flushed to disk, and the multipart upload on the server is not aborted. The 6,000 parts that made it stay there, under their upload ID, waiting. They are billed while they wait — every S3-compatible store charges for parts of an unfinished upload, and they are invisible in a normal listing — so RocketBucket shows the bucket's incomplete uploads and lets you delete them. But it never aborts one on its own. It may not be the one that created it; another client may own it.
What happens when you start again
You drag the same file to the same place. Before RocketBucket reads a single byte of it, it lists the incomplete uploads for that key on the server and matches them against the journal. No hashing, no disk reads — it is cheap enough to do before asking you anything. Then it asks: resume this one, start over (aborting only the upload you chose), or cancel.
Choose resume, and the first check is whether this is still the same file. Same size, same modification date, and the same inode — so a file that was replaced by a different one of exactly the same size fails here, and the journal is not trusted. Then the server's part list has to agree with the journal: every part we recorded must be there, with the ETag and size we recorded. Only then are those parts adopted.
Adopted parts are never read again. RocketBucket seeks past them and reads only the missing parts, by byte range, hashes each one into the journal, and uploads it. A resume where every part is already on the server reads zero bytes from disk and sends zero uploads; there is a test that asserts exactly that with 10,000 parts.
That is not how the first version worked, and the first version is how we learned what 500 GB means. It verified each part before skipping it by reading the whole file from byte zero and hashing every part, including the 9,800 already uploaded, in one loop that never paused. Memory climbed to 14.5 GB and macOS killed it. We suspected the journal first — it rewrote the whole file on every part, ten thousand rewrites of a document that grows each time — and batched it to every 64 parts. That was a real bug and a correct fix, and the resume still died at 14.5 GB. The actual cause was in the hashing loop: the hex encoding used String(format:), which allocates an Objective-C string per byte, and nothing ever drained the roughly 127,000 temporaries it left behind. Reading only the missing parts made the question moot.
How it finishes
When the last part is up, RocketBucket checks that parts 1 through 9,847 are all present and that their sizes add up to the file, and sends the Complete request. For a small file that is the end. For 500 GB on R2 it is the beginning of a strange half-minute: the server acknowledges the request, and then spends tens of seconds assembling 9,847 pieces into one object. During that window the object does not exist yet, and the multipart upload is already gone.
We know this because our first 500 GB completion reported upload completion is uncertain, please verify the object manually — and the object was there, complete, with our token on it. The reconciliation that runs when a Complete is ambiguous had asked "is the object there, with my token, at my size?" three times, four seconds apart, seen "upload gone, object absent" each time, and given up. It now asks six times — 2, 4, 8, 15, 15, 15 seconds — and treats a vanished upload as a question, not an answer. The definition of success did not move: the object must answer, carry our token, and be exactly the expected size. If the token or size is wrong we still say we are not sure, because the alternative is claiming an object that might be somebody else's.
Review found two more things in the resume path before it shipped: a range read was not checked for returning every byte it asked for, which could have completed a truncated object the server would happily accept; and the fast path called Complete directly, skipping the reconciliation above. Both fixed.
The run
Start the 500 GB upload. Stop it. Quit. Open RocketBucket, drag the file again, choose resume. It adopts the parts on the server, uploads the rest, completes, waits out the assembly, and confirms the object: 536,870,912,000 bytes, 9,847 parts, commit token present, no leftover multipart upload. Peak memory across the entire run, about 900 MB to 1 GB.
The design for all of this had been reviewed five times and was right. What broke was in none of it: an allocator, a journal that grew, a Stop button checked in the wrong loop, and a server that needs half a minute to glue nine thousand pieces together. You do not find those by reviewing. You find them by uploading 500 GB, stopping it, and starting again.
RocketBucket is a dual-pane file transfer app for S3, R2, B2 and MinIO, now on the Mac App Store. rocketbucket.app