Repository navigation
Conversation
s3.multipart_download_type, also read from S3_MULTIPART_DOWNLOAD_TYPE, defaults to part and validation accepts only part and range, case-insensitively, with a blank value keeping the part strategy. On the multipart download path the value is passed as GetObjectType: range selects byte-range GETs sized by the existing chunk calculation, and part keeps partNumber GETs. The README and changelog note that this runs only when allow_multipart_download is true and download_max_bytes_per_second is 0. Fixes Altinity#1632
partNumber GETs fail with HTTP 412 on Ceph RGW (Hetzner Object Storage), so keeping `part` as default leaves every RGW user broken until they find the knob. Byte ranges work on AWS S3 too and make PartSizeBytes effective. Empty value now maps to range as well; `part` stays as explicit opt-out. Add the new key to the testflows default_config snapshot. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
An explicit empty value (yaml "" or S3_MULTIPART_DOWNLOAD_TYPE=) silently picked a download strategy; require part or range instead. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Slach
self-requested a review
October 22, 2026 09:56
Slach
approved these changes
Oct 22, 2026
v2.8.2 is already tagged, the fix ships in the next release. Co-Authored-By: Claude Opus 5.5 <noreply@anthropic.com>
Contributor
Author
|
Thanks @Slach for the follow-ups and the approval. Making |
…ix/1632-s3-multipart-download-range
Resolve the cli snapshot conflict by keeping both the new multipart_download_type default and the timeout key from master.
mvanhorn
force-pushed
the
fix/1632-s3-multipart-download-range
branch
from
October 28, 2026 11:51
9d6b3a1 to
d2877c9
Compare
The merge resolution placed the timeout key from master under general instead of after http2_write_byte_timeout. The snapshot now matches master plus the multipart_download_type line.
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
With multipart download enabled, a backup download from Ceph RGW (including Hetzner Object Storage) fails for every part archive that was stored as a multipart object. The storage returns HTTP 412 PreconditionFailed and the download uses up its retries.
The multipart download sets concurrency and part size but leaves the transfer manager on its default, partNumber GETs. That mode takes the ETag from the first response and sends it as If-Match on later requests, while RGW answers a partNumber GET with that part's own ETag rather than the multipart object's ETag, so the precondition fails from the second part on. Byte-range GETs of the same object, with If-Match set to the object ETag, succeed.
What changed
s3.multipart_download_type, also read from S3_MULTIPART_DOWNLOAD_TYPE, defaults to part and validation accepts only part and range, case-insensitively, with a blank value keeping the part strategy. On the multipart download path the value is passed as GetObjectType: range selects byte-range GETs sized by the existing chunk calculation, and part keeps partNumber GETs. The README and changelog note that this runs only when allow_multipart_download is true and download_max_bytes_per_second is 0.
Tests
make testin a Linux container (golang:1.24) for the clickhouse-backup S3Config change; it passed.Fixes #1632