fix: passthrough Scylla manifest UploadPartCopy with full-object range#121
Merged
Conversation
Scylla Manager sends bytes=0-(size-1) on .sm_ manifest copies, which previously forced UPLOAD_PART_COPY_STREAMING and queued behind the 2-slot pipeline until the 300s client timeout. Normalize full-object ranges to whole-object copies and stop gating passthrough on the streaming semaphore. Co-authored-by: Cursor <cursoragent@cursor.com>
Update test_passthrough_encrypt_delta_vs_reencrypt to expect encrypt bytes proportional to the 32MB partial range, not the full 96MB QUICK_SIZE source. Fix ruff line-length and unused-variable lint. Co-authored-by: Cursor <cursoragent@cursor.com>
Co-authored-by: Cursor <cursoragent@cursor.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary
x-amz-copy-source-range: bytes=0-(size-1)to a whole-object copy so large encrypted SST manifests (.sm_*) useUPLOAD_PART_COPY_PASSTHROUGHinstead ofUPLOAD_PART_COPY_STREAMINGuploadId=2~prvq0MWZR...)Test plan
pytest tests/unit/test_upload_part_copy_passthrough.py(6 tests: range normalization, Scylla-shaped full-range passthrough, semaphore non-blocking)pytest tests/unit/test_copy_pipeline_cap.py tests/unit/test_copy_per_part_memory.py tests/unit/test_copy_clamp_and_concurrency.py(41 tests)tests/integration/test_upload_part_copy_passthrough_e2e.py(newtest_scylla_manifest_full_range_passthroughon 1280MB source)UPLOAD_PART_COPY_PASSTHROUGH+ 200 onbig-Data.db.sm_*manifest uploads during Scylla backupMade with Cursor