Repository navigation
Fix distributed cloud performance, security, and robustness concerns - #152
Merged
Merged
Conversation
…concerns - Fix POST creation-with-upload lock acquisition and watchdog leak: ensure distributed lease lock is acquired for newly created uploadId and released cleanly on stream completion or abortion across Tus and RUFH. - Enforce stream safety and rollback integrity: rollback uncommitted bytes on client stream interruptions and reset tentative length for RUFH deferred length uploads. - Prevent parent upload truncation on child error rollback in S3 and Azure Blob storage services. - Increase S3 server-side compose HTTP timeouts to 60s for large uploads and preserve fallback behavior. - Harden against HTTP request smuggling and DoS: enforce strict CRLF and chunk size validation in chunked transfer encoding stream and bound metadata length. - Optimize pattern compilation using ConcurrentHashMap caching. - Add preliminary request validation in getRawInterimResponse to prevent phantom upload creation on invalid requests. - Add integration test for chunked streaming with trailing checksum verification across Disk, S3, and Azure backends.
…nosniff header in download extension
- Evict pre-lock ThreadLocal cache entries on lock acquisition in ThreadLocalCachedStorageAndLockingService - Harden protocol handlers, validators, stream/header parsing, and Disk/S3/Azure storage, locking, and deduplication - Add unit tests covering new edge cases and error paths - Clean up CHANGELOG.md for 2.0.0 and update AGENTS.md and storage/architecture docs
…esponse, and improve coverage
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Summary of Changes
This PR addresses performance, security, and robustness concerns identified when using
tus-java-serverin distributed cloud environments with slow or unstable client connections across all storage backends (Disk, S3, Azure Blob) and protocol versions (Tus 1.0.0 and RUFH Draft-12).1. Lock Acquisition & Watchdog Leak Prevention on POST Requests
POST /fileswith payload), the initial upload was streamed without acquiring an upload lock for the newly generatedUploadId. If the client disconnected and sent a resume/terminate request to another instance, concurrent execution occurred without contention resolution. Furthermore, the active interruptible stream leaked intoAbstractLeaseLockingService.activeInputStreams, causing perpetual 500ms watchdog stat polling.CreationWithUploadPostRequestHandlerandRufhCreationPostRequestHandlerto acquire anUploadLockfor the generatedUploadIdviaacquireUploadLockWithRetry, registering the interruptible stream with lock contention resolution and ensuring timely cleanup infinallyblocks upon completion or failure.2. Stream Safety & Uncommitted Byte Rollback
UploadInfo.offsetoruploadInfo.lengthcould be left in an inconsistent tentative state.RufhCreationPostRequestHandlerandRufhAppendPatchRequestHandler, cleanly reset tentativeuploadInfo.setLength(null)when client streaming terminates prematurely, enabling subsequent resume requests.TusFileUploadService.processTusException, cap rollback to(uploadInfo.getOffset() - initialOffset)so that uncommitted temp chunks do not truncate earlier valid committed data.3. Safe Deduplication Handling in Cloud Storage
duplicatesUploadIdto parent uploads.S3StorageService.removeLastNumberOfBytesandAzureBlobStorageService.removeLastNumberOfBytes, target only the child upload's own physical files/blobs, preventing parent data truncation. Extracted expiration cutoff calculation intoS3Utils.calculateExpirationCutoffandAzureUtils.calculateExpirationCutoff.4. S3 Compose HTTP Timeout Calibration
5. Security & DoS Hardening
HttpChunkedEncodingInputStream.Upload-Metadataheader length (64KB) inUploadMetadataValidatorto prevent memory exhaustion DoS spikes, while preserving backward compatibility for non-ASCII characters without spaces or commas.ConcurrentHashMapcaching inUtils.java.6. Preliminary Validation in
getRawInterimResponsegetRawInterimResponseacross all protocol versions. When validation fails,nullis returned without touching storage, allowing the request to fall through to normal servlet processing where formal RFC 7807 problem details or Tus error responses are generated.7. Testing & Quality Gates
testChunkedPatchWithTrailingContentDigestValidationinAbstractITRufhProtocol, running end-to-end chunked streaming with trailing checksum validation across Disk, S3, and Azure backends.fmt-maven-plugin:check).origin/master(exceeding the 95.00% threshold).