Repository navigation
Conversation
The S3 tier resolved credentials once through aws-creds, which sends AssumeRoleWithWebIdentity only to sts.amazonaws.com. S3-compatible stores such as MinIO and SeaweedFS serve STS themselves. With --s3StsEndpoint (or AWS_ENDPOINT_URL_STS) set, serve exchanges AWS_WEB_IDENTITY_TOKEN_FILE for AWS_ROLE_ARN's credentials there and renews the session five minutes before it expires, re-reading the token file each time. Fixes Tinder#614
Author
|
This change is part of the following stack: Change managed by git-spice. |
This branch has not been deployed
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Fixes #614.
Problem
serve's S3 tier resolves credentials once withCredentials::default(). WithAWS_ROLE_ARNandAWS_WEB_IDENTITY_TOKEN_FILEset,aws-credssends the token tohttps://sts.amazonaws.com/, which is hardcoded inaws-creds0.38 and 0.40. S3-compatible stores that serve STS themselves (SeaweedFS, MinIO) can't be reached, so a pod with a Kubernetes ServiceAccount token and no static keys fails at startup withfailed to resolve AWS credentials.--s3Endpointchanges only where objects are read and written.Change
--s3StsEndpointflag, which also reads the standardAWS_ENDPOINT_URL_STS. When it is set,servesendsAssumeRoleWithWebIdentityto that endpoint as a form POST, withAWS_ROLE_ARN, the token fromAWS_WEB_IDENTITY_TOKEN_FILE, andAWS_ROLE_SESSION_NAME(defaultbazel-diff).rust-s3carry noexpiration. Otherwiserust-s3would replace them, once expired, from the default chain, which goes back tosts.amazonaws.com.quick-xml0.32 becomes a direct dependency to parse the STS answer. It was already inCargo.lockthroughaws-creds.RemoteCache::S3is now boxed like the GCS and Nexus variants, because the session state trippedclippy::large_enum_variant.tools/readme_template.mddocument the flag.Testing
bazelisk test //:rust_tests //:rust_clippy_check //:rust_format_check //tools/...passes on macOS arm64 (Bazel 8.8.2). New tests:web_identity::tests, against a local STS ontiny_http: the form POST contents, reuse of a session that is not near expiry, renewal of an expiring session with a rotated token, a refused exchange, an answer without credentials, a missing token file, a non-HTTP endpoint, and the renewal margin boundary.server::tests::s3_cache_signs_with_the_web_identity_session: an S3 GET throughS3Cachecarries the session token from the STS exchange.I checked the README help text against
bazel-diff serve --help.