feat: Set Feast user-agent on S3 and S3-compatible clients#6573
Draft
goanpeca wants to merge 1 commit into
Draft
feat: Set Feast user-agent on S3 and S3-compatible clients#6573goanpeca wants to merge 1 commit into
goanpeca wants to merge 1 commit into
Conversation
Signed-off-by: Gonzalo Peña-Castellanos <goanpeca@gmail.com>
9e9a1b3 to
ae0d78b
Compare
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
What this PR does / why we need it:
Summary
Feast talks to object storage through boto3 in several places (the S3 registry store, the AWS utils helpers, the ibis offline store lister, and the Spark event-log check). Most of these clients were created without a user-agent, so requests reached the storage service as generic boto3 traffic with no indication that Feast was the caller.
This PR threads Feast's existing user-agent (
feast-dev/feast/<version>, already defined asUSER_AGENTinsdk/python/feast/utils.py) onto those clients viabotocoreConfig(user_agent_extra=get_user_agent()). It is a small, additive change: no new dependencies, no new configuration, and no behavior change beyond an addedUser-Agentsuffix on outbound requests.Because these clients honor a custom
endpoint_url(for exampleFEAST_S3_ENDPOINT_URL), the same instrumentation applies whether Feast is pointed at Amazon S3 or at any S3-compatible object store (for example Backblaze B2, Cloudflare R2, or MinIO). This makes it easier for operators and storage providers to attribute traffic to Feast when debugging or reviewing usage.Changes
sdk/python/feast/infra/registry/s3.py: passConfig(user_agent_extra=get_user_agent())when constructing the S3 registry resource. Added a one-line note thatFEAST_S3_ENDPOINT_URLmay point at an S3-compatible endpoint.sdk/python/feast/infra/utils/aws_utils.py: adduser_agent_extrato the existingConfiginget_s3_resourceandlist_s3_files(alongside the region already set there).sdk/python/feast/infra/offline_stores/ibis.py: pass aConfigwithuser_agent_extrato the boto3 client used bylist_s3_files.sdk/python/feast/infra/compute_engines/spark/utils.py: adduser_agent_extrato the boto3Configused for the S3A event-log directory check.The value comes from the pre-existing
get_user_agent()helper; this PR only wires it into clients that previously omitted it.Which issue(s) this PR fixes:
Checks
git commit -s)Testing Strategy
The change is internal instrumentation: it only appends a
User-Agentsuffix to boto3 clients and does not alter request semantics, return values, or public APIs. It reuses the existingget_user_agent()helper (the same value Feast already advertises elsewhere), so no new test surface is introduced. Existing unit tests were run and pass, andruffis clean on the four changed files.Misc
Scope is intentionally minimal and additive: no new dependencies, no config changes, no user-facing API changes. Happy to add a
kind/*label and adjust the title or wording if maintainers prefer a different convention.