Skip to content

feat: Set Feast user-agent on S3 and S3-compatible clients#6573

Draft
goanpeca wants to merge 1 commit into
feast-dev:masterfrom
goanpeca:feat/b2-s3-user-agent
Draft

feat: Set Feast user-agent on S3 and S3-compatible clients#6573
goanpeca wants to merge 1 commit into
feast-dev:masterfrom
goanpeca:feat/b2-s3-user-agent

Conversation

@goanpeca

@goanpeca goanpeca commented Jul 1, 2026

Copy link
Copy Markdown

What this PR does / why we need it:

Summary

Feast talks to object storage through boto3 in several places (the S3 registry store, the AWS utils helpers, the ibis offline store lister, and the Spark event-log check). Most of these clients were created without a user-agent, so requests reached the storage service as generic boto3 traffic with no indication that Feast was the caller.

This PR threads Feast's existing user-agent (feast-dev/feast/<version>, already defined as USER_AGENT in sdk/python/feast/utils.py) onto those clients via botocore Config(user_agent_extra=get_user_agent()). It is a small, additive change: no new dependencies, no new configuration, and no behavior change beyond an added User-Agent suffix on outbound requests.

Because these clients honor a custom endpoint_url (for example FEAST_S3_ENDPOINT_URL), the same instrumentation applies whether Feast is pointed at Amazon S3 or at any S3-compatible object store (for example Backblaze B2, Cloudflare R2, or MinIO). This makes it easier for operators and storage providers to attribute traffic to Feast when debugging or reviewing usage.

Changes

  • sdk/python/feast/infra/registry/s3.py: pass Config(user_agent_extra=get_user_agent()) when constructing the S3 registry resource. Added a one-line note that FEAST_S3_ENDPOINT_URL may point at an S3-compatible endpoint.
  • sdk/python/feast/infra/utils/aws_utils.py: add user_agent_extra to the existing Config in get_s3_resource and list_s3_files (alongside the region already set there).
  • sdk/python/feast/infra/offline_stores/ibis.py: pass a Config with user_agent_extra to the boto3 client used by list_s3_files.
  • sdk/python/feast/infra/compute_engines/spark/utils.py: add user_agent_extra to the boto3 Config used for the S3A event-log directory check.

The value comes from the pre-existing get_user_agent() helper; this PR only wires it into clients that previously omitted it.

Which issue(s) this PR fixes:

Checks

  • I've made sure the tests are passing.
  • My commits are signed off (git commit -s)
  • My PR title follows conventional commits format

Testing Strategy

  • Unit tests
  • Integration tests
  • Manual tests
  • Testing is not required for this change

The change is internal instrumentation: it only appends a User-Agent suffix to boto3 clients and does not alter request semantics, return values, or public APIs. It reuses the existing get_user_agent() helper (the same value Feast already advertises elsewhere), so no new test surface is introduced. Existing unit tests were run and pass, and ruff is clean on the four changed files.

Misc

Scope is intentionally minimal and additive: no new dependencies, no config changes, no user-facing API changes. Happy to add a kind/* label and adjust the title or wording if maintainers prefer a different convention.

Signed-off-by: Gonzalo Peña-Castellanos <goanpeca@gmail.com>
@goanpeca goanpeca force-pushed the feat/b2-s3-user-agent branch from 9e9a1b3 to ae0d78b Compare July 1, 2026 13:55
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant