Skip to content

[python] Add SDK version to REST User-Agent - #10106

Open
hutiefang76 wants to merge 3 commits into
apache:masterfrom
hutiefang76:frank/paimon-8543-sdk-user-agent
Open

hutiefang76 wants to merge 3 commits into
apache:masterfrom
hutiefang76:frank/paimon-8543-sdk-user-agent

Conversation

@hutiefang76

@hutiefang76 hutiefang76 commented Sep 22, 2026 •

Copy link
Copy Markdown
Contributor

What is changed

Adds PyPaimon and Python version information to the default REST User-Agent. PVFS uses the standard header.User-Agent key and adds its PythonPVFS prefix without replacing an explicitly configured value.

The REST client now treats configured User-Agent keys case-insensitively, so header.user-agent is respected even when a request is prepared by requests. An explicit server override takes precedence over a client value, followed by server defaults and the SDK default. PVFS applies the same case-insensitive check before adding its own value.

Version metadata that cannot be read or parsed falls back to unknown, so requests continue normally.

Verification

  • 45 tests passed across user_agent_test.py, build_info_test.py, rest/api_test.py, and pvfs_oss_filesystem_test.py, including prepared-request and precedence regressions.
  • Flake8 passed for the changed files; changed Python files compiled successfully.

Closes #8543

@JingsongLi JingsongLi left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The linked #8543 is a concrete observability need, and the default version string now reaches real REST requests. I ran the four affected Python suites: 41 tests passed after allowing the local REST test server to bind its port. I also checked the prepared HTTP request, not only the in-memory header map.

[P2] Preserve explicitly configured User-Agent regardless of header casing. HTTP header names are case-insensitive, but base_headers.setdefault("User-Agent", ...) and PVFS's options.contains("header.User-Agent") are exact-case checks. With a valid configuration header.user-agent=custom-client/1.0, RESTApi keeps both user-agent: custom-client/1.0 and User-Agent: PyPaimon/...; requests.Session.prepare_request(...) sends the latter, silently overriding the explicit value. PVFS also inserts both keys. This violates the preservation requirement in #8543. Please detect existing User-Agent headers case-insensitively (also after REST config merge) and add a test that inspects a prepared request for a lowercase configured key.

Requirement fit: SUPPORTED. Requesting changes for the configuration contract before production use.

@hutiefang76

Copy link
Copy Markdown
Contributor Author

Addressed the case-insensitive User-Agent feedback in ca90868bf. The new tests inspect the prepared HTTP request for lowercase client headers, including config merge/override behavior, and cover the PVFS option path.

The latest CI's flake8 check passes. Python Tests and Native CI still report the same two failures in interval_partition_test.py::test_signed_zero_key_ranges_keep_versions_in_one_split (SimpleNamespace lacks path_factory). That test file is unchanged by this PR and is identical on current origin/master; the PR changes only the User-Agent files. I am keeping this CI failure visible rather than treating the PR as green.

@hutiefang76

Copy link
Copy Markdown
Contributor Author

Synced with master in d615576 to pick up the merged #10169 fix for the unrelated interval-partition CI failure. The targeted User-Agent, build-info, REST, PVFS and interval-partition tests pass locally (39 tests), and flake8 passes with the project config. The local Ray integration case was excluded because Ray is not installed. Fresh CI is now pending.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

[Feature] Add sdk version information to REST User-Agent

2 participants