Skip to content

[core] Support REST catalog branch management - #1141

Merged
JingsongLi merged 5 commits into
apache:mainfrom
JingsongLi:codex/rest-branch-management
Oct 17, 2026
Merged

JingsongLi merged 5 commits into
apache:mainfrom
JingsongLi:codex/rest-branch-management

Conversation

@JingsongLi

@JingsongLi JingsongLi commented Oct 17, 2026 •

Copy link
Copy Markdown
Contributor

Purpose

PyPaimon already supports REST branch creation, listing, deletion and fast-forward. Add the missing Rust Catalog, RESTApi and Native binding capabilities, following Java's RESTCatalog and RESTApi contracts.

Changes

  • Add Catalog::{create_branch, drop_branch, fast_forward, list_branches} with unsupported defaults and a REST implementation. Use Java's endpoints, branch/fromTag payload, empty forward request, nullable branch-list response, ordering and operation-specific errors.
  • Expose thin PaimonCatalog methods that accept string/tuple identifiers and release the GIL. config_required=False reuses resolved REST configuration without fetching /config again.
  • Provide structured CatalogError fields so PyPaimon can retain its public exception types. Preserve missing JSON status codes, unknown HTTP errors and transport errors; keep ForkSafetyError distinct.
  • Reject empty/dot path components before dispatch. Keep publication on the REST server without shared-data cleanup or replay through another backend.
  • Implement OpenAPI V4 (ACS4-HMAC-SHA256) in Rust core, selected by dlf.signing-algorithm=openapi-v4. Port Java's canonical request, RFC 3986 query encoding, payload hash, credential scope and 61 action mappings. Existing signing algorithms remain available.
  • Merge generated authentication headers case-insensitively and replace JSON Content-Type defaults rather than append duplicate values.
  • Accept arbitrary successful mutation response bodies. Retry only GET on 429/503, with at most five retries and Java's Retry-After/backoff behavior; do not status-retry POST/DELETE.
  • Align connection/read timeouts with Java's three-minute defaults instead of a 30-second total request deadline.

Verification

  • Rust API unit tests: 104 passed.
  • REST API/catalog/branch integration tests: 188 passed.
  • Binding catalog/branch/GIL/table tests: 83 passed.
  • PyPaimon Native branch/lifecycle tests against this binding: 109 passed, exercising 161 plans, 162 reads, 140 writes and 84 commits.
  • Ordinary PyPaimon branch/DTO/DLF signer tests: 82 passed.
  • Rust 1.98.0 Clippy for core and bindings with all targets and -D warnings, rustfmt, changed-file flake8 and git diff --check: passed.
  • A real HTTP mutation with a 31-second response delay succeeded after one request, validating removal of the old total deadline.

Repeated independent review and regression testing covered Java parity, V4 signatures, credential caching/refresh, header collisions, GET retries, transport errors and uncertain publication. V4 integration independently recomputes wire MACs with the Python signer while guarding against Python operation fallback. Paimon REST authenticates each request; no cross-client cookie/session synchronization or response observation API is introduced.

PyPaimon's mock server acknowledges fast-forward; request semantics are verified against Java and Rust protocol tests. This does not claim to test a production server's snapshot-switching implementation.

@leaves12138 leaves12138 left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

LGTM

@JingsongLi
JingsongLi merged commit 9568f39 into apache:main Oct 17, 2026
16 checks passed
sundapeng added a commit to sundapeng/paimon that referenced this pull request Oct 17, 2026
apache/paimon-rust#1141 made the Rust REST client retry GET requests on
HTTP 429 and 503, as Java does. Native CI builds pypaimon-rust from
paimon-rust main, so four tests that answer the snapshot request with 503
and expect a single call now see six and fail on master.

They check that search does not retry on its own, not HTTP retries, so
answer with the non-retryable 500 instead.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants