Skip to content

feat(catalog-rest): parse server-advertised endpoints from GET /v1/config#2692

Open
huan233usc wants to merge 1 commit into
apache:mainfrom
huan233usc:feat/rest-config-endpoints
Open

feat(catalog-rest): parse server-advertised endpoints from GET /v1/config#2692
huan233usc wants to merge 1 commit into
apache:mainfrom
huan233usc:feat/rest-config-endpoints

Conversation

@huan233usc

@huan233usc huan233usc commented Jun 22, 2026

Copy link
Copy Markdown
Contributor

Which issue does this PR close?

Related to #1690 (server-side scan planning), which relies on endpoint
negotiation. This PR lands the negotiation piece on its own so it can be
reviewed independently.

What changes are included in this PR?

The REST spec lets a server advertise the routes it supports in the endpoints
field of its GET /v1/config response, so clients can negotiate optional
capabilities instead of assuming every server implements every operation. The
Rust client currently ignores this field.

This mirrors the capability negotiation the Java client gained in
apache/iceberg#10929 (the endpoints field in the config response, the
Endpoint type, and the default-endpoint fallback when a server omits the list).

This PR:

  • Adds an Endpoint type (HTTP method + path template) parsed from the
    "<method> <path>" wire form via FromStr — which validates the single-space
    shape and the HTTP method — with serde delegating to it. The method is stored
    as a typed http::Method (kept out of the public API; method() returns
    &str).
  • Parses the config response's endpoints into Option<Vec<Endpoint>> so an
    absent field and an explicit empty list are modelled distinctly, and stores
    the negotiated set on the catalog's runtime context.
  • Resolves the set following the spec's optional-field semantics: an absent
    field → a standard base set of namespace/table operations is assumed (so an
    older server still resolves its core operations as supported); a present list
    — even an empty one — is used verbatim. Optional endpoints outside the base
    set stay unsupported unless explicitly advertised.
  • Exposes RestCatalog::supports_endpoint(&Endpoint) to query the negotiated
    set.

No existing behaviour changes for callers that don't consult supports_endpoint.

Are these changes tested?

Yes:

  • Unit tests for Endpoint: FromStr/serde round-trip, rejection of malformed
    input (no / extra / leading / trailing space, empty), HTTP-method
    normalization, and the default endpoint set.
  • mockito catalog tests: a server that advertises endpoints (asserting
    supports_endpoint is true/false for listed/unlisted routes), a server that
    omits the field (base operations resolve as supported), and a server that
    sends an explicit empty list (nothing is supported).

@huan233usc huan233usc marked this pull request as draft June 22, 2026 05:30
@huan233usc huan233usc force-pushed the feat/rest-config-endpoints branch 4 times, most recently from 9f2517e to a0837b8 Compare June 23, 2026 21:02
@huan233usc huan233usc marked this pull request as ready for review June 23, 2026 21:05
…nfig

The REST spec lets a server advertise the routes it supports in the `endpoints`
field of its `GET /v1/config` response, so clients can negotiate optional
capabilities instead of assuming every server implements every operation. The
Rust client currently ignores this field.

Add an `Endpoint` type (HTTP method + path template) parsed from the
`"<method> <path>"` wire form via `FromStr` — which validates the method and the
single-space shape — with serde delegating to it. The method is stored as a
typed `http::Method` (kept out of the public API). Parse the config response's
`endpoints` into `Option<Vec<Endpoint>>` so an absent field and an explicit empty
list are modelled distinctly, store the negotiated set on the catalog's runtime
context, and expose `RestCatalog::supports_endpoint`. When the field is absent a
standard base set of namespace/table operations is assumed; an explicit list —
even an empty one — is used verbatim.

Building block for endpoint-gated features (e.g. server-side scan planning),
mirroring the capability negotiation Java gained in apache/iceberg#10929.
@huan233usc huan233usc force-pushed the feat/rest-config-endpoints branch from 8d72566 to c769500 Compare June 24, 2026 02:24

@amogh-jahagirdar amogh-jahagirdar left a comment

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Overall looks great @huan233usc just had some questions on some comments and I think we probably want to handle the explicit empty case a bit differently by just falling back to the default.

}

#[test]
fn default_endpoints_cover_base_ops_but_not_optional_ones() {

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

I'm not sure tests like this are really helpful, we're basically just asserting what's in the DEFAULT_ENDPOINTS list.

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Yeah test_config_without_endpoints_falls_back_to_default_set tests the default fallback behavior how I'd expect, so I think I'd just remove this test.

Comment on lines +209 to +214
"GET", // no space
"GET /v1", // two spaces
" GET /v1", // leading space (empty method)
"GET ", // trailing space (empty path)
" /v1", // empty method
"", // empty

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor: I'd leave out the inline comments, it's apparent from the strings themselves why these are malformed.

/// It's could be different from the user config.
config: RestCatalogConfig,
/// Endpoints the server advertised in its `GET /v1/config` response, used
/// for capability negotiation. Empty when the server does not advertise an

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Is the "Empty when the server does not advertise an endpoints list" comment accurate? We set it to the default endpoints list no?

Comment on lines +423 to +426
let endpoints = match &catalog_config.endpoints {
Some(advertised) => advertised.iter().cloned().collect(),
None => crate::endpoint::DEFAULT_ENDPOINTS.clone(),
};

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

In case the server returns an explicit empty list , I think the right thing to do is still just use the default endpoints, those were established to be a minimum to be supported by a server so maybe smth like:

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

let endpoints =  match &catalog_config.endpoints {
      Some(advertised) if !advertised.is_empty() => advertised.iter().cloned().collect(),
      _ => crate::endpoint::DEFAULT_ENDPOINTS.clone(),
}

///
/// Servers advertise the routes they support in the `endpoints` field of
/// the `GET /v1/config` response. When that field is present it is used
/// verbatim (an explicit empty list therefore advertises nothing). When a

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Commented above, but I'm not sure about using verbatim the empty list case; why not just fallback to the default list because that's largely been established as the minimal support required? and even if a server happens to not support it, it'd just reject any subsequent requests.

@amogh-jahagirdar amogh-jahagirdar Jun 28, 2026

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

The reason for the fallback in the explicit empty case is that I'm largely looking at it as more likely that there are buggy servers out there that return empty lists even though they don't intend to, rather than servers that want to advertise "Hey i don't support anything" because in that case I'm not sure what a server is even doing

/// (the `endpoints` field of `GET /v1/config`). `None` when the field is
/// absent (e.g. an older server); `Some` — including an empty list — when
/// the server sends it explicitly.
#[serde(default)]

Copy link
Copy Markdown
Contributor

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Minor: Not a rust expert but my understanding is this serde(default) is redundant

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants