feat(catalog-rest): parse server-advertised endpoints from GET /v1/config#2692
feat(catalog-rest): parse server-advertised endpoints from GET /v1/config#2692huan233usc wants to merge 1 commit into
Conversation
9f2517e to
a0837b8
Compare
…nfig The REST spec lets a server advertise the routes it supports in the `endpoints` field of its `GET /v1/config` response, so clients can negotiate optional capabilities instead of assuming every server implements every operation. The Rust client currently ignores this field. Add an `Endpoint` type (HTTP method + path template) parsed from the `"<method> <path>"` wire form via `FromStr` — which validates the method and the single-space shape — with serde delegating to it. The method is stored as a typed `http::Method` (kept out of the public API). Parse the config response's `endpoints` into `Option<Vec<Endpoint>>` so an absent field and an explicit empty list are modelled distinctly, store the negotiated set on the catalog's runtime context, and expose `RestCatalog::supports_endpoint`. When the field is absent a standard base set of namespace/table operations is assumed; an explicit list — even an empty one — is used verbatim. Building block for endpoint-gated features (e.g. server-side scan planning), mirroring the capability negotiation Java gained in apache/iceberg#10929.
8d72566 to
c769500
Compare
amogh-jahagirdar
left a comment
There was a problem hiding this comment.
Overall looks great @huan233usc just had some questions on some comments and I think we probably want to handle the explicit empty case a bit differently by just falling back to the default.
| } | ||
|
|
||
| #[test] | ||
| fn default_endpoints_cover_base_ops_but_not_optional_ones() { |
There was a problem hiding this comment.
I'm not sure tests like this are really helpful, we're basically just asserting what's in the DEFAULT_ENDPOINTS list.
There was a problem hiding this comment.
Yeah test_config_without_endpoints_falls_back_to_default_set tests the default fallback behavior how I'd expect, so I think I'd just remove this test.
| "GET", // no space | ||
| "GET /v1", // two spaces | ||
| " GET /v1", // leading space (empty method) | ||
| "GET ", // trailing space (empty path) | ||
| " /v1", // empty method | ||
| "", // empty |
There was a problem hiding this comment.
Minor: I'd leave out the inline comments, it's apparent from the strings themselves why these are malformed.
| /// It's could be different from the user config. | ||
| config: RestCatalogConfig, | ||
| /// Endpoints the server advertised in its `GET /v1/config` response, used | ||
| /// for capability negotiation. Empty when the server does not advertise an |
There was a problem hiding this comment.
Is the "Empty when the server does not advertise an endpoints list" comment accurate? We set it to the default endpoints list no?
| let endpoints = match &catalog_config.endpoints { | ||
| Some(advertised) => advertised.iter().cloned().collect(), | ||
| None => crate::endpoint::DEFAULT_ENDPOINTS.clone(), | ||
| }; |
There was a problem hiding this comment.
In case the server returns an explicit empty list , I think the right thing to do is still just use the default endpoints, those were established to be a minimum to be supported by a server so maybe smth like:
There was a problem hiding this comment.
let endpoints = match &catalog_config.endpoints {
Some(advertised) if !advertised.is_empty() => advertised.iter().cloned().collect(),
_ => crate::endpoint::DEFAULT_ENDPOINTS.clone(),
}
| /// | ||
| /// Servers advertise the routes they support in the `endpoints` field of | ||
| /// the `GET /v1/config` response. When that field is present it is used | ||
| /// verbatim (an explicit empty list therefore advertises nothing). When a |
There was a problem hiding this comment.
Commented above, but I'm not sure about using verbatim the empty list case; why not just fallback to the default list because that's largely been established as the minimal support required? and even if a server happens to not support it, it'd just reject any subsequent requests.
There was a problem hiding this comment.
The reason for the fallback in the explicit empty case is that I'm largely looking at it as more likely that there are buggy servers out there that return empty lists even though they don't intend to, rather than servers that want to advertise "Hey i don't support anything" because in that case I'm not sure what a server is even doing
| /// (the `endpoints` field of `GET /v1/config`). `None` when the field is | ||
| /// absent (e.g. an older server); `Some` — including an empty list — when | ||
| /// the server sends it explicitly. | ||
| #[serde(default)] |
There was a problem hiding this comment.
Minor: Not a rust expert but my understanding is this serde(default) is redundant
Which issue does this PR close?
Related to #1690 (server-side scan planning), which relies on endpoint
negotiation. This PR lands the negotiation piece on its own so it can be
reviewed independently.
What changes are included in this PR?
The REST spec lets a server advertise the routes it supports in the
endpointsfield of its
GET /v1/configresponse, so clients can negotiate optionalcapabilities instead of assuming every server implements every operation. The
Rust client currently ignores this field.
This mirrors the capability negotiation the Java client gained in
apache/iceberg#10929 (the
endpointsfield in the config response, theEndpointtype, and the default-endpoint fallback when a server omits the list).This PR:
Endpointtype (HTTP method + path template) parsed from the"<method> <path>"wire form viaFromStr— which validates the single-spaceshape and the HTTP method — with serde delegating to it. The method is stored
as a typed
http::Method(kept out of the public API;method()returns&str).endpointsintoOption<Vec<Endpoint>>so anabsent field and an explicit empty list are modelled distinctly, and stores
the negotiated set on the catalog's runtime context.
field → a standard base set of namespace/table operations is assumed (so an
older server still resolves its core operations as supported); a present list
— even an empty one — is used verbatim. Optional endpoints outside the base
set stay unsupported unless explicitly advertised.
RestCatalog::supports_endpoint(&Endpoint)to query the negotiatedset.
No existing behaviour changes for callers that don't consult
supports_endpoint.Are these changes tested?
Yes:
Endpoint:FromStr/serde round-trip, rejection of malformedinput (no / extra / leading / trailing space, empty), HTTP-method
normalization, and the default endpoint set.
mockitocatalog tests: a server that advertisesendpoints(assertingsupports_endpointis true/false for listed/unlisted routes), a server thatomits the field (base operations resolve as supported), and a server that
sends an explicit empty list (nothing is supported).