From 1df41a7dc64e7c9701d21c400ef2b7b51b8a3454 Mon Sep 17 00:00:00 2001 From: Sergii Tkachenko Date: Wed, 24 Jan 2024 08:05:13 -0800 Subject: [PATCH 01/53] A77: xDS Server-Side Rate Limiting --- A77-xds-rate-limiting-rlqs.md | 120 ++++++++++++++++++++++++++++++++++ 1 file changed, 120 insertions(+) create mode 100644 A77-xds-rate-limiting-rlqs.md diff --git a/A77-xds-rate-limiting-rlqs.md b/A77-xds-rate-limiting-rlqs.md new file mode 100644 index 000000000..76fcf860f --- /dev/null +++ b/A77-xds-rate-limiting-rlqs.md @@ -0,0 +1,120 @@ +A77: xDS Server-Side Rate Limiting +====== + +* Author(s): Sergii Tkachenko (sergiitk@) +* Approver: Mark Roth (@markdroth) +* Status: Draft +* Implemented in: +* Last updated: 2024-01-24 +* Discussion at: `TODO(sergiitk): ` + +## Abstract + +This proposal introduces support for the +[quota-based global rate limit](https://www.envoyproxy.io/docs/envoy/latest/configuration/http/http_filters/rate_limit_quota_filter) +feature to [xDS-Enabled gRPC servers][A36].\ +Main characteristics of the feature: + +* Only covers server-side rate limiting. +* Can only be applied to HTTP-level (L7) requests. +* Rate limit decisions are offloaded to a remote Rate Limit Quota Service (RLQS) + asynchronously. +* RLQS maintains a global view of the shared rate limit resource, and fairly + distributes it among multiple servers (best-effort). + +## Background + +[Global rate limiting](https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/other_features/global_rate_limiting) +allows mesh users to manage fair consumption of their services and prevent +misbehaving clients from overloading the services. *Global* rate limiting +implies that there must be a single source of truth for the state of a finite +shared resource; a Rate Limiting Service. As of 2024-01-24, Envoy describes two +types of such service: + +1. [Rate Limiting Service (RLS)](https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/other_features/global_rate_limiting#per-connection-or-per-http-request-rate-limiting): + synchronous (blocking) per-HTTP-request rate limit check. Best suited for + precise cases, where even a single request over the limit must be throttled. +2. [Rate Limiting Quota Service (RLQS)](https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/other_features/global_rate_limiting#quota-based-rate-limiting): + asynchronous (non-blocking) quota-based check, with periodic load reports of + bucketed HTTP requests. Best suited for high-request-per-second applications, + where a certain margin of error is acceptable as long as expected average QPS + is achieved. + +gRPC only intends to support quota-based global rate limiting, Rate Limiting +Quota Service (RLQS), which is described in this document. + +Simplified RLQS flow happy-path: + +1. [Rate Limit Quota xDS HTTP filter][rate_limit_quota_filter] + is configured on a gRPC Server via an LDS update in the HTTP connection + manager filter chain. +2. gRPC Server establishes a gRPC channel to the RLQS as specified + in [RateLimitQuotaFilterConfig.rlqs_server]. +3. gRPC Server parses [RateLimitQuotaFilterConfig.bucket_matchers] tree and + caches it in the filter state. +4. gRPC Server installs a Server Interceptor (filter in C-core, later called + "the Interceptor" for simplicity) +5. Once a request is intercepted by the Interceptor: + 1. The request is matched into a Bucket by evaluating the `bucket_matchers` + tree against the request attributes. + 2. If the Bucket doesn't exist, gRPC server immediately sends the + initial `BucketQuotaUsage` report to the RLQS server. The request is + throttled according + to [RateLimitQuotaBucketSettings.no_assignment_behavior]. + 3. If the Bucket exists, the request is throttled according to Bucket's + quota assignment. Bucket's `num_requests_allowed` + or `num_requests_denied` request counter is increased by one. +6. For all existing buckets, a `BucketQuotaUsage` report is sent + every [RateLimitQuotaBucketSettings.reporting_interval] + to `RateLimitQuotaService.StreamRateLimitQuotas`. +7. RLQS may send Bucket quota assignments via `RateLimitQuotaResponse` at any + time. Once received, it must the quota must be applied to a Bucket with + matching [bucket_id]. + +### Related Proposals: + +* [A36: xDS-Enabled Servers][A36] + +[A36]: A36-xds-for-servers.md + +## Proposal + +TODO(sergiitk): A precise statement of the proposed change. + +### Temporary environment variable protection + +During initial development, this feature will be enabled via the +`GRPC_EXPERIMENTAL_XDS_ENABLE_RLQS` environment variable. This environment +variable protection will be removed once the feature has proven stable. + +## Rationale + +TODO(sergiitk): A discussion of alternate approaches and the trade offs, +advantages, and disadvantages of the specified approach. + +## Implementation + +The initial implementation will be in Java.\ +C-core, Python, and Go will follow after that. + +## Open issues (if applicable) + +TODO(sergiitk): A discussion of issues relating to this proposal for which the +author does not know the solution. This section may be omitted if there are +none. + + + +[rate_limit_quota_filter]: https://www.envoyproxy.io/docs/envoy/latest/configuration/http/http_filters/rate_limit_quota_filter + +[RateLimitQuotaBucketSettings.domain]: https://www.envoyproxy.io/docs/envoy/latest/api-v3/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto.html#envoy-v3-api-field-extensions-filters-http-rate-limit-quota-v3-ratelimitquotafilterconfig-domain + +[RateLimitQuotaBucketSettings.no_assignment_behavior]: https://www.envoyproxy.io/docs/envoy/latest/api-v3/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#envoy-v3-api-field-extensions-filters-http-rate-limit-quota-v3-ratelimitquotabucketsettings-no-assignment-behavior + +[RateLimitQuotaBucketSettings.reporting_interval]: https://www.envoyproxy.io/docs/envoy/latest/api-v3/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#envoy-v3-api-field-extensions-filters-http-rate-limit-quota-v3-ratelimitquotabucketsettings-reporting-interval + +[RateLimitQuotaFilterConfig.rlqs_server]: https://www.envoyproxy.io/docs/envoy/latest/api-v3/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#envoy-v3-api-field-extensions-filters-http-rate-limit-quota-v3-ratelimitquotafilterconfig-rlqs-server + +[RateLimitQuotaFilterConfig.bucket_matchers]: https://www.envoyproxy.io/docs/envoy/latest/api-v3/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#envoy-v3-api-field-extensions-filters-http-rate-limit-quota-v3-ratelimitquotafilterconfig-bucket-matchers + +[bucket_id]: https://www.envoyproxy.io/docs/envoy/latest/api-v3/service/rate_limit_quota/v3/rlqs.proto#envoy-v3-api-field-service-rate-limit-quota-v3-ratelimitquotaresponse-bucketaction-bucket-id From 2a219849eabce5616893ccf0a3c6c9cec21c183c Mon Sep 17 00:00:00 2001 From: Sergii Tkachenko Date: Wed, 7 Feb 2024 14:58:54 -0800 Subject: [PATCH 02/53] Simplified background --- A77-xds-rate-limiting-rlqs.md | 65 ++++++++++++++++++++++++----------- 1 file changed, 45 insertions(+), 20 deletions(-) diff --git a/A77-xds-rate-limiting-rlqs.md b/A77-xds-rate-limiting-rlqs.md index 76fcf860f..7b1cebcd3 100644 --- a/A77-xds-rate-limiting-rlqs.md +++ b/A77-xds-rate-limiting-rlqs.md @@ -43,7 +43,40 @@ types of such service: gRPC only intends to support quota-based global rate limiting, Rate Limiting Quota Service (RLQS), which is described in this document. -Simplified RLQS flow happy-path: +This proposal details how gRPC will add support +for [Rate Limit Quota xDS HTTP filter][rate_limit_quota_filter] +configured via [xDS HTTP Filters][A39] to [xDS-Enabled gRPC servers][A36]. It +will cover four major parts needed for language-specific gRPC implementations: + +1. Support missing xDS types needed to parse the Rate Limit Quota Filter config. +2. Implement the client side of RLQS protocol (RLQS + Client): `StreamRateLimitQuotas.StreamRateLimitQuotas`. It will establish + bidirectional gRPC stream to the remote [Rate Limit Quota Service][rlqs]. +3. Implement a Server Interceptor (filter in C-core, later called + "the Interceptor" for simplicity) that requests. +4. Send and receive updates. + +### Related Proposals: + +* [A36: xDS-Enabled Servers][A36] +* [A39: xDS HTTP Filter Support][A39] + +[A36]: A36-xds-for-servers.md + +[A39]: A39-xds-http-filters.md + +## Proposal + +### xDS types support +#### Unified Matchers +#### io.envoyproxy.envoy.config.core.v3.GrpcService.GoogleGrpc +#### Canonical CEL + +TODO(sergiitk): A precise statement of the proposed change. + +#### Simplified RLQS flow happy-path: + +// TODO(sergiitk): Insert graphics 1. [Rate Limit Quota xDS HTTP filter][rate_limit_quota_filter] is configured on a gRPC Server via an LDS update in the HTTP connection @@ -55,15 +88,15 @@ Simplified RLQS flow happy-path: 4. gRPC Server installs a Server Interceptor (filter in C-core, later called "the Interceptor" for simplicity) 5. Once a request is intercepted by the Interceptor: - 1. The request is matched into a Bucket by evaluating the `bucket_matchers` - tree against the request attributes. - 2. If the Bucket doesn't exist, gRPC server immediately sends the - initial `BucketQuotaUsage` report to the RLQS server. The request is - throttled according - to [RateLimitQuotaBucketSettings.no_assignment_behavior]. - 3. If the Bucket exists, the request is throttled according to Bucket's - quota assignment. Bucket's `num_requests_allowed` - or `num_requests_denied` request counter is increased by one. + - The request is matched into a Bucket by evaluating the `bucket_matchers` + tree against the request attributes. + - If the Bucket doesn't exist, gRPC server immediately sends the + initial `BucketQuotaUsage` report to the RLQS server. The request is + throttled according + to [RateLimitQuotaBucketSettings.no_assignment_behavior]. + - If the Bucket exists, the request is throttled according to Bucket's quota + assignment. Bucket's `num_requests_allowed` + or `num_requests_denied` request counter is increased by one. 6. For all existing buckets, a `BucketQuotaUsage` report is sent every [RateLimitQuotaBucketSettings.reporting_interval] to `RateLimitQuotaService.StreamRateLimitQuotas`. @@ -71,16 +104,6 @@ Simplified RLQS flow happy-path: time. Once received, it must the quota must be applied to a Bucket with matching [bucket_id]. -### Related Proposals: - -* [A36: xDS-Enabled Servers][A36] - -[A36]: A36-xds-for-servers.md - -## Proposal - -TODO(sergiitk): A precise statement of the proposed change. - ### Temporary environment variable protection During initial development, this feature will be enabled via the @@ -118,3 +141,5 @@ none. [RateLimitQuotaFilterConfig.bucket_matchers]: https://www.envoyproxy.io/docs/envoy/latest/api-v3/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#envoy-v3-api-field-extensions-filters-http-rate-limit-quota-v3-ratelimitquotafilterconfig-bucket-matchers [bucket_id]: https://www.envoyproxy.io/docs/envoy/latest/api-v3/service/rate_limit_quota/v3/rlqs.proto#envoy-v3-api-field-service-rate-limit-quota-v3-ratelimitquotaresponse-bucketaction-bucket-id + +[rlqs]: https://www.envoyproxy.io/docs/envoy/latest/api-v3/service/rate_limit_quota/v3/rlqs.proto From b012cfcdfce9f1a4b16505da38c1352ba0cb277d Mon Sep 17 00:00:00 2001 From: Sergii Tkachenko Date: Wed, 7 Feb 2024 15:28:05 -0800 Subject: [PATCH 03/53] add some notes from the sync with Mark --- A77-xds-rate-limiting-rlqs.md | 24 ++++++++++++++++++++---- 1 file changed, 20 insertions(+), 4 deletions(-) diff --git a/A77-xds-rate-limiting-rlqs.md b/A77-xds-rate-limiting-rlqs.md index 7b1cebcd3..484436319 100644 --- a/A77-xds-rate-limiting-rlqs.md +++ b/A77-xds-rate-limiting-rlqs.md @@ -52,9 +52,13 @@ will cover four major parts needed for language-specific gRPC implementations: 2. Implement the client side of RLQS protocol (RLQS Client): `StreamRateLimitQuotas.StreamRateLimitQuotas`. It will establish bidirectional gRPC stream to the remote [Rate Limit Quota Service][rlqs]. -3. Implement a Server Interceptor (filter in C-core, later called + 4. Send and receive updates. +3. Implement a HTTP filter (filter in C-core, later called "the Interceptor" for simplicity) that requests. -4. Send and receive updates. +5. Fault modes + + +/// per-host and per-route overrides ### Related Proposals: @@ -69,7 +73,13 @@ will cover four major parts needed for language-specific gRPC implementations: ### xDS types support #### Unified Matchers +add a note that we'll be using this in the future +list specific matcher +start from the assumption we implement both + #### io.envoyproxy.envoy.config.core.v3.GrpcService.GoogleGrpc +consider credential could be a security/ 3p trust issue. +TODO(sergiitk): meeting with leads about this #### Canonical CEL TODO(sergiitk): A precise statement of the proposed change. @@ -86,7 +96,7 @@ TODO(sergiitk): A precise statement of the proposed change. 3. gRPC Server parses [RateLimitQuotaFilterConfig.bucket_matchers] tree and caches it in the filter state. 4. gRPC Server installs a Server Interceptor (filter in C-core, later called - "the Interceptor" for simplicity) + "the Interceptor" for simplicity) // TODO(sergiitk): HTTP filter 5. Once a request is intercepted by the Interceptor: - The request is matched into a Bucket by evaluating the `bucket_matchers` tree against the request attributes. @@ -95,7 +105,7 @@ TODO(sergiitk): A precise statement of the proposed change. throttled according to [RateLimitQuotaBucketSettings.no_assignment_behavior]. - If the Bucket exists, the request is throttled according to Bucket's quota - assignment. Bucket's `num_requests_allowed` + assignment. Bucket's `num_requests_allowed` // TODO(sergiitk): explain what's throttle or `num_requests_denied` request counter is increased by one. 6. For all existing buckets, a `BucketQuotaUsage` report is sent every [RateLimitQuotaBucketSettings.reporting_interval] @@ -104,6 +114,12 @@ TODO(sergiitk): A precise statement of the proposed change. time. Once received, it must the quota must be applied to a Bucket with matching [bucket_id]. + +// TODO(sergiitk): think about it from the perspective what events needs to be handled: +1. parsing config +2. getting an update +can be described with pseudo-code + ### Temporary environment variable protection During initial development, this feature will be enabled via the From 79c977b6b9b9c01d6eee4458109685a21e4cf30e Mon Sep 17 00:00:00 2001 From: Sergii Tkachenko Date: Tue, 5 Nov 2024 12:12:30 -0800 Subject: [PATCH 04/53] Add gRFC content --- A77-xds-rate-limiting-rlqs.md | 581 ++++++++++++++++++++++++++-------- 1 file changed, 455 insertions(+), 126 deletions(-) diff --git a/A77-xds-rate-limiting-rlqs.md b/A77-xds-rate-limiting-rlqs.md index 484436319..99880eeda 100644 --- a/A77-xds-rate-limiting-rlqs.md +++ b/A77-xds-rate-limiting-rlqs.md @@ -1,161 +1,490 @@ A77: xDS Server-Side Rate Limiting ====== -* Author(s): Sergii Tkachenko (sergiitk@) +* Author(s): Sergii Tkachenko (@sergiitk) * Approver: Mark Roth (@markdroth) -* Status: Draft +* Status: In Review * Implemented in: -* Last updated: 2024-01-24 +* Last updated: 2024-11-04 * Discussion at: `TODO(sergiitk): ` ## Abstract -This proposal introduces support for the -[quota-based global rate limit](https://www.envoyproxy.io/docs/envoy/latest/configuration/http/http_filters/rate_limit_quota_filter) -feature to [xDS-Enabled gRPC servers][A36].\ -Main characteristics of the feature: - -* Only covers server-side rate limiting. -* Can only be applied to HTTP-level (L7) requests. -* Rate limit decisions are offloaded to a remote Rate Limit Quota Service (RLQS) - asynchronously. -* RLQS maintains a global view of the shared rate limit resource, and fairly - distributes it among multiple servers (best-effort). +We're adding support for global rate limiting to xDS-enabled gRPC servers. Users +will be able to configure per-time-unit quotas based on request metadata. Rate +Limit Quota Service will fairly distribute request quotas across participating +servers. ## Background [Global rate limiting](https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/other_features/global_rate_limiting) allows mesh users to manage fair consumption of their services and prevent -misbehaving clients from overloading the services. *Global* rate limiting -implies that there must be a single source of truth for the state of a finite -shared resource; a Rate Limiting Service. As of 2024-01-24, Envoy describes two -types of such service: - -1. [Rate Limiting Service (RLS)](https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/other_features/global_rate_limiting#per-connection-or-per-http-request-rate-limiting): - synchronous (blocking) per-HTTP-request rate limit check. Best suited for - precise cases, where even a single request over the limit must be throttled. -2. [Rate Limiting Quota Service (RLQS)](https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/other_features/global_rate_limiting#quota-based-rate-limiting): - asynchronous (non-blocking) quota-based check, with periodic load reports of - bucketed HTTP requests. Best suited for high-request-per-second applications, - where a certain margin of error is acceptable as long as expected average QPS - is achieved. - -gRPC only intends to support quota-based global rate limiting, Rate Limiting -Quota Service (RLQS), which is described in this document. - -This proposal details how gRPC will add support -for [Rate Limit Quota xDS HTTP filter][rate_limit_quota_filter] -configured via [xDS HTTP Filters][A39] to [xDS-Enabled gRPC servers][A36]. It -will cover four major parts needed for language-specific gRPC implementations: - -1. Support missing xDS types needed to parse the Rate Limit Quota Filter config. -2. Implement the client side of RLQS protocol (RLQS - Client): `StreamRateLimitQuotas.StreamRateLimitQuotas`. It will establish - bidirectional gRPC stream to the remote [Rate Limit Quota Service][rlqs]. - 4. Send and receive updates. -3. Implement a HTTP filter (filter in C-core, later called - "the Interceptor" for simplicity) that requests. -5. Fault modes - - -/// per-host and per-route overrides +misbehaving clients from overloading the services. We will +implement [quota-based rate limiting](https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/other_features/global_rate_limiting#quota-based-rate-limiting), +where rate-limiting decisions are asynchronously offloaded +to [Rate Limiting Quota Service][rlqs_proto] (RLQS). Requests are grouped into +buckets based on their metadata, and gRPC servers periodically report bucket +usages. RLQS aggregates the data from different gRPC servers, and fairly +distributes the quota among them. This approach is best suited for +high-request-per-second applications, where a certain margin of error is +acceptable as long as expected average QPS is achieved. + +To support RLQS, we'll need to implement several other xDS-related features, +which are covered in the proposal: + +1. xDS Control Plane will provide RLQS connection details + in [the filter config][rlqs_filter_proto] via [`GrpcService`] message. +2. Quota assignments will be configured + via [`TokenBucket`](https://www.envoyproxy.io/docs/envoy/latest/api-v3/type/v3/token_bucket.proto) + message. +3. RPCs will be matched into buckets using [Unified Matcher API]. +4. One of the matching mechanisms will be [CEL](https://cel.dev/) (Common + Expression Language). +5. RLQS filter state will persist across LDS/RDS updates using cache retention + mechanism similar to the one implemented for [gRFC A83]. ### Related Proposals: -* [A36: xDS-Enabled Servers][A36] -* [A39: xDS HTTP Filter Support][A39] +* [A27: xDS-Based Global Load Balancing][gRFC A27] +* [A36: xDS-Enabled Servers][gRFC A36] +* [A39: xDS HTTP Filter Support][gRFC A39] +* [A41: xDS RBAC Support][gRFC A41] +* [A83: xDS GCP Authentication Filter][gRFC A83] + +[gRFC A27]: A27-xds-global-load-balancing.md +[gRFC A41]: A41-xds-rbac.md +[gRFC A36]: A36-xds-for-servers.md +[gRFC A39]: A39-xds-http-filters.md +[gRFC A83]: A83-xds-gcp-authn-filter.md -[A36]: A36-xds-for-servers.md +[`GrpcService`]: https://www.envoyproxy.io/docs/envoy/latest/api-v3/config/core/v3/grpc_service.proto +[`GrpcService.GoogleGrpc`]: https://www.envoyproxy.io/docs/envoy/latest/api-v3/config/core/v3/grpc_service.proto#envoy-v3-api-msg-config-core-v3-grpcservice-googlegrpc +[Unified Matcher API]: https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/advanced/matching/matching_api.html +[Envoy CEL environment]: https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/advanced/attributes -[A39]: A39-xds-http-filters.md +[rlqs_proto]: https://www.envoyproxy.io/docs/envoy/latest/api-v3/service/rate_limit_quota/v3/rlqs.proto.html +[rlqs_filter_proto]: https://www.envoyproxy.io/docs/envoy/latest/api-v3/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto ## Proposal -### xDS types support -#### Unified Matchers -add a note that we'll be using this in the future -list specific matcher -start from the assumption we implement both - -#### io.envoyproxy.envoy.config.core.v3.GrpcService.GoogleGrpc -consider credential could be a security/ 3p trust issue. -TODO(sergiitk): meeting with leads about this -#### Canonical CEL - -TODO(sergiitk): A precise statement of the proposed change. - -#### Simplified RLQS flow happy-path: - -// TODO(sergiitk): Insert graphics - -1. [Rate Limit Quota xDS HTTP filter][rate_limit_quota_filter] - is configured on a gRPC Server via an LDS update in the HTTP connection - manager filter chain. -2. gRPC Server establishes a gRPC channel to the RLQS as specified - in [RateLimitQuotaFilterConfig.rlqs_server]. -3. gRPC Server parses [RateLimitQuotaFilterConfig.bucket_matchers] tree and - caches it in the filter state. -4. gRPC Server installs a Server Interceptor (filter in C-core, later called - "the Interceptor" for simplicity) // TODO(sergiitk): HTTP filter -5. Once a request is intercepted by the Interceptor: - - The request is matched into a Bucket by evaluating the `bucket_matchers` - tree against the request attributes. - - If the Bucket doesn't exist, gRPC server immediately sends the - initial `BucketQuotaUsage` report to the RLQS server. The request is - throttled according - to [RateLimitQuotaBucketSettings.no_assignment_behavior]. - - If the Bucket exists, the request is throttled according to Bucket's quota - assignment. Bucket's `num_requests_allowed` // TODO(sergiitk): explain what's throttle - or `num_requests_denied` request counter is increased by one. -6. For all existing buckets, a `BucketQuotaUsage` report is sent - every [RateLimitQuotaBucketSettings.reporting_interval] - to `RateLimitQuotaService.StreamRateLimitQuotas`. -7. RLQS may send Bucket quota assignments via `RateLimitQuotaResponse` at any - time. Once received, it must the quota must be applied to a Bucket with - matching [bucket_id]. - - -// TODO(sergiitk): think about it from the perspective what events needs to be handled: -1. parsing config -2. getting an update -can be described with pseudo-code +### RLQS Components Overview + +The diagram below shows the conceptual components of the RLQS Filter. Note that +the actual implementation may vary depending on the language. + +```mermaid +graph TD +%% RLQS Components Flowchart v7 + +%% == nodes == + rlqs_filter(RLQS HTTP Filter) + rlqs_cache(RLQS Cache) + rlqs_filter_state(RLQS Filter State) + rlqs_client(RLQS Client) + rlqs_server[(RLQS Server)] + rlqs_bucket_cache(RLQS Bucket Cache) + report_timers(Report Timers) + matcher_tree(Matcher Tree) + rlqs_bucket(RLQS Bucket) + rpc_handler("Filter's onClientCall handler") + request{{RPC}} +%% == edges == + rlqs_filter -- " Get RLQS Filter State
per unique config " --> rlqs_cache -- " getOrCreate(config) " --> rlqs_filter_state + rlqs_filter -- " Pass RLQS Filter State
for the route " --> rpc_handler -- " rateLimit(call) " --> rlqs_filter_state + request --> rpc_handler + rlqs_filter_state --o matcher_tree & report_timers + rlqs_filter_state -- sendUsageReports --> rlqs_client + rlqs_filter_state -- CRUD --> rlqs_bucket_cache + rlqs_client -- onBucketsUpdate --> rlqs_filter_state + rlqs_client <-. gRPC Stream .-> rlqs_server + rlqs_bucket_cache -- " getOrCreate(bucketId)
Atomic Updates " --> rlqs_bucket + style request stroke: RoyalBlue, stroke-width: 2px; + linkStyle 3,4 stroke: RoyalBlue, stroke-width: 2px; +``` + +##### RLQS HTTP Filter + +The filter parses the config, combines LDS filter config with RDS overrides, and +generates the `onClientCall` handlers (aka interceptors in Java and Go, and +filters in C++). + +##### RLQS Cache + +RLQS Cache persists across LDS/RDS updates. It maps unique filter configs to +RLQS Filter State instances, and provides the thread safety for creating and +accessing them. Each unique filter config generates a unique RLQS Filter state, +a 1:1 mapping. + +##### RLQS Filter State + +RLQS Filter State contains the business logic for rate limiting, and the current +state of rate limit assignments per bucket. RLQS Filter State is what's passed +to the `onCallHandler`. It exposes the public "`rateLimit()`" method, which +takes request metadata as an argument. + +##### Matching + +RLQS Filter State evaluates the metadata against the matcher tree to match the +request into a bucket. The Bucket holds the Rate Limit Quota assigned by the +RLQS server (f.e. 100 requests per minute), and aggregates the number of +requests it allowed/denied. This information is used to make the rate limiting +decision. + +##### Reporting + +The aggregated number of requests is reported to the RLQS server at configured +intervals. The report action is triggered by the Report Timers. RLQS Client +manages a gRPC stream to the RLQS server. It's used by the filter state to send +periodic bucket usage reports, and to receive new rate limit quota assignments +to the buckets. + +### Connecting to RLQS Control Plane + +xDS Control Plane provides RLQS connection details in [`GrpcService`] message ( +already supported by Envoy). `GrpcService` supports two modes: + +1. `GrpcService.EnvoyGrpc`, Envoy's minimal custom gRPC client implementation. +2. [`GrpcService.GoogleGrpc`], regular gRPC-cpp client. + +For obvious reasons, we'll only support the `GoogleGrpc` mode. + +#### Security Considerations + +In [`GrpcService.GoogleGrpc`], xDS Control Plane provides the target +URI, `channel_credentials`, and `call_credentials`. If the xDS Control Plane is +compromised, the attacker could configure the xDS clients to talk to other +malicious Control Plane, leading to such potential exploits as: + +1. Leaking customer's Application Default Credentials OAuth token. +2. Causing MalOut/DDoS by sending bad data from the compromised RLQS (f.e. set + rate limit policy to `ALLOW_ALL`/`DENY_ALL`). + +To prevent that, we'll introduce the allow-list to the bootstrap file introduced +in [gRFC A27]. This allow-list will be a map from the fully-qualified server +target URI to an object containing channel credentials to use. + +```javascript +// The allowlist of Control Planes allowed to be configured via xDS. +"allowed_grpc_services": { + // The key is fully-qualified server URI. + "dns:///xds.example.org:443": { + // The value is an object containing "channel_creds". + "channel_creds": [ + // The format is identical to xds_servers.channel_creds. + { + "type": "string containing channel cred type", + "config": "optional JSON object containing config for the type" + } + ] + } +} +``` + +When xDS Control Plane configures connection to another control plane +via [`GrpcService.GoogleGrpc`] message, we'll inspect the allow-list for the +matching target URI. + +1. If target URI is not present, we don't create the connection to the requested + Control Plane, and NACK the xDS resource. +2. If target URI is present, we create the connection to the requested Control + Plane using the channel credentials provided in the bootstrap file. Transport + security configuration provided by the TD is ignored. + +> [!IMPORTANT] +> This solution is not specific to RLQS, and should be used with any +> other Control Planes configured via [`GrpcService`] message. + +### Unified Matcher API + +RPCs will be matched into buckets using [Unified Matcher API] — an adaptable +framework that can be used in any xDS component that needs matching features. + +Envoy provides two syntactically equivalent Unified Matcher +definitions: [`envoy.config.common.matcher.v3.Matcher`](https://github.com/envoyproxy/envoy/blob/e3da7ebb16ad01c2ac7662758a75dba5cdc024ce/api/envoy/config/common/matcher/v3/matcher.proto) +and [`xds.type.matcher.v3.Matcher`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto). +We will only support the latter, which is the preferred version for all new APIs +using Unified Matcher. + +For RLQS, Unified Matcher tree will be provided in the filter config. Evaluating +the tree against RPC metadata will yield `RateLimitQuotaBucketSettings`, which +contains the information needed to associate the RPC with `bucket_id` and the +default rate limiting configuration. + +In this iteration the following Unified Mather extensions will be supported: + +1. Inputs: + 1. [`HttpRequestHeaderMatchInput`](https://www.envoyproxy.io/docs/envoy/latest/api-v3/type/matcher/v3/http_inputs.proto#type-matcher-v3-httprequestheadermatchinput) + 2. [`HttpAttributesCelMatchInput`](https://www.envoyproxy.io/docs/envoy/latest/xds/type/matcher/v3/http_inputs.proto#envoy-v3-api-msg-xds-type-matcher-v3-httpattributescelmatchinput) +2. Custom Matchers: + 1. [`CelMatcher`](https://www.envoyproxy.io/docs/envoy/latest/xds/type/matcher/v3/cel.proto.html) + +### CEL Integration + +We will support request metadata matching via CEL expressions. Only Canonical +CEL and only checked expressions will be supported (`cel.expr.CheckedExpr`). + +CEL evaluation environment is a set of available variables and extension +functions in a CEL program. We will match [Envoy CEL environment]. + +#### Supported CEL Functions + +Similar to Envoy, we will +support [standard CEL functions](https://github.com/google/cel-spec/blob/c629b2be086ed6b4c44ef4975e56945f66560677/doc/langdef.md#standard-definitions) +except comprehension-style macros. + +| CEL Method | Description | +|----------------------------------------------------|-----------------------------------------------------------------------------------------------| +| `size(x)` | Returns the length of a container x (string, bytes, list, map). | +| `x.matches(y)` | Returns true if the string x is partially matched by the specified [RE2][RE2_wiki] pattern y. | +| `x.contains(y)` | Returns true if the string x contains the substring y. | +| `x.startsWith(y)` | Returns true if the string x begins with the substring y. | +| `x.endsWith(y)` | Returns true if the string x ends with the substring y. | +| `timestamp(x)`, `timestamp.get*(x)`, `duration` | Date/time functions. | +| `in`, `[]` | Map/list indexing. | +| `has(m.x)` | (macro) Returns true if the map `m` has the string `"x"` as a key. | +| `int`, `uint`, `double`, `string`, `bytes`, `bool` | Conversions and identities. | +| `==`, `!=`, `>`, `<`, `<=`, `>=` | Comparisons. | +| `or`, `&&`, `+`, `-`, `/`, `*`, `%`, `!` | Basic functions. | + +[RE2_wiki]: https://en.wikipedia.org/wiki/RE2_(software) + +#### Supported CEL Variables + +For RLQS, only the `request` variable is supported in CEL expressions. We will +adapt [Envoy's Request Attributes](https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/advanced/attributes#request-attributes) +for gRPC. + +| Attribute | Type | gRPC source | Envoy Description | +|---------------------|-----------------------|------------------------------|-------------------------------------------------------------| +| `request.path` | `string` | Full method name1 | The path portion of the URL. | +| `request.url_path` | `string` | Same as `request.path` | The path portion of the URL without the query string. | +| `request.host` | `string` | Authority2 | The host portion of the URL. | +| `request.scheme` | `string` | Not set | The scheme portion of the URL. | +| `request.method` | `string` | `POST`3 | Request method. | +| `request.headers` | `map` | `metadata`4 | All request headers indexed by the lower-cased header name. | +| `request.referer` | `string` | `metadata["referer"]` | Referer request header. | +| `request.useragent` | `string` | `metadata["user-agent"]` | User agent request header. | +| `request.time` | `timestamp` | Not set | Time of the first byte received. | +| `request.id` | `string` | `metadata["x-request-id"]` | Request ID corresponding to `x-request-id` header value | +| `request.protocol` | `string` | Not set | Request protocol. | +| `request.query` | `string` | `""` | The query portion of the URL. | + +##### Footnotes + +**1 `request.path`** + +* CPP: `metadata[":path"]` +* Go: `grpc.Method(ctx)` +* Java: `"/" + serverCall.getMethodDescriptor().getFullMethodName()` + +**2 `request.host`** + +* CPP, Go: `metadata[":authority"]` +* Java: `serverCall.getAuthority()` + +**3 `request.method`**\ +Hard-coded to `"POST"` if unavailable and a code audit confirms the server +denies requests for all other method types. + +**4 `request.headers`**\ +As defined in [gRFC A41], "header" field. + +##### Implementation + +For performance reasons, CEL variables should be resolved on demand. CEL Runtime +provides the different variable resolving approaches based on the language: + +* CPP: [`BaseActivation::FindValue()`](https://github.com/google/cel-cpp/blob/9310c4910e598362695930f0e11b7f278f714755/eval/public/base_activation.h#L35) +* Go: [`Activation.ResolveName(string)`](https://github.com/google/cel-go/blob/3f12ecad39e2eb662bcd82b6391cfd0cb4cb1c5e/interpreter/activation.go#L30) +* Java: [`CelVariableResolver`](https://javadoc.io/doc/dev.cel/runtime/0.6.0/dev/cel/runtime/CelVariableResolver.html) + +### Persistent Filter Cache + +RLQS Filter State holds the bucket usage data, report timers and the +bidirectional stream to the RLQS server. To prevent the loss of state across +LDS/RDS updates, RLQS filter will require a cache retention mechanism similar to +the one implemented for [gRFC A83]. + +The scope of each RLQS Filter Cache instance will be per server instance (same +scope as the filter chain) and per filter name. + +RLQS implementations will provide a mechanism for new instances of the filter to +retain the cache from previous instances. There may be multiple instances of the +RLQS Filter State, each one mapped to a unique filter config generated from LDS +config, and RDS overrides. Consider the following example that demonstrates the +lifecycle of RLQS Filter Cache. + +```mermaid +--- +config: + sequence: + showSequenceNumbers: true + height: 46 + diagramMarginX: 40 + diagramMarginY: 40 +--- +sequenceDiagram + +%% gRFC: RLQS Filter Cache Lifecycle v1.1 + participant xds as Control Plane + participant filter as RLQS HTTP Filter + participant cache as RLQS Cache + participant e1 as RlqsFilterState(c1) + participant e2 as RlqsFilterState(c2) + +# Notes + Note right of xds: r1-4: routes
c1-2: unique filter configs +%% LDS 1 + xds ->> filter: LDS1
RLQS{r1=c1, r2=c2, r3=c2} + filter ->> cache: r1: getOrCreate(c1) + cache ->>+ e1: new RlqsFilterState(c1) + filter ->> cache: r2: getOrCreate(c2) + cache ->>+ e2: new RlqsFilterState(c2) + filter ->> cache: r3: getOrCreate(c2) + Note over filter: r1: RlqsFilterState(c1)
r2: RlqsFilterState(c2)
r3: RlqsFilterState(c2) +%% RDS 1 + xds ->> filter: RDS1
RLQS{r1=c2} + filter ->> cache: r1: getOrCreate(c2) + filter ->> cache: shutdownFilterState(c1) + cache -x e1: RlqsFilterState(c1).shutdown() + deactivate e1 + Note over filter: r1: RlqsFilterState(c2)
r2: RlqsFilterState(c2)
r3: RlqsFilterState(c2) +%% LDS 2 + xds ->> filter: LDS2
RLQS{r3=c2, r4=c2} + filter ->> cache: r4: getOrCreate(c2) + Note over filter: r3: RlqsFilterState(c2)
r4: RlqsFilterState(c2) +%% End + deactivate e2 +``` + +**LDS 1** + +In this example, the RLQS filter is configured for three routes: `r1`, `r2`, +and `r3`. Each unique config generates a unique RLQS Filter +State: `RlqsFilterState(c1)` for the config `c1`, and `RlqsFilterState(c2)` for +the config `c2`. After processing the first LDS update, we've generated +onCallHandlers for three routes: + +1. `r1`, referencing `RlqsFilterState(c1)`. +2. `r2`, referencing `RlqsFilterState(c2)`. +3. `r3`, also referencing `RlqsFilterState(c2)`. + +**RDS 1** + +RDS 1 updates RLQS config for the route `r1` so it's identical to config `c2`. +We retrieve `RlqsFilterState(c2)` from the RLQS Cache and generate new +onCallHandlers for route `r2`. `RlqsFilterState(c1)` is no longer referenced by +any onCallHandler, and can be destroyed with all associated resources. + +**LDS 2** + +LDS 2 update removes `r1` and `r2`, and adds new route r4 with the config +identical to `c2`. While onCallHandlers for routes `r1` and `r2` are +destroyed, `RlqsFilterState(c2)` is still used by two onCallHandlers, so it's +preserved in RLQS Cache. + +##### Future considerations + +With this proposal, the filter state is lost if change is made to the filter +config, including updates to inconsequential fields such as deny response +status. Additional logic can be introduced to handle updates to such fields +while preserving the filter state. + +### Multithreading + +There are several mutex-synchronized operations executed in +latency-sensitive `onCallHandler`: + +1. Inserting/reading a bucket from the bucket cache using `bucket_id`. Note + that `bucket_id` is represented as a `Map`, which may + introduce complexities in efficient cache sharding for certain programming + languages. +2. Incrementing `num_request_allowed`/`num_request_denied` bucket counters. + +Each gRPC implementation needs to consider what synchronization primitives are +available in their language to minimize the thread lock time. + +### Code Samples + +> [!NOTE] +> Not a reference implementation. Only for flow illustration purposes. + +#### On Call Handler +```java +final RlqsFilterState filterState = rlqsCache.getOrCreateFilterState(config); + +return new ServerInterceptor() { + @Override + public Listener interceptCall( + ServerCall call, + Metadata headers, ServerCallHandler next) { + // TODO: handle filter_enabled and filter_enforced + + // RlqsClient matches the request into a bucket, + // and returns the rate limiting result. + RlqsRateLimitResult result = + filterState.rateLimit(HttpMatchInput.create(headers, call)); + + // Allowed. + if (result.isAllowed()) { + return next.startCall(call, headers); + } + // Denied: fail the call with given Status. + call.close( + result.denyResponse().status(), + result.denyResponse().headersToAdd()); + return new ServerCall.Listener(){}; + } +}; +``` + +#### Bucket Matching +```java +public RlqsRateLimitResult rateLimit(HttpMatchInput input) { + // Perform request matching. The result is RlqsBucketSettings. + RlqsBucketSettings bucketSettings = bucketMatchers.match(input); + // BucketId may be dynamic (f.e. based on headers). + RlqsBucketId bucketId = bucketSettings.bucketIdForRequest(input); + + RlqsBucket bucket = bucketCache.getOrCreate(bucketId, bucketSettings, this::onNewBucket); + return bucket.rateLimit(); +} + +private void onNewBucket(RlqsBucket newBucket) { + // The report for the first RPC is sent immediately. + scheduleImmediateReport(newBucket); + registerReportTimer(newBucket.getReportingInterval()); +} +``` + ### Temporary environment variable protection -During initial development, this feature will be enabled via the -`GRPC_EXPERIMENTAL_XDS_ENABLE_RLQS` environment variable. This environment +During initial development, this feature will be enabled via +the `GRPC_EXPERIMENTAL_XDS_ENABLE_RLQS` environment variable. This environment variable protection will be removed once the feature has proven stable. ## Rationale -TODO(sergiitk): A discussion of alternate approaches and the trade offs, -advantages, and disadvantages of the specified approach. - -## Implementation - -The initial implementation will be in Java.\ -C-core, Python, and Go will follow after that. +#### Alternative protocol -## Open issues (if applicable) +[Rate Limiting Service (RLS)](https://www.envoyproxy.io/docs/envoy/latest/configuration/http/http_filters/rate_limit_filter) +is another Global Rate Limiting solution supported by Envoy. While it's best +suited for precise cases, where even a single request over the limit must be +throttled, this approach performs synchronous (blocking) per-HTTP-request rate +limit check. -TODO(sergiitk): A discussion of issues relating to this proposal for which the -author does not know the solution. This section may be omitted if there are -none. +#### xDS-configured Control Plane Trust - +The problem with a compromised xDS Control Plane configuring a connection to a +malicious RLQS server may be solved holistically by signing xDS messages +cryptographically. This feature would solve multiple problems in the same class, +but it's out-of-scope of RLQS. -[rate_limit_quota_filter]: https://www.envoyproxy.io/docs/envoy/latest/configuration/http/http_filters/rate_limit_quota_filter +The proposed solution with the change to the bootstrap file was designed to be +compatible with such future protocol extension. -[RateLimitQuotaBucketSettings.domain]: https://www.envoyproxy.io/docs/envoy/latest/api-v3/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto.html#envoy-v3-api-field-extensions-filters-http-rate-limit-quota-v3-ratelimitquotafilterconfig-domain - -[RateLimitQuotaBucketSettings.no_assignment_behavior]: https://www.envoyproxy.io/docs/envoy/latest/api-v3/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#envoy-v3-api-field-extensions-filters-http-rate-limit-quota-v3-ratelimitquotabucketsettings-no-assignment-behavior - -[RateLimitQuotaBucketSettings.reporting_interval]: https://www.envoyproxy.io/docs/envoy/latest/api-v3/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#envoy-v3-api-field-extensions-filters-http-rate-limit-quota-v3-ratelimitquotabucketsettings-reporting-interval - -[RateLimitQuotaFilterConfig.rlqs_server]: https://www.envoyproxy.io/docs/envoy/latest/api-v3/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#envoy-v3-api-field-extensions-filters-http-rate-limit-quota-v3-ratelimitquotafilterconfig-rlqs-server - -[RateLimitQuotaFilterConfig.bucket_matchers]: https://www.envoyproxy.io/docs/envoy/latest/api-v3/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#envoy-v3-api-field-extensions-filters-http-rate-limit-quota-v3-ratelimitquotafilterconfig-bucket-matchers - -[bucket_id]: https://www.envoyproxy.io/docs/envoy/latest/api-v3/service/rate_limit_quota/v3/rlqs.proto#envoy-v3-api-field-service-rate-limit-quota-v3-ratelimitquotaresponse-bucketaction-bucket-id +## Implementation -[rlqs]: https://www.envoyproxy.io/docs/envoy/latest/api-v3/service/rate_limit_quota/v3/rlqs.proto +* The initial implementation will be in Java. +* C-core, Python, Go are TBD. From 60260c0d88068a52b604c54e663d6b9fb245f247 Mon Sep 17 00:00:00 2001 From: Sergii Tkachenko Date: Mon, 11 Nov 2024 15:54:14 -0800 Subject: [PATCH 05/53] Add boxes around components in RLQS Components Flowchart --- A77-xds-rate-limiting-rlqs.md | 34 +++++++++++++++++++++------------- 1 file changed, 21 insertions(+), 13 deletions(-) diff --git a/A77-xds-rate-limiting-rlqs.md b/A77-xds-rate-limiting-rlqs.md index 99880eeda..93e186d4a 100644 --- a/A77-xds-rate-limiting-rlqs.md +++ b/A77-xds-rate-limiting-rlqs.md @@ -74,20 +74,28 @@ the actual implementation may vary depending on the language. ```mermaid graph TD -%% RLQS Components Flowchart v7 +%% RLQS Components Flowchart v8 %% == nodes == - rlqs_filter(RLQS HTTP Filter) - rlqs_cache(RLQS Cache) - rlqs_filter_state(RLQS Filter State) - rlqs_client(RLQS Client) - rlqs_server[(RLQS Server)] - rlqs_bucket_cache(RLQS Bucket Cache) - report_timers(Report Timers) - matcher_tree(Matcher Tree) - rlqs_bucket(RLQS Bucket) - rpc_handler("Filter's onClientCall handler") - request{{RPC}} + subgraph grpc_client_box [gRPC Client] + request{{RPC}} + end + subgraph rlqs_server_box [RLQS Server] + rlqs[(RLQS)] + end + subgraph grpc_server_box [gRPC Server] + rlqs_filter(RLQS HTTP Filter) + rlqs_cache(RLQS Cache) + subgraph rlqs_filter_state_box [RLQS Filter State] + rlqs_client(RLQS Client) + rlqs_filter_state(RLQS Filter State) + report_timers(Report Timers) + matcher_tree(Matcher Tree) + rlqs_bucket_cache(RLQS Bucket Cache) + rlqs_bucket(RLQS Bucket) + end + rpc_handler("Filter's onClientCall handler") + end %% == edges == rlqs_filter -- " Get RLQS Filter State
per unique config " --> rlqs_cache -- " getOrCreate(config) " --> rlqs_filter_state rlqs_filter -- " Pass RLQS Filter State
for the route " --> rpc_handler -- " rateLimit(call) " --> rlqs_filter_state @@ -96,8 +104,8 @@ graph TD rlqs_filter_state -- sendUsageReports --> rlqs_client rlqs_filter_state -- CRUD --> rlqs_bucket_cache rlqs_client -- onBucketsUpdate --> rlqs_filter_state - rlqs_client <-. gRPC Stream .-> rlqs_server rlqs_bucket_cache -- " getOrCreate(bucketId)
Atomic Updates " --> rlqs_bucket + rlqs_client <-. gRPC Stream .-> rlqs style request stroke: RoyalBlue, stroke-width: 2px; linkStyle 3,4 stroke: RoyalBlue, stroke-width: 2px; ``` From 0647faa2b7b1e1aa2308cbb61e1b3f0c74b8f699 Mon Sep 17 00:00:00 2001 From: Sergii Tkachenko Date: Mon, 11 Nov 2024 16:21:38 -0800 Subject: [PATCH 06/53] minor: switch rlqs components flowchart theme --- A77-xds-rate-limiting-rlqs.md | 6 ++++++ 1 file changed, 6 insertions(+) diff --git a/A77-xds-rate-limiting-rlqs.md b/A77-xds-rate-limiting-rlqs.md index 93e186d4a..2963e1e73 100644 --- a/A77-xds-rate-limiting-rlqs.md +++ b/A77-xds-rate-limiting-rlqs.md @@ -73,6 +73,12 @@ The diagram below shows the conceptual components of the RLQS Filter. Note that the actual implementation may vary depending on the language. ```mermaid +--- +config: + theme: base + themeVariables: + clusterBorder: "#777" +--- graph TD %% RLQS Components Flowchart v8 From 2b9bbc054a7f584099f5512c03e0ea33c99734b6 Mon Sep 17 00:00:00 2001 From: Sergii Tkachenko Date: Mon, 11 Nov 2024 16:32:53 -0800 Subject: [PATCH 07/53] minor: improve components flowchart link visibility in dark mode --- A77-xds-rate-limiting-rlqs.md | 1 + 1 file changed, 1 insertion(+) diff --git a/A77-xds-rate-limiting-rlqs.md b/A77-xds-rate-limiting-rlqs.md index 2963e1e73..d2747790b 100644 --- a/A77-xds-rate-limiting-rlqs.md +++ b/A77-xds-rate-limiting-rlqs.md @@ -114,6 +114,7 @@ graph TD rlqs_client <-. gRPC Stream .-> rlqs style request stroke: RoyalBlue, stroke-width: 2px; linkStyle 3,4 stroke: RoyalBlue, stroke-width: 2px; + linkStyle 11 stroke: Teal, stroke-width: 3px; ``` ##### RLQS HTTP Filter From 93ff79902fadae2cc367e42f8fc9ec5cdf868642 Mon Sep 17 00:00:00 2001 From: Sergii Tkachenko Date: Tue, 22 Apr 2025 12:24:37 -0700 Subject: [PATCH 08/53] feedback wip pre-print --- A77-xds-rate-limiting-rlqs.md | 40 ++++++++++++++++------------------- 1 file changed, 18 insertions(+), 22 deletions(-) diff --git a/A77-xds-rate-limiting-rlqs.md b/A77-xds-rate-limiting-rlqs.md index d2747790b..d269cd911 100644 --- a/A77-xds-rate-limiting-rlqs.md +++ b/A77-xds-rate-limiting-rlqs.md @@ -156,7 +156,7 @@ to the buckets. ### Connecting to RLQS Control Plane xDS Control Plane provides RLQS connection details in [`GrpcService`] message ( -already supported by Envoy). `GrpcService` supports two modes: +already supported by Envoy). `GrpcService` supports two modes: 1. `GrpcService.EnvoyGrpc`, Envoy's minimal custom gRPC client implementation. 2. [`GrpcService.GoogleGrpc`], regular gRPC-cpp client. @@ -165,20 +165,17 @@ For obvious reasons, we'll only support the `GoogleGrpc` mode. #### Security Considerations -In [`GrpcService.GoogleGrpc`], xDS Control Plane provides the target -URI, `channel_credentials`, and `call_credentials`. If the xDS Control Plane is -compromised, the attacker could configure the xDS clients to talk to other -malicious Control Plane, leading to such potential exploits as: +In [`GrpcService.GoogleGrpc`], the xDS Control Plane provides the target URI, +`channel_credentials`, and `call_credentials`. If the xDS Control Plane is +compromised, an attacker could configure the xDS clients to talk to other +malicious Control Planes. This could lead to potential exploits such as leaking +customer's Application Default Credentials OAuth token. -1. Leaking customer's Application Default Credentials OAuth token. -2. Causing MalOut/DDoS by sending bad data from the compromised RLQS (f.e. set - rate limit policy to `ALLOW_ALL`/`DENY_ALL`). +To prevent that, we'll introduce an allow-list to the bootstrap file introduced +in [gRFC A27]. This allow-list will be a map from a fully-qualified server +target URI to an object containing channel credentials to use for that target. -To prevent that, we'll introduce the allow-list to the bootstrap file introduced -in [gRFC A27]. This allow-list will be a map from the fully-qualified server -target URI to an object containing channel credentials to use. - -```javascript +```js // The allowlist of Control Planes allowed to be configured via xDS. "allowed_grpc_services": { // The key is fully-qualified server URI. @@ -186,7 +183,7 @@ target URI to an object containing channel credentials to use. // The value is an object containing "channel_creds". "channel_creds": [ // The format is identical to xds_servers.channel_creds. - { + { "type": "string containing channel cred type", "config": "optional JSON object containing config for the type" } @@ -378,7 +375,7 @@ In this example, the RLQS filter is configured for three routes: `r1`, `r2`, and `r3`. Each unique config generates a unique RLQS Filter State: `RlqsFilterState(c1)` for the config `c1`, and `RlqsFilterState(c2)` for the config `c2`. After processing the first LDS update, we've generated -onCallHandlers for three routes: +`onCallHandlers` for three routes: 1. `r1`, referencing `RlqsFilterState(c1)`. 2. `r2`, referencing `RlqsFilterState(c2)`. @@ -388,22 +385,22 @@ onCallHandlers for three routes: RDS 1 updates RLQS config for the route `r1` so it's identical to config `c2`. We retrieve `RlqsFilterState(c2)` from the RLQS Cache and generate new -onCallHandlers for route `r2`. `RlqsFilterState(c1)` is no longer referenced by -any onCallHandler, and can be destroyed with all associated resources. +`onCallHandlers` for route `r2`. `RlqsFilterState(c1)` is no longer referenced +by any `onCallHandler`, and can be destroyed with all associated resources. **LDS 2** LDS 2 update removes `r1` and `r2`, and adds new route r4 with the config -identical to `c2`. While onCallHandlers for routes `r1` and `r2` are -destroyed, `RlqsFilterState(c2)` is still used by two onCallHandlers, so it's +identical to `c2`. While `onCallHandlers` for routes `r1` and `r2` are +destroyed, `RlqsFilterState(c2)` is still used by two `onCallHandlers`, so it's preserved in RLQS Cache. ##### Future considerations With this proposal, the filter state is lost if change is made to the filter config, including updates to inconsequential fields such as deny response -status. Additional logic can be introduced to handle updates to such fields -while preserving the filter state. +status. If this becomes a problem, additional logic can be introduced to handle +updates to such fields while preserving the filter state. ### Multithreading @@ -472,7 +469,6 @@ private void onNewBucket(RlqsBucket newBucket) { } ``` - ### Temporary environment variable protection During initial development, this feature will be enabled via From f05db3179344ed5693239fec88d2f1d25c8ba558 Mon Sep 17 00:00:00 2001 From: Sergii Tkachenko Date: Tue, 22 Apr 2025 15:45:43 -0700 Subject: [PATCH 09/53] continue addressing feedback --- A77-xds-rate-limiting-rlqs.md | 246 ++++++++++++++++++++++------------ 1 file changed, 162 insertions(+), 84 deletions(-) diff --git a/A77-xds-rate-limiting-rlqs.md b/A77-xds-rate-limiting-rlqs.md index d269cd911..2e833904c 100644 --- a/A77-xds-rate-limiting-rlqs.md +++ b/A77-xds-rate-limiting-rlqs.md @@ -117,35 +117,177 @@ graph TD linkStyle 11 stroke: Teal, stroke-width: 3px; ``` -##### RLQS HTTP Filter +#### RLQS HTTP Filter The filter parses the config, combines LDS filter config with RDS overrides, and generates the `onClientCall` handlers (aka interceptors in Java and Go, and filters in C++). -##### RLQS Cache +#### RLQS HTTP Filter: Channel Level + +#### RLQS HTTP Filter: Call Level + +##### Code Sample: On Call Handler + +> [!NOTE] +> Not a reference implementation. Only for flow illustration purposes. + +```java +final RlqsFilterState filterState = rlqsCache.getOrCreateFilterState(config); + +return new ServerInterceptor() { + @Override + public Listener interceptCall( + ServerCall call, + Metadata headers, ServerCallHandler next) { + // TODO: handle filter_enabled and filter_enforced + + // RlqsClient matches the request into a bucket, + // and returns the rate limiting result. + RlqsRateLimitResult result = + filterState.rateLimit(HttpMatchInput.create(headers, call)); + + // Allowed. + if (result.isAllowed()) { + return next.startCall(call, headers); + } + // Denied: fail the call with given Status. + call.close( + result.denyResponse().status(), + result.denyResponse().headersToAdd()); + return new ServerCall.Listener(){}; + } +}; +``` + +#### RLQS Cache RLQS Cache persists across LDS/RDS updates. It maps unique filter configs to RLQS Filter State instances, and provides the thread safety for creating and accessing them. Each unique filter config generates a unique RLQS Filter state, a 1:1 mapping. -##### RLQS Filter State - -RLQS Filter State contains the business logic for rate limiting, and the current +#### RLQS Filter State + +In order to retain filter state across LDS/RDS updates, the actual logic for the +RLQS filter will be moved into a separate object called RLQS Filter State, which +will be stored in the persistent filter state mechanism described in [gRFC A83]. +The key in the persistent filter state will be the RLQS xDS HTTP filter config, +which ensures that two RLQS filter instances with the same config will share +filter state but two RLQS filter instances with different configs will each have +their own filter state. + +RLQS Filter State object will include the following data members: + +- Matcher Tree: From filter config, initialized at instantiation, constant. + Used to identify the bucket map entry for each data plane RPC. +- Bucket Map: Accessed on each data plane RPC, when we get a response from the + RLQS server, and when report timers fire. +- RLQS Client: Accessed when we get the first data plane RPC for a given + bucket and when a report timer fires. Notifies RLQS Filter State of + responses received from the RLQS server. +- Report Timer: Created/modified when we get an RLQS response for a given + bucket, or when a previous timer fires. + + + +##### Code Sample: Bucket Matching + +> [!NOTE] +> Not a reference implementation. Only for flow illustration purposes. + +```java +public RlqsRateLimitResult rateLimit(HttpMatchInput input) { + // Perform request matching. The result is RlqsBucketSettings. + RlqsBucketSettings bucketSettings = bucketMatchers.match(input); + // BucketId may be dynamic (f.e. based on headers). + RlqsBucketId bucketId = bucketSettings.bucketIdForRequest(input); + + RlqsBucket bucket = bucketCache.getOrCreate(bucketId, bucketSettings, this::onNewBucket); + return bucket.rateLimit(); +} + +private void onNewBucket(RlqsBucket newBucket) { + // The report for the first RPC is sent immediately. + scheduleImmediateReport(newBucket); + registerReportTimer(newBucket.getReportingInterval()); +} +``` + +#### RLQS Client + +TODO + +#### RLQS Bucket Map + +TODO + +##### Multithreading + +There are several mutex-synchronized operations executed in +latency-sensitive `onCallHandler`: + +1. Inserting/reading a bucket from the bucket cache using `bucket_id`. Note + that `bucket_id` is represented as a `Map`, which may + introduce complexities in efficient cache sharding for certain programming + languages. +2. Incrementing `num_request_allowed`/`num_request_denied` bucket counters. + +Each gRPC implementation needs to consider what synchronization primitives are +available in their language to minimize the thread lock time. -##### Matching +### Handling Events +#### On LDS/RDS Updates + + +When receiving an LDS/RDS update, the RLQS filter will: + +1. Parse the new filter config. +2. Retrieve the corresponding RLQS Filter State from the RLQS Cache, creating + it if it doesn't exist. +3. Generate new `onClientCall` handlers (interceptors) using the retrieved + RLQS Filter State. +4. Update the filter chain with the new `onClientCall` handlers. +5. Release the RLQS Filter State from the cache if it's no longer referenced + by any `onClientCall` handlers. + +#### On Data Plane RPC + +When processing each data plane RPC, the RLQS filter will ask the RLQS filter +state for a rate-limit decision for the RPC. The RLQS filter state uses the +matcher tree from the filter config to determine which bucket to use for the +RPC. It then looks for that bucket in the bucket map, creating it if it doesn't +already exist. If a new bucket was created, it sends a request on the RLQS +stream informing the server of the bucket's creation. Finally, it returns the +resulting rate-limit decision based on the bucket contents. + + + +#### On RLQS Server Response + + + +When receiving a response from the RLQS server, the RLQS Client will: + +1. Parse the response. +2. Match the response to the corresponding buckets. +3. Update the rate limit quota assignments in the corresponding buckets. +4. Notify the RLQS Filter State of the new assignments. + +#### On Report Timers -##### Reporting + The aggregated number of requests is reported to the RLQS server at configured intervals. The report action is triggered by the Report Timers. RLQS Client @@ -153,7 +295,9 @@ manages a gRPC stream to the RLQS server. It's used by the filter state to send periodic bucket usage reports, and to receive new rate limit quota assignments to the buckets. -### Connecting to RLQS Control Plane +### Integrations + +#### Connecting to RLQS Control Plane xDS Control Plane provides RLQS connection details in [`GrpcService`] message ( already supported by Envoy). `GrpcService` supports two modes: @@ -163,7 +307,7 @@ already supported by Envoy). `GrpcService` supports two modes: For obvious reasons, we'll only support the `GoogleGrpc` mode. -#### Security Considerations +##### Security Considerations In [`GrpcService.GoogleGrpc`], the xDS Control Plane provides the target URI, `channel_credentials`, and `call_credentials`. If the xDS Control Plane is @@ -175,7 +319,7 @@ To prevent that, we'll introduce an allow-list to the bootstrap file introduced in [gRFC A27]. This allow-list will be a map from a fully-qualified server target URI to an object containing channel credentials to use for that target. -```js +```json5 // The allowlist of Control Planes allowed to be configured via xDS. "allowed_grpc_services": { // The key is fully-qualified server URI. @@ -206,7 +350,7 @@ matching target URI. > This solution is not specific to RLQS, and should be used with any > other Control Planes configured via [`GrpcService`] message. -### Unified Matcher API +#### Unified Matcher API RPCs will be matched into buckets using [Unified Matcher API] — an adaptable framework that can be used in any xDS component that needs matching features. @@ -230,7 +374,7 @@ In this iteration the following Unified Mather extensions will be supported: 2. Custom Matchers: 1. [`CelMatcher`](https://www.envoyproxy.io/docs/envoy/latest/xds/type/matcher/v3/cel.proto.html) -### CEL Integration +#### CEL Integration We will support request metadata matching via CEL expressions. Only Canonical CEL and only checked expressions will be supported (`cel.expr.CheckedExpr`). @@ -238,7 +382,7 @@ CEL and only checked expressions will be supported (`cel.expr.CheckedExpr`). CEL evaluation environment is a set of available variables and extension functions in a CEL program. We will match [Envoy CEL environment]. -#### Supported CEL Functions +##### Supported CEL Functions Similar to Envoy, we will support [standard CEL functions](https://github.com/google/cel-spec/blob/c629b2be086ed6b4c44ef4975e56945f66560677/doc/langdef.md#standard-definitions) @@ -260,7 +404,7 @@ except comprehension-style macros. [RE2_wiki]: https://en.wikipedia.org/wiki/RE2_(software) -#### Supported CEL Variables +##### Supported CEL Variables For RLQS, only the `request` variable is supported in CEL expressions. We will adapt [Envoy's Request Attributes](https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/advanced/attributes#request-attributes) @@ -281,7 +425,7 @@ for gRPC. | `request.protocol` | `string` | Not set | Request protocol. | | `request.query` | `string` | `""` | The query portion of the URL. | -##### Footnotes +###### Footnotes **1 `request.path`** @@ -310,7 +454,8 @@ provides the different variable resolving approaches based on the language: * Go: [`Activation.ResolveName(string)`](https://github.com/google/cel-go/blob/3f12ecad39e2eb662bcd82b6391cfd0cb4cb1c5e/interpreter/activation.go#L30) * Java: [`CelVariableResolver`](https://javadoc.io/doc/dev.cel/runtime/0.6.0/dev/cel/runtime/CelVariableResolver.html) -### Persistent Filter Cache +#### Persistent Filter Cache + RLQS Filter State holds the bucket usage data, report timers and the bidirectional stream to the RLQS server. To prevent the loss of state across @@ -402,73 +547,6 @@ config, including updates to inconsequential fields such as deny response status. If this becomes a problem, additional logic can be introduced to handle updates to such fields while preserving the filter state. -### Multithreading - -There are several mutex-synchronized operations executed in -latency-sensitive `onCallHandler`: - -1. Inserting/reading a bucket from the bucket cache using `bucket_id`. Note - that `bucket_id` is represented as a `Map`, which may - introduce complexities in efficient cache sharding for certain programming - languages. -2. Incrementing `num_request_allowed`/`num_request_denied` bucket counters. - -Each gRPC implementation needs to consider what synchronization primitives are -available in their language to minimize the thread lock time. - -### Code Samples - -> [!NOTE] -> Not a reference implementation. Only for flow illustration purposes. - -#### On Call Handler -```java -final RlqsFilterState filterState = rlqsCache.getOrCreateFilterState(config); - -return new ServerInterceptor() { - @Override - public Listener interceptCall( - ServerCall call, - Metadata headers, ServerCallHandler next) { - // TODO: handle filter_enabled and filter_enforced - - // RlqsClient matches the request into a bucket, - // and returns the rate limiting result. - RlqsRateLimitResult result = - filterState.rateLimit(HttpMatchInput.create(headers, call)); - - // Allowed. - if (result.isAllowed()) { - return next.startCall(call, headers); - } - // Denied: fail the call with given Status. - call.close( - result.denyResponse().status(), - result.denyResponse().headersToAdd()); - return new ServerCall.Listener(){}; - } -}; -``` - -#### Bucket Matching -```java -public RlqsRateLimitResult rateLimit(HttpMatchInput input) { - // Perform request matching. The result is RlqsBucketSettings. - RlqsBucketSettings bucketSettings = bucketMatchers.match(input); - // BucketId may be dynamic (f.e. based on headers). - RlqsBucketId bucketId = bucketSettings.bucketIdForRequest(input); - - RlqsBucket bucket = bucketCache.getOrCreate(bucketId, bucketSettings, this::onNewBucket); - return bucket.rateLimit(); -} - -private void onNewBucket(RlqsBucket newBucket) { - // The report for the first RPC is sent immediately. - scheduleImmediateReport(newBucket); - registerReportTimer(newBucket.getReportingInterval()); -} -``` - ### Temporary environment variable protection During initial development, this feature will be enabled via From b2f4a0eb7bf3cc364d2a75dcccf05ace70a8d405 Mon Sep 17 00:00:00 2001 From: Sergii Tkachenko Date: Tue, 22 Apr 2025 16:15:09 -0700 Subject: [PATCH 10/53] should be all high-level comments, now getting ready to todos --- A77-xds-rate-limiting-rlqs.md | 28 ++++++++++++++++------------ 1 file changed, 16 insertions(+), 12 deletions(-) diff --git a/A77-xds-rate-limiting-rlqs.md b/A77-xds-rate-limiting-rlqs.md index 2e833904c..ee2e2d02e 100644 --- a/A77-xds-rate-limiting-rlqs.md +++ b/A77-xds-rate-limiting-rlqs.md @@ -5,7 +5,7 @@ A77: xDS Server-Side Rate Limiting * Approver: Mark Roth (@markdroth) * Status: In Review * Implemented in: -* Last updated: 2024-11-04 +* Last updated: 2025-04-22 * Discussion at: `TODO(sergiitk): ` ## Abstract @@ -117,17 +117,27 @@ graph TD linkStyle 11 stroke: Teal, stroke-width: 3px; ``` -#### RLQS HTTP Filter +##### RLQS xDS HTTP Filter: Channel Level + + The filter parses the config, combines LDS filter config with RDS overrides, and generates the `onClientCall` handlers (aka interceptors in Java and Go, and filters in C++). -#### RLQS HTTP Filter: Channel Level +###### RLQS Cache + + +RLQS Cache persists across LDS/RDS updates. It maps unique filter configs to +RLQS Filter State instances, and provides the thread safety for creating and +accessing them. Each unique filter config generates a unique RLQS Filter state, +a 1:1 mapping. + +##### RLQS xDS HTTP Filter: Call Level -#### RLQS HTTP Filter: Call Level + -##### Code Sample: On Call Handler +###### Code Sample: On Call Handler > [!NOTE] > Not a reference implementation. Only for flow illustration purposes. @@ -160,13 +170,6 @@ return new ServerInterceptor() { }; ``` -#### RLQS Cache - -RLQS Cache persists across LDS/RDS updates. It maps unique filter configs to -RLQS Filter State instances, and provides the thread safety for creating and -accessing them. Each unique filter config generates a unique RLQS Filter state, -a 1:1 mapping. - #### RLQS Filter State In order to retain filter state across LDS/RDS updates, the actual logic for the @@ -456,6 +459,7 @@ provides the different variable resolving approaches based on the language: #### Persistent Filter Cache + RLQS Filter State holds the bucket usage data, report timers and the bidirectional stream to the RLQS server. To prevent the loss of state across From 6cb008fc3b7119e727b75c771e52494a08071599 Mon Sep 17 00:00:00 2001 From: Sergii Tkachenko Date: Tue, 22 Apr 2025 16:31:51 -0700 Subject: [PATCH 11/53] make todos visible --- A77-xds-rate-limiting-rlqs.md | 32 +++++++++++++++++--------------- 1 file changed, 17 insertions(+), 15 deletions(-) diff --git a/A77-xds-rate-limiting-rlqs.md b/A77-xds-rate-limiting-rlqs.md index ee2e2d02e..e30280534 100644 --- a/A77-xds-rate-limiting-rlqs.md +++ b/A77-xds-rate-limiting-rlqs.md @@ -6,7 +6,8 @@ A77: xDS Server-Side Rate Limiting * Status: In Review * Implemented in: * Last updated: 2025-04-22 -* Discussion at: `TODO(sergiitk): ` +* Discussion at: + - [ ] TODO(sergiitk): insert google group thread ## Abstract @@ -43,7 +44,7 @@ which are covered in the proposal: 5. RLQS filter state will persist across LDS/RDS updates using cache retention mechanism similar to the one implemented for [gRFC A83]. -### Related Proposals: +### Related Proposals * [A27: xDS-Based Global Load Balancing][gRFC A27] * [A36: xDS-Enabled Servers][gRFC A36] @@ -119,14 +120,15 @@ graph TD ##### RLQS xDS HTTP Filter: Channel Level - +- [ ] TODO(sergiitk): clearly clarify what is xDS HTTP filter: java vs core The filter parses the config, combines LDS filter config with RDS overrides, and generates the `onClientCall` handlers (aka interceptors in Java and Go, and filters in C++). ###### RLQS Cache - + +- [ ] TODO(sergiitk): verivy/update this block - merge cache into filter? RLQS Cache persists across LDS/RDS updates. It maps unique filter configs to RLQS Filter State instances, and provides the thread safety for creating and @@ -135,7 +137,7 @@ a 1:1 mapping. ##### RLQS xDS HTTP Filter: Call Level - +- [ ] TODO(sergiitk): explain interceptor fields/role ###### Code Sample: On Call Handler @@ -222,11 +224,11 @@ private void onNewBucket(RlqsBucket newBucket) { #### RLQS Client -TODO +- [ ] TODO(sergiitk): explain RLQS Client #### RLQS Bucket Map -TODO +- [ ] TODO(sergiitk): explain bucket map ##### Multithreading @@ -246,7 +248,8 @@ available in their language to minimize the thread lock time. #### On LDS/RDS Updates - +- [ ] TODO(sergiitk): verify mlgen + When receiving an LDS/RDS update, the RLQS filter will: 1. Parse the new filter config. @@ -268,18 +271,16 @@ already exist. If a new bucket was created, it sends a request on the RLQS stream informing the server of the bucket's creation. Finally, it returns the resulting rate-limit decision based on the bucket contents. - #### On RLQS Server Response - +- [ ] TODO(sergiitk): verify mlgen When receiving a response from the RLQS server, the RLQS Client will: @@ -290,7 +291,7 @@ When receiving a response from the RLQS server, the RLQS Client will: #### On Report Timers - +- [ ] TODO(sergiitk): verify - moved from another place The aggregated number of requests is reported to the RLQS server at configured intervals. The report action is triggered by the Report Timers. RLQS Client @@ -458,8 +459,9 @@ provides the different variable resolving approaches based on the language: * Java: [`CelVariableResolver`](https://javadoc.io/doc/dev.cel/runtime/0.6.0/dev/cel/runtime/CelVariableResolver.html) #### Persistent Filter Cache - - + +- [ ] TODO(sergiitk): move to A83 +- [ ] TODO(sergiitk): clearly clarify what is xDS HTTP filter: java vs core RLQS Filter State holds the bucket usage data, report timers and the bidirectional stream to the RLQS server. To prevent the loss of state across From 60862c0b073c9064698e3000d4df04371e39678f Mon Sep 17 00:00:00 2001 From: Sergii Tkachenko Date: Wed, 23 Apr 2025 13:09:50 -0700 Subject: [PATCH 12/53] RLQS Components Flowchart v9: remove separate "RLQS Cache" --- A77-xds-rate-limiting-rlqs.md | 11 +++++------ 1 file changed, 5 insertions(+), 6 deletions(-) diff --git a/A77-xds-rate-limiting-rlqs.md b/A77-xds-rate-limiting-rlqs.md index e30280534..5256e7c2b 100644 --- a/A77-xds-rate-limiting-rlqs.md +++ b/A77-xds-rate-limiting-rlqs.md @@ -81,7 +81,7 @@ config: clusterBorder: "#777" --- graph TD -%% RLQS Components Flowchart v8 +%% RLQS Components Flowchart v9 %% == nodes == subgraph grpc_client_box [gRPC Client] @@ -91,8 +91,7 @@ graph TD rlqs[(RLQS)] end subgraph grpc_server_box [gRPC Server] - rlqs_filter(RLQS HTTP Filter) - rlqs_cache(RLQS Cache) + rlqs_filter(RLQS xDS HTTP Filter) subgraph rlqs_filter_state_box [RLQS Filter State] rlqs_client(RLQS Client) rlqs_filter_state(RLQS Filter State) @@ -104,7 +103,7 @@ graph TD rpc_handler("Filter's onClientCall handler") end %% == edges == - rlqs_filter -- " Get RLQS Filter State
per unique config " --> rlqs_cache -- " getOrCreate(config) " --> rlqs_filter_state + rlqs_filter -- " GetOrCreate RLQS Filter State
per unique config " --> rlqs_filter_state rlqs_filter -- " Pass RLQS Filter State
for the route " --> rpc_handler -- " rateLimit(call) " --> rlqs_filter_state request --> rpc_handler rlqs_filter_state --o matcher_tree & report_timers @@ -114,8 +113,8 @@ graph TD rlqs_bucket_cache -- " getOrCreate(bucketId)
Atomic Updates " --> rlqs_bucket rlqs_client <-. gRPC Stream .-> rlqs style request stroke: RoyalBlue, stroke-width: 2px; - linkStyle 3,4 stroke: RoyalBlue, stroke-width: 2px; - linkStyle 11 stroke: Teal, stroke-width: 3px; + linkStyle 2,3 stroke: RoyalBlue, stroke-width: 2px; + linkStyle 10 stroke: Teal, stroke-width: 3px; ``` ##### RLQS xDS HTTP Filter: Channel Level From 9dd22fa9fd3c089f2054413eec6f6880fcfaa937 Mon Sep 17 00:00:00 2001 From: Sergii Tkachenko Date: Fri, 25 Apr 2025 19:33:49 -0700 Subject: [PATCH 13/53] Finish "RLQS xDS HTTP Filter" component docs --- A77-xds-rate-limiting-rlqs.md | 93 ++++++++++++++++++++++++++--------- 1 file changed, 71 insertions(+), 22 deletions(-) diff --git a/A77-xds-rate-limiting-rlqs.md b/A77-xds-rate-limiting-rlqs.md index 5256e7c2b..2c0d1afb9 100644 --- a/A77-xds-rate-limiting-rlqs.md +++ b/A77-xds-rate-limiting-rlqs.md @@ -119,56 +119,105 @@ graph TD ##### RLQS xDS HTTP Filter: Channel Level -- [ ] TODO(sergiitk): clearly clarify what is xDS HTTP filter: java vs core +For each route, the filter creates an internal RLQS Filter Config object by +combining the LDS filter config with any RDS config overrides. Two routes with +the same combined filter configuration should result in identical RLQS Filter +Config objects. + +For each unique RLQS Filter Config, the filter creates a corresponding RLQS +Filter State object. To track the uniqueness of RLQS Filter Configs, the filter +will maintain a map from RLQS Filter Configs to their respective RLQS Filter +State instances. This map will be stored in the persistent filter state +mechanism described in [gRFC A83]. + +Channel-level RLQS xDS HTTP Filter object will include the following data +members: -The filter parses the config, combines LDS filter config with RDS overrides, and -generates the `onClientCall` handlers (aka interceptors in Java and Go, and -filters in C++). +- RLQS Filter State Cache: from filter config, initialized at instantiation, + retained across LDS/RDS updates. A 1:1 mapping between unique RLQS Filter + Config instances and corresponding unique RLQS Filter State instances. + +> [!NOTE] +> Not a reference implementation. Only for flow illustration purposes. -###### RLQS Cache +```java +final class RlqsFilter implements Filter { + private final ConcurrentMap + filterStateCache = new ConcurrentHashMap<>(); + // ... -- [ ] TODO(sergiitk): verivy/update this block - merge cache into filter? + @Override + public ServerInterceptor buildServerInterceptor( + FilterConfig config, @Nullable FilterConfig overrideConfig) { + // Parse the filter config. + RlqsFilterConfig.Builder rlqsConfigBuilder = parseRlqsConfig(config); + + // Merge with per-route overrides if provided. + if (overrideConfig instanceof RlqsConfigOverride rlqsConfigOverride) { + // Only domain and matchers can be overriden. + if (!rlqsConfigOverride.domain().isEmpty()) { + rlqsConfigBuilder.domain(rlqsConfigOverride.domain()); + } + if (rlqsConfigOverride.bucketMatchers() != null) { + rlqsConfigBuilder.bucketMatchers(rlqsConfigOverride.bucketMatchers()); + } + } -RLQS Cache persists across LDS/RDS updates. It maps unique filter configs to -RLQS Filter State instances, and provides the thread safety for creating and -accessing them. Each unique filter config generates a unique RLQS Filter state, -a 1:1 mapping. + // Get or Create RLQS Filter Config instance from A83 cache. + RlqsFilterState rlqsFilterState = filterStateCache.computeIfAbsent( + rlqsConfigBuilder.build(), + (cfg) -> new RlqsFilterState(cfg, getRlqsServerInfo(cfg.rlqsService())) + ); + return new RlqsServerInterceptor(rlqsFilterState); + } +} +``` ##### RLQS xDS HTTP Filter: Call Level -- [ ] TODO(sergiitk): explain interceptor fields/role +When processing a data plane RPC for a given route, the filter passes the RPC +metadata to the corresponding RLQS Filter State instance for evaluation. Based +on the evaluation result, the filter either allows the RPC to proceed, or denies +it with a specific gRPC status. The evaluation result may also contain a list of +HTTP headers to add to the original request, or the deny response. + +Call-level RLQS xDS HTTP Filter object will include the following data members: -###### Code Sample: On Call Handler +- RLQS Filter State instance corresponding route's RLQS Filter Config: + implementation-dependent. > [!NOTE] > Not a reference implementation. Only for flow illustration purposes. ```java -final RlqsFilterState filterState = rlqsCache.getOrCreateFilterState(config); +private static class RlqsServerInterceptor implements ServerInterceptor { + private final RlqsFilterState rlqsFilterState; -return new ServerInterceptor() { - @Override - public Listener interceptCall( - ServerCall call, - Metadata headers, ServerCallHandler next) { - // TODO: handle filter_enabled and filter_enforced + public RlqsServerInterceptor(RlqsFilterState rlqsFilterState) { + this.rlqsFilterState = rlqsFilterState; + } + @Override + public ServerCall.Listener interceptCall( + ServerCall call, Metadata headers, + ServerCallHandler next) { // RlqsClient matches the request into a bucket, // and returns the rate limiting result. RlqsRateLimitResult result = - filterState.rateLimit(HttpMatchInput.create(headers, call)); + rlqsFilterState.rateLimit(HttpMatchInput.create(headers, call)); - // Allowed. if (result.isAllowed()) { + // TODO: append request_headers_to_add_when_not_enforced return next.startCall(call, headers); } + // Denied: fail the call with given Status. call.close( result.denyResponse().status(), result.denyResponse().headersToAdd()); return new ServerCall.Listener(){}; } -}; +} ``` #### RLQS Filter State From eb244cabbdbaa5a4799efa47bb8e1d832f15bc58 Mon Sep 17 00:00:00 2001 From: Sergii Tkachenko Date: Mon, 28 Apr 2025 11:53:55 -0700 Subject: [PATCH 14/53] explain RLQS Client --- A77-xds-rate-limiting-rlqs.md | 31 +++++++++++++++++++++---------- 1 file changed, 21 insertions(+), 10 deletions(-) diff --git a/A77-xds-rate-limiting-rlqs.md b/A77-xds-rate-limiting-rlqs.md index 2c0d1afb9..e36c9f13b 100644 --- a/A77-xds-rate-limiting-rlqs.md +++ b/A77-xds-rate-limiting-rlqs.md @@ -117,7 +117,7 @@ graph TD linkStyle 10 stroke: Teal, stroke-width: 3px; ``` -##### RLQS xDS HTTP Filter: Channel Level +#### RLQS xDS HTTP Filter: Channel Level For each route, the filter creates an internal RLQS Filter Config object by combining the LDS filter config with any RDS config overrides. Two routes with @@ -173,7 +173,7 @@ final class RlqsFilter implements Filter { } ``` -##### RLQS xDS HTTP Filter: Call Level +#### RLQS xDS HTTP Filter: Call Level When processing a data plane RPC for a given route, the filter passes the RPC metadata to the corresponding RLQS Filter State instance for evaluation. Based @@ -242,16 +242,11 @@ RLQS Filter State object will include the following data members: - Report Timer: Created/modified when we get an RLQS response for a given bucket, or when a previous timer fires. - - -##### Code Sample: Bucket Matching - > [!NOTE] > Not a reference implementation. Only for flow illustration purposes. +- [ ] TODO(sergiitk): update + ```java public RlqsRateLimitResult rateLimit(HttpMatchInput input) { // Perform request matching. The result is RlqsBucketSettings. @@ -272,7 +267,23 @@ private void onNewBucket(RlqsBucket newBucket) { #### RLQS Client -- [ ] TODO(sergiitk): explain RLQS Client +The RLQS Client is responsible for communication with the Rate Limit Quota +Service (RLQS) server. It manages a gRPC stream to the RLQS server, used for +sending periodic bucket usage reports and receiving new rate limit quota +assignments. The client handles reconnects and manages the lifecycle of the +stream. + +RLQS Client object will include the following data members: + +- gRPC Channel: From filter config, initialized at instantiation, constant. A + channel to the RLQS server. +- gRPC Stream: Initialized at instantiation, replaced any time the stream + fails (with backoff). Accessed when we get the first data plane RPC for a + given bucket and when a report timer fires. A bidirectional gRPC stream to + the RLQS server. +- Buckets Update Callback: Provided by RLQS Filter state at instantiation, + constant, accessed on responses from the RLQS server. A callback to notify + the RLQS Filter State of updates to quota assignments. #### RLQS Bucket Map From c5b53a7e4b694999e822e81c039d35d0ec1871f9 Mon Sep 17 00:00:00 2001 From: Sergii Tkachenko Date: Mon, 28 Apr 2025 17:01:56 -0700 Subject: [PATCH 15/53] RLQS Bucket Cache --- A77-xds-rate-limiting-rlqs.md | 74 +++++++++++++++++++++++++++-------- 1 file changed, 57 insertions(+), 17 deletions(-) diff --git a/A77-xds-rate-limiting-rlqs.md b/A77-xds-rate-limiting-rlqs.md index e36c9f13b..6d717986f 100644 --- a/A77-xds-rate-limiting-rlqs.md +++ b/A77-xds-rate-limiting-rlqs.md @@ -233,9 +233,9 @@ their own filter state. RLQS Filter State object will include the following data members: - Matcher Tree: From filter config, initialized at instantiation, constant. - Used to identify the bucket map entry for each data plane RPC. -- Bucket Map: Accessed on each data plane RPC, when we get a response from the - RLQS server, and when report timers fire. + Used to identify the RLQS Bucket for each data plane RPC. +- RLQS Bucket Cache: Accessed on each data plane RPC, when we get a response + from the RLQS server, and when report timers fire. - RLQS Client: Accessed when we get the first data plane RPC for a given bucket and when a report timer fires. Notifies RLQS Filter State of responses received from the RLQS server. @@ -285,16 +285,56 @@ RLQS Client object will include the following data members: constant, accessed on responses from the RLQS server. A callback to notify the RLQS Filter State of updates to quota assignments. -#### RLQS Bucket Map - -- [ ] TODO(sergiitk): explain bucket map - -##### Multithreading - -There are several mutex-synchronized operations executed in -latency-sensitive `onCallHandler`: - -1. Inserting/reading a bucket from the bucket cache using `bucket_id`. Note +#### RLQS Bucket Cache + +The RLQS Bucket Cache is responsible for storing and managing the lifecycle of +RLQS Buckets. It provides a thread-safe way to access and update buckets, +ensuring that only one instance of a RLQS bucket exists for a given `bucket_id`. +The cache also allows to retrieve all buckets that need to be reported to the +RLQS server at a given reporting interval. Depending on implementation, this +component may be inlined into RLQS Filter State. + +RLQS Bucket Cache object will include the following data members: + +- Buckets Map: Initialized empty at instantiation, thread-safe. Entries + retrieved on each data plane RPC and when report timers fire. Entries + inserted when we get the first data plane RPC for a given bucket. Entries + are deleted either by RLQS server via `abandon_action`, or on report timers + if a bucket's active assignment expires. +- Buckets Per Interval Map: Initialized empty at instantiation, thread-safe. + Entries retrieved when a report timer fires. Entries inserted and deleted at + the same time as Buckets Map. Used to efficiently access the set of buckets + for a given report interval. A map from reporting interval to a set of + `RlqsBucket` instances. + +##### RLQS Bucket + +The RLQS Bucket tracks the current rate limiting assignment and the quota +usage for a specific bucket, identified by a `bucket_id`. It uses this data to +determine whether to allow or deny a request. The bucket also provides a +thread-safe mechanism to snapshot and reset its quota usage when building usage +reports. + +RLQS Bucket object will include the following data members: + +- `bucket_id`: From filter config and RPC metadata, constant, initialized at + bucket creation. A unique identifier for the bucket. +- "No Assignment" and "Expired Assignment" strategies: Initialized at first + RPC for the bucket from filter config, constant. Fallback rate limiting + strategies. +- Active Assignment: Initialized at first RPC for the bucket from filter + config, updated on RLQS server responses. The current rate limiting strategy + to apply to requests, and associated expiration timestamps. +- Request Counters: Initialized at 0, updated on data plane RPCs, reset on + report timers and RLQS server responses. Tracks the number allowed/denied + requests for the bucket. + +##### RLQS Bucket Cache Multithreading + +There are several mutex-synchronized operations on RLQS buckets that are +executed during latency-sensitive data plane RPC processing: + +1. Inserting/reading a bucket from the RLQS Bucket Cache using `bucket_id`. Note that `bucket_id` is represented as a `Map`, which may introduce complexities in efficient cache sharding for certain programming languages. @@ -325,10 +365,10 @@ When receiving an LDS/RDS update, the RLQS filter will: When processing each data plane RPC, the RLQS filter will ask the RLQS filter state for a rate-limit decision for the RPC. The RLQS filter state uses the matcher tree from the filter config to determine which bucket to use for the -RPC. It then looks for that bucket in the bucket map, creating it if it doesn't -already exist. If a new bucket was created, it sends a request on the RLQS -stream informing the server of the bucket's creation. Finally, it returns the -resulting rate-limit decision based on the bucket contents. +RPC. It then looks for that bucket in the bucket cache map, creating it if it +doesn't already exist. If a new bucket was created, it sends a request on the +RLQS stream informing the server of the bucket's creation. Finally, it returns +the resulting rate-limit decision based on the bucket contents. - [ ] TODO(sergiitk): redo the next block?` RLQS Filter State evaluates the metadata against the matcher tree to match the From d181d813a813fe546e0b1afe51078ecf998cf1a3 Mon Sep 17 00:00:00 2001 From: Sergii Tkachenko Date: Mon, 28 Apr 2025 17:03:28 -0700 Subject: [PATCH 16/53] move rlqs client below the cache --- A77-xds-rate-limiting-rlqs.md | 40 +++++++++++++++++------------------ 1 file changed, 20 insertions(+), 20 deletions(-) diff --git a/A77-xds-rate-limiting-rlqs.md b/A77-xds-rate-limiting-rlqs.md index 6d717986f..6bab84049 100644 --- a/A77-xds-rate-limiting-rlqs.md +++ b/A77-xds-rate-limiting-rlqs.md @@ -265,26 +265,6 @@ private void onNewBucket(RlqsBucket newBucket) { } ``` -#### RLQS Client - -The RLQS Client is responsible for communication with the Rate Limit Quota -Service (RLQS) server. It manages a gRPC stream to the RLQS server, used for -sending periodic bucket usage reports and receiving new rate limit quota -assignments. The client handles reconnects and manages the lifecycle of the -stream. - -RLQS Client object will include the following data members: - -- gRPC Channel: From filter config, initialized at instantiation, constant. A - channel to the RLQS server. -- gRPC Stream: Initialized at instantiation, replaced any time the stream - fails (with backoff). Accessed when we get the first data plane RPC for a - given bucket and when a report timer fires. A bidirectional gRPC stream to - the RLQS server. -- Buckets Update Callback: Provided by RLQS Filter state at instantiation, - constant, accessed on responses from the RLQS server. A callback to notify - the RLQS Filter State of updates to quota assignments. - #### RLQS Bucket Cache The RLQS Bucket Cache is responsible for storing and managing the lifecycle of @@ -343,6 +323,26 @@ executed during latency-sensitive data plane RPC processing: Each gRPC implementation needs to consider what synchronization primitives are available in their language to minimize the thread lock time. +#### RLQS Client + +The RLQS Client is responsible for communication with the Rate Limit Quota +Service (RLQS) server. It manages a gRPC stream to the RLQS server, used for +sending periodic bucket usage reports and receiving new rate limit quota +assignments. The client handles reconnects and manages the lifecycle of the +stream. + +RLQS Client object will include the following data members: + +- gRPC Channel: From filter config, initialized at instantiation, constant. A + channel to the RLQS server. +- gRPC Stream: Initialized at instantiation, replaced any time the stream + fails (with backoff). Accessed when we get the first data plane RPC for a + given bucket and when a report timer fires. A bidirectional gRPC stream to + the RLQS server. +- Buckets Update Callback: Provided by RLQS Filter state at instantiation, + constant, accessed on responses from the RLQS server. A callback to notify + the RLQS Filter State of updates to quota assignments. + ### Handling Events #### On LDS/RDS Updates From 851606a89be289da83aefd2a53d793388b8ac2f5 Mon Sep 17 00:00:00 2001 From: Sergii Tkachenko Date: Mon, 28 Apr 2025 19:31:47 -0700 Subject: [PATCH 17/53] finish draft 2 --- A77-xds-rate-limiting-rlqs.md | 245 +++++++++++++--------------------- 1 file changed, 92 insertions(+), 153 deletions(-) diff --git a/A77-xds-rate-limiting-rlqs.md b/A77-xds-rate-limiting-rlqs.md index 6bab84049..8c4c09e1b 100644 --- a/A77-xds-rate-limiting-rlqs.md +++ b/A77-xds-rate-limiting-rlqs.md @@ -173,6 +173,14 @@ final class RlqsFilter implements Filter { } ``` +##### Future considerations + +This proposal uses the entire RLQS Filter Config to identify the corresponding +unique Filter State. As a result, a Filter State will be recreated even if only +inconsequential fields, such as the deny response status, are changed. If this +becomes a problem, additional logic should be introduced to exclude such fields +when comparing Filter Config objects. + #### RLQS xDS HTTP Filter: Call Level When processing a data plane RPC for a given route, the filter passes the RPC @@ -239,29 +247,53 @@ RLQS Filter State object will include the following data members: - RLQS Client: Accessed when we get the first data plane RPC for a given bucket and when a report timer fires. Notifies RLQS Filter State of responses received from the RLQS server. -- Report Timer: Created/modified when we get an RLQS response for a given - bucket, or when a previous timer fires. - -> [!NOTE] -> Not a reference implementation. Only for flow illustration purposes. +- Report Timers Map: Initialized at instantiation. Used to track discovered + reporting intervals and their execution handlers. + - Entries inserted we get the first data plane RPC for a given bucket and + there's no key for bucket reporting interval. + - Entries deleted when there's more buckets with given reporting interval + in RLQS Bucket Cache. -- [ ] TODO(sergiitk): update +Pseudo-code for RLQS Filter State RPC rate limiting: ```java -public RlqsRateLimitResult rateLimit(HttpMatchInput input) { - // Perform request matching. The result is RlqsBucketSettings. - RlqsBucketSettings bucketSettings = bucketMatchers.match(input); - // BucketId may be dynamic (f.e. based on headers). - RlqsBucketId bucketId = bucketSettings.bucketIdForRequest(input); - - RlqsBucket bucket = bucketCache.getOrCreate(bucketId, bucketSettings, this::onNewBucket); - return bucket.rateLimit(); -} +final class RlqsFilterState { + private final RlqsClient rlqsClient; + private final Matcher bucketMatchers; + private final RlqsBucketCache bucketCache; + private final ScheduledExecutorService scheduler; + private final ConcurrentMap> + timers = new ConcurrentHashMap<>(); + // ... + + public RlqsRateLimitResult rateLimit(HttpMatchInput input) { + // Perform request matching. The result is RlqsBucketSettings. + RlqsBucketSettings bucketSettings = bucketMatchers.match(input); + // BucketId may be dynamic (f.e. based on headers). + RlqsBucketId bucketId = bucketSettings.bucketIdForRequest(input); + // Get or create RLQS Bucket. + RlqsBucket bucket = bucketCache.getOrCreate( + bucketId, bucketSettings, this::onNewBucket); + return bucket.rateLimit(); + } -private void onNewBucket(RlqsBucket newBucket) { - // The report for the first RPC is sent immediately. - scheduleImmediateReport(newBucket); - registerReportTimer(newBucket.getReportingInterval()); + private void onNewBucket(RlqsBucket newBucket) { + // The report for the first RPC is sent immediately. + scheduleImmediateReport(newBucket); + registerReportTimer(newBucket.getReportingInterval()); + } + + private void scheduleImmediateReport(RlqsBucket newBucket) { + // Do not block data plane RPC on sending the report. + scheduler.schedule( + () -> rlqsClient.sendUsageReports(newBucket.snapshotAndResetUsage()), + 1, TimeUnit.MICROSECONDS) + } + + private void registerReportTimer(final long intervalMillis) { + // TODO: bound / roundup the report interval for better grouping. + timers.computeIfAbsent(intervalMillis, k -> newTimer(intervalMillis)); + } } ``` @@ -278,9 +310,10 @@ RLQS Bucket Cache object will include the following data members: - Buckets Map: Initialized empty at instantiation, thread-safe. Entries retrieved on each data plane RPC and when report timers fire. Entries - inserted when we get the first data plane RPC for a given bucket. Entries - are deleted either by RLQS server via `abandon_action`, or on report timers - if a bucket's active assignment expires. + inserted when we get the first data plane RPC for a given bucket, or when + instructed by the RLQS Server. Entries are deleted either by RLQS server via + `abandon_action`, or on report timers if a bucket's active assignment + expires. - Buckets Per Interval Map: Initialized empty at instantiation, thread-safe. Entries retrieved when a report timer fires. Entries inserted and deleted at the same time as Buckets Map. Used to efficiently access the set of buckets @@ -347,56 +380,57 @@ RLQS Client object will include the following data members: #### On LDS/RDS Updates -- [ ] TODO(sergiitk): verify mlgen +When receiving an LDS/RDS update, the RLQS filter will perform the following +steps for each route: -When receiving an LDS/RDS update, the RLQS filter will: +1. Parse the new filter config and per-route overrides into an internal RLQS + Filter Config object. +2. Retrieve the corresponding RLQS Filter State from the RLQS Filter State + Cache, creating it if it doesn't exist. +3. Generate new data plane RPC handler using the retrieved RLQS Filter State. -1. Parse the new filter config. -2. Retrieve the corresponding RLQS Filter State from the RLQS Cache, creating - it if it doesn't exist. -3. Generate new `onClientCall` handlers (interceptors) using the retrieved - RLQS Filter State. -4. Update the filter chain with the new `onClientCall` handlers. -5. Release the RLQS Filter State from the cache if it's no longer referenced - by any `onClientCall` handlers. +Once all routes are parsed, the RLQS filter will release from the RLQS Filter +State Cache all RLQS Filter State instances no longer referenced by any data +plane RPC handlers. #### On Data Plane RPC -When processing each data plane RPC, the RLQS filter will ask the RLQS filter -state for a rate-limit decision for the RPC. The RLQS filter state uses the +When processing each data plane RPC, the RLQS Filter will ask the RLQS Filter +State for a rate-limit decision for the RPC. The RLQS Filter State uses the matcher tree from the filter config to determine which bucket to use for the -RPC. It then looks for that bucket in the bucket cache map, creating it if it -doesn't already exist. If a new bucket was created, it sends a request on the -RLQS stream informing the server of the bucket's creation. Finally, it returns -the resulting rate-limit decision based on the bucket contents. - -- [ ] TODO(sergiitk): redo the next block?` -RLQS Filter State evaluates the metadata against the matcher tree to match the -request into a bucket. The Bucket holds the Rate Limit Quota assigned by the -RLQS server (f.e. 100 requests per minute), and aggregates the number of -requests it allowed/denied. This information is used to make the rate limiting -decision. +RPC. It then looks for that RLQS Bucket in the RLQS Bucket Cache map, creating +it if it doesn't already exist. -#### On RLQS Server Response +If a new bucket has been created, the RPC is rate-limited according to bucket's +default configuration. Then the filter schedules a report on the RLQS stream +informing the server of the bucket's creation, and registers the report timers +for bucket's reporting interval. + +If the bucket exists, the RPC is rate-limited according to its active quota +assignment and usage counters. The bucket increments corresponding bucket usage +counter. -- [ ] TODO(sergiitk): verify mlgen +#### On RLQS Server Response -When receiving a response from the RLQS server, the RLQS Client will: +When receiving an RLQS Server Response, the RLQS Client passes parsed response +to the RLQS Filter State. The RLQS Filter State iterates through the bucket +assignments in the response, and updates the corresponding buckets in the RLQS +Bucket Cache. Buckets marked to be abandoned are purged from the cache. -1. Parse the response. -2. Match the response to the corresponding buckets. -3. Update the rate limit quota assignments in the corresponding buckets. -4. Notify the RLQS Filter State of the new assignments. +If a bucket has a new rate limit assignment, the bucket's active assignment is +updated and bucket usage counters are reset. Otherwise, only the assignment +expiration is updated. #### On Report Timers -- [ ] TODO(sergiitk): verify - moved from another place +When a report timer fires, the RLQS Filter State retrieves all buckets with the +corresponding reporting interval from the RLQS Bucket Cache. For each bucket, +the filter snapshots the current usage counters, and resets them. The filter +then sends the snapshot to RLQS server using RLQS Client. -The aggregated number of requests is reported to the RLQS server at configured -intervals. The report action is triggered by the Report Timers. RLQS Client -manages a gRPC stream to the RLQS server. It's used by the filter state to send -periodic bucket usage reports, and to receive new rate limit quota assignments -to the buckets. +If a bucket's active assignment has expired, the bucket is removed from the +cache. If there are no more buckets with the given reporting interval, the +corresponding timer is removed. ### Integrations @@ -557,101 +591,6 @@ provides the different variable resolving approaches based on the language: * Go: [`Activation.ResolveName(string)`](https://github.com/google/cel-go/blob/3f12ecad39e2eb662bcd82b6391cfd0cb4cb1c5e/interpreter/activation.go#L30) * Java: [`CelVariableResolver`](https://javadoc.io/doc/dev.cel/runtime/0.6.0/dev/cel/runtime/CelVariableResolver.html) -#### Persistent Filter Cache - -- [ ] TODO(sergiitk): move to A83 -- [ ] TODO(sergiitk): clearly clarify what is xDS HTTP filter: java vs core - -RLQS Filter State holds the bucket usage data, report timers and the -bidirectional stream to the RLQS server. To prevent the loss of state across -LDS/RDS updates, RLQS filter will require a cache retention mechanism similar to -the one implemented for [gRFC A83]. - -The scope of each RLQS Filter Cache instance will be per server instance (same -scope as the filter chain) and per filter name. - -RLQS implementations will provide a mechanism for new instances of the filter to -retain the cache from previous instances. There may be multiple instances of the -RLQS Filter State, each one mapped to a unique filter config generated from LDS -config, and RDS overrides. Consider the following example that demonstrates the -lifecycle of RLQS Filter Cache. - -```mermaid ---- -config: - sequence: - showSequenceNumbers: true - height: 46 - diagramMarginX: 40 - diagramMarginY: 40 ---- -sequenceDiagram - -%% gRFC: RLQS Filter Cache Lifecycle v1.1 - participant xds as Control Plane - participant filter as RLQS HTTP Filter - participant cache as RLQS Cache - participant e1 as RlqsFilterState(c1) - participant e2 as RlqsFilterState(c2) - -# Notes - Note right of xds: r1-4: routes
c1-2: unique filter configs -%% LDS 1 - xds ->> filter: LDS1
RLQS{r1=c1, r2=c2, r3=c2} - filter ->> cache: r1: getOrCreate(c1) - cache ->>+ e1: new RlqsFilterState(c1) - filter ->> cache: r2: getOrCreate(c2) - cache ->>+ e2: new RlqsFilterState(c2) - filter ->> cache: r3: getOrCreate(c2) - Note over filter: r1: RlqsFilterState(c1)
r2: RlqsFilterState(c2)
r3: RlqsFilterState(c2) -%% RDS 1 - xds ->> filter: RDS1
RLQS{r1=c2} - filter ->> cache: r1: getOrCreate(c2) - filter ->> cache: shutdownFilterState(c1) - cache -x e1: RlqsFilterState(c1).shutdown() - deactivate e1 - Note over filter: r1: RlqsFilterState(c2)
r2: RlqsFilterState(c2)
r3: RlqsFilterState(c2) -%% LDS 2 - xds ->> filter: LDS2
RLQS{r3=c2, r4=c2} - filter ->> cache: r4: getOrCreate(c2) - Note over filter: r3: RlqsFilterState(c2)
r4: RlqsFilterState(c2) -%% End - deactivate e2 -``` - -**LDS 1** - -In this example, the RLQS filter is configured for three routes: `r1`, `r2`, -and `r3`. Each unique config generates a unique RLQS Filter -State: `RlqsFilterState(c1)` for the config `c1`, and `RlqsFilterState(c2)` for -the config `c2`. After processing the first LDS update, we've generated -`onCallHandlers` for three routes: - -1. `r1`, referencing `RlqsFilterState(c1)`. -2. `r2`, referencing `RlqsFilterState(c2)`. -3. `r3`, also referencing `RlqsFilterState(c2)`. - -**RDS 1** - -RDS 1 updates RLQS config for the route `r1` so it's identical to config `c2`. -We retrieve `RlqsFilterState(c2)` from the RLQS Cache and generate new -`onCallHandlers` for route `r2`. `RlqsFilterState(c1)` is no longer referenced -by any `onCallHandler`, and can be destroyed with all associated resources. - -**LDS 2** - -LDS 2 update removes `r1` and `r2`, and adds new route r4 with the config -identical to `c2`. While `onCallHandlers` for routes `r1` and `r2` are -destroyed, `RlqsFilterState(c2)` is still used by two `onCallHandlers`, so it's -preserved in RLQS Cache. - -##### Future considerations - -With this proposal, the filter state is lost if change is made to the filter -config, including updates to inconsequential fields such as deny response -status. If this becomes a problem, additional logic can be introduced to handle -updates to such fields while preserving the filter state. - ### Temporary environment variable protection During initial development, this feature will be enabled via From e480c030a80e9d1688c4f283f7b2c05ebd28410f Mon Sep 17 00:00:00 2001 From: Sergii Tkachenko Date: Tue, 29 Apr 2025 12:21:14 -0700 Subject: [PATCH 18/53] Move filter state cache description to filter channel level --- A77-xds-rate-limiting-rlqs.md | 39 +++++++++++++++-------------------- 1 file changed, 17 insertions(+), 22 deletions(-) diff --git a/A77-xds-rate-limiting-rlqs.md b/A77-xds-rate-limiting-rlqs.md index 8c4c09e1b..0b4f911a1 100644 --- a/A77-xds-rate-limiting-rlqs.md +++ b/A77-xds-rate-limiting-rlqs.md @@ -58,6 +58,8 @@ which are covered in the proposal: [gRFC A39]: A39-xds-http-filters.md [gRFC A83]: A83-xds-gcp-authn-filter.md +[RLQS xDS HTTP Filter: Channel Level]: #rlqs-xds-http-filter-channel-level + [`GrpcService`]: https://www.envoyproxy.io/docs/envoy/latest/api-v3/config/core/v3/grpc_service.proto [`GrpcService.GoogleGrpc`]: https://www.envoyproxy.io/docs/envoy/latest/api-v3/config/core/v3/grpc_service.proto#envoy-v3-api-msg-config-core-v3-grpcservice-googlegrpc [Unified Matcher API]: https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/advanced/matching/matching_api.html @@ -119,16 +121,13 @@ graph TD #### RLQS xDS HTTP Filter: Channel Level -For each route, the filter creates an internal RLQS Filter Config object by -combining the LDS filter config with any RDS config overrides. Two routes with -the same combined filter configuration should result in identical RLQS Filter -Config objects. - -For each unique RLQS Filter Config, the filter creates a corresponding RLQS -Filter State object. To track the uniqueness of RLQS Filter Configs, the filter -will maintain a map from RLQS Filter Configs to their respective RLQS Filter -State instances. This map will be stored in the persistent filter state -mechanism described in [gRFC A83]. +In order to retain filter state across LDS/RDS updates, the actual logic for the +RLQS filter will be moved into a separate object called RLQS Filter State, which +will be stored in the persistent filter state mechanism described in [gRFC A83]. +The key in the persistent filter state will be the RLQS xDS HTTP filter config, +which ensures that two RLQS filter instances with the same config will share +filter state but two RLQS filter instances with different configs will each have +their own filter state. Channel-level RLQS xDS HTTP Filter object will include the following data members: @@ -137,8 +136,7 @@ members: retained across LDS/RDS updates. A 1:1 mapping between unique RLQS Filter Config instances and corresponding unique RLQS Filter State instances. -> [!NOTE] -> Not a reference implementation. Only for flow illustration purposes. +Pseudo-code: ```java final class RlqsFilter implements Filter { @@ -163,7 +161,7 @@ final class RlqsFilter implements Filter { } } - // Get or Create RLQS Filter Config instance from A83 cache. + // Get or Create RLQS Filter Config from persistent filter state. RlqsFilterState rlqsFilterState = filterStateCache.computeIfAbsent( rlqsConfigBuilder.build(), (cfg) -> new RlqsFilterState(cfg, getRlqsServerInfo(cfg.rlqsService())) @@ -194,8 +192,7 @@ Call-level RLQS xDS HTTP Filter object will include the following data members: - RLQS Filter State instance corresponding route's RLQS Filter Config: implementation-dependent. -> [!NOTE] -> Not a reference implementation. Only for flow illustration purposes. +Pseudo-code: ```java private static class RlqsServerInterceptor implements ServerInterceptor { @@ -230,13 +227,11 @@ private static class RlqsServerInterceptor implements ServerInterceptor { #### RLQS Filter State -In order to retain filter state across LDS/RDS updates, the actual logic for the -RLQS filter will be moved into a separate object called RLQS Filter State, which -will be stored in the persistent filter state mechanism described in [gRFC A83]. -The key in the persistent filter state will be the RLQS xDS HTTP filter config, -which ensures that two RLQS filter instances with the same config will share -filter state but two RLQS filter instances with different configs will each have -their own filter state. +RLQS Filter State manages the runtime state and logic for a unique RLQS Filter +Config. It's responsible for matching data plane RPCs to buckets, managing the +lifecycle of those buckets, and communicating with the RLQS server to report +usage and receive quota assignments. Filter state instances are persisted across +LDS/RDS updates, see [RLQS xDS HTTP Filter: Channel Level]. RLQS Filter State object will include the following data members: From d1c78463a066c21646ac9d8ba7ebfe23840cd65e Mon Sep 17 00:00:00 2001 From: Sergii Tkachenko Date: Tue, 29 Apr 2025 12:51:40 -0700 Subject: [PATCH 19/53] rlqs bucket cache -> rlqs bucket map --- A77-xds-rate-limiting-rlqs.md | 70 +++++++++++++++++------------------ 1 file changed, 35 insertions(+), 35 deletions(-) diff --git a/A77-xds-rate-limiting-rlqs.md b/A77-xds-rate-limiting-rlqs.md index 0b4f911a1..feae859e3 100644 --- a/A77-xds-rate-limiting-rlqs.md +++ b/A77-xds-rate-limiting-rlqs.md @@ -83,7 +83,7 @@ config: clusterBorder: "#777" --- graph TD -%% RLQS Components Flowchart v9 +%% RLQS Components Flowchart v10 %% == nodes == subgraph grpc_client_box [gRPC Client] @@ -93,30 +93,31 @@ graph TD rlqs[(RLQS)] end subgraph grpc_server_box [gRPC Server] - rlqs_filter(RLQS xDS HTTP Filter) + rlqs_filter(RLQS Filter: Channel Level) + rlqs_filter_state_cache(RLQS Filter State Cache) subgraph rlqs_filter_state_box [RLQS Filter State] rlqs_client(RLQS Client) rlqs_filter_state(RLQS Filter State) report_timers(Report Timers) matcher_tree(Matcher Tree) - rlqs_bucket_cache(RLQS Bucket Cache) + rlqs_bucket_map(RLQS Bucket Map) rlqs_bucket(RLQS Bucket) end - rpc_handler("Filter's onClientCall handler") + rpc_handler("RLQS Filter: Call Level") end %% == edges == - rlqs_filter -- " GetOrCreate RLQS Filter State
per unique config " --> rlqs_filter_state - rlqs_filter -- " Pass RLQS Filter State
for the route " --> rpc_handler -- " rateLimit(call) " --> rlqs_filter_state + rlqs_filter -- "Get RLQS Filter State
per unique config" --> rlqs_filter_state_cache -- "getOrCreate(config)" --> rlqs_filter_state + rlqs_filter -- " Generate per-route RPC handlers
using RLQS Filter State " --> rpc_handler -- " rlqsFilterState.rateLimit(call) " --> rlqs_filter_state request --> rpc_handler rlqs_filter_state --o matcher_tree & report_timers rlqs_filter_state -- sendUsageReports --> rlqs_client - rlqs_filter_state -- CRUD --> rlqs_bucket_cache + rlqs_filter_state -- CRUD --> rlqs_bucket_map rlqs_client -- onBucketsUpdate --> rlqs_filter_state - rlqs_bucket_cache -- " getOrCreate(bucketId)
Atomic Updates " --> rlqs_bucket + rlqs_bucket_map -- " getOrCreate(bucketId)
Atomic Updates " --> rlqs_bucket rlqs_client <-. gRPC Stream .-> rlqs style request stroke: RoyalBlue, stroke-width: 2px; - linkStyle 2,3 stroke: RoyalBlue, stroke-width: 2px; - linkStyle 10 stroke: Teal, stroke-width: 3px; + linkStyle 3,4 stroke: RoyalBlue, stroke-width: 2px; + linkStyle 11 stroke: Teal, stroke-width: 3px; ``` #### RLQS xDS HTTP Filter: Channel Level @@ -237,7 +238,7 @@ RLQS Filter State object will include the following data members: - Matcher Tree: From filter config, initialized at instantiation, constant. Used to identify the RLQS Bucket for each data plane RPC. -- RLQS Bucket Cache: Accessed on each data plane RPC, when we get a response +- RLQS Bucket Map: Accessed on each data plane RPC, when we get a response from the RLQS server, and when report timers fire. - RLQS Client: Accessed when we get the first data plane RPC for a given bucket and when a report timer fires. Notifies RLQS Filter State of @@ -247,7 +248,7 @@ RLQS Filter State object will include the following data members: - Entries inserted we get the first data plane RPC for a given bucket and there's no key for bucket reporting interval. - Entries deleted when there's more buckets with given reporting interval - in RLQS Bucket Cache. + in RLQS Bucket Map. Pseudo-code for RLQS Filter State RPC rate limiting: @@ -292,28 +293,27 @@ final class RlqsFilterState { } ``` -#### RLQS Bucket Cache +#### RLQS Bucket Map -The RLQS Bucket Cache is responsible for storing and managing the lifecycle of -RLQS Buckets. It provides a thread-safe way to access and update buckets, +The RLQS Bucket Map class is responsible for storing and managing the lifecycle +of RLQS Buckets. It provides a thread-safe way to access and update buckets, ensuring that only one instance of a RLQS bucket exists for a given `bucket_id`. -The cache also allows to retrieve all buckets that need to be reported to the +RLQS Bucket Map allows to retrieve all buckets that need to be reported to the RLQS server at a given reporting interval. Depending on implementation, this component may be inlined into RLQS Filter State. -RLQS Bucket Cache object will include the following data members: +RLQS Bucket Map object will include the following data members: -- Buckets Map: Initialized empty at instantiation, thread-safe. Entries - retrieved on each data plane RPC and when report timers fire. Entries - inserted when we get the first data plane RPC for a given bucket, or when - instructed by the RLQS Server. Entries are deleted either by RLQS server via - `abandon_action`, or on report timers if a bucket's active assignment - expires. +- Bucket Map: Initialized empty at instantiation, thread-safe. + - Entries retrieved on each data plane RPC and when report timers fire. + - Entries inserted when we get the first data plane RPC for a given + bucket, or when instructed by the RLQS Server. + - Entries deleted either by RLQS server via `abandon_action`, or on report + timers if a bucket's active assignment expires. - Buckets Per Interval Map: Initialized empty at instantiation, thread-safe. - Entries retrieved when a report timer fires. Entries inserted and deleted at - the same time as Buckets Map. Used to efficiently access the set of buckets - for a given report interval. A map from reporting interval to a set of - `RlqsBucket` instances. + Used to track the buckets per discovered report intervals. + - Entries retrieved when a report timer fires. + - Entries inserted and deleted in lock step with Buckets Map updates. ##### RLQS Bucket @@ -337,12 +337,12 @@ RLQS Bucket object will include the following data members: report timers and RLQS server responses. Tracks the number allowed/denied requests for the bucket. -##### RLQS Bucket Cache Multithreading +##### RLQS Buckets and Multithreading There are several mutex-synchronized operations on RLQS buckets that are executed during latency-sensitive data plane RPC processing: -1. Inserting/reading a bucket from the RLQS Bucket Cache using `bucket_id`. Note +1. Inserting/reading a bucket from the RLQS Bucket Map using `bucket_id`. Note that `bucket_id` is represented as a `Map`, which may introduce complexities in efficient cache sharding for certain programming languages. @@ -393,8 +393,8 @@ plane RPC handlers. When processing each data plane RPC, the RLQS Filter will ask the RLQS Filter State for a rate-limit decision for the RPC. The RLQS Filter State uses the matcher tree from the filter config to determine which bucket to use for the -RPC. It then looks for that RLQS Bucket in the RLQS Bucket Cache map, creating -it if it doesn't already exist. +RPC. It then looks for that RLQS Bucket in the RLQS Bucket Map, creating it if +it doesn't already exist. If a new bucket has been created, the RPC is rate-limited according to bucket's default configuration. Then the filter schedules a report on the RLQS stream @@ -410,7 +410,7 @@ counter. When receiving an RLQS Server Response, the RLQS Client passes parsed response to the RLQS Filter State. The RLQS Filter State iterates through the bucket assignments in the response, and updates the corresponding buckets in the RLQS -Bucket Cache. Buckets marked to be abandoned are purged from the cache. +Bucket Map. Buckets marked to be abandoned are purged from the cache. If a bucket has a new rate limit assignment, the bucket's active assignment is updated and bucket usage counters are reset. Otherwise, only the assignment @@ -419,9 +419,9 @@ expiration is updated. #### On Report Timers When a report timer fires, the RLQS Filter State retrieves all buckets with the -corresponding reporting interval from the RLQS Bucket Cache. For each bucket, -the filter snapshots the current usage counters, and resets them. The filter -then sends the snapshot to RLQS server using RLQS Client. +corresponding reporting interval from the RLQS Bucket Map. For each bucket, the +filter snapshots the current usage counters, and resets them. The filter then +sends the snapshot to RLQS server using RLQS Client. If a bucket's active assignment has expired, the bucket is removed from the cache. If there are no more buckets with the given reporting interval, the From 57c11b7c1861db9fca3bea89dcb524bdc7d5c480 Mon Sep 17 00:00:00 2001 From: Sergii Tkachenko Date: Tue, 29 Apr 2025 12:59:32 -0700 Subject: [PATCH 20/53] proofreading --- A77-xds-rate-limiting-rlqs.md | 6 +++--- 1 file changed, 3 insertions(+), 3 deletions(-) diff --git a/A77-xds-rate-limiting-rlqs.md b/A77-xds-rate-limiting-rlqs.md index feae859e3..e0308ed83 100644 --- a/A77-xds-rate-limiting-rlqs.md +++ b/A77-xds-rate-limiting-rlqs.md @@ -41,8 +41,8 @@ which are covered in the proposal: 3. RPCs will be matched into buckets using [Unified Matcher API]. 4. One of the matching mechanisms will be [CEL](https://cel.dev/) (Common Expression Language). -5. RLQS filter state will persist across LDS/RDS updates using cache retention - mechanism similar to the one implemented for [gRFC A83]. +5. RLQS Filter State will persist across LDS/RDS updates using cache retention + mechanism described in [gRFC A83]. ### Related Proposals @@ -90,7 +90,7 @@ graph TD request{{RPC}} end subgraph rlqs_server_box [RLQS Server] - rlqs[(RLQS)] + rlqs[(RLQS
Service)] end subgraph grpc_server_box [gRPC Server] rlqs_filter(RLQS Filter: Channel Level) From 6ab4494809306970e8a20ce7142e73f7ef2cade3 Mon Sep 17 00:00:00 2001 From: Sergii Tkachenko Date: Tue, 22 Apr 2025 12:24:37 -0700 Subject: [PATCH 21/53] Address Mark's feedback --- A77-xds-rate-limiting-rlqs.md | 557 ++++++++++++++++++++-------------- 1 file changed, 335 insertions(+), 222 deletions(-) diff --git a/A77-xds-rate-limiting-rlqs.md b/A77-xds-rate-limiting-rlqs.md index d2747790b..3cb7e3ecf 100644 --- a/A77-xds-rate-limiting-rlqs.md +++ b/A77-xds-rate-limiting-rlqs.md @@ -5,8 +5,9 @@ A77: xDS Server-Side Rate Limiting * Approver: Mark Roth (@markdroth) * Status: In Review * Implemented in: -* Last updated: 2024-11-04 -* Discussion at: `TODO(sergiitk): ` +* Last updated: 2025-04-29 +* Discussion at: + - [ ] TODO(sergiitk): insert google group thread ## Abstract @@ -40,10 +41,10 @@ which are covered in the proposal: 3. RPCs will be matched into buckets using [Unified Matcher API]. 4. One of the matching mechanisms will be [CEL](https://cel.dev/) (Common Expression Language). -5. RLQS filter state will persist across LDS/RDS updates using cache retention - mechanism similar to the one implemented for [gRFC A83]. +5. RLQS Filter State will persist across LDS/RDS updates using cache retention + mechanism described in [gRFC A83]. -### Related Proposals: +### Related Proposals * [A27: xDS-Based Global Load Balancing][gRFC A27] * [A36: xDS-Enabled Servers][gRFC A36] @@ -57,6 +58,8 @@ which are covered in the proposal: [gRFC A39]: A39-xds-http-filters.md [gRFC A83]: A83-xds-gcp-authn-filter.md +[RLQS xDS HTTP Filter: Channel Level]: #rlqs-xds-http-filter-channel-level + [`GrpcService`]: https://www.envoyproxy.io/docs/envoy/latest/api-v3/config/core/v3/grpc_service.proto [`GrpcService.GoogleGrpc`]: https://www.envoyproxy.io/docs/envoy/latest/api-v3/config/core/v3/grpc_service.proto#envoy-v3-api-msg-config-core-v3-grpcservice-googlegrpc [Unified Matcher API]: https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/advanced/matching/matching_api.html @@ -80,105 +83,375 @@ config: clusterBorder: "#777" --- graph TD -%% RLQS Components Flowchart v8 +%% RLQS Components Flowchart v10 %% == nodes == subgraph grpc_client_box [gRPC Client] request{{RPC}} end subgraph rlqs_server_box [RLQS Server] - rlqs[(RLQS)] + rlqs[(RLQS
Service)] end subgraph grpc_server_box [gRPC Server] - rlqs_filter(RLQS HTTP Filter) - rlqs_cache(RLQS Cache) + rlqs_filter(RLQS Filter: Channel Level) + rlqs_filter_state_cache(RLQS Filter State Cache) subgraph rlqs_filter_state_box [RLQS Filter State] rlqs_client(RLQS Client) rlqs_filter_state(RLQS Filter State) report_timers(Report Timers) matcher_tree(Matcher Tree) - rlqs_bucket_cache(RLQS Bucket Cache) + rlqs_bucket_map(RLQS Bucket Map) rlqs_bucket(RLQS Bucket) end - rpc_handler("Filter's onClientCall handler") + rpc_handler("RLQS Filter: Call Level") end %% == edges == - rlqs_filter -- " Get RLQS Filter State
per unique config " --> rlqs_cache -- " getOrCreate(config) " --> rlqs_filter_state - rlqs_filter -- " Pass RLQS Filter State
for the route " --> rpc_handler -- " rateLimit(call) " --> rlqs_filter_state + rlqs_filter -- "Get RLQS Filter State
per unique config" --> rlqs_filter_state_cache -- "getOrCreate(config)" --> rlqs_filter_state + rlqs_filter -- " Generate per-route RPC handlers
using RLQS Filter State " --> rpc_handler -- " rlqsFilterState.rateLimit(call) " --> rlqs_filter_state request --> rpc_handler rlqs_filter_state --o matcher_tree & report_timers rlqs_filter_state -- sendUsageReports --> rlqs_client - rlqs_filter_state -- CRUD --> rlqs_bucket_cache + rlqs_filter_state -- CRUD --> rlqs_bucket_map rlqs_client -- onBucketsUpdate --> rlqs_filter_state - rlqs_bucket_cache -- " getOrCreate(bucketId)
Atomic Updates " --> rlqs_bucket + rlqs_bucket_map -- " getOrCreate(bucketId)
Atomic Updates " --> rlqs_bucket rlqs_client <-. gRPC Stream .-> rlqs style request stroke: RoyalBlue, stroke-width: 2px; linkStyle 3,4 stroke: RoyalBlue, stroke-width: 2px; linkStyle 11 stroke: Teal, stroke-width: 3px; ``` -##### RLQS HTTP Filter +#### RLQS xDS HTTP Filter: Channel Level + +In order to retain filter state across LDS/RDS updates, the actual logic for the +RLQS filter will be moved into a separate object called RLQS Filter State, which +will be stored in the persistent filter state mechanism described in [gRFC A83]. +The key in the persistent filter state will be the RLQS xDS HTTP filter config, +which ensures that two RLQS filter instances with the same config will share +filter state but two RLQS filter instances with different configs will each have +their own filter state. + +Channel-level RLQS xDS HTTP Filter object will include the following data +members: + +- RLQS Filter State Cache: from filter config, initialized at instantiation, + retained across LDS/RDS updates. A 1:1 mapping between unique RLQS Filter + Config instances and corresponding unique RLQS Filter State instances. + +Pseudo-code: + +```java +final class RlqsFilter implements Filter { + private final ConcurrentMap + filterStateCache = new ConcurrentHashMap<>(); + // ... + + @Override + public ServerInterceptor buildServerInterceptor( + FilterConfig config, @Nullable FilterConfig overrideConfig) { + // Parse the filter config. + RlqsFilterConfig.Builder rlqsConfigBuilder = parseRlqsConfig(config); + + // Merge with per-route overrides if provided. + if (overrideConfig instanceof RlqsConfigOverride rlqsConfigOverride) { + // Only domain and matchers can be overriden. + if (!rlqsConfigOverride.domain().isEmpty()) { + rlqsConfigBuilder.domain(rlqsConfigOverride.domain()); + } + if (rlqsConfigOverride.bucketMatchers() != null) { + rlqsConfigBuilder.bucketMatchers(rlqsConfigOverride.bucketMatchers()); + } + } + + // Get or Create RLQS Filter Config from persistent filter state. + RlqsFilterState rlqsFilterState = filterStateCache.computeIfAbsent( + rlqsConfigBuilder.build(), + (cfg) -> new RlqsFilterState(cfg, getRlqsServerInfo(cfg.rlqsService())) + ); + return new RlqsServerInterceptor(rlqsFilterState); + } +} +``` + +##### Future considerations + +This proposal uses the entire RLQS Filter Config to identify the corresponding +unique Filter State. As a result, a Filter State will be recreated even if only +inconsequential fields, such as the deny response status, are changed. If this +becomes a problem, additional logic should be introduced to exclude such fields +when comparing Filter Config objects. + +#### RLQS xDS HTTP Filter: Call Level + +When processing a data plane RPC for a given route, the filter passes the RPC +metadata to the corresponding RLQS Filter State instance for evaluation. Based +on the evaluation result, the filter either allows the RPC to proceed, or denies +it with a specific gRPC status. The evaluation result may also contain a list of +HTTP headers to add to the original request, or the deny response. + +Call-level RLQS xDS HTTP Filter object will include the following data members: + +- RLQS Filter State instance corresponding route's RLQS Filter Config: + implementation-dependent. + +Pseudo-code: + +```java +private static class RlqsServerInterceptor implements ServerInterceptor { + private final RlqsFilterState rlqsFilterState; + + public RlqsServerInterceptor(RlqsFilterState rlqsFilterState) { + this.rlqsFilterState = rlqsFilterState; + } + + @Override + public ServerCall.Listener interceptCall( + ServerCall call, Metadata headers, + ServerCallHandler next) { + // RlqsClient matches the request into a bucket, + // and returns the rate limiting result. + RlqsRateLimitResult result = + rlqsFilterState.rateLimit(HttpMatchInput.create(headers, call)); + + if (result.isAllowed()) { + // TODO: append request_headers_to_add_when_not_enforced + return next.startCall(call, headers); + } + + // Denied: fail the call with given Status. + call.close( + result.denyResponse().status(), + result.denyResponse().headersToAdd()); + return new ServerCall.Listener(){}; + } +} +``` + +#### RLQS Filter State + +RLQS Filter State manages the runtime state and logic for a unique RLQS Filter +Config. It's responsible for matching data plane RPCs to buckets, managing the +lifecycle of those buckets, and communicating with the RLQS server to report +usage and receive quota assignments. Filter state instances are persisted across +LDS/RDS updates, see [RLQS xDS HTTP Filter: Channel Level]. + +RLQS Filter State object will include the following data members: + +- Matcher Tree: From filter config, initialized at instantiation, constant. + Used to identify the RLQS Bucket for each data plane RPC. +- RLQS Bucket Map: Accessed on each data plane RPC, when we get a response + from the RLQS server, and when report timers fire. +- RLQS Client: Accessed when we get the first data plane RPC for a given + bucket and when a report timer fires. Notifies RLQS Filter State of + responses received from the RLQS server. +- Report Timers Map: Initialized at instantiation. Used to track discovered + reporting intervals and their execution handlers. + - Entries inserted we get the first data plane RPC for a given bucket and + there's no key for bucket reporting interval. + - Entries deleted when there's more buckets with given reporting interval + in RLQS Bucket Map. + +Pseudo-code for RLQS Filter State RPC rate limiting: + +```java +final class RlqsFilterState { + private final RlqsClient rlqsClient; + private final Matcher bucketMatchers; + private final RlqsBucketCache bucketCache; + private final ScheduledExecutorService scheduler; + private final ConcurrentMap> + timers = new ConcurrentHashMap<>(); + // ... + + public RlqsRateLimitResult rateLimit(HttpMatchInput input) { + // Perform request matching. The result is RlqsBucketSettings. + RlqsBucketSettings bucketSettings = bucketMatchers.match(input); + // BucketId may be dynamic (f.e. based on headers). + RlqsBucketId bucketId = bucketSettings.bucketIdForRequest(input); + // Get or create RLQS Bucket. + RlqsBucket bucket = bucketCache.getOrCreate( + bucketId, bucketSettings, this::onNewBucket); + return bucket.rateLimit(); + } + + private void onNewBucket(RlqsBucket newBucket) { + // The report for the first RPC is sent immediately. + scheduleImmediateReport(newBucket); + registerReportTimer(newBucket.getReportingInterval()); + } + + private void scheduleImmediateReport(RlqsBucket newBucket) { + // Do not block data plane RPC on sending the report. + scheduler.schedule( + () -> rlqsClient.sendUsageReports(newBucket.snapshotAndResetUsage()), + 1, TimeUnit.MICROSECONDS) + } + + private void registerReportTimer(final long intervalMillis) { + // TODO: bound / roundup the report interval for better grouping. + timers.computeIfAbsent(intervalMillis, k -> newTimer(intervalMillis)); + } +} +``` + +#### RLQS Bucket Map + +The RLQS Bucket Map class is responsible for storing and managing the lifecycle +of RLQS Buckets. It provides a thread-safe way to access and update buckets, +ensuring that only one instance of a RLQS bucket exists for a given `bucket_id`. +RLQS Bucket Map allows to retrieve all buckets that need to be reported to the +RLQS server at a given reporting interval. Depending on implementation, this +component may be inlined into RLQS Filter State. + +RLQS Bucket Map object will include the following data members: + +- Bucket Map: Initialized empty at instantiation, thread-safe. + - Entries retrieved on each data plane RPC and when report timers fire. + - Entries inserted when we get the first data plane RPC for a given + bucket, or when instructed by the RLQS Server. + - Entries deleted either by RLQS server via `abandon_action`, or on report + timers if a bucket's active assignment expires. +- Buckets Per Interval Map: Initialized empty at instantiation, thread-safe. + Used to track the buckets per discovered report intervals. + - Entries retrieved when a report timer fires. + - Entries inserted and deleted in lock step with Buckets Map updates. + +##### RLQS Bucket + +The RLQS Bucket tracks the current rate limiting assignment and the quota +usage for a specific bucket, identified by a `bucket_id`. It uses this data to +determine whether to allow or deny a request. The bucket also provides a +thread-safe mechanism to snapshot and reset its quota usage when building usage +reports. + +RLQS Bucket object will include the following data members: + +- `bucket_id`: From filter config and RPC metadata, constant, initialized at + bucket creation. A unique identifier for the bucket. +- "No Assignment" and "Expired Assignment" strategies: Initialized at first + RPC for the bucket from filter config, constant. Fallback rate limiting + strategies. +- Active Assignment: Initialized at first RPC for the bucket from filter + config, updated on RLQS server responses. The current rate limiting strategy + to apply to requests, and associated expiration timestamps. +- Request Counters: Initialized at 0, updated on data plane RPCs, reset on + report timers and RLQS server responses. Tracks the number allowed/denied + requests for the bucket. + +##### RLQS Buckets and Multithreading + +There are several mutex-synchronized operations on RLQS buckets that are +executed during latency-sensitive data plane RPC processing: + +1. Inserting/reading a bucket from the RLQS Bucket Map using `bucket_id`. Note + that `bucket_id` is represented as a `Map`, which may + introduce complexities in efficient cache sharding for certain programming + languages. +2. Incrementing `num_request_allowed`/`num_request_denied` bucket counters. + +Each gRPC implementation needs to consider what synchronization primitives are +available in their language to minimize the thread lock time. + +#### RLQS Client + +The RLQS Client is responsible for communication with the Rate Limit Quota +Service (RLQS) server. It manages a gRPC stream to the RLQS server, used for +sending periodic bucket usage reports and receiving new rate limit quota +assignments. The client handles reconnects and manages the lifecycle of the +stream. + +RLQS Client object will include the following data members: + +- gRPC Channel: From filter config, initialized at instantiation, constant. A + channel to the RLQS server. +- gRPC Stream: Initialized at instantiation, replaced any time the stream + fails (with backoff). Accessed when we get the first data plane RPC for a + given bucket and when a report timer fires. A bidirectional gRPC stream to + the RLQS server. +- Buckets Update Callback: Provided by RLQS Filter state at instantiation, + constant, accessed on responses from the RLQS server. A callback to notify + the RLQS Filter State of updates to quota assignments. + +### Handling Events + +#### On LDS/RDS Updates + +When receiving an LDS/RDS update, the RLQS filter will perform the following +steps for each route: + +1. Parse the new filter config and per-route overrides into an internal RLQS + Filter Config object. +2. Retrieve the corresponding RLQS Filter State from the RLQS Filter State + Cache, creating it if it doesn't exist. +3. Generate new data plane RPC handler using the retrieved RLQS Filter State. + +Once all routes are parsed, the RLQS filter will release from the RLQS Filter +State Cache all RLQS Filter State instances no longer referenced by any data +plane RPC handlers. -The filter parses the config, combines LDS filter config with RDS overrides, and -generates the `onClientCall` handlers (aka interceptors in Java and Go, and -filters in C++). +#### On Data Plane RPC -##### RLQS Cache +When processing each data plane RPC, the RLQS Filter will ask the RLQS Filter +State for a rate-limit decision for the RPC. The RLQS Filter State uses the +matcher tree from the filter config to determine which bucket to use for the +RPC. It then looks for that RLQS Bucket in the RLQS Bucket Map, creating it if +it doesn't already exist. -RLQS Cache persists across LDS/RDS updates. It maps unique filter configs to -RLQS Filter State instances, and provides the thread safety for creating and -accessing them. Each unique filter config generates a unique RLQS Filter state, -a 1:1 mapping. +If a new bucket has been created, the RPC is rate-limited according to bucket's +default configuration. Then the filter schedules a report on the RLQS stream +informing the server of the bucket's creation, and registers the report timers +for bucket's reporting interval. -##### RLQS Filter State +If the bucket exists, the RPC is rate-limited according to its active quota +assignment and usage counters. The bucket increments corresponding bucket usage +counter. -RLQS Filter State contains the business logic for rate limiting, and the current -state of rate limit assignments per bucket. RLQS Filter State is what's passed -to the `onCallHandler`. It exposes the public "`rateLimit()`" method, which -takes request metadata as an argument. +#### On RLQS Server Response -##### Matching +When receiving an RLQS Server Response, the RLQS Client passes parsed response +to the RLQS Filter State. The RLQS Filter State iterates through the bucket +assignments in the response, and updates the corresponding buckets in the RLQS +Bucket Map. Buckets marked to be abandoned are purged from the cache. -RLQS Filter State evaluates the metadata against the matcher tree to match the -request into a bucket. The Bucket holds the Rate Limit Quota assigned by the -RLQS server (f.e. 100 requests per minute), and aggregates the number of -requests it allowed/denied. This information is used to make the rate limiting -decision. +If a bucket has a new rate limit assignment, the bucket's active assignment is +updated and bucket usage counters are reset. Otherwise, only the assignment +expiration is updated. -##### Reporting +#### On Report Timers -The aggregated number of requests is reported to the RLQS server at configured -intervals. The report action is triggered by the Report Timers. RLQS Client -manages a gRPC stream to the RLQS server. It's used by the filter state to send -periodic bucket usage reports, and to receive new rate limit quota assignments -to the buckets. +When a report timer fires, the RLQS Filter State retrieves all buckets with the +corresponding reporting interval from the RLQS Bucket Map. For each bucket, the +filter snapshots the current usage counters, and resets them. The filter then +sends the snapshot to RLQS server using RLQS Client. -### Connecting to RLQS Control Plane +If a bucket's active assignment has expired, the bucket is removed from the +cache. If there are no more buckets with the given reporting interval, the +corresponding timer is removed. + +### Integrations + +#### Connecting to RLQS Control Plane xDS Control Plane provides RLQS connection details in [`GrpcService`] message ( -already supported by Envoy). `GrpcService` supports two modes: +already supported by Envoy). `GrpcService` supports two modes: 1. `GrpcService.EnvoyGrpc`, Envoy's minimal custom gRPC client implementation. 2. [`GrpcService.GoogleGrpc`], regular gRPC-cpp client. For obvious reasons, we'll only support the `GoogleGrpc` mode. -#### Security Considerations - -In [`GrpcService.GoogleGrpc`], xDS Control Plane provides the target -URI, `channel_credentials`, and `call_credentials`. If the xDS Control Plane is -compromised, the attacker could configure the xDS clients to talk to other -malicious Control Plane, leading to such potential exploits as: +##### Security Considerations -1. Leaking customer's Application Default Credentials OAuth token. -2. Causing MalOut/DDoS by sending bad data from the compromised RLQS (f.e. set - rate limit policy to `ALLOW_ALL`/`DENY_ALL`). +In [`GrpcService.GoogleGrpc`], the xDS Control Plane provides the target URI, +`channel_credentials`, and `call_credentials`. If the xDS Control Plane is +compromised, an attacker could configure the xDS clients to talk to other +malicious Control Planes. This could lead to potential exploits such as leaking +customer's Application Default Credentials OAuth token. -To prevent that, we'll introduce the allow-list to the bootstrap file introduced -in [gRFC A27]. This allow-list will be a map from the fully-qualified server -target URI to an object containing channel credentials to use. +To prevent that, we'll introduce an allow-list to the bootstrap file introduced +in [gRFC A27]. This allow-list will be a map from a fully-qualified server +target URI to an object containing channel credentials to use for that target. -```javascript +```json5 // The allowlist of Control Planes allowed to be configured via xDS. "allowed_grpc_services": { // The key is fully-qualified server URI. @@ -186,7 +459,7 @@ target URI to an object containing channel credentials to use. // The value is an object containing "channel_creds". "channel_creds": [ // The format is identical to xds_servers.channel_creds. - { + { "type": "string containing channel cred type", "config": "optional JSON object containing config for the type" } @@ -209,7 +482,7 @@ matching target URI. > This solution is not specific to RLQS, and should be used with any > other Control Planes configured via [`GrpcService`] message. -### Unified Matcher API +#### Unified Matcher API RPCs will be matched into buckets using [Unified Matcher API] — an adaptable framework that can be used in any xDS component that needs matching features. @@ -233,7 +506,7 @@ In this iteration the following Unified Mather extensions will be supported: 2. Custom Matchers: 1. [`CelMatcher`](https://www.envoyproxy.io/docs/envoy/latest/xds/type/matcher/v3/cel.proto.html) -### CEL Integration +#### CEL Integration We will support request metadata matching via CEL expressions. Only Canonical CEL and only checked expressions will be supported (`cel.expr.CheckedExpr`). @@ -241,7 +514,7 @@ CEL and only checked expressions will be supported (`cel.expr.CheckedExpr`). CEL evaluation environment is a set of available variables and extension functions in a CEL program. We will match [Envoy CEL environment]. -#### Supported CEL Functions +##### Supported CEL Functions Similar to Envoy, we will support [standard CEL functions](https://github.com/google/cel-spec/blob/c629b2be086ed6b4c44ef4975e56945f66560677/doc/langdef.md#standard-definitions) @@ -263,7 +536,7 @@ except comprehension-style macros. [RE2_wiki]: https://en.wikipedia.org/wiki/RE2_(software) -#### Supported CEL Variables +##### Supported CEL Variables For RLQS, only the `request` variable is supported in CEL expressions. We will adapt [Envoy's Request Attributes](https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/advanced/attributes#request-attributes) @@ -284,7 +557,7 @@ for gRPC. | `request.protocol` | `string` | Not set | Request protocol. | | `request.query` | `string` | `""` | The query portion of the URL. | -##### Footnotes +###### Footnotes **1 `request.path`** @@ -313,166 +586,6 @@ provides the different variable resolving approaches based on the language: * Go: [`Activation.ResolveName(string)`](https://github.com/google/cel-go/blob/3f12ecad39e2eb662bcd82b6391cfd0cb4cb1c5e/interpreter/activation.go#L30) * Java: [`CelVariableResolver`](https://javadoc.io/doc/dev.cel/runtime/0.6.0/dev/cel/runtime/CelVariableResolver.html) -### Persistent Filter Cache - -RLQS Filter State holds the bucket usage data, report timers and the -bidirectional stream to the RLQS server. To prevent the loss of state across -LDS/RDS updates, RLQS filter will require a cache retention mechanism similar to -the one implemented for [gRFC A83]. - -The scope of each RLQS Filter Cache instance will be per server instance (same -scope as the filter chain) and per filter name. - -RLQS implementations will provide a mechanism for new instances of the filter to -retain the cache from previous instances. There may be multiple instances of the -RLQS Filter State, each one mapped to a unique filter config generated from LDS -config, and RDS overrides. Consider the following example that demonstrates the -lifecycle of RLQS Filter Cache. - -```mermaid ---- -config: - sequence: - showSequenceNumbers: true - height: 46 - diagramMarginX: 40 - diagramMarginY: 40 ---- -sequenceDiagram - -%% gRFC: RLQS Filter Cache Lifecycle v1.1 - participant xds as Control Plane - participant filter as RLQS HTTP Filter - participant cache as RLQS Cache - participant e1 as RlqsFilterState(c1) - participant e2 as RlqsFilterState(c2) - -# Notes - Note right of xds: r1-4: routes
c1-2: unique filter configs -%% LDS 1 - xds ->> filter: LDS1
RLQS{r1=c1, r2=c2, r3=c2} - filter ->> cache: r1: getOrCreate(c1) - cache ->>+ e1: new RlqsFilterState(c1) - filter ->> cache: r2: getOrCreate(c2) - cache ->>+ e2: new RlqsFilterState(c2) - filter ->> cache: r3: getOrCreate(c2) - Note over filter: r1: RlqsFilterState(c1)
r2: RlqsFilterState(c2)
r3: RlqsFilterState(c2) -%% RDS 1 - xds ->> filter: RDS1
RLQS{r1=c2} - filter ->> cache: r1: getOrCreate(c2) - filter ->> cache: shutdownFilterState(c1) - cache -x e1: RlqsFilterState(c1).shutdown() - deactivate e1 - Note over filter: r1: RlqsFilterState(c2)
r2: RlqsFilterState(c2)
r3: RlqsFilterState(c2) -%% LDS 2 - xds ->> filter: LDS2
RLQS{r3=c2, r4=c2} - filter ->> cache: r4: getOrCreate(c2) - Note over filter: r3: RlqsFilterState(c2)
r4: RlqsFilterState(c2) -%% End - deactivate e2 -``` - -**LDS 1** - -In this example, the RLQS filter is configured for three routes: `r1`, `r2`, -and `r3`. Each unique config generates a unique RLQS Filter -State: `RlqsFilterState(c1)` for the config `c1`, and `RlqsFilterState(c2)` for -the config `c2`. After processing the first LDS update, we've generated -onCallHandlers for three routes: - -1. `r1`, referencing `RlqsFilterState(c1)`. -2. `r2`, referencing `RlqsFilterState(c2)`. -3. `r3`, also referencing `RlqsFilterState(c2)`. - -**RDS 1** - -RDS 1 updates RLQS config for the route `r1` so it's identical to config `c2`. -We retrieve `RlqsFilterState(c2)` from the RLQS Cache and generate new -onCallHandlers for route `r2`. `RlqsFilterState(c1)` is no longer referenced by -any onCallHandler, and can be destroyed with all associated resources. - -**LDS 2** - -LDS 2 update removes `r1` and `r2`, and adds new route r4 with the config -identical to `c2`. While onCallHandlers for routes `r1` and `r2` are -destroyed, `RlqsFilterState(c2)` is still used by two onCallHandlers, so it's -preserved in RLQS Cache. - -##### Future considerations - -With this proposal, the filter state is lost if change is made to the filter -config, including updates to inconsequential fields such as deny response -status. Additional logic can be introduced to handle updates to such fields -while preserving the filter state. - -### Multithreading - -There are several mutex-synchronized operations executed in -latency-sensitive `onCallHandler`: - -1. Inserting/reading a bucket from the bucket cache using `bucket_id`. Note - that `bucket_id` is represented as a `Map`, which may - introduce complexities in efficient cache sharding for certain programming - languages. -2. Incrementing `num_request_allowed`/`num_request_denied` bucket counters. - -Each gRPC implementation needs to consider what synchronization primitives are -available in their language to minimize the thread lock time. - -### Code Samples - -> [!NOTE] -> Not a reference implementation. Only for flow illustration purposes. - -#### On Call Handler -```java -final RlqsFilterState filterState = rlqsCache.getOrCreateFilterState(config); - -return new ServerInterceptor() { - @Override - public Listener interceptCall( - ServerCall call, - Metadata headers, ServerCallHandler next) { - // TODO: handle filter_enabled and filter_enforced - - // RlqsClient matches the request into a bucket, - // and returns the rate limiting result. - RlqsRateLimitResult result = - filterState.rateLimit(HttpMatchInput.create(headers, call)); - - // Allowed. - if (result.isAllowed()) { - return next.startCall(call, headers); - } - // Denied: fail the call with given Status. - call.close( - result.denyResponse().status(), - result.denyResponse().headersToAdd()); - return new ServerCall.Listener(){}; - } -}; -``` - -#### Bucket Matching -```java -public RlqsRateLimitResult rateLimit(HttpMatchInput input) { - // Perform request matching. The result is RlqsBucketSettings. - RlqsBucketSettings bucketSettings = bucketMatchers.match(input); - // BucketId may be dynamic (f.e. based on headers). - RlqsBucketId bucketId = bucketSettings.bucketIdForRequest(input); - - RlqsBucket bucket = bucketCache.getOrCreate(bucketId, bucketSettings, this::onNewBucket); - return bucket.rateLimit(); -} - -private void onNewBucket(RlqsBucket newBucket) { - // The report for the first RPC is sent immediately. - scheduleImmediateReport(newBucket); - registerReportTimer(newBucket.getReportingInterval()); -} -``` - - ### Temporary environment variable protection During initial development, this feature will be enabled via From b010cbf3475e3f50421e8ccb8133b051925ee974 Mon Sep 17 00:00:00 2001 From: Sergii Tkachenko Date: Tue, 29 Apr 2025 13:12:06 -0700 Subject: [PATCH 22/53] --- synced with origin/a77-rlqs --- --- A77-xds-rate-limiting-rlqs.md | 2 +- 1 file changed, 1 insertion(+), 1 deletion(-) diff --git a/A77-xds-rate-limiting-rlqs.md b/A77-xds-rate-limiting-rlqs.md index e0308ed83..3cb7e3ecf 100644 --- a/A77-xds-rate-limiting-rlqs.md +++ b/A77-xds-rate-limiting-rlqs.md @@ -5,7 +5,7 @@ A77: xDS Server-Side Rate Limiting * Approver: Mark Roth (@markdroth) * Status: In Review * Implemented in: -* Last updated: 2025-04-22 +* Last updated: 2025-04-29 * Discussion at: - [ ] TODO(sergiitk): insert google group thread From 30fce7a73c579366f44a90fd0472d5e549ead798 Mon Sep 17 00:00:00 2001 From: Sergii Tkachenko Date: Mon, 12 May 2025 11:41:57 -0700 Subject: [PATCH 23/53] r2 initial feedback - minor comments / spelling --- A77-xds-rate-limiting-rlqs.md | 31 ++++++++++--------------------- 1 file changed, 10 insertions(+), 21 deletions(-) diff --git a/A77-xds-rate-limiting-rlqs.md b/A77-xds-rate-limiting-rlqs.md index 3cb7e3ecf..3b46cb1c1 100644 --- a/A77-xds-rate-limiting-rlqs.md +++ b/A77-xds-rate-limiting-rlqs.md @@ -247,8 +247,8 @@ RLQS Filter State object will include the following data members: reporting intervals and their execution handlers. - Entries inserted we get the first data plane RPC for a given bucket and there's no key for bucket reporting interval. - - Entries deleted when there's more buckets with given reporting interval - in RLQS Bucket Map. + - Entries deleted when there's no more buckets with given reporting + interval in RLQS Bucket Map. Pseudo-code for RLQS Filter State RPC rate limiting: @@ -375,7 +375,7 @@ RLQS Client object will include the following data members: #### On LDS/RDS Updates -When receiving an LDS/RDS update, the RLQS filter will perform the following +When receiving an LDS/RDS update, the RLQS filter will perform the following steps for each route: 1. Parse the new filter config and per-route overrides into an internal RLQS @@ -476,7 +476,7 @@ matching target URI. Control Plane, and NACK the xDS resource. 2. If target URI is present, we create the connection to the requested Control Plane using the channel credentials provided in the bootstrap file. Transport - security configuration provided by the TD is ignored. + security configuration provided by the control plane is ignored. > [!IMPORTANT] > This solution is not specific to RLQS, and should be used with any @@ -544,12 +544,12 @@ for gRPC. | Attribute | Type | gRPC source | Envoy Description | |---------------------|-----------------------|------------------------------|-------------------------------------------------------------| -| `request.path` | `string` | Full method name1 | The path portion of the URL. | +| `request.path` | `string` | Full method name | The path portion of the URL. | | `request.url_path` | `string` | Same as `request.path` | The path portion of the URL without the query string. | -| `request.host` | `string` | Authority2 | The host portion of the URL. | +| `request.host` | `string` | Authority | The host portion of the URL. | | `request.scheme` | `string` | Not set | The scheme portion of the URL. | -| `request.method` | `string` | `POST`3 | Request method. | -| `request.headers` | `map` | `metadata`4 | All request headers indexed by the lower-cased header name. | +| `request.method` | `string` | `POST`1 | Request method. | +| `request.headers` | `map` | `metadata`2 | All request headers indexed by the lower-cased header name. | | `request.referer` | `string` | `metadata["referer"]` | Referer request header. | | `request.useragent` | `string` | `metadata["user-agent"]` | User agent request header. | | `request.time` | `timestamp` | Not set | Time of the first byte received. | @@ -559,22 +559,11 @@ for gRPC. ###### Footnotes -**1 `request.path`** - -* CPP: `metadata[":path"]` -* Go: `grpc.Method(ctx)` -* Java: `"/" + serverCall.getMethodDescriptor().getFullMethodName()` - -**2 `request.host`** - -* CPP, Go: `metadata[":authority"]` -* Java: `serverCall.getAuthority()` - -**3 `request.method`**\ +**1 `request.method`**\ Hard-coded to `"POST"` if unavailable and a code audit confirms the server denies requests for all other method types. -**4 `request.headers`**\ +**2 `request.headers`**\ As defined in [gRFC A41], "header" field. ##### Implementation From c28b4a28e128d0818a1e4357600fc8bdcb72370e Mon Sep 17 00:00:00 2001 From: Sergii Tkachenko Date: Mon, 12 May 2025 13:14:27 -0700 Subject: [PATCH 24/53] Filter Configuration wip --- A77-xds-rate-limiting-rlqs.md | 70 ++++++++++++++++++++++++++++++++++- 1 file changed, 68 insertions(+), 2 deletions(-) diff --git a/A77-xds-rate-limiting-rlqs.md b/A77-xds-rate-limiting-rlqs.md index 3b46cb1c1..4847e8474 100644 --- a/A77-xds-rate-limiting-rlqs.md +++ b/A77-xds-rate-limiting-rlqs.md @@ -59,18 +59,84 @@ which are covered in the proposal: [gRFC A83]: A83-xds-gcp-authn-filter.md [RLQS xDS HTTP Filter: Channel Level]: #rlqs-xds-http-filter-channel-level +[GrpcService Security Considerations]: #grpcservice-security-considerations [`GrpcService`]: https://www.envoyproxy.io/docs/envoy/latest/api-v3/config/core/v3/grpc_service.proto [`GrpcService.GoogleGrpc`]: https://www.envoyproxy.io/docs/envoy/latest/api-v3/config/core/v3/grpc_service.proto#envoy-v3-api-msg-config-core-v3-grpcservice-googlegrpc [Unified Matcher API]: https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/advanced/matching/matching_api.html [Envoy CEL environment]: https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/advanced/attributes +[google.protobuf.Duration]: https://developers.google.com/protocol-buffers/docs/reference/google.protobuf#google.protobuf.Duration + [rlqs_proto]: https://www.envoyproxy.io/docs/envoy/latest/api-v3/service/rate_limit_quota/v3/rlqs.proto.html [rlqs_filter_proto]: https://www.envoyproxy.io/docs/envoy/latest/api-v3/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto +[RateLimitQuotaFilterConfig]: https://github.com/envoyproxy/envoy/blob/82bc63199ff429490260e52794bb3095f17bcdae/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L37 +[proto_rlqs_server]: https://github.com/envoyproxy/envoy/blob/82bc63199ff429490260e52794bb3095f17bcdae/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L39 +[proto_domain]: https://github.com/envoyproxy/envoy/blob/82bc63199ff429490260e52794bb3095f17bcdae/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L44 +[proto_bucket_matchers]: https://github.com/envoyproxy/envoy/blob/82bc63199ff429490260e52794bb3095f17bcdae/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L115 +[proto_filter_enabled]: https://github.com/envoyproxy/envoy/blob/82bc63199ff429490260e52794bb3095f17bcdae/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L121 +[proto_filter_enforced]: https://github.com/envoyproxy/envoy/blob/82bc63199ff429490260e52794bb3095f17bcdae/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L132 +[proto_request_headers_to_add_when_not_enforced]: https://github.com/envoyproxy/envoy/blob/82bc63199ff429490260e52794bb3095f17bcdae/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L137 + +[proto_grpc_svc_google_grpc]: https://github.com/envoyproxy/envoy/blob/82bc63199ff429490260e52794bb3095f17bcdae/api/envoy/config/core/v3/grpc_service.proto#L303 +[proto_grpc_svc_target_uri]: https://github.com/envoyproxy/envoy/blob/82bc63199ff429490260e52794bb3095f17bcdae/api/envoy/config/core/v3/grpc_service.proto#L254 +[proto_grpc_svc_timeout]: https://github.com/envoyproxy/envoy/blob/82bc63199ff429490260e52794bb3095f17bcdae/api/envoy/config/core/v3/grpc_service.proto#L308 + +[proto_frac_default_value]: https://github.com/envoyproxy/envoy/blob/82bc63199ff429490260e52794bb3095f17bcdae/api/envoy/config/core/v3/base.proto#L648 +[proto_frac_numerator]: https://github.com/envoyproxy/envoy/blob/82bc63199ff429490260e52794bb3095f17bcdae/api/envoy/type/v3/percent.proto#L52 +[proto_frac_denominator]: https://github.com/envoyproxy/envoy/blob/82bc63199ff429490260e52794bb3095f17bcdae/api/envoy/type/v3/percent.proto#L56 + ## Proposal -### RLQS Components Overview +### Filter Configuration + +We will support the following fields in the [RateLimitQuotaFilterConfig][] +proto: + +- [rlqs_server][proto_rlqs_server]: This field must be present. Inside of it: + - [google_grpc][proto_grpc_svc_google_grpc]: This field must be present. + Inside of it: + - [target_uri][proto_grpc_svc_target_uri]: This field must be + non-empty and must be a valid target URI. The value specified here + must be present in the `allowed_grpc_services` map in the bootstrap + config, which will also determine the credentials to use, as + described in [GrpcService Security Considerations][]. + - All other fields are ignored. + - [timeout][proto_grpc_svc_timeout]: Specifies the deadline for the RPCs + sent to the RLQS server. If unset, there is no deadline. The value must + obey the restrictions specified in the [google.protobuf.Duration][], + documentation and it must have a positive value. + - All other fields are ignored. +- [domain][proto_domain]: This field must be present and non-empty. +- [bucket_matchers][proto_bucket_matchers]: This field must be present. Inside + of it, there must be a valid matchers structure. `TODO(sergiitk): add more + info`. +- [filter_enabled][proto_filter_enabled]: Optional, defaults to 100%. Inside + of it: + - [default_value][proto_frac_default_value]: Required if `filter_enabled` + is set. Specifies the fraction of requests for which the rate limiting + is enabled. Inside of it: + - [numerator][proto_frac_numerator]: Optional, defaults to 0. + - [denominator][proto_frac_denominator]: Optional, defaults to + HUNDRED. + - All other fields are ignored. +- [filter_enforced][proto_filter_enforced]: Optional, defaults to 100%. Inside + of it: + - [default_value][proto_frac_default_value]: Required if `filter_enforced` + is set. Specifies the fraction of requests for which the rate limiting + is enforced. Inside of it: + - [numerator][proto_frac_numerator]: Optional, defaults to 0. + - [denominator][proto_frac_denominator]: Optional, defaults to + HUNDRED. + - All other fields are ignored. +- [request_headers_to_add_when_not_enforced][proto_request_headers_to_add_when_not_enforced]: + Optional. Inside of it, a list of the following items: + - HeaderValueOption `TODO(sergiitk): finish`. + +### RLQS Components + +#### RLQS Components Overview The diagram below shows the conceptual components of the RLQS Filter. Note that the actual implementation may vary depending on the language. @@ -439,7 +505,7 @@ already supported by Envoy). `GrpcService` supports two modes: For obvious reasons, we'll only support the `GoogleGrpc` mode. -##### Security Considerations +##### GrpcService Security Considerations In [`GrpcService.GoogleGrpc`], the xDS Control Plane provides the target URI, `channel_credentials`, and `call_credentials`. If the xDS Control Plane is From d36c209f888982e83c01856cafa8b7f5e9cc3ed4 Mon Sep 17 00:00:00 2001 From: Sergii Tkachenko Date: Tue, 13 May 2025 16:50:52 -0700 Subject: [PATCH 25/53] break down into config messages --- A77-xds-rate-limiting-rlqs.md | 216 +++++++++++++++++++++------------- 1 file changed, 136 insertions(+), 80 deletions(-) diff --git a/A77-xds-rate-limiting-rlqs.md b/A77-xds-rate-limiting-rlqs.md index 4847e8474..9891d4890 100644 --- a/A77-xds-rate-limiting-rlqs.md +++ b/A77-xds-rate-limiting-rlqs.md @@ -20,12 +20,12 @@ servers. [Global rate limiting](https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/other_features/global_rate_limiting) allows mesh users to manage fair consumption of their services and prevent -misbehaving clients from overloading the services. We will -implement [quota-based rate limiting](https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/other_features/global_rate_limiting#quota-based-rate-limiting), -where rate-limiting decisions are asynchronously offloaded -to [Rate Limiting Quota Service][rlqs_proto] (RLQS). Requests are grouped into -buckets based on their metadata, and gRPC servers periodically report bucket -usages. RLQS aggregates the data from different gRPC servers, and fairly +misbehaving clients from overloading the services. We will implement +[quota-based rate limiting](https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/other_features/global_rate_limiting#quota-based-rate-limiting), +where rate-limiting decisions are asynchronously offloaded to +[Rate Limiting Quota Service][rlqs_service_proto] (RLQS). Requests are grouped +into buckets based on their metadata, and gRPC servers periodically report +bucket usages. RLQS aggregates the data from different gRPC servers, and fairly distributes the quota among them. This approach is best suited for high-request-per-second applications, where a certain margin of error is acceptable as long as expected average QPS is achieved. @@ -33,16 +33,15 @@ acceptable as long as expected average QPS is achieved. To support RLQS, we'll need to implement several other xDS-related features, which are covered in the proposal: -1. xDS Control Plane will provide RLQS connection details - in [the filter config][rlqs_filter_proto] via [`GrpcService`] message. -2. Quota assignments will be configured - via [`TokenBucket`](https://www.envoyproxy.io/docs/envoy/latest/api-v3/type/v3/token_bucket.proto) - message. -3. RPCs will be matched into buckets using [Unified Matcher API]. -4. One of the matching mechanisms will be [CEL](https://cel.dev/) (Common - Expression Language). -5. RLQS Filter State will persist across LDS/RDS updates using cache retention - mechanism described in [gRFC A83]. +1. xDS Control Plane will provide RLQS connection details in + [the filter config][Config: RateLimitQuotaFilterConfig] via + [GrpcService][Config: GrpcService] message. +2. Quota assignments will be configured via [TokenBucket][] message. +3. RPCs will be matched into buckets using [Unified Matcher API]. +4. One of the matching mechanisms will be [CEL](https://cel.dev/) (Common + Expression Language). +5. RLQS Filter State will persist across LDS/RDS updates using cache retention + mechanism described in [gRFC A83]. ### Related Proposals @@ -60,79 +59,135 @@ which are covered in the proposal: [RLQS xDS HTTP Filter: Channel Level]: #rlqs-xds-http-filter-channel-level [GrpcService Security Considerations]: #grpcservice-security-considerations +[Config: GrpcService]: #config-grpcservice +[Config: Bucket Matchers]: #config-bucket-matchers +[Config: RuntimeFractionalPercent]: #config-runtimefractionalpercent +[Config: HeaderValueOption]: #config-headervalueoption +[Config: RateLimitQuotaFilterConfig]: #ratelimitquotafilterconfig -[`GrpcService`]: https://www.envoyproxy.io/docs/envoy/latest/api-v3/config/core/v3/grpc_service.proto -[`GrpcService.GoogleGrpc`]: https://www.envoyproxy.io/docs/envoy/latest/api-v3/config/core/v3/grpc_service.proto#envoy-v3-api-msg-config-core-v3-grpcservice-googlegrpc [Unified Matcher API]: https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/advanced/matching/matching_api.html [Envoy CEL environment]: https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/advanced/attributes -[google.protobuf.Duration]: https://developers.google.com/protocol-buffers/docs/reference/google.protobuf#google.protobuf.Duration +[TokenBucket]: https://github.com/envoyproxy/envoy/blob/82bc63199ff429490260e52794bb3095f17bcdae/api/envoy/type/v3/token_bucket.proto +[GrpcService]: https://github.com/envoyproxy/envoy/blob/82bc63199ff429490260e52794bb3095f17bcdae/api/envoy/config/core/v3/grpc_service.proto +[GrpcService.GoogleGrpc]: https://github.com/envoyproxy/envoy/blob/82bc63199ff429490260e52794bb3095f17bcdae/api/envoy/config/core/v3/grpc_service.proto#L73 -[rlqs_proto]: https://www.envoyproxy.io/docs/envoy/latest/api-v3/service/rate_limit_quota/v3/rlqs.proto.html -[rlqs_filter_proto]: https://www.envoyproxy.io/docs/envoy/latest/api-v3/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto +[rlqs_service_proto]: https://github.com/envoyproxy/envoy/blob/82bc63199ff429490260e52794bb3095f17bcdae/api/envoy/service/rate_limit_quota/v3/rlqs.proto +[rate_limit_quota.proto]: https://github.com/envoyproxy/envoy/blob/82bc63199ff429490260e52794bb3095f17bcdae/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto [RateLimitQuotaFilterConfig]: https://github.com/envoyproxy/envoy/blob/82bc63199ff429490260e52794bb3095f17bcdae/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L37 -[proto_rlqs_server]: https://github.com/envoyproxy/envoy/blob/82bc63199ff429490260e52794bb3095f17bcdae/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L39 -[proto_domain]: https://github.com/envoyproxy/envoy/blob/82bc63199ff429490260e52794bb3095f17bcdae/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L44 -[proto_bucket_matchers]: https://github.com/envoyproxy/envoy/blob/82bc63199ff429490260e52794bb3095f17bcdae/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L115 -[proto_filter_enabled]: https://github.com/envoyproxy/envoy/blob/82bc63199ff429490260e52794bb3095f17bcdae/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L121 -[proto_filter_enforced]: https://github.com/envoyproxy/envoy/blob/82bc63199ff429490260e52794bb3095f17bcdae/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L132 -[proto_request_headers_to_add_when_not_enforced]: https://github.com/envoyproxy/envoy/blob/82bc63199ff429490260e52794bb3095f17bcdae/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L137 - -[proto_grpc_svc_google_grpc]: https://github.com/envoyproxy/envoy/blob/82bc63199ff429490260e52794bb3095f17bcdae/api/envoy/config/core/v3/grpc_service.proto#L303 -[proto_grpc_svc_target_uri]: https://github.com/envoyproxy/envoy/blob/82bc63199ff429490260e52794bb3095f17bcdae/api/envoy/config/core/v3/grpc_service.proto#L254 -[proto_grpc_svc_timeout]: https://github.com/envoyproxy/envoy/blob/82bc63199ff429490260e52794bb3095f17bcdae/api/envoy/config/core/v3/grpc_service.proto#L308 -[proto_frac_default_value]: https://github.com/envoyproxy/envoy/blob/82bc63199ff429490260e52794bb3095f17bcdae/api/envoy/config/core/v3/base.proto#L648 -[proto_frac_numerator]: https://github.com/envoyproxy/envoy/blob/82bc63199ff429490260e52794bb3095f17bcdae/api/envoy/type/v3/percent.proto#L52 -[proto_frac_denominator]: https://github.com/envoyproxy/envoy/blob/82bc63199ff429490260e52794bb3095f17bcdae/api/envoy/type/v3/percent.proto#L56 +[google.protobuf.Duration]: https://developers.google.com/protocol-buffers/docs/reference/google.protobuf#google.protobuf.Duration ## Proposal ### Filter Configuration +The RLQS Filter API is defined in [rate_limit_quota.proto][]. + +#### RateLimitQuotaFilterConfig + We will support the following fields in the [RateLimitQuotaFilterConfig][] proto: -- [rlqs_server][proto_rlqs_server]: This field must be present. Inside of it: - - [google_grpc][proto_grpc_svc_google_grpc]: This field must be present. - Inside of it: - - [target_uri][proto_grpc_svc_target_uri]: This field must be - non-empty and must be a valid target URI. The value specified here - must be present in the `allowed_grpc_services` map in the bootstrap - config, which will also determine the credentials to use, as - described in [GrpcService Security Considerations][]. - - All other fields are ignored. - - [timeout][proto_grpc_svc_timeout]: Specifies the deadline for the RPCs - sent to the RLQS server. If unset, there is no deadline. The value must - obey the restrictions specified in the [google.protobuf.Duration][], - documentation and it must have a positive value. +- [rlqs_server](https://github.com/envoyproxy/envoy/blob/82bc63199ff429490260e52794bb3095f17bcdae/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L39): + This field must be present. Inside of it, [Config: GrpcService][]. +- [domain](https://github.com/envoyproxy/envoy/blob/82bc63199ff429490260e52794bb3095f17bcdae/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L44): + This field must be present and non-empty. +- [bucket_matchers](https://github.com/envoyproxy/envoy/blob/82bc63199ff429490260e52794bb3095f17bcdae/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L115): + This field must be present. Inside of it, there must be a valid matchers + structure. Details described in [Config: Bucket Matchers][]. +- [filter_enabled](https://github.com/envoyproxy/envoy/blob/82bc63199ff429490260e52794bb3095f17bcdae/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L121): + Specifies the fraction of requests for which the rate limiting is enabled. + When not present, the filter is enabled for all requests (100%). Otherwise, + the fraction is determined from [Config: RuntimeFractionalPercent][]. +- [filter_enforced](https://github.com/envoyproxy/envoy/blob/82bc63199ff429490260e52794bb3095f17bcdae/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L132): + Specifies the fraction of requests for which the rate limiting is enforced. + When not present, the filter is enforced for all requests (100%). Otherwise, + the fraction is determined from [Config: RuntimeFractionalPercent][]. +- [request_headers_to_add_when_not_enforced](https://github.com/envoyproxy/envoy/blob/82bc63199ff429490260e52794bb3095f17bcdae/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L137): + If present, must be a list of [Config: HeaderValueOption][]. Must not + contain more than 10 items. + +#### Config: GrpcService + +We will support the following fields in the [GrpcService][] proto: + +- [google_grpc](https://github.com/envoyproxy/envoy/blob/82bc63199ff429490260e52794bb3095f17bcdae/api/envoy/config/core/v3/grpc_service.proto#L303): + This field must be present. Inside of it: + - [target_uri](https://github.com/envoyproxy/envoy/blob/82bc63199ff429490260e52794bb3095f17bcdae/api/envoy/config/core/v3/grpc_service.proto#L254): + This field must be non-empty and must be a valid target URI. The value + specified here must be present in the `allowed_grpc_services` map in the + bootstrap config, which will also determine the credentials to use, as + described in [GrpcService Security Considerations][]. - All other fields are ignored. -- [domain][proto_domain]: This field must be present and non-empty. -- [bucket_matchers][proto_bucket_matchers]: This field must be present. Inside - of it, there must be a valid matchers structure. `TODO(sergiitk): add more - info`. -- [filter_enabled][proto_filter_enabled]: Optional, defaults to 100%. Inside - of it: - - [default_value][proto_frac_default_value]: Required if `filter_enabled` - is set. Specifies the fraction of requests for which the rate limiting - is enabled. Inside of it: - - [numerator][proto_frac_numerator]: Optional, defaults to 0. - - [denominator][proto_frac_denominator]: Optional, defaults to - HUNDRED. - - All other fields are ignored. -- [filter_enforced][proto_filter_enforced]: Optional, defaults to 100%. Inside - of it: - - [default_value][proto_frac_default_value]: Required if `filter_enforced` - is set. Specifies the fraction of requests for which the rate limiting - is enforced. Inside of it: - - [numerator][proto_frac_numerator]: Optional, defaults to 0. - - [denominator][proto_frac_denominator]: Optional, defaults to - HUNDRED. - - All other fields are ignored. -- [request_headers_to_add_when_not_enforced][proto_request_headers_to_add_when_not_enforced]: - Optional. Inside of it, a list of the following items: - - HeaderValueOption `TODO(sergiitk): finish`. +- [timeout](https://github.com/envoyproxy/envoy/blob/82bc63199ff429490260e52794bb3095f17bcdae/api/envoy/config/core/v3/grpc_service.proto#L308): + Specifies the deadline for the RPCs sent to the RLQS server. If unset, there + is no deadline. The value must obey the restrictions specified in the + [google.protobuf.Duration][], documentation and it must have a positive + value. +- All other fields are ignored. + +The following fields will be ignored by gRPC: + +- http_service: It doesn't make sense for gRPC to support non-gRPC mechanisms + for contacting the ext_authz server. +- stat_prefix: This does not apply to gRPC. +- initial_metadata: `TODO(sergiitk): confirm`. +- retry_policy: `TODO(sergiitk): confirm`. + +#### Config: RuntimeFractionalPercent + +We will support the following fields in the [RuntimeFractionalPercent][] proto: + +- [default_value](https://github.com/envoyproxy/envoy/blob/82bc63199ff429490260e52794bb3095f17bcdae/api/envoy/config/core/v3/base.proto#L648): + This field must be present. Specifies the fraction of requests for which + given setting will apply. Inside of it: + - [numerator](https://github.com/envoyproxy/envoy/blob/82bc63199ff429490260e52794bb3095f17bcdae/api/envoy/type/v3/percent.proto#L52): + Optional, defaults to 0. + - [denominator](https://github.com/envoyproxy/envoy/blob/82bc63199ff429490260e52794bb3095f17bcdae/api/envoy/type/v3/percent.proto#L56): + Must be one of the [DenominatorType][] values: + - `HUNDRED` (default) + - `TEN_THOUSAND` + - `MILLION` +- All other fields are ignored. + +The following fields will be ignored by gRPC: + +- runtime_key: gRPC does not have Envoy's concept of runtime settings. + +#### Config: HeaderValueOption + +We will support the following fields in the [HeaderValueOption][] proto: + +- [header](https://github.com/envoyproxy/envoy/blob/82bc63199ff429490260e52794bb3095f17bcdae/api/envoy/config/core/v3/base.proto#L458): + Must be present. + - [key](https://github.com/envoyproxy/envoy/blob/82bc63199ff429490260e52794bb3095f17bcdae/api/envoy/config/core/v3/base.proto#L404): + Must contain a valid header field name. + - [value](https://github.com/envoyproxy/envoy/blob/82bc63199ff429490260e52794bb3095f17bcdae/api/envoy/config/core/v3/base.proto#L415): + May be empty. Otherwise, must contain a valid header value. + - [raw_value](https://github.com/envoyproxy/envoy/blob/82bc63199ff429490260e52794bb3095f17bcdae/api/envoy/config/core/v3/base.proto#L422): + `TODO(sergiitk): confirm`. +- [append_action](https://github.com/envoyproxy/envoy/blob/82bc63199ff429490260e52794bb3095f17bcdae/api/envoy/config/core/v3/base.proto#L408): + Must be of the + [HeaderAppendAction](https://github.com/envoyproxy/envoy/blob/82bc63199ff429490260e52794bb3095f17bcdae/api/envoy/config/core/v3/base.proto#L434) + values: + - `APPEND_IF_EXISTS_OR_ADD` (default) + - `ADD_IF_ABSENT` + - `OVERWRITE_IF_EXISTS_OR_ADD` + - `OVERWRITE_IF_EXISTS` +- [keep_empty_value](https://github.com/envoyproxy/envoy/blob/82bc63199ff429490260e52794bb3095f17bcdae/api/envoy/config/core/v3/base.proto#L480) +- All other fields are ignored. + +The following fields will be ignored by gRPC: + +- append: Deprecated in favor of `append_action`. + +#### Config: Bucket Matchers + +TODO(sergiitk): add details + +--- ### RLQS Components @@ -141,6 +196,7 @@ proto: The diagram below shows the conceptual components of the RLQS Filter. Note that the actual implementation may vary depending on the language. + ```mermaid --- config: @@ -497,17 +553,17 @@ corresponding timer is removed. #### Connecting to RLQS Control Plane -xDS Control Plane provides RLQS connection details in [`GrpcService`] message ( -already supported by Envoy). `GrpcService` supports two modes: +xDS Control Plane provides RLQS connection details in [GrpcService] message ( +already supported by Envoy). GrpcService supports two modes: -1. `GrpcService.EnvoyGrpc`, Envoy's minimal custom gRPC client implementation. -2. [`GrpcService.GoogleGrpc`], regular gRPC-cpp client. +1. GrpcService.EnvoyGrpc, Envoy's minimal custom gRPC client implementation. +2. [GrpcService.GoogleGrpc], regular gRPC-cpp client. -For obvious reasons, we'll only support the `GoogleGrpc` mode. +For obvious reasons, we'll only support the GoogleGrpc mode. ##### GrpcService Security Considerations -In [`GrpcService.GoogleGrpc`], the xDS Control Plane provides the target URI, +In [GrpcService.GoogleGrpc], the xDS Control Plane provides the target URI, `channel_credentials`, and `call_credentials`. If the xDS Control Plane is compromised, an attacker could configure the xDS clients to talk to other malicious Control Planes. This could lead to potential exploits such as leaking @@ -535,7 +591,7 @@ target URI to an object containing channel credentials to use for that target. ``` When xDS Control Plane configures connection to another control plane -via [`GrpcService.GoogleGrpc`] message, we'll inspect the allow-list for the +via [GrpcService.GoogleGrpc] message, we'll inspect the allow-list for the matching target URI. 1. If target URI is not present, we don't create the connection to the requested @@ -546,7 +602,7 @@ matching target URI. > [!IMPORTANT] > This solution is not specific to RLQS, and should be used with any -> other Control Planes configured via [`GrpcService`] message. +> other Control Planes configured via [GrpcService] message. #### Unified Matcher API From 45739121f3831e692f518c170598dbf7f1a8c35d Mon Sep 17 00:00:00 2001 From: Sergii Tkachenko Date: Tue, 13 May 2025 17:26:21 -0700 Subject: [PATCH 26/53] fix more links --- A77-xds-rate-limiting-rlqs.md | 11 ++++++----- 1 file changed, 6 insertions(+), 5 deletions(-) diff --git a/A77-xds-rate-limiting-rlqs.md b/A77-xds-rate-limiting-rlqs.md index 9891d4890..90b572a79 100644 --- a/A77-xds-rate-limiting-rlqs.md +++ b/A77-xds-rate-limiting-rlqs.md @@ -69,7 +69,7 @@ which are covered in the proposal: [Envoy CEL environment]: https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/advanced/attributes [TokenBucket]: https://github.com/envoyproxy/envoy/blob/82bc63199ff429490260e52794bb3095f17bcdae/api/envoy/type/v3/token_bucket.proto -[GrpcService]: https://github.com/envoyproxy/envoy/blob/82bc63199ff429490260e52794bb3095f17bcdae/api/envoy/config/core/v3/grpc_service.proto +[GrpcService]: https://github.com/envoyproxy/envoy/blob/82bc63199ff429490260e52794bb3095f17bcdae/api/envoy/config/core/v3/grpc_service.proto#L29 [GrpcService.GoogleGrpc]: https://github.com/envoyproxy/envoy/blob/82bc63199ff429490260e52794bb3095f17bcdae/api/envoy/config/core/v3/grpc_service.proto#L73 [rlqs_service_proto]: https://github.com/envoyproxy/envoy/blob/82bc63199ff429490260e52794bb3095f17bcdae/api/envoy/service/rate_limit_quota/v3/rlqs.proto @@ -96,7 +96,7 @@ proto: This field must be present and non-empty. - [bucket_matchers](https://github.com/envoyproxy/envoy/blob/82bc63199ff429490260e52794bb3095f17bcdae/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L115): This field must be present. Inside of it, there must be a valid matchers - structure. Details described in [Config: Bucket Matchers][]. + structure as described in [Config: Bucket Matchers][]. - [filter_enabled](https://github.com/envoyproxy/envoy/blob/82bc63199ff429490260e52794bb3095f17bcdae/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L121): Specifies the fraction of requests for which the rate limiting is enabled. When not present, the filter is enabled for all requests (100%). Otherwise, @@ -138,7 +138,9 @@ The following fields will be ignored by gRPC: #### Config: RuntimeFractionalPercent -We will support the following fields in the [RuntimeFractionalPercent][] proto: +We will support the following fields in the +[RuntimeFractionalPercent](https://github.com/envoyproxy/envoy/blob/82bc63199ff429490260e52794bb3095f17bcdae/api/envoy/config/core/v3/base.proto#L643) +proto: - [default_value](https://github.com/envoyproxy/envoy/blob/82bc63199ff429490260e52794bb3095f17bcdae/api/envoy/config/core/v3/base.proto#L648): This field must be present. Specifies the fraction of requests for which @@ -158,7 +160,7 @@ The following fields will be ignored by gRPC: #### Config: HeaderValueOption -We will support the following fields in the [HeaderValueOption][] proto: +We will support the following fields in the [HeaderValueOption](https://github.com/envoyproxy/envoy/blob/82bc63199ff429490260e52794bb3095f17bcdae/api/envoy/config/core/v3/base.proto#L429) proto: - [header](https://github.com/envoyproxy/envoy/blob/82bc63199ff429490260e52794bb3095f17bcdae/api/envoy/config/core/v3/base.proto#L458): Must be present. @@ -196,7 +198,6 @@ TODO(sergiitk): add details The diagram below shows the conceptual components of the RLQS Filter. Note that the actual implementation may vary depending on the language. - ```mermaid --- config: From 3e15c9c47c83b4984278868e915251c569b775a9 Mon Sep 17 00:00:00 2001 From: Sergii Tkachenko Date: Tue, 13 May 2025 17:36:33 -0700 Subject: [PATCH 27/53] add validation info to frac percent --- A77-xds-rate-limiting-rlqs.md | 17 ++++++++++------- 1 file changed, 10 insertions(+), 7 deletions(-) diff --git a/A77-xds-rate-limiting-rlqs.md b/A77-xds-rate-limiting-rlqs.md index 90b572a79..dd1dd00a7 100644 --- a/A77-xds-rate-limiting-rlqs.md +++ b/A77-xds-rate-limiting-rlqs.md @@ -143,15 +143,18 @@ We will support the following fields in the proto: - [default_value](https://github.com/envoyproxy/envoy/blob/82bc63199ff429490260e52794bb3095f17bcdae/api/envoy/config/core/v3/base.proto#L648): - This field must be present. Specifies the fraction of requests for which - given setting will apply. Inside of it: + This field must be present. If the denominator specified is less than the + numerator, the final fractional percentage is capped at 1 (100%). The + fraction specified with: - [numerator](https://github.com/envoyproxy/envoy/blob/82bc63199ff429490260e52794bb3095f17bcdae/api/envoy/type/v3/percent.proto#L52): - Optional, defaults to 0. + Non-negative integer, 0 by default. - [denominator](https://github.com/envoyproxy/envoy/blob/82bc63199ff429490260e52794bb3095f17bcdae/api/envoy/type/v3/percent.proto#L56): - Must be one of the [DenominatorType][] values: - - `HUNDRED` (default) - - `TEN_THOUSAND` - - `MILLION` + Must be one of the + [DenominatorType](https://github.com/envoyproxy/envoy/blob/82bc63199ff429490260e52794bb3095f17bcdae/api/envoy/type/v3/percent.proto#L34) + values: + - `HUNDRED` (default) + - `TEN_THOUSAND` + - `MILLION` - All other fields are ignored. The following fields will be ignored by gRPC: From 23708260a17eb03993ff975b409ef77eec3441c1 Mon Sep 17 00:00:00 2001 From: Sergii Tkachenko Date: Fri, 3 Oct 2025 15:24:51 -0700 Subject: [PATCH 28/53] more updates on filter config, bucket matchers wip --- A77-xds-rate-limiting-rlqs.md | 219 ++++++++++++---------------------- 1 file changed, 79 insertions(+), 140 deletions(-) diff --git a/A77-xds-rate-limiting-rlqs.md b/A77-xds-rate-limiting-rlqs.md index dd1dd00a7..3670bf25f 100644 --- a/A77-xds-rate-limiting-rlqs.md +++ b/A77-xds-rate-limiting-rlqs.md @@ -5,7 +5,7 @@ A77: xDS Server-Side Rate Limiting * Approver: Mark Roth (@markdroth) * Status: In Review * Implemented in: -* Last updated: 2025-04-29 +* Last updated: 2025-10-03 * Discussion at: - [ ] TODO(sergiitk): insert google group thread @@ -23,7 +23,7 @@ allows mesh users to manage fair consumption of their services and prevent misbehaving clients from overloading the services. We will implement [quota-based rate limiting](https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/other_features/global_rate_limiting#quota-based-rate-limiting), where rate-limiting decisions are asynchronously offloaded to -[Rate Limiting Quota Service][rlqs_service_proto] (RLQS). Requests are grouped +[Rate Limiting Quota Service][rlqs_proto] (RLQS). Requests are grouped into buckets based on their metadata, and gRPC servers periodically report bucket usages. RLQS aggregates the data from different gRPC servers, and fairly distributes the quota among them. This approach is best suited for @@ -34,123 +34,108 @@ To support RLQS, we'll need to implement several other xDS-related features, which are covered in the proposal: 1. xDS Control Plane will provide RLQS connection details in - [the filter config][Config: RateLimitQuotaFilterConfig] via - [GrpcService][Config: GrpcService] message. -2. Quota assignments will be configured via [TokenBucket][] message. + [the filter config][RateLimitQuotaFilterConfig] via GrpcService message as + described in [A102]. +2. Quota assignments will be configured via [TokenBucket] message. 3. RPCs will be matched into buckets using [Unified Matcher API]. 4. One of the matching mechanisms will be [CEL](https://cel.dev/) (Common Expression Language). 5. RLQS Filter State will persist across LDS/RDS updates using cache retention - mechanism described in [gRFC A83]. + mechanism described in [A83]. ### Related Proposals -* [A27: xDS-Based Global Load Balancing][gRFC A27] -* [A36: xDS-Enabled Servers][gRFC A36] -* [A39: xDS HTTP Filter Support][gRFC A39] -* [A41: xDS RBAC Support][gRFC A41] -* [A83: xDS GCP Authentication Filter][gRFC A83] - -[gRFC A27]: A27-xds-global-load-balancing.md -[gRFC A41]: A41-xds-rbac.md -[gRFC A36]: A36-xds-for-servers.md -[gRFC A39]: A39-xds-http-filters.md -[gRFC A83]: A83-xds-gcp-authn-filter.md - -[RLQS xDS HTTP Filter: Channel Level]: #rlqs-xds-http-filter-channel-level -[GrpcService Security Considerations]: #grpcservice-security-considerations -[Config: GrpcService]: #config-grpcservice -[Config: Bucket Matchers]: #config-bucket-matchers +* [gRFC A27: xDS-Based Global Load Balancing][A27] +* [gRFC A36: xDS-Enabled Servers][A36] +* [gRFC A39: xDS HTTP Filter Support][A39] +* [gRFC A41: xDS RBAC Support][A41] +* [gRFC A83: xDS GCP Authentication Filter][A83] +* [gRFC A102: xDS GrpcService Support][A102] + +[A27]: A27-xds-global-load-balancing.md +[A41]: A41-xds-rbac.md +[A36]: A36-xds-for-servers.md +[A39]: A39-xds-http-filters.md +[A83]: A83-xds-gcp-authn-filter.md +[A102]: https://github.com/grpc/proposal/pull/510 + +[RateLimitQuotaFilterConfig]: #ratelimitquotafilterconfig +[RateLimitQuotaOverride]: #ratelimitquotaoverride [Config: RuntimeFractionalPercent]: #config-runtimefractionalpercent [Config: HeaderValueOption]: #config-headervalueoption -[Config: RateLimitQuotaFilterConfig]: #ratelimitquotafilterconfig +[Config: Bucket Matchers]: #config-bucket-matchers +[RLQS xDS HTTP Filter: Channel Level]: #rlqs-xds-http-filter-channel-level [Unified Matcher API]: https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/advanced/matching/matching_api.html [Envoy CEL environment]: https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/advanced/attributes -[TokenBucket]: https://github.com/envoyproxy/envoy/blob/82bc63199ff429490260e52794bb3095f17bcdae/api/envoy/type/v3/token_bucket.proto -[GrpcService]: https://github.com/envoyproxy/envoy/blob/82bc63199ff429490260e52794bb3095f17bcdae/api/envoy/config/core/v3/grpc_service.proto#L29 -[GrpcService.GoogleGrpc]: https://github.com/envoyproxy/envoy/blob/82bc63199ff429490260e52794bb3095f17bcdae/api/envoy/config/core/v3/grpc_service.proto#L73 +[TokenBucket]: https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/type/v3/token_bucket.proto +[GrpcService.GoogleGrpc]: https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/config/core/v3/grpc_service.proto#L68 -[rlqs_service_proto]: https://github.com/envoyproxy/envoy/blob/82bc63199ff429490260e52794bb3095f17bcdae/api/envoy/service/rate_limit_quota/v3/rlqs.proto - -[rate_limit_quota.proto]: https://github.com/envoyproxy/envoy/blob/82bc63199ff429490260e52794bb3095f17bcdae/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto -[RateLimitQuotaFilterConfig]: https://github.com/envoyproxy/envoy/blob/82bc63199ff429490260e52794bb3095f17bcdae/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L37 - -[google.protobuf.Duration]: https://developers.google.com/protocol-buffers/docs/reference/google.protobuf#google.protobuf.Duration +[rlqs_proto]: https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/service/rate_limit_quota/v3/rlqs.proto ## Proposal ### Filter Configuration -The RLQS Filter API is defined in [rate_limit_quota.proto][]. +The RLQS Filter API is defined in +[rate_limit_quota.proto](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto). #### RateLimitQuotaFilterConfig -We will support the following fields in the [RateLimitQuotaFilterConfig][] -proto: +We will support the following fields in the +[RateLimitQuotaFilterConfig](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L37) +message: -- [rlqs_server](https://github.com/envoyproxy/envoy/blob/82bc63199ff429490260e52794bb3095f17bcdae/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L39): - This field must be present. Inside of it, [Config: GrpcService][]. -- [domain](https://github.com/envoyproxy/envoy/blob/82bc63199ff429490260e52794bb3095f17bcdae/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L44): +- [rlqs_server](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L39): + This field must be present. Inside of it, GrpcService as described in + [A102]. +- [domain](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L44): This field must be present and non-empty. -- [bucket_matchers](https://github.com/envoyproxy/envoy/blob/82bc63199ff429490260e52794bb3095f17bcdae/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L115): +- [bucket_matchers](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L115): This field must be present. Inside of it, there must be a valid matchers - structure as described in [Config: Bucket Matchers][]. -- [filter_enabled](https://github.com/envoyproxy/envoy/blob/82bc63199ff429490260e52794bb3095f17bcdae/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L121): + structure as described in [Config: Bucket Matchers]. +- [filter_enabled](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L121): Specifies the fraction of requests for which the rate limiting is enabled. When not present, the filter is enabled for all requests (100%). Otherwise, - the fraction is determined from [Config: RuntimeFractionalPercent][]. -- [filter_enforced](https://github.com/envoyproxy/envoy/blob/82bc63199ff429490260e52794bb3095f17bcdae/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L132): + the fraction is determined from [Config: RuntimeFractionalPercent]. +- [filter_enforced](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L132): Specifies the fraction of requests for which the rate limiting is enforced. When not present, the filter is enforced for all requests (100%). Otherwise, - the fraction is determined from [Config: RuntimeFractionalPercent][]. -- [request_headers_to_add_when_not_enforced](https://github.com/envoyproxy/envoy/blob/82bc63199ff429490260e52794bb3095f17bcdae/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L137): - If present, must be a list of [Config: HeaderValueOption][]. Must not - contain more than 10 items. - -#### Config: GrpcService - -We will support the following fields in the [GrpcService][] proto: - -- [google_grpc](https://github.com/envoyproxy/envoy/blob/82bc63199ff429490260e52794bb3095f17bcdae/api/envoy/config/core/v3/grpc_service.proto#L303): - This field must be present. Inside of it: - - [target_uri](https://github.com/envoyproxy/envoy/blob/82bc63199ff429490260e52794bb3095f17bcdae/api/envoy/config/core/v3/grpc_service.proto#L254): - This field must be non-empty and must be a valid target URI. The value - specified here must be present in the `allowed_grpc_services` map in the - bootstrap config, which will also determine the credentials to use, as - described in [GrpcService Security Considerations][]. - - All other fields are ignored. -- [timeout](https://github.com/envoyproxy/envoy/blob/82bc63199ff429490260e52794bb3095f17bcdae/api/envoy/config/core/v3/grpc_service.proto#L308): - Specifies the deadline for the RPCs sent to the RLQS server. If unset, there - is no deadline. The value must obey the restrictions specified in the - [google.protobuf.Duration][], documentation and it must have a positive - value. -- All other fields are ignored. + the fraction is determined from [Config: RuntimeFractionalPercent]. +- [request_headers_to_add_when_not_enforced](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L137): + If present, must be a list of [Config: HeaderValueOption]. Must not contain + more than 10 items. -The following fields will be ignored by gRPC: +#### RateLimitQuotaOverride + +We will support the following fields in the +[RateLimitQuotaFilterConfig](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L143) +message: -- http_service: It doesn't make sense for gRPC to support non-gRPC mechanisms - for contacting the ext_authz server. -- stat_prefix: This does not apply to gRPC. -- initial_metadata: `TODO(sergiitk): confirm`. -- retry_policy: `TODO(sergiitk): confirm`. +- [domain](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L44): + If non-empty, overrides the domain value provided on the less specific + definition. +- [bucket_matchers](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L115): + If present, must be a valid matchers structure as described in + [Config: Bucket Matchers], and fully overrides the matchers provided on the + less specific definition. #### Config: RuntimeFractionalPercent We will support the following fields in the -[RuntimeFractionalPercent](https://github.com/envoyproxy/envoy/blob/82bc63199ff429490260e52794bb3095f17bcdae/api/envoy/config/core/v3/base.proto#L643) +[RuntimeFractionalPercent](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/config/core/v3/base.proto#L643) proto: -- [default_value](https://github.com/envoyproxy/envoy/blob/82bc63199ff429490260e52794bb3095f17bcdae/api/envoy/config/core/v3/base.proto#L648): +- [default_value](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/config/core/v3/base.proto#L648): This field must be present. If the denominator specified is less than the numerator, the final fractional percentage is capped at 1 (100%). The fraction specified with: - - [numerator](https://github.com/envoyproxy/envoy/blob/82bc63199ff429490260e52794bb3095f17bcdae/api/envoy/type/v3/percent.proto#L52): + - [numerator](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/type/v3/percent.proto#L52): Non-negative integer, 0 by default. - - [denominator](https://github.com/envoyproxy/envoy/blob/82bc63199ff429490260e52794bb3095f17bcdae/api/envoy/type/v3/percent.proto#L56): + - [denominator](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/type/v3/percent.proto#L56): Must be one of the - [DenominatorType](https://github.com/envoyproxy/envoy/blob/82bc63199ff429490260e52794bb3095f17bcdae/api/envoy/type/v3/percent.proto#L34) + [DenominatorType](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/type/v3/percent.proto#L34) values: - `HUNDRED` (default) - `TEN_THOUSAND` @@ -163,25 +148,25 @@ The following fields will be ignored by gRPC: #### Config: HeaderValueOption -We will support the following fields in the [HeaderValueOption](https://github.com/envoyproxy/envoy/blob/82bc63199ff429490260e52794bb3095f17bcdae/api/envoy/config/core/v3/base.proto#L429) proto: +We will support the following fields in the [HeaderValueOption](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/config/core/v3/base.proto#L429) proto: -- [header](https://github.com/envoyproxy/envoy/blob/82bc63199ff429490260e52794bb3095f17bcdae/api/envoy/config/core/v3/base.proto#L458): +- [header](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/config/core/v3/base.proto#L458): Must be present. - - [key](https://github.com/envoyproxy/envoy/blob/82bc63199ff429490260e52794bb3095f17bcdae/api/envoy/config/core/v3/base.proto#L404): + - [key](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/config/core/v3/base.proto#L404): Must contain a valid header field name. - - [value](https://github.com/envoyproxy/envoy/blob/82bc63199ff429490260e52794bb3095f17bcdae/api/envoy/config/core/v3/base.proto#L415): + - [value](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/config/core/v3/base.proto#L415): May be empty. Otherwise, must contain a valid header value. - - [raw_value](https://github.com/envoyproxy/envoy/blob/82bc63199ff429490260e52794bb3095f17bcdae/api/envoy/config/core/v3/base.proto#L422): + - [raw_value](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/config/core/v3/base.proto#L422): `TODO(sergiitk): confirm`. -- [append_action](https://github.com/envoyproxy/envoy/blob/82bc63199ff429490260e52794bb3095f17bcdae/api/envoy/config/core/v3/base.proto#L408): +- [append_action](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/config/core/v3/base.proto#L476): Must be of the - [HeaderAppendAction](https://github.com/envoyproxy/envoy/blob/82bc63199ff429490260e52794bb3095f17bcdae/api/envoy/config/core/v3/base.proto#L434) + [HeaderAppendAction](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/config/core/v3/base.proto#L434) values: - `APPEND_IF_EXISTS_OR_ADD` (default) - `ADD_IF_ABSENT` - `OVERWRITE_IF_EXISTS_OR_ADD` - `OVERWRITE_IF_EXISTS` -- [keep_empty_value](https://github.com/envoyproxy/envoy/blob/82bc63199ff429490260e52794bb3095f17bcdae/api/envoy/config/core/v3/base.proto#L480) +- [keep_empty_value](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/config/core/v3/base.proto#L480) - All other fields are ignored. The following fields will be ignored by gRPC: @@ -250,7 +235,7 @@ graph TD In order to retain filter state across LDS/RDS updates, the actual logic for the RLQS filter will be moved into a separate object called RLQS Filter State, which -will be stored in the persistent filter state mechanism described in [gRFC A83]. +will be stored in the persistent filter state mechanism described in [A83]. The key in the persistent filter state will be the RLQS xDS HTTP filter config, which ensures that two RLQS filter instances with the same config will share filter state but two RLQS filter instances with different configs will each have @@ -409,7 +394,7 @@ final class RlqsFilterState { // Do not block data plane RPC on sending the report. scheduler.schedule( () -> rlqsClient.sendUsageReports(newBucket.snapshotAndResetUsage()), - 1, TimeUnit.MICROSECONDS) + 1, TimeUnit.MICROSECONDS); } private void registerReportTimer(final long intervalMillis) { @@ -504,8 +489,10 @@ RLQS Client object will include the following data members: When receiving an LDS/RDS update, the RLQS filter will perform the following steps for each route: -1. Parse the new filter config and per-route overrides into an internal RLQS - Filter Config object. +1. Parse the [filter config][RateLimitQuotaFilterConfig] and + [per-route overrides][RateLimitQuotaOverride] into an internal RLQS Filter + Config object. + 1. Invalid filter configuration results in gRPC NACKing the xDS resource. 2. Retrieve the corresponding RLQS Filter State from the RLQS Filter State Cache, creating it if it doesn't exist. 3. Generate new data plane RPC handler using the retrieved RLQS Filter State. @@ -557,56 +544,8 @@ corresponding timer is removed. #### Connecting to RLQS Control Plane -xDS Control Plane provides RLQS connection details in [GrpcService] message ( -already supported by Envoy). GrpcService supports two modes: - -1. GrpcService.EnvoyGrpc, Envoy's minimal custom gRPC client implementation. -2. [GrpcService.GoogleGrpc], regular gRPC-cpp client. - -For obvious reasons, we'll only support the GoogleGrpc mode. - -##### GrpcService Security Considerations - -In [GrpcService.GoogleGrpc], the xDS Control Plane provides the target URI, -`channel_credentials`, and `call_credentials`. If the xDS Control Plane is -compromised, an attacker could configure the xDS clients to talk to other -malicious Control Planes. This could lead to potential exploits such as leaking -customer's Application Default Credentials OAuth token. - -To prevent that, we'll introduce an allow-list to the bootstrap file introduced -in [gRFC A27]. This allow-list will be a map from a fully-qualified server -target URI to an object containing channel credentials to use for that target. - -```json5 -// The allowlist of Control Planes allowed to be configured via xDS. -"allowed_grpc_services": { - // The key is fully-qualified server URI. - "dns:///xds.example.org:443": { - // The value is an object containing "channel_creds". - "channel_creds": [ - // The format is identical to xds_servers.channel_creds. - { - "type": "string containing channel cred type", - "config": "optional JSON object containing config for the type" - } - ] - } -} -``` - -When xDS Control Plane configures connection to another control plane -via [GrpcService.GoogleGrpc] message, we'll inspect the allow-list for the -matching target URI. - -1. If target URI is not present, we don't create the connection to the requested - Control Plane, and NACK the xDS resource. -2. If target URI is present, we create the connection to the requested Control - Plane using the channel credentials provided in the bootstrap file. Transport - security configuration provided by the control plane is ignored. - -> [!IMPORTANT] -> This solution is not specific to RLQS, and should be used with any -> other Control Planes configured via [GrpcService] message. +xDS Control Plane provides RLQS connection details in [GrpcService.GoogleGrpc] +message as specified in [A102]. #### Unified Matcher API @@ -690,7 +629,7 @@ Hard-coded to `"POST"` if unavailable and a code audit confirms the server denies requests for all other method types. **2 `request.headers`**\ -As defined in [gRFC A41], "header" field. +As defined in [A41], "header" field. ##### Implementation From 2d6fd0bd77ce302b87ef056d4d007024b7ac4a29 Mon Sep 17 00:00:00 2001 From: Sergii Tkachenko Date: Fri, 3 Oct 2025 15:51:51 -0700 Subject: [PATCH 29/53] clarify header raw_value --- A77-xds-rate-limiting-rlqs.md | 12 +++++++++--- 1 file changed, 9 insertions(+), 3 deletions(-) diff --git a/A77-xds-rate-limiting-rlqs.md b/A77-xds-rate-limiting-rlqs.md index 3670bf25f..36ac08090 100644 --- a/A77-xds-rate-limiting-rlqs.md +++ b/A77-xds-rate-limiting-rlqs.md @@ -58,6 +58,7 @@ which are covered in the proposal: [A39]: A39-xds-http-filters.md [A83]: A83-xds-gcp-authn-filter.md [A102]: https://github.com/grpc/proposal/pull/510 +[G1]: G1-true-binary-metadata.md [RateLimitQuotaFilterConfig]: #ratelimitquotafilterconfig [RateLimitQuotaOverride]: #ratelimitquotaoverride @@ -153,11 +154,16 @@ We will support the following fields in the [HeaderValueOption](https://github.c - [header](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/config/core/v3/base.proto#L458): Must be present. - [key](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/config/core/v3/base.proto#L404): - Must contain a valid header field name. + Value length must be in the range `[1, 16384)`. Must be a valid HTTP/2 + header name. - [value](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/config/core/v3/base.proto#L415): - May be empty. Otherwise, must contain a valid header value. + Specifies the header value. Must be shorter than 16384 bytes. Must be a + valid HTTP/2 header value. Not used if `key` ends in `-bin` and + `raw_value` is set. - [raw_value](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/config/core/v3/base.proto#L422): - `TODO(sergiitk): confirm`. + Used only if `key` ends in `-bin`. Must be shorter than 16384 bytes. + Will be base64-encoded on the wire, unless the pure binary metadata + extension from [gRFC G1: True Binary Metadata][G1] is used. - [append_action](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/config/core/v3/base.proto#L476): Must be of the [HeaderAppendAction](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/config/core/v3/base.proto#L434) From eb99cf9284bc58368f2f128b782570a224c2cc9e Mon Sep 17 00:00:00 2001 From: Sergii Tkachenko Date: Mon, 12 May 2025 11:41:57 -0700 Subject: [PATCH 30/53] r2 initial feedback - minor comments / spelling --- A77-xds-rate-limiting-rlqs.md | 31 ++++++++++--------------------- 1 file changed, 10 insertions(+), 21 deletions(-) diff --git a/A77-xds-rate-limiting-rlqs.md b/A77-xds-rate-limiting-rlqs.md index 3cb7e3ecf..3b46cb1c1 100644 --- a/A77-xds-rate-limiting-rlqs.md +++ b/A77-xds-rate-limiting-rlqs.md @@ -247,8 +247,8 @@ RLQS Filter State object will include the following data members: reporting intervals and their execution handlers. - Entries inserted we get the first data plane RPC for a given bucket and there's no key for bucket reporting interval. - - Entries deleted when there's more buckets with given reporting interval - in RLQS Bucket Map. + - Entries deleted when there's no more buckets with given reporting + interval in RLQS Bucket Map. Pseudo-code for RLQS Filter State RPC rate limiting: @@ -375,7 +375,7 @@ RLQS Client object will include the following data members: #### On LDS/RDS Updates -When receiving an LDS/RDS update, the RLQS filter will perform the following +When receiving an LDS/RDS update, the RLQS filter will perform the following steps for each route: 1. Parse the new filter config and per-route overrides into an internal RLQS @@ -476,7 +476,7 @@ matching target URI. Control Plane, and NACK the xDS resource. 2. If target URI is present, we create the connection to the requested Control Plane using the channel credentials provided in the bootstrap file. Transport - security configuration provided by the TD is ignored. + security configuration provided by the control plane is ignored. > [!IMPORTANT] > This solution is not specific to RLQS, and should be used with any @@ -544,12 +544,12 @@ for gRPC. | Attribute | Type | gRPC source | Envoy Description | |---------------------|-----------------------|------------------------------|-------------------------------------------------------------| -| `request.path` | `string` | Full method name1 | The path portion of the URL. | +| `request.path` | `string` | Full method name | The path portion of the URL. | | `request.url_path` | `string` | Same as `request.path` | The path portion of the URL without the query string. | -| `request.host` | `string` | Authority2 | The host portion of the URL. | +| `request.host` | `string` | Authority | The host portion of the URL. | | `request.scheme` | `string` | Not set | The scheme portion of the URL. | -| `request.method` | `string` | `POST`3 | Request method. | -| `request.headers` | `map` | `metadata`4 | All request headers indexed by the lower-cased header name. | +| `request.method` | `string` | `POST`1 | Request method. | +| `request.headers` | `map` | `metadata`2 | All request headers indexed by the lower-cased header name. | | `request.referer` | `string` | `metadata["referer"]` | Referer request header. | | `request.useragent` | `string` | `metadata["user-agent"]` | User agent request header. | | `request.time` | `timestamp` | Not set | Time of the first byte received. | @@ -559,22 +559,11 @@ for gRPC. ###### Footnotes -**1 `request.path`** - -* CPP: `metadata[":path"]` -* Go: `grpc.Method(ctx)` -* Java: `"/" + serverCall.getMethodDescriptor().getFullMethodName()` - -**2 `request.host`** - -* CPP, Go: `metadata[":authority"]` -* Java: `serverCall.getAuthority()` - -**3 `request.method`**\ +**1 `request.method`**\ Hard-coded to `"POST"` if unavailable and a code audit confirms the server denies requests for all other method types. -**4 `request.headers`**\ +**2 `request.headers`**\ As defined in [gRFC A41], "header" field. ##### Implementation From ed2fbaa24e13d4a8b6e0a2673090635cbfe7ed59 Mon Sep 17 00:00:00 2001 From: Sergii Tkachenko Date: Mon, 12 May 2025 13:14:27 -0700 Subject: [PATCH 31/53] Filter Config fields and validation, except Bucket Matchers --- A77-xds-rate-limiting-rlqs.md | 249 ++++++++++++++++++++++------------ 1 file changed, 160 insertions(+), 89 deletions(-) diff --git a/A77-xds-rate-limiting-rlqs.md b/A77-xds-rate-limiting-rlqs.md index 3b46cb1c1..36ac08090 100644 --- a/A77-xds-rate-limiting-rlqs.md +++ b/A77-xds-rate-limiting-rlqs.md @@ -5,7 +5,7 @@ A77: xDS Server-Side Rate Limiting * Approver: Mark Roth (@markdroth) * Status: In Review * Implemented in: -* Last updated: 2025-04-29 +* Last updated: 2025-10-03 * Discussion at: - [ ] TODO(sergiitk): insert google group thread @@ -20,12 +20,12 @@ servers. [Global rate limiting](https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/other_features/global_rate_limiting) allows mesh users to manage fair consumption of their services and prevent -misbehaving clients from overloading the services. We will -implement [quota-based rate limiting](https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/other_features/global_rate_limiting#quota-based-rate-limiting), -where rate-limiting decisions are asynchronously offloaded -to [Rate Limiting Quota Service][rlqs_proto] (RLQS). Requests are grouped into -buckets based on their metadata, and gRPC servers periodically report bucket -usages. RLQS aggregates the data from different gRPC servers, and fairly +misbehaving clients from overloading the services. We will implement +[quota-based rate limiting](https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/other_features/global_rate_limiting#quota-based-rate-limiting), +where rate-limiting decisions are asynchronously offloaded to +[Rate Limiting Quota Service][rlqs_proto] (RLQS). Requests are grouped +into buckets based on their metadata, and gRPC servers periodically report +bucket usages. RLQS aggregates the data from different gRPC servers, and fairly distributes the quota among them. This approach is best suited for high-request-per-second applications, where a certain margin of error is acceptable as long as expected average QPS is achieved. @@ -33,44 +33,161 @@ acceptable as long as expected average QPS is achieved. To support RLQS, we'll need to implement several other xDS-related features, which are covered in the proposal: -1. xDS Control Plane will provide RLQS connection details - in [the filter config][rlqs_filter_proto] via [`GrpcService`] message. -2. Quota assignments will be configured - via [`TokenBucket`](https://www.envoyproxy.io/docs/envoy/latest/api-v3/type/v3/token_bucket.proto) - message. -3. RPCs will be matched into buckets using [Unified Matcher API]. -4. One of the matching mechanisms will be [CEL](https://cel.dev/) (Common - Expression Language). -5. RLQS Filter State will persist across LDS/RDS updates using cache retention - mechanism described in [gRFC A83]. +1. xDS Control Plane will provide RLQS connection details in + [the filter config][RateLimitQuotaFilterConfig] via GrpcService message as + described in [A102]. +2. Quota assignments will be configured via [TokenBucket] message. +3. RPCs will be matched into buckets using [Unified Matcher API]. +4. One of the matching mechanisms will be [CEL](https://cel.dev/) (Common + Expression Language). +5. RLQS Filter State will persist across LDS/RDS updates using cache retention + mechanism described in [A83]. ### Related Proposals -* [A27: xDS-Based Global Load Balancing][gRFC A27] -* [A36: xDS-Enabled Servers][gRFC A36] -* [A39: xDS HTTP Filter Support][gRFC A39] -* [A41: xDS RBAC Support][gRFC A41] -* [A83: xDS GCP Authentication Filter][gRFC A83] - -[gRFC A27]: A27-xds-global-load-balancing.md -[gRFC A41]: A41-xds-rbac.md -[gRFC A36]: A36-xds-for-servers.md -[gRFC A39]: A39-xds-http-filters.md -[gRFC A83]: A83-xds-gcp-authn-filter.md - +* [gRFC A27: xDS-Based Global Load Balancing][A27] +* [gRFC A36: xDS-Enabled Servers][A36] +* [gRFC A39: xDS HTTP Filter Support][A39] +* [gRFC A41: xDS RBAC Support][A41] +* [gRFC A83: xDS GCP Authentication Filter][A83] +* [gRFC A102: xDS GrpcService Support][A102] + +[A27]: A27-xds-global-load-balancing.md +[A41]: A41-xds-rbac.md +[A36]: A36-xds-for-servers.md +[A39]: A39-xds-http-filters.md +[A83]: A83-xds-gcp-authn-filter.md +[A102]: https://github.com/grpc/proposal/pull/510 +[G1]: G1-true-binary-metadata.md + +[RateLimitQuotaFilterConfig]: #ratelimitquotafilterconfig +[RateLimitQuotaOverride]: #ratelimitquotaoverride +[Config: RuntimeFractionalPercent]: #config-runtimefractionalpercent +[Config: HeaderValueOption]: #config-headervalueoption +[Config: Bucket Matchers]: #config-bucket-matchers [RLQS xDS HTTP Filter: Channel Level]: #rlqs-xds-http-filter-channel-level -[`GrpcService`]: https://www.envoyproxy.io/docs/envoy/latest/api-v3/config/core/v3/grpc_service.proto -[`GrpcService.GoogleGrpc`]: https://www.envoyproxy.io/docs/envoy/latest/api-v3/config/core/v3/grpc_service.proto#envoy-v3-api-msg-config-core-v3-grpcservice-googlegrpc [Unified Matcher API]: https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/advanced/matching/matching_api.html [Envoy CEL environment]: https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/advanced/attributes -[rlqs_proto]: https://www.envoyproxy.io/docs/envoy/latest/api-v3/service/rate_limit_quota/v3/rlqs.proto.html -[rlqs_filter_proto]: https://www.envoyproxy.io/docs/envoy/latest/api-v3/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto +[TokenBucket]: https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/type/v3/token_bucket.proto +[GrpcService.GoogleGrpc]: https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/config/core/v3/grpc_service.proto#L68 + +[rlqs_proto]: https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/service/rate_limit_quota/v3/rlqs.proto ## Proposal -### RLQS Components Overview +### Filter Configuration + +The RLQS Filter API is defined in +[rate_limit_quota.proto](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto). + +#### RateLimitQuotaFilterConfig + +We will support the following fields in the +[RateLimitQuotaFilterConfig](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L37) +message: + +- [rlqs_server](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L39): + This field must be present. Inside of it, GrpcService as described in + [A102]. +- [domain](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L44): + This field must be present and non-empty. +- [bucket_matchers](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L115): + This field must be present. Inside of it, there must be a valid matchers + structure as described in [Config: Bucket Matchers]. +- [filter_enabled](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L121): + Specifies the fraction of requests for which the rate limiting is enabled. + When not present, the filter is enabled for all requests (100%). Otherwise, + the fraction is determined from [Config: RuntimeFractionalPercent]. +- [filter_enforced](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L132): + Specifies the fraction of requests for which the rate limiting is enforced. + When not present, the filter is enforced for all requests (100%). Otherwise, + the fraction is determined from [Config: RuntimeFractionalPercent]. +- [request_headers_to_add_when_not_enforced](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L137): + If present, must be a list of [Config: HeaderValueOption]. Must not contain + more than 10 items. + +#### RateLimitQuotaOverride + +We will support the following fields in the +[RateLimitQuotaFilterConfig](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L143) +message: + +- [domain](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L44): + If non-empty, overrides the domain value provided on the less specific + definition. +- [bucket_matchers](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L115): + If present, must be a valid matchers structure as described in + [Config: Bucket Matchers], and fully overrides the matchers provided on the + less specific definition. + +#### Config: RuntimeFractionalPercent + +We will support the following fields in the +[RuntimeFractionalPercent](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/config/core/v3/base.proto#L643) +proto: + +- [default_value](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/config/core/v3/base.proto#L648): + This field must be present. If the denominator specified is less than the + numerator, the final fractional percentage is capped at 1 (100%). The + fraction specified with: + - [numerator](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/type/v3/percent.proto#L52): + Non-negative integer, 0 by default. + - [denominator](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/type/v3/percent.proto#L56): + Must be one of the + [DenominatorType](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/type/v3/percent.proto#L34) + values: + - `HUNDRED` (default) + - `TEN_THOUSAND` + - `MILLION` +- All other fields are ignored. + +The following fields will be ignored by gRPC: + +- runtime_key: gRPC does not have Envoy's concept of runtime settings. + +#### Config: HeaderValueOption + +We will support the following fields in the [HeaderValueOption](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/config/core/v3/base.proto#L429) proto: + +- [header](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/config/core/v3/base.proto#L458): + Must be present. + - [key](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/config/core/v3/base.proto#L404): + Value length must be in the range `[1, 16384)`. Must be a valid HTTP/2 + header name. + - [value](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/config/core/v3/base.proto#L415): + Specifies the header value. Must be shorter than 16384 bytes. Must be a + valid HTTP/2 header value. Not used if `key` ends in `-bin` and + `raw_value` is set. + - [raw_value](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/config/core/v3/base.proto#L422): + Used only if `key` ends in `-bin`. Must be shorter than 16384 bytes. + Will be base64-encoded on the wire, unless the pure binary metadata + extension from [gRFC G1: True Binary Metadata][G1] is used. +- [append_action](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/config/core/v3/base.proto#L476): + Must be of the + [HeaderAppendAction](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/config/core/v3/base.proto#L434) + values: + - `APPEND_IF_EXISTS_OR_ADD` (default) + - `ADD_IF_ABSENT` + - `OVERWRITE_IF_EXISTS_OR_ADD` + - `OVERWRITE_IF_EXISTS` +- [keep_empty_value](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/config/core/v3/base.proto#L480) +- All other fields are ignored. + +The following fields will be ignored by gRPC: + +- append: Deprecated in favor of `append_action`. + +#### Config: Bucket Matchers + +TODO(sergiitk): add details + +--- + +### RLQS Components + +#### RLQS Components Overview The diagram below shows the conceptual components of the RLQS Filter. Note that the actual implementation may vary depending on the language. @@ -124,7 +241,7 @@ graph TD In order to retain filter state across LDS/RDS updates, the actual logic for the RLQS filter will be moved into a separate object called RLQS Filter State, which -will be stored in the persistent filter state mechanism described in [gRFC A83]. +will be stored in the persistent filter state mechanism described in [A83]. The key in the persistent filter state will be the RLQS xDS HTTP filter config, which ensures that two RLQS filter instances with the same config will share filter state but two RLQS filter instances with different configs will each have @@ -283,7 +400,7 @@ final class RlqsFilterState { // Do not block data plane RPC on sending the report. scheduler.schedule( () -> rlqsClient.sendUsageReports(newBucket.snapshotAndResetUsage()), - 1, TimeUnit.MICROSECONDS) + 1, TimeUnit.MICROSECONDS); } private void registerReportTimer(final long intervalMillis) { @@ -378,8 +495,10 @@ RLQS Client object will include the following data members: When receiving an LDS/RDS update, the RLQS filter will perform the following steps for each route: -1. Parse the new filter config and per-route overrides into an internal RLQS - Filter Config object. +1. Parse the [filter config][RateLimitQuotaFilterConfig] and + [per-route overrides][RateLimitQuotaOverride] into an internal RLQS Filter + Config object. + 1. Invalid filter configuration results in gRPC NACKing the xDS resource. 2. Retrieve the corresponding RLQS Filter State from the RLQS Filter State Cache, creating it if it doesn't exist. 3. Generate new data plane RPC handler using the retrieved RLQS Filter State. @@ -431,56 +550,8 @@ corresponding timer is removed. #### Connecting to RLQS Control Plane -xDS Control Plane provides RLQS connection details in [`GrpcService`] message ( -already supported by Envoy). `GrpcService` supports two modes: - -1. `GrpcService.EnvoyGrpc`, Envoy's minimal custom gRPC client implementation. -2. [`GrpcService.GoogleGrpc`], regular gRPC-cpp client. - -For obvious reasons, we'll only support the `GoogleGrpc` mode. - -##### Security Considerations - -In [`GrpcService.GoogleGrpc`], the xDS Control Plane provides the target URI, -`channel_credentials`, and `call_credentials`. If the xDS Control Plane is -compromised, an attacker could configure the xDS clients to talk to other -malicious Control Planes. This could lead to potential exploits such as leaking -customer's Application Default Credentials OAuth token. - -To prevent that, we'll introduce an allow-list to the bootstrap file introduced -in [gRFC A27]. This allow-list will be a map from a fully-qualified server -target URI to an object containing channel credentials to use for that target. - -```json5 -// The allowlist of Control Planes allowed to be configured via xDS. -"allowed_grpc_services": { - // The key is fully-qualified server URI. - "dns:///xds.example.org:443": { - // The value is an object containing "channel_creds". - "channel_creds": [ - // The format is identical to xds_servers.channel_creds. - { - "type": "string containing channel cred type", - "config": "optional JSON object containing config for the type" - } - ] - } -} -``` - -When xDS Control Plane configures connection to another control plane -via [`GrpcService.GoogleGrpc`] message, we'll inspect the allow-list for the -matching target URI. - -1. If target URI is not present, we don't create the connection to the requested - Control Plane, and NACK the xDS resource. -2. If target URI is present, we create the connection to the requested Control - Plane using the channel credentials provided in the bootstrap file. Transport - security configuration provided by the control plane is ignored. - -> [!IMPORTANT] -> This solution is not specific to RLQS, and should be used with any -> other Control Planes configured via [`GrpcService`] message. +xDS Control Plane provides RLQS connection details in [GrpcService.GoogleGrpc] +message as specified in [A102]. #### Unified Matcher API @@ -564,7 +635,7 @@ Hard-coded to `"POST"` if unavailable and a code audit confirms the server denies requests for all other method types. **2 `request.headers`**\ -As defined in [gRFC A41], "header" field. +As defined in [A41], "header" field. ##### Implementation From a115e1b8913504622aaa82425d44c65d62706e20 Mon Sep 17 00:00:00 2001 From: Sergii Tkachenko Date: Mon, 13 Oct 2025 14:01:30 -0700 Subject: [PATCH 32/53] On Sending Usage Reports --- A77-xds-rate-limiting-rlqs.md | 83 +++++++++++++++++++++++++++++++++-- 1 file changed, 79 insertions(+), 4 deletions(-) diff --git a/A77-xds-rate-limiting-rlqs.md b/A77-xds-rate-limiting-rlqs.md index 36ac08090..3073358b5 100644 --- a/A77-xds-rate-limiting-rlqs.md +++ b/A77-xds-rate-limiting-rlqs.md @@ -66,6 +66,10 @@ which are covered in the proposal: [Config: HeaderValueOption]: #config-headervalueoption [Config: Bucket Matchers]: #config-bucket-matchers [RLQS xDS HTTP Filter: Channel Level]: #rlqs-xds-http-filter-channel-level +[RLQS Buckets and Multithreading]: #rlqs-buckets-and-multithreading +[On Data Plane RPC]: #on-data-plane-rpc +[On Report Timers]: #on-report-timers +[On Sending Usage Reports]: #on-sending-usage-reports [Unified Matcher API]: https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/advanced/matching/matching_api.html [Envoy CEL environment]: https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/advanced/attributes @@ -517,8 +521,9 @@ it doesn't already exist. If a new bucket has been created, the RPC is rate-limited according to bucket's default configuration. Then the filter schedules a report on the RLQS stream -informing the server of the bucket's creation, and registers the report timers -for bucket's reporting interval. +informing the server of the bucket's creation as described in +[On Sending Usage Reports], and registers the report timers for bucket's +reporting interval. If the bucket exists, the RPC is rate-limited according to its active quota assignment and usage counters. The bucket increments corresponding bucket usage @@ -539,13 +544,83 @@ expiration is updated. When a report timer fires, the RLQS Filter State retrieves all buckets with the corresponding reporting interval from the RLQS Bucket Map. For each bucket, the -filter snapshots the current usage counters, and resets them. The filter then -sends the snapshot to RLQS server using RLQS Client. +filter snapshots the current usage counters, and resets them as detailed in +[On Sending Usage Reports]. The filter then sends the snapshot to RLQS server +using RLQS Client. If a bucket's active assignment has expired, the bucket is removed from the cache. If there are no more buckets with the given reporting interval, the corresponding timer is removed. +#### On Sending Usage Reports + +When preparing bucket reports, the implementer should keep in mind that bucket +usage counters may be updated concurrently by other threads, see +[RLQS Buckets and Multithreading]. + +One potential approach is preserving the current state of the bucket in a +snapshot and immediately resetting the usage counters. The snapshot would +contain time delta between the new and previous snapshot creation times, and the +number of allowed/denied requests. The pseudo-code for this approach may look +something like this: + +```java + private final AtomicLong numRequestsAllowed = new AtomicLong(); + private final AtomicLong numRequestsDenied = new AtomicLong(); + private final AtomicLong lastSnapshotTimeNanos = new AtomicLong(-1); + + public RlqsBucketUsage snapshotAndResetUsage() { + long snapAllowed = numRequestsAllowed.get(); + long snapDenied = numRequestsDenied.get(); + long snapTime = nanoTimeNow(); + + // Reset stats. + numRequestsAllowed.addAndGet(-snapAllowed); + numRequestsDenied.addAndGet(-snapDenied); + + long lastSnapTime = lastSnapshotTimeNanos.getAndSet(snapTime); + // First snapshot. + if (lastSnapTime < 0) { + lastSnapTime = snapTime; + } + // RlqsBucketUsage snapshots the current bucket state, and will later be transformed + // to the report sent to RLQS Server. + return RlqsBucketUsage.create(bucketId, snapAllowed, snapDenied, snapTime - lastSnapTime); + } +``` + +The `RateLimitQuotaUsageReports` message is sent to the RLQS server via the +`StreamRateLimitQuotas` RPC defined in [rlqs.proto][rlqs_proto]. Each message +contains usage reports for one or more buckets. + +The following fields will be populated in the `RateLimitQuotaUsageReports`: + +- `domain`: Populated from the `domain` field in the + [RateLimitQuotaFilterConfig]. This field is only sent in the first + `RateLimitQuotaUsageReports` message on a new gRPC stream to the RLQS + server. Subsequent messages on the same stream will omit this field. +- `bucket_quota_usages`: A list of `BucketQuotaUsage` messages, each + representing the usage report for a specific bucket. Each `BucketQuotaUsage` + message will have the following fields populated: + - `bucket_id`: Populated from the `bucket_id` of the `RlqsBucket`. This + identifies the bucket for which the usage is being reported. + - `time_elapsed`: A `google.protobuf.Duration` representing the time since + the last usage report was sent for this specific `bucket_id`. + - `num_requests_allowed`: The number of requests allowed for this bucket + since the last report. This comes from the `Request Counters` in the + `RlqsBucket`. + - `num_requests_denied`: The number of requests denied for this bucket + since the last report. This also comes from the `Request Counters` in + the `RlqsBucket`. + +Usage reports are sent in following scenarios: + +1. **On New Bucket Creation**: When the first RPC for a new `bucket_id` is + processed, an immediate report is scheduled to inform the RLQS server of the + new bucket subscription, see [On Data Plane RPC]. +2. **On Report Timers**: When a report timer fires, see [On Report Timers]. +3. **Replacing the assignment**: TODO(sergiitk) + ### Integrations #### Connecting to RLQS Control Plane From 840695d886c5a90fd06a220e50f3345473b4b83f Mon Sep 17 00:00:00 2001 From: Sergii Tkachenko Date: Mon, 13 Oct 2025 14:12:23 -0700 Subject: [PATCH 33/53] on reports: link protos --- A77-xds-rate-limiting-rlqs.md | 48 ++++++++++++++++++++--------------- 1 file changed, 28 insertions(+), 20 deletions(-) diff --git a/A77-xds-rate-limiting-rlqs.md b/A77-xds-rate-limiting-rlqs.md index 3073358b5..54bb5b203 100644 --- a/A77-xds-rate-limiting-rlqs.md +++ b/A77-xds-rate-limiting-rlqs.md @@ -76,6 +76,7 @@ which are covered in the proposal: [TokenBucket]: https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/type/v3/token_bucket.proto [GrpcService.GoogleGrpc]: https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/config/core/v3/grpc_service.proto#L68 +[`google.protobuf.Duration`]: https://developers.google.com/protocol-buffers/docs/reference/google.protobuf#google.protobuf.Duration [rlqs_proto]: https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/service/rate_limit_quota/v3/rlqs.proto @@ -544,7 +545,7 @@ expiration is updated. When a report timer fires, the RLQS Filter State retrieves all buckets with the corresponding reporting interval from the RLQS Bucket Map. For each bucket, the -filter snapshots the current usage counters, and resets them as detailed in +filter snapshots the current usage counters, and resets them as described in [On Sending Usage Reports]. The filter then sends the snapshot to RLQS server using RLQS Client. @@ -593,25 +594,32 @@ The `RateLimitQuotaUsageReports` message is sent to the RLQS server via the `StreamRateLimitQuotas` RPC defined in [rlqs.proto][rlqs_proto]. Each message contains usage reports for one or more buckets. -The following fields will be populated in the `RateLimitQuotaUsageReports`: - -- `domain`: Populated from the `domain` field in the - [RateLimitQuotaFilterConfig]. This field is only sent in the first - `RateLimitQuotaUsageReports` message on a new gRPC stream to the RLQS - server. Subsequent messages on the same stream will omit this field. -- `bucket_quota_usages`: A list of `BucketQuotaUsage` messages, each - representing the usage report for a specific bucket. Each `BucketQuotaUsage` - message will have the following fields populated: - - `bucket_id`: Populated from the `bucket_id` of the `RlqsBucket`. This - identifies the bucket for which the usage is being reported. - - `time_elapsed`: A `google.protobuf.Duration` representing the time since - the last usage report was sent for this specific `bucket_id`. - - `num_requests_allowed`: The number of requests allowed for this bucket - since the last report. This comes from the `Request Counters` in the - `RlqsBucket`. - - `num_requests_denied`: The number of requests denied for this bucket - since the last report. This also comes from the `Request Counters` in - the `RlqsBucket`. +The following fields will be populated in the +[`RateLimitQuotaUsageReports`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/service/rate_limit_quota/v3/rlqs.proto#L66): + +- [`domain`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/service/rate_limit_quota/v3/rlqs.proto#L96): + Populated from the `domain` field in the [RateLimitQuotaFilterConfig]. This + field is only sent in the first `RateLimitQuotaUsageReports` message on a + new gRPC stream to the RLQS server. Subsequent messages on the same stream + will omit this field. +- [`bucket_quota_usages`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/service/rate_limit_quota/v3/rlqs.proto#L100): + A list of + [`BucketQuotaUsage`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/service/rate_limit_quota/v3/rlqs.proto#L72) + messages, each representing the usage report for a specific bucket. Each + `BucketQuotaUsage` message will have the following fields populated: + - [`bucket_id`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/service/rate_limit_quota/v3/rlqs.proto#L74): + Populated from the `bucket_id` of the `RlqsBucket`. This identifies the + bucket for which the usage is being reported. + - [`time_elapsed`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/service/rate_limit_quota/v3/rlqs.proto#L77): + A [`google.protobuf.Duration`] representing the time difference between + the current time and the last usage report was sent for this specific + `bucket_id`. + - [`num_requests_allowed`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/service/rate_limit_quota/v3/rlqs.proto#L83): + The number of requests allowed for this bucket since the last report. + This comes from the `Request Counters` in the `RlqsBucket`. + - [`num_requests_denied`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/service/rate_limit_quota/v3/rlqs.proto#L86): + The number of requests denied for this bucket since the last report. + This also comes from the `Request Counters` in the `RlqsBucket`. Usage reports are sent in following scenarios: From 343c49c4e2847ef1f30685018cfc50c89fe72f35 Mon Sep 17 00:00:00 2001 From: Sergii Tkachenko Date: Mon, 13 Oct 2025 15:50:38 -0700 Subject: [PATCH 34/53] On RLQS Server Response WIP --- A77-xds-rate-limiting-rlqs.md | 17 +++++++++++++---- 1 file changed, 13 insertions(+), 4 deletions(-) diff --git a/A77-xds-rate-limiting-rlqs.md b/A77-xds-rate-limiting-rlqs.md index 54bb5b203..26e480f15 100644 --- a/A77-xds-rate-limiting-rlqs.md +++ b/A77-xds-rate-limiting-rlqs.md @@ -65,9 +65,12 @@ which are covered in the proposal: [Config: RuntimeFractionalPercent]: #config-runtimefractionalpercent [Config: HeaderValueOption]: #config-headervalueoption [Config: Bucket Matchers]: #config-bucket-matchers + [RLQS xDS HTTP Filter: Channel Level]: #rlqs-xds-http-filter-channel-level [RLQS Buckets and Multithreading]: #rlqs-buckets-and-multithreading + [On Data Plane RPC]: #on-data-plane-rpc +[On RLQS Server Response]: #on-rlqs-server-response [On Report Timers]: #on-report-timers [On Sending Usage Reports]: #on-sending-usage-reports @@ -535,11 +538,16 @@ counter. When receiving an RLQS Server Response, the RLQS Client passes parsed response to the RLQS Filter State. The RLQS Filter State iterates through the bucket assignments in the response, and updates the corresponding buckets in the RLQS -Bucket Map. Buckets marked to be abandoned are purged from the cache. +Bucket Map as described in +[`RateLimitQuotaResponse.QuotaAssignmentAction`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/service/rate_limit_quota/v3/rlqs.proto#L106-L139). If a bucket has a new rate limit assignment, the bucket's active assignment is -updated and bucket usage counters are reset. Otherwise, only the assignment -expiration is updated. +updated and an immediate bucket usage report is sent, see +[On Sending Usage Reports]. Otherwise, only the assignment expiration is +updated. + +Buckets marked to be abandoned are purged from the cache as described in +[`RateLimitQuotaResponse.AbandonAction`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/service/rate_limit_quota/v3/rlqs.proto#L169-L199). #### On Report Timers @@ -627,7 +635,8 @@ Usage reports are sent in following scenarios: processed, an immediate report is scheduled to inform the RLQS server of the new bucket subscription, see [On Data Plane RPC]. 2. **On Report Timers**: When a report timer fires, see [On Report Timers]. -3. **Replacing the assignment**: TODO(sergiitk) +3. **Replacing the assignment**: When bucket's active assignment is replaced, + as described in [On RLQS Server Response]. ### Integrations From 79f1bf668f51ce2528f6fd9fd6a86137855e0fb4 Mon Sep 17 00:00:00 2001 From: Sergii Tkachenko Date: Tue, 14 Oct 2025 14:31:17 -0700 Subject: [PATCH 35/53] bucket matchers wip --- A77-xds-rate-limiting-rlqs.md | 103 ++++++++++++++++++++++++++++++++-- 1 file changed, 99 insertions(+), 4 deletions(-) diff --git a/A77-xds-rate-limiting-rlqs.md b/A77-xds-rate-limiting-rlqs.md index 26e480f15..d8f482fcb 100644 --- a/A77-xds-rate-limiting-rlqs.md +++ b/A77-xds-rate-limiting-rlqs.md @@ -65,9 +65,11 @@ which are covered in the proposal: [Config: RuntimeFractionalPercent]: #config-runtimefractionalpercent [Config: HeaderValueOption]: #config-headervalueoption [Config: Bucket Matchers]: #config-bucket-matchers +[Config: RateLimitQuotaBucketSettings]: #config-ratelimitquotabucketsettings [RLQS xDS HTTP Filter: Channel Level]: #rlqs-xds-http-filter-channel-level [RLQS Buckets and Multithreading]: #rlqs-buckets-and-multithreading +[Unified Matcher API Support]: #unified-matcher-api-support [On Data Plane RPC]: #on-data-plane-rpc [On RLQS Server Response]: #on-rlqs-server-response @@ -157,7 +159,9 @@ The following fields will be ignored by gRPC: #### Config: HeaderValueOption -We will support the following fields in the [HeaderValueOption](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/config/core/v3/base.proto#L429) proto: +We will support the following fields in the +[HeaderValueOption](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/config/core/v3/base.proto#L429) +proto: - [header](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/config/core/v3/base.proto#L458): Must be present. @@ -189,9 +193,100 @@ The following fields will be ignored by gRPC: #### Config: Bucket Matchers -TODO(sergiitk): add details +The `bucket_matchers` field is a +[`xds.type.matcher.v3.Matcher`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L22) +(see [Unified Matcher API Support]) used to assign requests based on their +metadata to bucket configurations as defined in +[Config: RateLimitQuotaBucketSettings]. + +We will support the following fields: + +- `matcher_type`: One of the following must be set: + - `matcher_list`: A + [`MatcherList`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L43) + containing a list of matchers to evaluate in order. The first one that + matches wins. + - `matcher_tree`: A + [`MatcherTree`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L99) + that maps input values to actions. +- `on_no_match`: If set, specifies an action to take when no matcher succeeds. + Must be a valid + [`OnMatch`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L24) + message. If not set, requests that do not match are allowed and not reported + to the RLQS server. + +##### Bucket Matchers: OnMatch + +A match action is defined by an `OnMatch` message, which contains either a +nested `Matcher` or an `action`. For RLQS, the `action` must be a +[`TypedExtensionConfig`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/core/v3/extension.proto#L14) +containing a `RateLimitQuotaBucketSettings` message as described in +[Config: RateLimitQuotaBucketSettings]. ---- +The following fields will be ignored by gRPC: + +- `keep_matching`: Not supported in the initial implementation, may be added + later. + +##### Bucket Matchers: MatcherList + +A matcher's `predicate` determines if a request matches. We will support the +following predicate types: + +- `single_predicate`: A single condition to evaluate. + - `input`: A `TypedExtensionConfig` specifying what part of the request to + match against. See the [Unified Matcher API Support] for supported + inputs. + - `value_match`: A + [`StringMatcher`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/string.proto#L19) + to match the input against a string value. All string match types are + supported. + - `custom_match`: A `TypedExtensionConfig` for custom matching logic. See + the [Unified Matcher API Support] for supported custom matchers. +- `or_matcher`: A list of predicates, returns true if any of them are true. +- `and_matcher`: A list of predicates, returns true if all of them are true. +- `not_matcher`: Inverts the result of a predicate. + +#### Config: RateLimitQuotaBucketSettings + +The `RateLimitQuotaBucketSettings` message configures the behavior of a bucket. +We will support the following fields: + +- `bucket_id_builder`: Used to construct the `BucketId` for each request. If + not set, requests are not assigned to a bucket, not reported to the RLQS + server, and are handled by `no_assignment_behavior`. + - `bucket_id_builder`: A map from string to `ValueBuilder`. + - `ValueBuilder`: One of the following must be set: + - `string_value`: A static string value. + - `custom_value`: A `TypedExtensionConfig` that resolves to a + string. The supported extensions are the same as for matcher + inputs. +- `reporting_interval`: Must be present. The interval at which the client + reports usage to the RLQS server. Must be greater than 100ms. +- `deny_response_settings`: Customizes the response for denied requests. + - `http_status`: HTTP status code for denied HTTP requests. Defaults to + 1. gRPC requests are not affected. + - `http_body`: HTTP response body for denied HTTP requests. gRPC requests + are not affected. + - `grpc_status`: `google.rpc.Status` for denied gRPC requests. Defaults to + `UNAVAILABLE`. + - `response_headers_to_add`: A list of up to 10 headers to add to the deny + response, as described in [Config: HeaderValueOption]. +- `no_assignment_behavior`: Behavior before the first quota assignment is + received. If not set, all requests are allowed. + - `fallback_rate_limit`: A + [`RateLimitStrategy`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/type/v3/ratelimit_strategy.proto#L16) + to apply. + - `blanket_rule`: One of `ALLOW_ALL` or `DENY_ALL`. + - `requests_per_time_unit`: A token bucket-style rate limit. +- `expired_assignment_behavior`: Behavior when a quota assignment expires and + cannot be refreshed. If not set, the bucket is abandoned. + - `expired_assignment_behavior_timeout`: How long to apply the expired + assignment behavior before abandoning the bucket. Defaults to 0 (abandon + immediately). + - One of the following must be set: + - `fallback_rate_limit`: A `RateLimitStrategy` to apply. + - `reuse_last_assignment`: Reuse the last known quota assignment. ### RLQS Components @@ -645,7 +740,7 @@ Usage reports are sent in following scenarios: xDS Control Plane provides RLQS connection details in [GrpcService.GoogleGrpc] message as specified in [A102]. -#### Unified Matcher API +#### Unified Matcher API Support RPCs will be matched into buckets using [Unified Matcher API] — an adaptable framework that can be used in any xDS component that needs matching features. From b28d12a258950ea22909e4e795c8a0e3ee6077d1 Mon Sep 17 00:00:00 2001 From: Sergii Tkachenko Date: Wed, 15 Oct 2025 12:18:11 -0700 Subject: [PATCH 36/53] Backticks in filter config for consistency with GrpcService gRFC --- A77-xds-rate-limiting-rlqs.md | 78 +++++++++++++++++------------------ 1 file changed, 39 insertions(+), 39 deletions(-) diff --git a/A77-xds-rate-limiting-rlqs.md b/A77-xds-rate-limiting-rlqs.md index d8f482fcb..eeaf8cd84 100644 --- a/A77-xds-rate-limiting-rlqs.md +++ b/A77-xds-rate-limiting-rlqs.md @@ -62,10 +62,10 @@ which are covered in the proposal: [RateLimitQuotaFilterConfig]: #ratelimitquotafilterconfig [RateLimitQuotaOverride]: #ratelimitquotaoverride -[Config: RuntimeFractionalPercent]: #config-runtimefractionalpercent -[Config: HeaderValueOption]: #config-headervalueoption +[Config: `RuntimeFractionalPercent`]: #config-runtimefractionalpercent +[Config: `HeaderValueOption`]: #config-headervalueoption [Config: Bucket Matchers]: #config-bucket-matchers -[Config: RateLimitQuotaBucketSettings]: #config-ratelimitquotabucketsettings +[Config: `RateLimitQuotaBucketSettings`]: #config-ratelimitquotabucketsettings [RLQS xDS HTTP Filter: Channel Level]: #rlqs-xds-http-filter-channel-level [RLQS Buckets and Multithreading]: #rlqs-buckets-and-multithreading @@ -92,61 +92,61 @@ which are covered in the proposal: The RLQS Filter API is defined in [rate_limit_quota.proto](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto). -#### RateLimitQuotaFilterConfig +#### `RateLimitQuotaFilterConfig` We will support the following fields in the -[RateLimitQuotaFilterConfig](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L37) +[`RateLimitQuotaFilterConfig`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L37) message: -- [rlqs_server](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L39): +- [`rlqs_server`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L39): This field must be present. Inside of it, GrpcService as described in [A102]. -- [domain](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L44): +- [`domain`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L44): This field must be present and non-empty. -- [bucket_matchers](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L115): +- [`bucket_matchers`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L115): This field must be present. Inside of it, there must be a valid matchers structure as described in [Config: Bucket Matchers]. -- [filter_enabled](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L121): +- [`filter_enabled`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L121): Specifies the fraction of requests for which the rate limiting is enabled. When not present, the filter is enabled for all requests (100%). Otherwise, - the fraction is determined from [Config: RuntimeFractionalPercent]. -- [filter_enforced](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L132): + the fraction is determined from [Config: `RuntimeFractionalPercent`]. +- [`filter_enforced`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L132): Specifies the fraction of requests for which the rate limiting is enforced. When not present, the filter is enforced for all requests (100%). Otherwise, - the fraction is determined from [Config: RuntimeFractionalPercent]. -- [request_headers_to_add_when_not_enforced](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L137): - If present, must be a list of [Config: HeaderValueOption]. Must not contain - more than 10 items. + the fraction is determined from [Config: `RuntimeFractionalPercent`]. +- [`request_headers_to_add_when_not_enforced`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L137): + If present, must be a list of [Config: `HeaderValueOption`]. Must not + contain more than 10 items. -#### RateLimitQuotaOverride +#### `RateLimitQuotaOverride` We will support the following fields in the -[RateLimitQuotaFilterConfig](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L143) +[`RateLimitQuotaOverride`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L143) message: -- [domain](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L44): +- [`domain`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L44): If non-empty, overrides the domain value provided on the less specific definition. -- [bucket_matchers](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L115): +- [`bucket_matchers`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L115): If present, must be a valid matchers structure as described in [Config: Bucket Matchers], and fully overrides the matchers provided on the less specific definition. -#### Config: RuntimeFractionalPercent +#### Config: `RuntimeFractionalPercent` We will support the following fields in the [RuntimeFractionalPercent](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/config/core/v3/base.proto#L643) proto: -- [default_value](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/config/core/v3/base.proto#L648): +- [`default_value`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/config/core/v3/base.proto#L648): This field must be present. If the denominator specified is less than the numerator, the final fractional percentage is capped at 1 (100%). The fraction specified with: - - [numerator](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/type/v3/percent.proto#L52): + - [`numerator`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/type/v3/percent.proto#L52): Non-negative integer, 0 by default. - - [denominator](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/type/v3/percent.proto#L56): + - [`denominator`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/type/v3/percent.proto#L56): Must be one of the - [DenominatorType](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/type/v3/percent.proto#L34) + [`DenominatorType`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/type/v3/percent.proto#L34) values: - `HUNDRED` (default) - `TEN_THOUSAND` @@ -155,41 +155,41 @@ proto: The following fields will be ignored by gRPC: -- runtime_key: gRPC does not have Envoy's concept of runtime settings. +- `runtime_key`: gRPC does not have Envoy's concept of runtime settings. -#### Config: HeaderValueOption +#### Config: `HeaderValueOption` We will support the following fields in the -[HeaderValueOption](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/config/core/v3/base.proto#L429) +[`HeaderValueOption`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/config/core/v3/base.proto#L429) proto: -- [header](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/config/core/v3/base.proto#L458): +- [`header`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/config/core/v3/base.proto#L458): Must be present. - - [key](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/config/core/v3/base.proto#L404): + - [`key`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/config/core/v3/base.proto#L404): Value length must be in the range `[1, 16384)`. Must be a valid HTTP/2 header name. - - [value](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/config/core/v3/base.proto#L415): + - [`value`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/config/core/v3/base.proto#L415): Specifies the header value. Must be shorter than 16384 bytes. Must be a valid HTTP/2 header value. Not used if `key` ends in `-bin` and `raw_value` is set. - - [raw_value](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/config/core/v3/base.proto#L422): + - [`raw_value`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/config/core/v3/base.proto#L422): Used only if `key` ends in `-bin`. Must be shorter than 16384 bytes. Will be base64-encoded on the wire, unless the pure binary metadata extension from [gRFC G1: True Binary Metadata][G1] is used. -- [append_action](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/config/core/v3/base.proto#L476): +- [`append_action`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/config/core/v3/base.proto#L476): Must be of the - [HeaderAppendAction](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/config/core/v3/base.proto#L434) + [`HeaderAppendAction`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/config/core/v3/base.proto#L434) values: - `APPEND_IF_EXISTS_OR_ADD` (default) - `ADD_IF_ABSENT` - `OVERWRITE_IF_EXISTS_OR_ADD` - `OVERWRITE_IF_EXISTS` -- [keep_empty_value](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/config/core/v3/base.proto#L480) +- [`keep_empty_value`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/config/core/v3/base.proto#L480) - All other fields are ignored. The following fields will be ignored by gRPC: -- append: Deprecated in favor of `append_action`. +- `append`: Deprecated in favor of `append_action`. #### Config: Bucket Matchers @@ -197,7 +197,7 @@ The `bucket_matchers` field is a [`xds.type.matcher.v3.Matcher`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L22) (see [Unified Matcher API Support]) used to assign requests based on their metadata to bucket configurations as defined in -[Config: RateLimitQuotaBucketSettings]. +[Config: `RateLimitQuotaBucketSettings`]. We will support the following fields: @@ -221,7 +221,7 @@ A match action is defined by an `OnMatch` message, which contains either a nested `Matcher` or an `action`. For RLQS, the `action` must be a [`TypedExtensionConfig`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/core/v3/extension.proto#L14) containing a `RateLimitQuotaBucketSettings` message as described in -[Config: RateLimitQuotaBucketSettings]. +[Config: `RateLimitQuotaBucketSettings`]. The following fields will be ignored by gRPC: @@ -247,7 +247,7 @@ following predicate types: - `and_matcher`: A list of predicates, returns true if all of them are true. - `not_matcher`: Inverts the result of a predicate. -#### Config: RateLimitQuotaBucketSettings +#### Config: `RateLimitQuotaBucketSettings` The `RateLimitQuotaBucketSettings` message configures the behavior of a bucket. We will support the following fields: @@ -271,7 +271,7 @@ We will support the following fields: - `grpc_status`: `google.rpc.Status` for denied gRPC requests. Defaults to `UNAVAILABLE`. - `response_headers_to_add`: A list of up to 10 headers to add to the deny - response, as described in [Config: HeaderValueOption]. + response, as described in [Config: `HeaderValueOption`]. - `no_assignment_behavior`: Behavior before the first quota assignment is received. If not set, all requests are allowed. - `fallback_rate_limit`: A From b6553642810f19332f968854c1b792e3483f8c51 Mon Sep 17 00:00:00 2001 From: Sergii Tkachenko Date: Sun, 26 Oct 2025 16:03:27 -0700 Subject: [PATCH 37/53] wip Config: Bucket Matchers reorder --- A77-xds-rate-limiting-rlqs.md | 54 ++++++++++++++++++++++++++++++++++- 1 file changed, 53 insertions(+), 1 deletion(-) diff --git a/A77-xds-rate-limiting-rlqs.md b/A77-xds-rate-limiting-rlqs.md index eeaf8cd84..dd9866514 100644 --- a/A77-xds-rate-limiting-rlqs.md +++ b/A77-xds-rate-limiting-rlqs.md @@ -79,9 +79,13 @@ which are covered in the proposal: [Unified Matcher API]: https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/advanced/matching/matching_api.html [Envoy CEL environment]: https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/advanced/attributes +[`xds.type.matcher.v3.HttpAttributesCelMatchInput`]: https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/http_inputs.proto#L22 +[`xds.type.matcher.v3.CelMatcher`]: https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/cel.proto#L30 + [TokenBucket]: https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/type/v3/token_bucket.proto [GrpcService.GoogleGrpc]: https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/config/core/v3/grpc_service.proto#L68 [`google.protobuf.Duration`]: https://developers.google.com/protocol-buffers/docs/reference/google.protobuf#google.protobuf.Duration +[`TypedExtensionConfig`]: https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/core/v3/extension.proto#L14 [rlqs_proto]: https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/service/rate_limit_quota/v3/rlqs.proto @@ -193,6 +197,50 @@ The following fields will be ignored by gRPC: #### Config: Bucket Matchers +RPCs will be matched into buckets using [Unified Matcher API] — an adaptable +framework for xDS components requiring matching features. General details on +parsing and validating the proto are described in [Unified Matcher API Support]. + +For RLQS, the `bucket_matchers` field in the filter config will contain a +Unified Matcher tree restricted to the following protocol-specific types (packed +as [`TypedExtensionConfig`]): + +1. Input specification: [`xds.type.matcher.v3.HttpAttributesCelMatchInput`]. +2. Custom matching logic: [`xds.type.matcher.v3.CelMatcher`]. +3. Protocol-specific action: [Config: `RateLimitQuotaBucketSettings`]. + +Any other types are considered invalid and will result in gRPC NACKing the xDS +resource. + +--- + + + When evaluated against RPC metadata, this tree must yield +an `OnMatch.action` that contains a [Config: `RateLimitQuotaBucketSettings`][] +message (packed as a [`TypedExtensionConfig`]). Any other action type is +considered invalid and will result in gRPC NACKing the xDS resource. + + +RLQS will only support `HttpAttributesCelMatchInput` +The RPC metadata will serve as an input to +[`HttpAttributesCelMatchInput`](https://www.envoyproxy.io/docs/envoy/latest/xds/type/matcher/v3/http_inputs.proto#envoy-v3-api-msg-xds-type-matcher-v3-httpattributescelmatchinput). + +A match action is defined by an `OnMatch` message, which contains either a +nested `Matcher` or an `action`. For RLQS, the `action` must be a +[`TypedExtensionConfig`]() +containing a `RateLimitQuotaBucketSettings` message as described in +[Config: `RateLimitQuotaBucketSettings`]. + + +In this iteration the following Unified Mather extensions will be supported: + +1. Inputs: + 1. [`HttpRequestHeaderMatchInput`](https://www.envoyproxy.io/docs/envoy/latest/api-v3/type/matcher/v3/http_inputs.proto#type-matcher-v3-httprequestheadermatchinput) + 2. [`HttpAttributesCelMatchInput`](https://www.envoyproxy.io/docs/envoy/latest/xds/type/matcher/v3/http_inputs.proto#envoy-v3-api-msg-xds-type-matcher-v3-httpattributescelmatchinput) +2. Custom Matchers: + 1. [`CelMatcher`](https://www.envoyproxy.io/docs/envoy/latest/xds/type/matcher/v3/cel.proto.html) + + The `bucket_matchers` field is a [`xds.type.matcher.v3.Matcher`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L22) (see [Unified Matcher API Support]) used to assign requests based on their @@ -249,6 +297,10 @@ following predicate types: #### Config: `RateLimitQuotaBucketSettings` +which +contains the information needed to associate the RPC with `bucket_id` and the +default rate limiting configuration. + The `RateLimitQuotaBucketSettings` message configures the behavior of a bucket. We will support the following fields: @@ -373,7 +425,7 @@ final class RlqsFilter implements Filter { // Merge with per-route overrides if provided. if (overrideConfig instanceof RlqsConfigOverride rlqsConfigOverride) { - // Only domain and matchers can be overriden. + // Only domain and matchers can be overridden. if (!rlqsConfigOverride.domain().isEmpty()) { rlqsConfigBuilder.domain(rlqsConfigOverride.domain()); } From 6527980a800291d091c04466b159c36d05655383 Mon Sep 17 00:00:00 2001 From: Sergii Tkachenko Date: Sun, 26 Oct 2025 16:03:50 -0700 Subject: [PATCH 38/53] finish reorder Config: Bucket Matchers --- A77-xds-rate-limiting-rlqs.md | 108 ++++++++++++---------------------- 1 file changed, 38 insertions(+), 70 deletions(-) diff --git a/A77-xds-rate-limiting-rlqs.md b/A77-xds-rate-limiting-rlqs.md index dd9866514..c08782f41 100644 --- a/A77-xds-rate-limiting-rlqs.md +++ b/A77-xds-rate-limiting-rlqs.md @@ -69,7 +69,11 @@ which are covered in the proposal: [RLQS xDS HTTP Filter: Channel Level]: #rlqs-xds-http-filter-channel-level [RLQS Buckets and Multithreading]: #rlqs-buckets-and-multithreading + [Unified Matcher API Support]: #unified-matcher-api-support +[Unified Matcher Extension: `HttpRequestHeaderMatchInput`]: #unified-matcher-extension-httprequestheadermatchinput +[Unified Matcher Extension: `HttpAttributesCelMatchInput`]: #unified-matcher-extension-httpattributescelmatchinput +[Unified Matcher Extension: `CelMatcher`]: #unified-matcher-extension-celmatcher [On Data Plane RPC]: #on-data-plane-rpc [On RLQS Server Response]: #on-rlqs-server-response @@ -79,9 +83,6 @@ which are covered in the proposal: [Unified Matcher API]: https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/advanced/matching/matching_api.html [Envoy CEL environment]: https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/advanced/attributes -[`xds.type.matcher.v3.HttpAttributesCelMatchInput`]: https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/http_inputs.proto#L22 -[`xds.type.matcher.v3.CelMatcher`]: https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/cel.proto#L30 - [TokenBucket]: https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/type/v3/token_bucket.proto [GrpcService.GoogleGrpc]: https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/config/core/v3/grpc_service.proto#L68 [`google.protobuf.Duration`]: https://developers.google.com/protocol-buffers/docs/reference/google.protobuf#google.protobuf.Duration @@ -197,72 +198,22 @@ The following fields will be ignored by gRPC: #### Config: Bucket Matchers -RPCs will be matched into buckets using [Unified Matcher API] — an adaptable -framework for xDS components requiring matching features. General details on -parsing and validating the proto are described in [Unified Matcher API Support]. +RPCs are matched into buckets using the [Unified Matcher API] — an adaptable +framework for xDS components requiring matching features. For details on general +Unified Matcher proto parsing and validation, see [Unified Matcher API Support]. -For RLQS, the `bucket_matchers` field in the filter config will contain a -Unified Matcher tree restricted to the following protocol-specific types (packed -as [`TypedExtensionConfig`]): +Specifically for RLQS, the `bucket_matchers` field in the filter config will +contain a Unified Matcher restricted to the following protocol-specific types, +packed as a [`TypedExtensionConfig`]: -1. Input specification: [`xds.type.matcher.v3.HttpAttributesCelMatchInput`]. -2. Custom matching logic: [`xds.type.matcher.v3.CelMatcher`]. -3. Protocol-specific action: [Config: `RateLimitQuotaBucketSettings`]. +1. Input specification: + [Unified Matcher Extension: `HttpAttributesCelMatchInput`]. +2. Custom matching logic: [Unified Matcher Extension: `CelMatcher`]. +3. Protocol-specific action: [Config: `RateLimitQuotaBucketSettings`]. Any other types are considered invalid and will result in gRPC NACKing the xDS resource. ---- - - - When evaluated against RPC metadata, this tree must yield -an `OnMatch.action` that contains a [Config: `RateLimitQuotaBucketSettings`][] -message (packed as a [`TypedExtensionConfig`]). Any other action type is -considered invalid and will result in gRPC NACKing the xDS resource. - - -RLQS will only support `HttpAttributesCelMatchInput` -The RPC metadata will serve as an input to -[`HttpAttributesCelMatchInput`](https://www.envoyproxy.io/docs/envoy/latest/xds/type/matcher/v3/http_inputs.proto#envoy-v3-api-msg-xds-type-matcher-v3-httpattributescelmatchinput). - -A match action is defined by an `OnMatch` message, which contains either a -nested `Matcher` or an `action`. For RLQS, the `action` must be a -[`TypedExtensionConfig`]() -containing a `RateLimitQuotaBucketSettings` message as described in -[Config: `RateLimitQuotaBucketSettings`]. - - -In this iteration the following Unified Mather extensions will be supported: - -1. Inputs: - 1. [`HttpRequestHeaderMatchInput`](https://www.envoyproxy.io/docs/envoy/latest/api-v3/type/matcher/v3/http_inputs.proto#type-matcher-v3-httprequestheadermatchinput) - 2. [`HttpAttributesCelMatchInput`](https://www.envoyproxy.io/docs/envoy/latest/xds/type/matcher/v3/http_inputs.proto#envoy-v3-api-msg-xds-type-matcher-v3-httpattributescelmatchinput) -2. Custom Matchers: - 1. [`CelMatcher`](https://www.envoyproxy.io/docs/envoy/latest/xds/type/matcher/v3/cel.proto.html) - - -The `bucket_matchers` field is a -[`xds.type.matcher.v3.Matcher`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L22) -(see [Unified Matcher API Support]) used to assign requests based on their -metadata to bucket configurations as defined in -[Config: `RateLimitQuotaBucketSettings`]. - -We will support the following fields: - -- `matcher_type`: One of the following must be set: - - `matcher_list`: A - [`MatcherList`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L43) - containing a list of matchers to evaluate in order. The first one that - matches wins. - - `matcher_tree`: A - [`MatcherTree`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L99) - that maps input values to actions. -- `on_no_match`: If set, specifies an action to take when no matcher succeeds. - Must be a valid - [`OnMatch`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L24) - message. If not set, requests that do not match are allowed and not reported - to the RLQS server. - ##### Bucket Matchers: OnMatch A match action is defined by an `OnMatch` message, which contains either a @@ -297,10 +248,6 @@ following predicate types: #### Config: `RateLimitQuotaBucketSettings` -which -contains the information needed to associate the RPC with `bucket_id` and the -default rate limiting configuration. - The `RateLimitQuotaBucketSettings` message configures the behavior of a bucket. We will support the following fields: @@ -811,10 +758,31 @@ default rate limiting configuration. In this iteration the following Unified Mather extensions will be supported: 1. Inputs: - 1. [`HttpRequestHeaderMatchInput`](https://www.envoyproxy.io/docs/envoy/latest/api-v3/type/matcher/v3/http_inputs.proto#type-matcher-v3-httprequestheadermatchinput) - 2. [`HttpAttributesCelMatchInput`](https://www.envoyproxy.io/docs/envoy/latest/xds/type/matcher/v3/http_inputs.proto#envoy-v3-api-msg-xds-type-matcher-v3-httpattributescelmatchinput) + 1. [Unified Matcher Extension: `HttpRequestHeaderMatchInput`] + 2. [Unified Matcher Extension: `HttpAttributesCelMatchInput`] 2. Custom Matchers: - 1. [`CelMatcher`](https://www.envoyproxy.io/docs/envoy/latest/xds/type/matcher/v3/cel.proto.html) + 1. [Unified Matcher Extension: `CelMatcher`] + +##### Unified Matcher Extension: `HttpRequestHeaderMatchInput` + +* [`xds.type.matcher.v3.HttpAttributesCelMatchInput`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/http_inputs.proto#L22) +* [`HttpRequestHeaderMatchInput`](https://www.envoyproxy.io/docs/envoy/latest/api-v3/type/matcher/v3/http_inputs.proto#type-matcher-v3-httprequestheadermatchinput) + +TODO(sergiitk): finish + +##### Unified Matcher Extension: `HttpAttributesCelMatchInput` + +* [`xds.type.matcher.v3.HttpAttributesCelMatchInput`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/http_inputs.proto#L22) +* [`HttpAttributesCelMatchInput`](https://www.envoyproxy.io/docs/envoy/latest/xds/type/matcher/v3/http_inputs.proto#envoy-v3-api-msg-xds-type-matcher-v3-httpattributescelmatchinput) + +TODO(sergiitk): finish + +##### Unified Matcher Extension: `CelMatcher` + +* [`xds.type.matcher.v3.CelMatcher`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/cel.proto#L30) +* [`CelMatcher`](https://www.envoyproxy.io/docs/envoy/latest/xds/type/matcher/v3/cel.proto.html) + +TODO(sergiitk): finish #### CEL Integration From 99e1eb65774bbd6ec071ef38885b5bd497a5439a Mon Sep 17 00:00:00 2001 From: Sergii Tkachenko Date: Sun, 26 Oct 2025 20:37:48 -0700 Subject: [PATCH 39/53] unified matcher api wip --- A77-xds-rate-limiting-rlqs.md | 213 ++++++++++++++++++++++++++-------- 1 file changed, 165 insertions(+), 48 deletions(-) diff --git a/A77-xds-rate-limiting-rlqs.md b/A77-xds-rate-limiting-rlqs.md index c08782f41..3c99ead9d 100644 --- a/A77-xds-rate-limiting-rlqs.md +++ b/A77-xds-rate-limiting-rlqs.md @@ -71,6 +71,14 @@ which are covered in the proposal: [RLQS Buckets and Multithreading]: #rlqs-buckets-and-multithreading [Unified Matcher API Support]: #unified-matcher-api-support +[Unified Matcher: `Matcher`]: #unified-matcher-matcher +[Unified Matcher: `OnMatch`]: #unified-matcher-onmatch +[Unified Matcher: `MatcherList`]: #unified-matcher-matcherlist +[Unified Matcher: `MatcherList.Predicate`]: #unified-matcher-matcherlistpredicate +[Unified Matcher: `MatcherList.Predicate.SinglePredicate`]: #unified-matcher-matcherlistpredicatesinglepredicate +[Unified Matcher: `MatcherList.Predicate.PredicateList`]: #unified-matcher-matcherlistpredicatepredicatelist +[Unified Matcher: `MatcherTree`]: #unified-matcher-matchertree +[Unified Matcher: `MatcherTree.MatchMap`]: #unified-matcher-matchertreematchmap [Unified Matcher Extension: `HttpRequestHeaderMatchInput`]: #unified-matcher-extension-httprequestheadermatchinput [Unified Matcher Extension: `HttpAttributesCelMatchInput`]: #unified-matcher-extension-httpattributescelmatchinput [Unified Matcher Extension: `CelMatcher`]: #unified-matcher-extension-celmatcher @@ -202,49 +210,25 @@ RPCs are matched into buckets using the [Unified Matcher API] — an adaptable framework for xDS components requiring matching features. For details on general Unified Matcher proto parsing and validation, see [Unified Matcher API Support]. -Specifically for RLQS, the `bucket_matchers` field in the filter config will -contain a Unified Matcher restricted to the following protocol-specific types, -packed as a [`TypedExtensionConfig`]: +For RLQS, the `bucket_matchers` field in the filter config will contain a +Unified Matcher restricted to the following protocol-specific types, packed as a +[`TypedExtensionConfig`]: -1. Input specification: +1. Input specification (`input` fields): [Unified Matcher Extension: `HttpAttributesCelMatchInput`]. -2. Custom matching logic: [Unified Matcher Extension: `CelMatcher`]. -3. Protocol-specific action: [Config: `RateLimitQuotaBucketSettings`]. +2. Custom matching logic (`custom_match` fields): + [Unified Matcher Extension: `CelMatcher`]. +3. Protocol-specific action (`OnMatch.action` field): + [Config: `RateLimitQuotaBucketSettings`]. Any other types are considered invalid and will result in gRPC NACKing the xDS resource. -##### Bucket Matchers: OnMatch +TODO(sergiitk): describe inputs/output? -A match action is defined by an `OnMatch` message, which contains either a -nested `Matcher` or an `action`. For RLQS, the `action` must be a -[`TypedExtensionConfig`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/core/v3/extension.proto#L14) -containing a `RateLimitQuotaBucketSettings` message as described in -[Config: `RateLimitQuotaBucketSettings`]. - -The following fields will be ignored by gRPC: - -- `keep_matching`: Not supported in the initial implementation, may be added - later. - -##### Bucket Matchers: MatcherList - -A matcher's `predicate` determines if a request matches. We will support the -following predicate types: - -- `single_predicate`: A single condition to evaluate. - - `input`: A `TypedExtensionConfig` specifying what part of the request to - match against. See the [Unified Matcher API Support] for supported - inputs. - - `value_match`: A - [`StringMatcher`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/string.proto#L19) - to match the input against a string value. All string match types are - supported. - - `custom_match`: A `TypedExtensionConfig` for custom matching logic. See - the [Unified Matcher API Support] for supported custom matchers. -- `or_matcher`: A list of predicates, returns true if any of them are true. -- `and_matcher`: A list of predicates, returns true if all of them are true. -- `not_matcher`: Inverts the result of a predicate. +Evaluating the tree against RPC metadata yields `RateLimitQuotaBucketSettings`, +which contains the information needed to associate the RPC with `bucket_id` and +the default rate limiting configuration. #### Config: `RateLimitQuotaBucketSettings` @@ -741,28 +725,161 @@ message as specified in [A102]. #### Unified Matcher API Support -RPCs will be matched into buckets using [Unified Matcher API] — an adaptable -framework that can be used in any xDS component that needs matching features. +[Unified Matcher API] is an adaptable framework that can be used in any xDS +component that needs matching features. + +Envoy provides two syntactically equivalent Unified Matcher definitions: +[`envoy.config.common.matcher.v3.Matcher`](https://github.com/envoyproxy/envoy/blob/e3da7ebb16ad01c2ac7662758a75dba5cdc024ce/api/envoy/config/common/matcher/v3/matcher.proto) +and +[`xds.type.matcher.v3.Matcher`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto), +which is the preferred version for all new APIs using Unified Matcher. If +`envoy.config.common.matcher.v3.Matcher` is provided, we will interpret it as is +`xds.type.matcher.v3.Matcher`. -Envoy provides two syntactically equivalent Unified Matcher -definitions: [`envoy.config.common.matcher.v3.Matcher`](https://github.com/envoyproxy/envoy/blob/e3da7ebb16ad01c2ac7662758a75dba5cdc024ce/api/envoy/config/common/matcher/v3/matcher.proto) -and [`xds.type.matcher.v3.Matcher`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto). -We will only support the latter, which is the preferred version for all new APIs -using Unified Matcher. +When implementing Unified Matcher API, a filter must define the following: -For RLQS, Unified Matcher tree will be provided in the filter config. Evaluating -the tree against RPC metadata will yield `RateLimitQuotaBucketSettings`, which -contains the information needed to associate the RPC with `bucket_id` and the -default rate limiting configuration. +- Supported protocol-specific actions (see [Unified Matcher: `OnMatch`]). +- Supported input extensions. +- Supported custom matcher extensions, and any limitations on their inputs. +- Filter-specific default no-match behavior (f.e. xDS resource NACK). In this iteration the following Unified Mather extensions will be supported: -1. Inputs: +1. Inputs: 1. [Unified Matcher Extension: `HttpRequestHeaderMatchInput`] 2. [Unified Matcher Extension: `HttpAttributesCelMatchInput`] 2. Custom Matchers: 1. [Unified Matcher Extension: `CelMatcher`] +##### Unified Matcher: `Matcher` + +We will support the following fields in the +[`xds.type.matcher.v3.Matcher`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L22) +message: + +- `matcher_type`: One of the following must be present and valid: + - [`matcher_list`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L126): + A valid [Unified Matcher: `MatcherList`] message. + - [`matcher_tree`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L129): + A valid [Unified Matcher: `MatcherTree`] message. +- [`on_no_match`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L135): + Specifies the action executed if no match is found in the `matcher_list` or + `matcher_tree`. If set, must be a valid [Unified Matcher: `OnMatch`]. + If not set, refer to filter's default no-match behavior. + +##### Unified Matcher: `OnMatch` + +We will support the following fields in the +[`xds.type.matcher.v3.Matcher.OnMatch`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L24) +message: + +- `on_match`: One of the following must be present and valid: + - [`matcher`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L33): + A nested [Unified Matcher: `OnMatch`] for more complex, tree-like + matching logic. + - [`action`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L36): + A [`TypedExtensionConfig`] containing a protocol-specific action to + take. + +The following fields will be ignored by gRPC: + +- `keep_matching`: Not supported in the initial implementation, may be added + later. + +##### Unified Matcher: `MatcherList` + +We will support the following fields in the +[`xds.type.matcher.v3.Matcher.MatcherList`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L43) +message: + +- [`matchers`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L96): + A list of + [`Matcher.MatcherList.FieldMatcher`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L87) + messages. Must contain at least 1 item. If `min_items` is violated, or any + of the fields are invalid, the xDS resource will be NACKed. + - [`predicate`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L89): + Must be set and contain a valid + [`Matcher.MatcherList.Predicate`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L45) + message. + - `match_type`: One of the following must be present and valid: + - [`single_predicate`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L73): + A Unified Matcher: `MatcherList.Predicate.SinglePredicate` + message. + - [`or_matcher`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L76): + A + [`Matcher.MatcherList.Predicate.PredicateList`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L65) + message. + - [`predicate`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L66): + A list of `Matcher.MatcherList.Predicate` messages. Must + contain at least 2 items. Returns true if any of them are + true. + - [`and_matcher`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L79): + A + [`Matcher.MatcherList.Predicate.PredicateList`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L65) + message. + - [`predicate`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L66): + A list of `Matcher.MatcherList.Predicate` messages. Must + contain at least 2 items. Returns true if all of them are + true. + - [`not_matcher`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L82): + A nested `Matcher.MatcherList.Predicate` message. + - [`on_match`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L92): + Must be set and contain a valid [Unified Matcher: `OnMatch`] message. + +##### Unified Matcher: `MatcherList.Predicate.SinglePredicate` + +We will support the following fields in the [`SinglePredicate`](#unified-matcher-matcherlistpredicatesinglepredicate) +[`Matcher.MatcherList.Predicate.SinglePredicate`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L47) +message: + +- [`input`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L50): + A `core.v3.TypedExtensionConfig`. Must be present. If not present, the xDS + resource will be NACKed. +- `matcher`: Oneof field. Must be present. If not present, the xDS + resource will be NACKed. + - [`value_match`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L56): + A `type.matcher.v3.StringMatcher`. + - [`custom_match`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L60): + A `core.v3.TypedExtensionConfig`. + +##### Unified Matcher: `MatcherList.Predicate.PredicateList` + +We will support the following fields in the [`PredicateList`](#unified-matcher-matcherlistpredicatepredicatelist) +[`Matcher.MatcherList.Predicate.PredicateList`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L65) +message: + +- [`predicate`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L66): + A repeated list of `Predicate` messages. Must contain at least 2 items. + If `min_items` is violated, the xDS resource will be NACKed. + +##### Unified Matcher: `MatcherTree` + +We will support the following fields in the [`MatcherTree`](#unified-matcher-matchertree) +[`Matcher.MatcherTree`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L99) +message: + +- [`input`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L106): + A `core.v3.TypedExtensionConfig`. Must be present. If not present, the xDS + resource will be NACKed. +- `tree_type`: Oneof field. Must be present. If not present, the xDS + resource will be NACKed. + - [`exact_match_map`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L114): + A `MatchMap` message. + - [`prefix_match_map`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L117): + A `MatchMap` message. + - [`custom_match`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L120): + A `core.v3.TypedExtensionConfig`. + +##### Unified Matcher: `MatcherTree.MatchMap` + +We will support the following fields in the [`MatchMap`](#unified-matcher-matchertreematchmap) +[`Matcher.MatcherTree.MatchMap`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L101) +message: + +- [`map`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L102): + A map from string to `OnMatch` messages. Must contain at least 1 pair. + If `min_pairs` is violated, the xDS resource will be NACKed. + ##### Unified Matcher Extension: `HttpRequestHeaderMatchInput` * [`xds.type.matcher.v3.HttpAttributesCelMatchInput`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/http_inputs.proto#L22) From b1f1e70395b378536eccc0601a1d502d1218b5cd Mon Sep 17 00:00:00 2001 From: Sergii Tkachenko Date: Sun, 26 Oct 2025 22:34:13 -0700 Subject: [PATCH 40/53] MatcherList and MatcherTree --- A77-xds-rate-limiting-rlqs.md | 132 +++++++++++++++++----------------- 1 file changed, 64 insertions(+), 68 deletions(-) diff --git a/A77-xds-rate-limiting-rlqs.md b/A77-xds-rate-limiting-rlqs.md index 3c99ead9d..0bcc4de06 100644 --- a/A77-xds-rate-limiting-rlqs.md +++ b/A77-xds-rate-limiting-rlqs.md @@ -74,14 +74,11 @@ which are covered in the proposal: [Unified Matcher: `Matcher`]: #unified-matcher-matcher [Unified Matcher: `OnMatch`]: #unified-matcher-onmatch [Unified Matcher: `MatcherList`]: #unified-matcher-matcherlist -[Unified Matcher: `MatcherList.Predicate`]: #unified-matcher-matcherlistpredicate -[Unified Matcher: `MatcherList.Predicate.SinglePredicate`]: #unified-matcher-matcherlistpredicatesinglepredicate -[Unified Matcher: `MatcherList.Predicate.PredicateList`]: #unified-matcher-matcherlistpredicatepredicatelist [Unified Matcher: `MatcherTree`]: #unified-matcher-matchertree -[Unified Matcher: `MatcherTree.MatchMap`]: #unified-matcher-matchertreematchmap -[Unified Matcher Extension: `HttpRequestHeaderMatchInput`]: #unified-matcher-extension-httprequestheadermatchinput -[Unified Matcher Extension: `HttpAttributesCelMatchInput`]: #unified-matcher-extension-httpattributescelmatchinput -[Unified Matcher Extension: `CelMatcher`]: #unified-matcher-extension-celmatcher +[Unified Matcher: `HttpRequestHeaderMatchInput`]: #unified-matcher-httprequestheadermatchinput +[Unified Matcher: `HttpAttributesCelMatchInput`]: #unified-matcher-httpattributescelmatchinput +[Unified Matcher: `StringMatcher`]: #unified-matcher-stringmatcher +[Unified Matcher: `CelMatcher`]: #unified-matcher-celmatcher [On Data Plane RPC]: #on-data-plane-rpc [On RLQS Server Response]: #on-rlqs-server-response @@ -215,9 +212,9 @@ Unified Matcher restricted to the following protocol-specific types, packed as a [`TypedExtensionConfig`]: 1. Input specification (`input` fields): - [Unified Matcher Extension: `HttpAttributesCelMatchInput`]. + [Unified Matcher: `HttpAttributesCelMatchInput`]. 2. Custom matching logic (`custom_match` fields): - [Unified Matcher Extension: `CelMatcher`]. + [Unified Matcher: `CelMatcher`]. 3. Protocol-specific action (`OnMatch.action` field): [Config: `RateLimitQuotaBucketSettings`]. @@ -241,7 +238,7 @@ We will support the following fields: - `bucket_id_builder`: A map from string to `ValueBuilder`. - `ValueBuilder`: One of the following must be set: - `string_value`: A static string value. - - `custom_value`: A `TypedExtensionConfig` that resolves to a + - `custom_value`: A [`TypedExtensionConfig`] that resolves to a string. The supported extensions are the same as for matcher inputs. - `reporting_interval`: Must be present. The interval at which the client @@ -746,10 +743,11 @@ When implementing Unified Matcher API, a filter must define the following: In this iteration the following Unified Mather extensions will be supported: 1. Inputs: - 1. [Unified Matcher Extension: `HttpRequestHeaderMatchInput`] - 2. [Unified Matcher Extension: `HttpAttributesCelMatchInput`] -2. Custom Matchers: - 1. [Unified Matcher Extension: `CelMatcher`] + 1. [Unified Matcher: `HttpRequestHeaderMatchInput`] + 2. [Unified Matcher: `HttpAttributesCelMatchInput`] +2. Matchers: + 1. [Unified Matcher: `StringMatcher`] (standard matcher) + 1. [Unified Matcher: `CelMatcher`] ##### Unified Matcher: `Matcher` @@ -775,7 +773,7 @@ message: - `on_match`: One of the following must be present and valid: - [`matcher`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L33): - A nested [Unified Matcher: `OnMatch`] for more complex, tree-like + A nested [Unified Matcher: `Matcher`] for more complex, tree-like matching logic. - [`action`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L36): A [`TypedExtensionConfig`] containing a protocol-specific action to @@ -795,16 +793,30 @@ message: - [`matchers`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L96): A list of [`Matcher.MatcherList.FieldMatcher`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L87) - messages. Must contain at least 1 item. If `min_items` is violated, or any - of the fields are invalid, the xDS resource will be NACKed. + messages. Must contain at least 1 item. - [`predicate`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L89): Must be set and contain a valid [`Matcher.MatcherList.Predicate`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L45) message. - `match_type`: One of the following must be present and valid: - [`single_predicate`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L73): - A Unified Matcher: `MatcherList.Predicate.SinglePredicate` - message. + A + [`Matcher.MatcherList.Predicate.SinglePredicate`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L47) + message. The return type of the input must match the input type + of the matcher. + - [`input`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L50): + A valid [`TypedExtensionConfig`]. Must be present and + contain one of the input extensions supported by the filter. + Must have return type compatible with the `matcher`. + - `matcher`: One of the following must be present and valid: + - [`value_match`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L56): + A valid [Unified Matcher: `StringMatcher`]. Only + compatible with `input` that returns a string. + - [`custom_match`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L60): + A valid [`TypedExtensionConfig`] containing one of the + custom matcher extensions supported by the filter. Must + have input type compatible with the `input`. Must return + a boolean indicating the status of the match. - [`or_matcher`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L76): A [`Matcher.MatcherList.Predicate.PredicateList`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L65) @@ -822,79 +834,63 @@ message: contain at least 2 items. Returns true if all of them are true. - [`not_matcher`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L82): - A nested `Matcher.MatcherList.Predicate` message. + A nested `Matcher.MatcherList.Predicate` message. Returns the + inverted result of predicate evaluation. - [`on_match`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L92): Must be set and contain a valid [Unified Matcher: `OnMatch`] message. -##### Unified Matcher: `MatcherList.Predicate.SinglePredicate` - -We will support the following fields in the [`SinglePredicate`](#unified-matcher-matcherlistpredicatesinglepredicate) -[`Matcher.MatcherList.Predicate.SinglePredicate`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L47) -message: - -- [`input`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L50): - A `core.v3.TypedExtensionConfig`. Must be present. If not present, the xDS - resource will be NACKed. -- `matcher`: Oneof field. Must be present. If not present, the xDS - resource will be NACKed. - - [`value_match`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L56): - A `type.matcher.v3.StringMatcher`. - - [`custom_match`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L60): - A `core.v3.TypedExtensionConfig`. - -##### Unified Matcher: `MatcherList.Predicate.PredicateList` - -We will support the following fields in the [`PredicateList`](#unified-matcher-matcherlistpredicatepredicatelist) -[`Matcher.MatcherList.Predicate.PredicateList`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L65) -message: - -- [`predicate`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L66): - A repeated list of `Predicate` messages. Must contain at least 2 items. - If `min_items` is violated, the xDS resource will be NACKed. - ##### Unified Matcher: `MatcherTree` -We will support the following fields in the [`MatcherTree`](#unified-matcher-matchertree) -[`Matcher.MatcherTree`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L99) +We will support the following fields in the +[`xds.type.matcher.v3.Matcher.MatcherTree`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L99) message: - [`input`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L106): - A `core.v3.TypedExtensionConfig`. Must be present. If not present, the xDS - resource will be NACKed. -- `tree_type`: Oneof field. Must be present. If not present, the xDS - resource will be NACKed. + A valid [`TypedExtensionConfig`]. Must be present and contain one of the + input extensions supported by the filter. Must have return type compatible + with the `matcher`. +- `tree_type`: One of the following must be present and valid: - [`exact_match_map`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L114): - A `MatchMap` message. + A + [`Matcher.MatcherTree.MatchMap`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L101) + message. Only compatible with `input` that returns a string. + - [`map`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L102): + A map from string to `OnMatch` messages. Must contain at least 1 + pair. - [`prefix_match_map`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L117): - A `MatchMap` message. + A + [`Matcher.MatcherTree.MatchMap`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L101) + message. Only compatible with `input` that returns a string. + - [`map`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L102): + A map from string to `OnMatch` messages. Must contain at least 1 + pair. - [`custom_match`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L120): - A `core.v3.TypedExtensionConfig`. - -##### Unified Matcher: `MatcherTree.MatchMap` + A valid [`TypedExtensionConfig`] containing one of the custom matcher + extensions supported by the filter. Must have input type compatible with + the `input`. Must return a boolean indicating the status of the match. -We will support the following fields in the [`MatchMap`](#unified-matcher-matchertreematchmap) -[`Matcher.MatcherTree.MatchMap`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L101) -message: +##### Unified Matcher Inputs -- [`map`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L102): - A map from string to `OnMatch` messages. Must contain at least 1 pair. - If `min_pairs` is violated, the xDS resource will be NACKed. +###### Unified Matcher: `HttpRequestHeaderMatchInput` -##### Unified Matcher Extension: `HttpRequestHeaderMatchInput` - -* [`xds.type.matcher.v3.HttpAttributesCelMatchInput`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/http_inputs.proto#L22) * [`HttpRequestHeaderMatchInput`](https://www.envoyproxy.io/docs/envoy/latest/api-v3/type/matcher/v3/http_inputs.proto#type-matcher-v3-httprequestheadermatchinput) TODO(sergiitk): finish -##### Unified Matcher Extension: `HttpAttributesCelMatchInput` +###### Unified Matcher: `HttpAttributesCelMatchInput` * [`xds.type.matcher.v3.HttpAttributesCelMatchInput`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/http_inputs.proto#L22) * [`HttpAttributesCelMatchInput`](https://www.envoyproxy.io/docs/envoy/latest/xds/type/matcher/v3/http_inputs.proto#envoy-v3-api-msg-xds-type-matcher-v3-httpattributescelmatchinput) TODO(sergiitk): finish -##### Unified Matcher Extension: `CelMatcher` +##### Unified Matcher Matchers + +##### Unified Matcher: `StringMatcher` + +TODO(sergiitk): finish + +##### Unified Matcher: `CelMatcher` * [`xds.type.matcher.v3.CelMatcher`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/cel.proto#L30) * [`CelMatcher`](https://www.envoyproxy.io/docs/envoy/latest/xds/type/matcher/v3/cel.proto.html) From 24ce7614f8f7f2cb18d932438227548333e6b4d2 Mon Sep 17 00:00:00 2001 From: Sergii Tkachenko Date: Mon, 27 Oct 2025 08:53:02 -0700 Subject: [PATCH 41/53] unified matcher extensions --- A77-xds-rate-limiting-rlqs.md | 73 ++++++++++++++++++++++++++--------- 1 file changed, 55 insertions(+), 18 deletions(-) diff --git a/A77-xds-rate-limiting-rlqs.md b/A77-xds-rate-limiting-rlqs.md index 0bcc4de06..f5830ff7c 100644 --- a/A77-xds-rate-limiting-rlqs.md +++ b/A77-xds-rate-limiting-rlqs.md @@ -70,7 +70,9 @@ which are covered in the proposal: [RLQS xDS HTTP Filter: Channel Level]: #rlqs-xds-http-filter-channel-level [RLQS Buckets and Multithreading]: #rlqs-buckets-and-multithreading +[Unified Matcher API]: https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/advanced/matching/matching_api.html [Unified Matcher API Support]: #unified-matcher-api-support +[Unified Matcher Inputs]: #unified-matcher-inputs [Unified Matcher: `Matcher`]: #unified-matcher-matcher [Unified Matcher: `OnMatch`]: #unified-matcher-onmatch [Unified Matcher: `MatcherList`]: #unified-matcher-matcherlist @@ -80,14 +82,14 @@ which are covered in the proposal: [Unified Matcher: `StringMatcher`]: #unified-matcher-stringmatcher [Unified Matcher: `CelMatcher`]: #unified-matcher-celmatcher +[Envoy CEL environment]: https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/advanced/attributes +[Supported CEL Variables]: #supported-cel-variables + [On Data Plane RPC]: #on-data-plane-rpc [On RLQS Server Response]: #on-rlqs-server-response [On Report Timers]: #on-report-timers [On Sending Usage Reports]: #on-sending-usage-reports -[Unified Matcher API]: https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/advanced/matching/matching_api.html -[Envoy CEL environment]: https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/advanced/attributes - [TokenBucket]: https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/type/v3/token_bucket.proto [GrpcService.GoogleGrpc]: https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/config/core/v3/grpc_service.proto#L68 [`google.protobuf.Duration`]: https://developers.google.com/protocol-buffers/docs/reference/google.protobuf#google.protobuf.Duration @@ -855,15 +857,15 @@ message: [`Matcher.MatcherTree.MatchMap`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L101) message. Only compatible with `input` that returns a string. - [`map`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L102): - A map from string to `OnMatch` messages. Must contain at least 1 - pair. + A map from a string to a valid [Unified Matcher: `OnMatch`] message. + Must contain at least 1 pair. - [`prefix_match_map`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L117): A [`Matcher.MatcherTree.MatchMap`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L101) message. Only compatible with `input` that returns a string. - [`map`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L102): - A map from string to `OnMatch` messages. Must contain at least 1 - pair. + A map from a string to a valid [Unified Matcher: `OnMatch`] message. + Must contain at least 1 pair. - [`custom_match`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L120): A valid [`TypedExtensionConfig`] containing one of the custom matcher extensions supported by the filter. Must have input type compatible with @@ -873,34 +875,68 @@ message: ###### Unified Matcher: `HttpRequestHeaderMatchInput` -* [`HttpRequestHeaderMatchInput`](https://www.envoyproxy.io/docs/envoy/latest/api-v3/type/matcher/v3/http_inputs.proto#type-matcher-v3-httprequestheadermatchinput) +Returns a `string` containing the value of the header with name specified in +`header_name`. -TODO(sergiitk): finish +We will support the following fields in the +[`envoy.type.matcher.v3.HttpRequestHeaderMatchInput`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/type/matcher/v3/http_inputs.proto#L22) +message: + +- [`header_name`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L106): + Must be present. Value length must be in the range `[1, 16384)`. Must be a + valid HTTP/2 header name. ###### Unified Matcher: `HttpAttributesCelMatchInput` -* [`xds.type.matcher.v3.HttpAttributesCelMatchInput`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/http_inputs.proto#L22) -* [`HttpAttributesCelMatchInput`](https://www.envoyproxy.io/docs/envoy/latest/xds/type/matcher/v3/http_inputs.proto#envoy-v3-api-msg-xds-type-matcher-v3-httpattributescelmatchinput) +Returns a language-specific interface that allows to access request RPC metadata +as defined in [Supported CEL Variables]. -TODO(sergiitk): finish +We will support the following fields in the +[`xds.type.matcher.v3.HttpAttributesCelMatchInput`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/http_inputs.proto#L22) +message: + +- no fields. ##### Unified Matcher Matchers ##### Unified Matcher: `StringMatcher` -TODO(sergiitk): finish +Compatible with [Unified Matcher Inputs] that return a `string`. + +We will support the following fields in the +[`xds.type.matcher.v3.StringMatcher`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/string.proto#L19) +message: + +- `match_pattern`: One of the following must be present and valid: + - [`exact`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/string.proto#L28): + The input string must match exactly. An empty string is a valid value. + - [`prefix`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/string.proto#L36): + The input string must have this prefix. Must be non-empty. + - [`suffix`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/string.proto#L44): + The input string must have this suffix. Must be non-empty. + - [`contains`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/string.proto#L55): + The input string must contain this substring. Must be non-empty. +- [`ignore_case`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/string.proto#L65): + If `true`, the matching is case-insensitive. + +The following are not supported by gRPC in the initial implementation and will +result in xDS resource NACK: + +- `safe_regex` +- `custom` ##### Unified Matcher: `CelMatcher` -* [`xds.type.matcher.v3.CelMatcher`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/cel.proto#L30) -* [`CelMatcher`](https://www.envoyproxy.io/docs/envoy/latest/xds/type/matcher/v3/cel.proto.html) +Compatible with [Unified Matcher: `HttpAttributesCelMatchInput`]. + + TODO(sergiitk): finish #### CEL Integration We will support request metadata matching via CEL expressions. Only Canonical -CEL and only checked expressions will be supported (`cel.expr.CheckedExpr`). +CEL and only checked expressions will be supported [`cel.expr.CheckedExpr`]. CEL evaluation environment is a set of available variables and extension functions in a CEL program. We will match [Envoy CEL environment]. @@ -929,8 +965,9 @@ except comprehension-style macros. ##### Supported CEL Variables -For RLQS, only the `request` variable is supported in CEL expressions. We will -adapt [Envoy's Request Attributes](https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/advanced/attributes#request-attributes) +In the initial implementation only the `request` variable is supported in CEL +expressions. We will adapt +[Envoy's Request Attributes](https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/advanced/attributes#request-attributes) for gRPC. | Attribute | Type | gRPC source | Envoy Description | From c34ed83543b7bc9a2df4bebd81d8899b72b5ec2c Mon Sep 17 00:00:00 2001 From: Sergii Tkachenko Date: Mon, 27 Oct 2025 12:58:32 -0700 Subject: [PATCH 42/53] CEL Matcher --- A77-xds-rate-limiting-rlqs.md | 64 +++++++++++++++++++++++++++++++---- 1 file changed, 58 insertions(+), 6 deletions(-) diff --git a/A77-xds-rate-limiting-rlqs.md b/A77-xds-rate-limiting-rlqs.md index f5830ff7c..7243bf6ae 100644 --- a/A77-xds-rate-limiting-rlqs.md +++ b/A77-xds-rate-limiting-rlqs.md @@ -82,7 +82,9 @@ which are covered in the proposal: [Unified Matcher: `StringMatcher`]: #unified-matcher-stringmatcher [Unified Matcher: `CelMatcher`]: #unified-matcher-celmatcher -[Envoy CEL environment]: https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/advanced/attributes +[`cel.expr.CheckedExpr`]: https://github.com/google/cel-spec/blob/master/proto/cel/expr/checked.proto +[CEL Integration]: #cel-integration +[CEL Runtime Restrictions]: #cel-runtime-restrictions [Supported CEL Variables]: #supported-cel-variables [On Data Plane RPC]: #on-data-plane-rpc @@ -374,7 +376,7 @@ final class RlqsFilter implements Filter { } ``` -##### Future considerations +###### Future considerations This proposal uses the entire RLQS Filter Config to identify the corresponding unique Filter State. As a result, a Filter State will be recreated even if only @@ -929,9 +931,33 @@ result in xDS resource NACK: Compatible with [Unified Matcher: `HttpAttributesCelMatchInput`]. +Performs a match by evaluating a Common Expression Language (CEL) expression. +See [CEL Integration] for details. +We will support the following fields in the +[`xds.type.matcher.v3.CelMatcher`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/cel.proto#L30) +message: -TODO(sergiitk): finish +- [`expr_match`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/cel.proto#L32): + Must be present and contain a valid + [`xds.type.v3.CelExpression`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/v3/cel.proto#L26) + message. + - [`cel_expr_checked`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/v3/cel.proto#L49): + Must be present and contain a valid [`cel.expr.CheckedExpr`] message. + This message will be converted into a native CEL Abstract Syntax Tree + (AST) using the language-specific CEL library. The AST's output (return) + type must be boolean. The resulting CEL program must also be validated + to conform to [CEL Runtime Restrictions]. If any of these conversion or + validation steps fail, gRPC will NACK the xDS resource. +- [`description`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/cel.proto#L36): + An optional string. May be ignored or used for testing/debugging. + +The following fields will be ignored by gRPC: + +- `CelExpression.parsed_expr` - deprecated, only Canonical CEL is supported. +- `CelExpression.checked_expr` - deprecated, only Canonical CEL is supported. +- `CelExpression.cel_expr_parsed` - only Checked CEL expressions are + supported. #### CEL Integration @@ -939,7 +965,33 @@ We will support request metadata matching via CEL expressions. Only Canonical CEL and only checked expressions will be supported [`cel.expr.CheckedExpr`]. CEL evaluation environment is a set of available variables and extension -functions in a CEL program. We will match [Envoy CEL environment]. +functions in a CEL program. We will match +[Envoy CEL environment](https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/advanced/attributes) +and CEL interpreter configuration. + +##### CEL Runtime Restrictions + +Certain CEL features can lead to superlinear time complexity or memory +exhaustion. To ensure consistent behavior with Envoy and maintain security, +gRPC will configure the CEL runtime +[similar to Envoy](https://github.com/envoyproxy/envoy/blob/c57801c2afbe26dd6fad7f5ce764f267d07fbd04/source/extensions/filters/common/expr/evaluator.cc#L17-L23): + +```c +// Disables comprehension expressions, e.g. exists(), all(). +options.enable_comprehension = false; + +// Limits the maximum program size for RE2 regex to 100. +options.regex_max_program_size = 100; + +// Disables string() overloads. +options.enable_string_conversion = false; + +// Disables string concatenation overload. +options.enable_string_concat = false; + +// Disables list concatenation overload. +options.enable_list_concat = false; +``` ##### Supported CEL Functions @@ -994,7 +1046,7 @@ denies requests for all other method types. **2 `request.headers`**\ As defined in [A41], "header" field. -##### Implementation +###### CEL Variable Implementation Details For performance reasons, CEL variables should be resolved on demand. CEL Runtime provides the different variable resolving approaches based on the language: @@ -1003,7 +1055,7 @@ provides the different variable resolving approaches based on the language: * Go: [`Activation.ResolveName(string)`](https://github.com/google/cel-go/blob/3f12ecad39e2eb662bcd82b6391cfd0cb4cb1c5e/interpreter/activation.go#L30) * Java: [`CelVariableResolver`](https://javadoc.io/doc/dev.cel/runtime/0.6.0/dev/cel/runtime/CelVariableResolver.html) -### Temporary environment variable protection +### Temporary Environment Variable Protection During initial development, this feature will be enabled via the `GRPC_EXPERIMENTAL_XDS_ENABLE_RLQS` environment variable. This environment From d820d1518799b12d1896458a0fa1c0b318f9bceb Mon Sep 17 00:00:00 2001 From: Sergii Tkachenko Date: Mon, 27 Oct 2025 14:01:43 -0700 Subject: [PATCH 43/53] Unified Matcher: Filter Specifications --- A77-xds-rate-limiting-rlqs.md | 77 +++++++++++++++++++++-------------- 1 file changed, 46 insertions(+), 31 deletions(-) diff --git a/A77-xds-rate-limiting-rlqs.md b/A77-xds-rate-limiting-rlqs.md index 7243bf6ae..5661dd978 100644 --- a/A77-xds-rate-limiting-rlqs.md +++ b/A77-xds-rate-limiting-rlqs.md @@ -72,6 +72,7 @@ which are covered in the proposal: [Unified Matcher API]: https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/advanced/matching/matching_api.html [Unified Matcher API Support]: #unified-matcher-api-support +[Unified Matcher: Filter Specifications]: #unified-matcher-filter-specifications [Unified Matcher Inputs]: #unified-matcher-inputs [Unified Matcher: `Matcher`]: #unified-matcher-matcher [Unified Matcher: `OnMatch`]: #unified-matcher-onmatch @@ -211,26 +212,33 @@ RPCs are matched into buckets using the [Unified Matcher API] — an adaptable framework for xDS components requiring matching features. For details on general Unified Matcher proto parsing and validation, see [Unified Matcher API Support]. -For RLQS, the `bucket_matchers` field in the filter config will contain a -Unified Matcher restricted to the following protocol-specific types, packed as a -[`TypedExtensionConfig`]: - -1. Input specification (`input` fields): - [Unified Matcher: `HttpAttributesCelMatchInput`]. -2. Custom matching logic (`custom_match` fields): - [Unified Matcher: `CelMatcher`]. -3. Protocol-specific action (`OnMatch.action` field): - [Config: `RateLimitQuotaBucketSettings`]. +The `bucket_matchers` field in the filter config will contain a Unified Matcher +restricted to the protocol-specific types, packed as a [`TypedExtensionConfig`]. + +Evaluating the tree against RPC metadata yields +[Config: `RateLimitQuotaBucketSettings`], which contains the information needed +to associate the RPC with `bucket_id` and the default rate limiting +configuration. + +Filter-specific Unified Matcher configuration per +[Unified Matcher: Filter Specifications]: + +- Supported protocol-specific actions: + 1. [Config: `RateLimitQuotaBucketSettings`]. +- Supported input extensions: + 1. [Unified Matcher: `HttpRequestHeaderMatchInput`]. + 2. [Unified Matcher: `HttpAttributesCelMatchInput`]. +- Supported matching extensions, and any limitations on their inputs: + 1. [Unified Matcher: `StringMatcher`], input restricted to + `HttpRequestHeaderMatchInput`. + 1. [Unified Matcher: `CelMatcher`], input restricted to + `HttpAttributesCelMatchInput`. +- Filter-specific default no-match behavior: + - The RPC is allowed by default and not reported to the RLQS server. Any other types are considered invalid and will result in gRPC NACKing the xDS resource. -TODO(sergiitk): describe inputs/output? - -Evaluating the tree against RPC metadata yields `RateLimitQuotaBucketSettings`, -which contains the information needed to associate the RPC with `bucket_id` and -the default rate limiting configuration. - #### Config: `RateLimitQuotaBucketSettings` The `RateLimitQuotaBucketSettings` message configures the behavior of a bucket. @@ -737,13 +745,6 @@ which is the preferred version for all new APIs using Unified Matcher. If `envoy.config.common.matcher.v3.Matcher` is provided, we will interpret it as is `xds.type.matcher.v3.Matcher`. -When implementing Unified Matcher API, a filter must define the following: - -- Supported protocol-specific actions (see [Unified Matcher: `OnMatch`]). -- Supported input extensions. -- Supported custom matcher extensions, and any limitations on their inputs. -- Filter-specific default no-match behavior (f.e. xDS resource NACK). - In this iteration the following Unified Mather extensions will be supported: 1. Inputs: @@ -753,6 +754,15 @@ In this iteration the following Unified Mather extensions will be supported: 1. [Unified Matcher: `StringMatcher`] (standard matcher) 1. [Unified Matcher: `CelMatcher`] +#### Unified Matcher: Filter Specifications + +When implementing Unified Matcher API, a filter must define the following: + +- Supported protocol-specific actions (see [Unified Matcher: `OnMatch`]). +- Supported input extensions. +- Supported matching extensions, and any limitations on their inputs. +- Filter-specific default no-match behavior (f.e. xDS resource NACK). + ##### Unified Matcher: `Matcher` We will support the following fields in the @@ -810,7 +820,8 @@ message: of the matcher. - [`input`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L50): A valid [`TypedExtensionConfig`]. Must be present and - contain one of the input extensions supported by the filter. + contain one of the input extensions + [supported by the filter][Unified Matcher: Filter Specifications]. Must have return type compatible with the `matcher`. - `matcher`: One of the following must be present and valid: - [`value_match`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L56): @@ -818,9 +829,10 @@ message: compatible with `input` that returns a string. - [`custom_match`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L60): A valid [`TypedExtensionConfig`] containing one of the - custom matcher extensions supported by the filter. Must - have input type compatible with the `input`. Must return - a boolean indicating the status of the match. + custom matcher extensions + [supported by the filter][Unified Matcher: Filter Specifications]. + Must have input type compatible with the `input`. Must + return a boolean indicating the status of the match. - [`or_matcher`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L76): A [`Matcher.MatcherList.Predicate.PredicateList`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L65) @@ -851,8 +863,9 @@ message: - [`input`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L106): A valid [`TypedExtensionConfig`]. Must be present and contain one of the - input extensions supported by the filter. Must have return type compatible - with the `matcher`. + input extensions + [supported by the filter][Unified Matcher: Filter Specifications]. Must have + return type compatible with the `matcher`. - `tree_type`: One of the following must be present and valid: - [`exact_match_map`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L114): A @@ -870,8 +883,10 @@ message: Must contain at least 1 pair. - [`custom_match`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L120): A valid [`TypedExtensionConfig`] containing one of the custom matcher - extensions supported by the filter. Must have input type compatible with - the `input`. Must return a boolean indicating the status of the match. + extensions + [supported by the filter][Unified Matcher: Filter Specifications]. Must + have input type compatible with the `input`. Must return a boolean + indicating the status of the match. ##### Unified Matcher Inputs From be2fb021065a799fea1705449375c8cf5e688663 Mon Sep 17 00:00:00 2001 From: Sergii Tkachenko Date: Mon, 27 Oct 2025 15:38:43 -0700 Subject: [PATCH 44/53] RateLimitQuotaBucketSettings --- A77-xds-rate-limiting-rlqs.md | 149 +++++++++++++++++++++++----------- 1 file changed, 103 insertions(+), 46 deletions(-) diff --git a/A77-xds-rate-limiting-rlqs.md b/A77-xds-rate-limiting-rlqs.md index 5661dd978..10b53f9a9 100644 --- a/A77-xds-rate-limiting-rlqs.md +++ b/A77-xds-rate-limiting-rlqs.md @@ -62,10 +62,12 @@ which are covered in the proposal: [RateLimitQuotaFilterConfig]: #ratelimitquotafilterconfig [RateLimitQuotaOverride]: #ratelimitquotaoverride -[Config: `RuntimeFractionalPercent`]: #config-runtimefractionalpercent -[Config: `HeaderValueOption`]: #config-headervalueoption [Config: Bucket Matchers]: #config-bucket-matchers [Config: `RateLimitQuotaBucketSettings`]: #config-ratelimitquotabucketsettings +[Config: `RateLimitStrategy`]: #config-ratelimitstrategy +[Config: `TokenBucket`]: #config-tokenbucket +[Config: `HeaderValueOption`]: #config-headervalueoption +[Config: `RuntimeFractionalPercent`]: #config-runtimefractionalpercent [RLQS xDS HTTP Filter: Channel Level]: #rlqs-xds-http-filter-channel-level [RLQS Buckets and Multithreading]: #rlqs-buckets-and-multithreading @@ -95,9 +97,12 @@ which are covered in the proposal: [TokenBucket]: https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/type/v3/token_bucket.proto [GrpcService.GoogleGrpc]: https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/config/core/v3/grpc_service.proto#L68 -[`google.protobuf.Duration`]: https://developers.google.com/protocol-buffers/docs/reference/google.protobuf#google.protobuf.Duration [`TypedExtensionConfig`]: https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/core/v3/extension.proto#L14 +[`google.rpc.Status`]: https://github.com/googleapis/googleapis/blob/7a87bf05880470b360f42e2b7f9ff5b28fa6cbe0/google/rpc/status.proto +[gRPC Status Codes]: https://grpc.github.io/grpc/core/md_doc_statuscodes.html +[`google.protobuf.Duration`]: https://developers.google.com/protocol-buffers/docs/reference/google.protobuf#google.protobuf.Duration + [rlqs_proto]: https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/service/rate_limit_quota/v3/rlqs.proto ## Proposal @@ -206,6 +211,99 @@ The following fields will be ignored by gRPC: - `append`: Deprecated in favor of `append_action`. +#### Config: `RateLimitQuotaBucketSettings` + +We will support the following fields in the +[`RateLimitQuotaBucketSettings`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L169) +message: + +- [`bucket_id_builder`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L348): + If present, must be a valid `BucketIdBuilder` message. This is used to + construct a `BucketId` for each request, which may involve dynamic values + from request metadata. If not set, requests are not reported to the RLQS + server and are handled according to `no_assignment_behavior`. + - `bucket_id_builder`: A map from string to + [`ValueBuilder`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L265). + Must contain at least one entry. In addition to the specification, gRPC + will restrict the total number of key-value pairs to 30. + - `ValueBuilder` can be a static + [`string_value`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L272) + or a dynamic + [`custom_value`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L276). + For `custom_value`, we support [`TypedExtensionConfig`] containing + [Unified Matcher: `HttpRequestHeaderMatchInput`]. +- [`reporting_interval`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L398): + Must be present. A [`google.protobuf.Duration`] specifying the interval for + reporting quota usage. Must be greater than 100ms. Note that gRPC will apply + a best-effort approach to report at the configured interval, and may report + earlier or later in certain situations. See [On Report Timers] for details. +- [`deny_response_settings`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L407): + If not set, RPCs will be denied as specified in the default value of + `grpc_status`. If present, must be a valid + [`DenyResponseSettings`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L227) + message: + - `grpc_status`: A [`google.rpc.Status`] for + [gRPC responses][gRPC Status Codes]. Defaults to `UNAVAILABLE`. + - `response_headers_to_add`: A list of up to 10 + [Config: `HeaderValueOption`] to add to the deny response. +- [`no_assignment_behavior`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L413): + If not set, all requests are allowed. If set, must be a valid + [`NoAssignmentBehavior`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L172) + message. + - `fallback_rate_limit`: Must be present and valid + [Config: `RateLimitStrategy`] to apply. +- [`expired_assignment_behavior`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L422): + Specifies behavior when a quota assignment expires and cannot be refreshed. + If not set, the bucket is abandoned when its quota expires. + - [`expired_assignment_behavior_timeout`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L206): + A [`google.protobuf.Duration`] that limits how long this behavior is + applied. Must be a positive duration if set. If not set, it defaults to + zero and the bucket is abandoned immediately. + - One of the following must be set: + - [`fallback_rate_limit`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L216): + A [Config: `RateLimitStrategy`] to apply. + - [`reuse_last_assignment`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L222): + Reuses the rate limit strategy specified in the last known quota + assignment, effectively extending it by + `expired_assignment_behavior_timeout`. If a quota has never been + assigned to te bucket, the bucket is abandoned immediately. + +The following fields will be ignored by gRPC: + +- `DenyResponseSettings.http_status` +- `DenyResponseSettings.http_body` + +#### Config: `RateLimitStrategy` + +We will support the following fields in the +[`RateLimitStrategy`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/type/v3/ratelimit_strategy.proto#L20) +message: + +- `strategy`: One of the following must be present: + - [`blanket_rule`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/type/v3/ratelimit_strategy.proto#L31): + An enum that can be `ALLOW_ALL` or `DENY_ALL`. + - [`token_bucket`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/type/v3/ratelimit_strategy.proto#L37): + A [Config: `TokenBucket`] strategy. + +The following fields will be ignored by gRPC: + +- `requests_per_time_unit`: Deprecated in favor of `token_bucket`. + +#### Config: `TokenBucket` + +We will support the following fields in the +[`TokenBucket`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/type/v3/token_bucket.proto#L19) +message: + +- [`max_tokens`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/type/v3/token_bucket.proto#L24): + Must be present and greater than 0. The maximum number of tokens that the + bucket can hold. +- [`tokens_per_fill`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/type/v3/token_bucket.proto#L30): + The number of tokens added to the bucket during each fill interval. Must be + greater than 0. Defaults to 1. +- [`fill_interval`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/type/v3/token_bucket.proto#L36): + Must be present. The interval at which tokens are added to the bucket. + #### Config: Bucket Matchers RPCs are matched into buckets using the [Unified Matcher API] — an adaptable @@ -239,47 +337,6 @@ Filter-specific Unified Matcher configuration per Any other types are considered invalid and will result in gRPC NACKing the xDS resource. -#### Config: `RateLimitQuotaBucketSettings` - -The `RateLimitQuotaBucketSettings` message configures the behavior of a bucket. -We will support the following fields: - -- `bucket_id_builder`: Used to construct the `BucketId` for each request. If - not set, requests are not assigned to a bucket, not reported to the RLQS - server, and are handled by `no_assignment_behavior`. - - `bucket_id_builder`: A map from string to `ValueBuilder`. - - `ValueBuilder`: One of the following must be set: - - `string_value`: A static string value. - - `custom_value`: A [`TypedExtensionConfig`] that resolves to a - string. The supported extensions are the same as for matcher - inputs. -- `reporting_interval`: Must be present. The interval at which the client - reports usage to the RLQS server. Must be greater than 100ms. -- `deny_response_settings`: Customizes the response for denied requests. - - `http_status`: HTTP status code for denied HTTP requests. Defaults to - 1. gRPC requests are not affected. - - `http_body`: HTTP response body for denied HTTP requests. gRPC requests - are not affected. - - `grpc_status`: `google.rpc.Status` for denied gRPC requests. Defaults to - `UNAVAILABLE`. - - `response_headers_to_add`: A list of up to 10 headers to add to the deny - response, as described in [Config: `HeaderValueOption`]. -- `no_assignment_behavior`: Behavior before the first quota assignment is - received. If not set, all requests are allowed. - - `fallback_rate_limit`: A - [`RateLimitStrategy`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/type/v3/ratelimit_strategy.proto#L16) - to apply. - - `blanket_rule`: One of `ALLOW_ALL` or `DENY_ALL`. - - `requests_per_time_unit`: A token bucket-style rate limit. -- `expired_assignment_behavior`: Behavior when a quota assignment expires and - cannot be refreshed. If not set, the bucket is abandoned. - - `expired_assignment_behavior_timeout`: How long to apply the expired - assignment behavior before abandoning the bucket. Defaults to 0 (abandon - immediately). - - One of the following must be set: - - `fallback_rate_limit`: A `RateLimitStrategy` to apply. - - `reuse_last_assignment`: Reuse the last known quota assignment. - ### RLQS Components #### RLQS Components Overview @@ -829,7 +886,7 @@ message: compatible with `input` that returns a string. - [`custom_match`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L60): A valid [`TypedExtensionConfig`] containing one of the - custom matcher extensions + matching extensions [supported by the filter][Unified Matcher: Filter Specifications]. Must have input type compatible with the `input`. Must return a boolean indicating the status of the match. @@ -882,7 +939,7 @@ message: A map from a string to a valid [Unified Matcher: `OnMatch`] message. Must contain at least 1 pair. - [`custom_match`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L120): - A valid [`TypedExtensionConfig`] containing one of the custom matcher + A valid [`TypedExtensionConfig`] containing one of the matching extensions [supported by the filter][Unified Matcher: Filter Specifications]. Must have input type compatible with the `input`. Must return a boolean From e6ee1021b1357ae96bc40b12eba9961716d6380c Mon Sep 17 00:00:00 2001 From: Sergii Tkachenko Date: Mon, 27 Oct 2025 16:23:27 -0700 Subject: [PATCH 45/53] RateLimitStrategy and TokenBucket --- A77-xds-rate-limiting-rlqs.md | 69 +++++++++++++++++++---------------- 1 file changed, 38 insertions(+), 31 deletions(-) diff --git a/A77-xds-rate-limiting-rlqs.md b/A77-xds-rate-limiting-rlqs.md index 10b53f9a9..2e79df689 100644 --- a/A77-xds-rate-limiting-rlqs.md +++ b/A77-xds-rate-limiting-rlqs.md @@ -36,7 +36,7 @@ which are covered in the proposal: 1. xDS Control Plane will provide RLQS connection details in [the filter config][RateLimitQuotaFilterConfig] via GrpcService message as described in [A102]. -2. Quota assignments will be configured via [TokenBucket] message. +2. Quota assignments will be configured via [`TokenBucket`] message. 3. RPCs will be matched into buckets using [Unified Matcher API]. 4. One of the matching mechanisms will be [CEL](https://cel.dev/) (Common Expression Language). @@ -65,7 +65,6 @@ which are covered in the proposal: [Config: Bucket Matchers]: #config-bucket-matchers [Config: `RateLimitQuotaBucketSettings`]: #config-ratelimitquotabucketsettings [Config: `RateLimitStrategy`]: #config-ratelimitstrategy -[Config: `TokenBucket`]: #config-tokenbucket [Config: `HeaderValueOption`]: #config-headervalueoption [Config: `RuntimeFractionalPercent`]: #config-runtimefractionalpercent @@ -95,12 +94,11 @@ which are covered in the proposal: [On Report Timers]: #on-report-timers [On Sending Usage Reports]: #on-sending-usage-reports -[TokenBucket]: https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/type/v3/token_bucket.proto -[GrpcService.GoogleGrpc]: https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/config/core/v3/grpc_service.proto#L68 +[`TokenBucket`]: https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/type/v3/token_bucket.proto [`TypedExtensionConfig`]: https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/core/v3/extension.proto#L14 -[`google.rpc.Status`]: https://github.com/googleapis/googleapis/blob/7a87bf05880470b360f42e2b7f9ff5b28fa6cbe0/google/rpc/status.proto [gRPC Status Codes]: https://grpc.github.io/grpc/core/md_doc_statuscodes.html +[`google.rpc.Status`]: https://github.com/googleapis/googleapis/blob/7a87bf05880470b360f42e2b7f9ff5b28fa6cbe0/google/rpc/status.proto [`google.protobuf.Duration`]: https://developers.google.com/protocol-buffers/docs/reference/google.protobuf#google.protobuf.Duration [rlqs_proto]: https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/service/rate_limit_quota/v3/rlqs.proto @@ -231,7 +229,8 @@ message: or a dynamic [`custom_value`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L276). For `custom_value`, we support [`TypedExtensionConfig`] containing - [Unified Matcher: `HttpRequestHeaderMatchInput`]. + [Unified Matcher: `HttpRequestHeaderMatchInput`] in the initial + implementation. - [`reporting_interval`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L398): Must be present. A [`google.protobuf.Duration`] specifying the interval for reporting quota usage. Must be greater than 100ms. Note that gRPC will apply @@ -276,33 +275,40 @@ The following fields will be ignored by gRPC: #### Config: `RateLimitStrategy` We will support the following fields in the -[`RateLimitStrategy`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/type/v3/ratelimit_strategy.proto#L20) +[`RateLimitStrategy`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/type/v3/ratelimit_strategy.proto#L22) message: -- `strategy`: One of the following must be present: - - [`blanket_rule`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/type/v3/ratelimit_strategy.proto#L31): - An enum that can be `ALLOW_ALL` or `DENY_ALL`. +- `strategy`: One of the following must be present and valid: + - [`blanket_rule`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/type/v3/ratelimit_strategy.proto#L67): + An + [`BlanketRule`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/type/v3/ratelimit_strategy.proto#L24) + enum that can be `ALLOW_ALL` or `DENY_ALL`. Defaults to `ALLOW_ALL` when + the field is defined, but its value is not specified. + - [`requests_per_time_unit`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/type/v3/ratelimit_strategy.proto#L72): + This field allows to specify the rate limit without mandating a specific + algorithm. gRPC language implementation may choose an algorithm best + suitable to their language, or use a [`TokenBucket`] of an equivalent + rate. The rate limit parameters are specified in a + [`RequestsPerTimeUnit`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/type/v3/ratelimit_strategy.proto#L43) + message: + - [`requests_per_time_unit`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/type/v3/ratelimit_strategy.proto#L54): + Must be present. If set to `0`, all requests are denied. + - [`time_unit`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/type/v3/ratelimit_strategy.proto#L52): + Must be a valid + [`RateLimitUnit`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/type/v3/ratelimit_unit.proto#L16) + enum value. Ignored if `requests_per_time_unit` is `0`. - [`token_bucket`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/type/v3/ratelimit_strategy.proto#L37): - A [Config: `TokenBucket`] strategy. - -The following fields will be ignored by gRPC: - -- `requests_per_time_unit`: Deprecated in favor of `token_bucket`. - -#### Config: `TokenBucket` - -We will support the following fields in the -[`TokenBucket`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/type/v3/token_bucket.proto#L19) -message: - -- [`max_tokens`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/type/v3/token_bucket.proto#L24): - Must be present and greater than 0. The maximum number of tokens that the - bucket can hold. -- [`tokens_per_fill`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/type/v3/token_bucket.proto#L30): - The number of tokens added to the bucket during each fill interval. Must be - greater than 0. Defaults to 1. -- [`fill_interval`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/type/v3/token_bucket.proto#L36): - Must be present. The interval at which tokens are added to the bucket. + A [`TokenBucket`] message containing configuration for a + [Token Bucket](https://en.wikipedia.org/wiki/Token_bucket) rate-limiting + algorithm: + - [`max_tokens`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/type/v3/token_bucket.proto#L26): + Must be present and greater than 0. + - [`tokens_per_fill`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/type/v3/token_bucket.proto#L30): + Must be greater than 0. Defaults to 1. + - [`fill_interval`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/type/v3/token_bucket.proto#L35): + Must be present and contain a positive [`google.protobuf.Duration`]. + In addition to the specification, gRPC will reject values less than + 100ms second. #### Config: Bucket Matchers @@ -786,7 +792,8 @@ Usage reports are sent in following scenarios: #### Connecting to RLQS Control Plane -xDS Control Plane provides RLQS connection details in [GrpcService.GoogleGrpc] +xDS Control Plane provides RLQS connection details in +[`GrpcService.GoogleGrpc`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/config/core/v3/grpc_service.proto#L68) message as specified in [A102]. #### Unified Matcher API Support From e7f55e0ac2c7ed41cda277f47959b08f2eac5ade Mon Sep 17 00:00:00 2001 From: Sergii Tkachenko Date: Mon, 27 Oct 2025 16:43:18 -0700 Subject: [PATCH 46/53] Unified Matcher: Filter Integration rename --- A77-xds-rate-limiting-rlqs.md | 44 ++++++++++++++++++++--------------- 1 file changed, 25 insertions(+), 19 deletions(-) diff --git a/A77-xds-rate-limiting-rlqs.md b/A77-xds-rate-limiting-rlqs.md index 2e79df689..87a3cec40 100644 --- a/A77-xds-rate-limiting-rlqs.md +++ b/A77-xds-rate-limiting-rlqs.md @@ -73,8 +73,9 @@ which are covered in the proposal: [Unified Matcher API]: https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/advanced/matching/matching_api.html [Unified Matcher API Support]: #unified-matcher-api-support -[Unified Matcher: Filter Specifications]: #unified-matcher-filter-specifications -[Unified Matcher Inputs]: #unified-matcher-inputs +[Unified Matcher: Filter Integration]: #unified-matcher-filter-integration +[Unified Matcher: Input Extensions]: #unified-matcher-input-extensions +[Unified Matcher: Matching Extensions]: #unified-matcher-matching-extensions [Unified Matcher: `Matcher`]: #unified-matcher-matcher [Unified Matcher: `OnMatch`]: #unified-matcher-onmatch [Unified Matcher: `MatcherList`]: #unified-matcher-matcherlist @@ -325,17 +326,17 @@ to associate the RPC with `bucket_id` and the default rate limiting configuration. Filter-specific Unified Matcher configuration per -[Unified Matcher: Filter Specifications]: +[Unified Matcher: Filter Integration]: - Supported protocol-specific actions: 1. [Config: `RateLimitQuotaBucketSettings`]. -- Supported input extensions: +- Supported [Unified Matcher: Input Extensions]: 1. [Unified Matcher: `HttpRequestHeaderMatchInput`]. 2. [Unified Matcher: `HttpAttributesCelMatchInput`]. -- Supported matching extensions, and any limitations on their inputs: - 1. [Unified Matcher: `StringMatcher`], input restricted to +- Supported [Unified Matcher: Matching Extensions]. + 1. [Unified Matcher: `StringMatcher`], input effectively restricted to `HttpRequestHeaderMatchInput`. - 1. [Unified Matcher: `CelMatcher`], input restricted to + 1. [Unified Matcher: `CelMatcher`], input effectively restricted to `HttpAttributesCelMatchInput`. - Filter-specific default no-match behavior: - The RPC is allowed by default and not reported to the RLQS server. @@ -350,6 +351,7 @@ resource. The diagram below shows the conceptual components of the RLQS Filter. Note that the actual implementation may vary depending on the language. + ```mermaid --- config: @@ -818,17 +820,21 @@ In this iteration the following Unified Mather extensions will be supported: 1. [Unified Matcher: `StringMatcher`] (standard matcher) 1. [Unified Matcher: `CelMatcher`] -#### Unified Matcher: Filter Specifications +#### Unified Matcher: Filter Integration When implementing Unified Matcher API, a filter must define the following: - Supported protocol-specific actions (see [Unified Matcher: `OnMatch`]). -- Supported input extensions. -- Supported matching extensions, and any limitations on their inputs. +- Supported [Unified Matcher: Input Extensions]. +- Supported [Unified Matcher: Matching Extensions], including any additional + limitations on their inputs. - Filter-specific default no-match behavior (f.e. xDS resource NACK). ##### Unified Matcher: `Matcher` +Unified Matcher API allows to build matcher trees of unrestricted depth. gRPC +will reject any matcher definition with + We will support the following fields in the [`xds.type.matcher.v3.Matcher`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L22) message: @@ -885,7 +891,7 @@ message: - [`input`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L50): A valid [`TypedExtensionConfig`]. Must be present and contain one of the input extensions - [supported by the filter][Unified Matcher: Filter Specifications]. + [supported by the filter][Unified Matcher: Filter Integration]. Must have return type compatible with the `matcher`. - `matcher`: One of the following must be present and valid: - [`value_match`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L56): @@ -894,7 +900,7 @@ message: - [`custom_match`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L60): A valid [`TypedExtensionConfig`] containing one of the matching extensions - [supported by the filter][Unified Matcher: Filter Specifications]. + [supported by the filter][Unified Matcher: Filter Integration]. Must have input type compatible with the `input`. Must return a boolean indicating the status of the match. - [`or_matcher`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L76): @@ -928,7 +934,7 @@ message: - [`input`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L106): A valid [`TypedExtensionConfig`]. Must be present and contain one of the input extensions - [supported by the filter][Unified Matcher: Filter Specifications]. Must have + [supported by the filter][Unified Matcher: Filter Integration]. Must have return type compatible with the `matcher`. - `tree_type`: One of the following must be present and valid: - [`exact_match_map`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L114): @@ -948,11 +954,11 @@ message: - [`custom_match`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L120): A valid [`TypedExtensionConfig`] containing one of the matching extensions - [supported by the filter][Unified Matcher: Filter Specifications]. Must + [supported by the filter][Unified Matcher: Filter Integration]. Must have input type compatible with the `input`. Must return a boolean indicating the status of the match. -##### Unified Matcher Inputs +##### Unified Matcher: Input Extensions ###### Unified Matcher: `HttpRequestHeaderMatchInput` @@ -978,11 +984,11 @@ message: - no fields. -##### Unified Matcher Matchers +##### Unified Matcher: Matching Extensions -##### Unified Matcher: `StringMatcher` +###### Unified Matcher: `StringMatcher` -Compatible with [Unified Matcher Inputs] that return a `string`. +Compatible with [Unified Matcher: Input Extensions] that return a `string`. We will support the following fields in the [`xds.type.matcher.v3.StringMatcher`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/string.proto#L19) @@ -1006,7 +1012,7 @@ result in xDS resource NACK: - `safe_regex` - `custom` -##### Unified Matcher: `CelMatcher` +###### Unified Matcher: `CelMatcher` Compatible with [Unified Matcher: `HttpAttributesCelMatchInput`]. From c08a9d98a7d77fd91c855fd3c9853c167378bc4d Mon Sep 17 00:00:00 2001 From: Sergii Tkachenko Date: Mon, 27 Oct 2025 16:46:27 -0700 Subject: [PATCH 47/53] reorder filter config sections --- A77-xds-rate-limiting-rlqs.md | 157 +++++++++++++++++----------------- 1 file changed, 78 insertions(+), 79 deletions(-) diff --git a/A77-xds-rate-limiting-rlqs.md b/A77-xds-rate-limiting-rlqs.md index 87a3cec40..78b706bb8 100644 --- a/A77-xds-rate-limiting-rlqs.md +++ b/A77-xds-rate-limiting-rlqs.md @@ -151,64 +151,38 @@ message: [Config: Bucket Matchers], and fully overrides the matchers provided on the less specific definition. -#### Config: `RuntimeFractionalPercent` - -We will support the following fields in the -[RuntimeFractionalPercent](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/config/core/v3/base.proto#L643) -proto: - -- [`default_value`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/config/core/v3/base.proto#L648): - This field must be present. If the denominator specified is less than the - numerator, the final fractional percentage is capped at 1 (100%). The - fraction specified with: - - [`numerator`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/type/v3/percent.proto#L52): - Non-negative integer, 0 by default. - - [`denominator`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/type/v3/percent.proto#L56): - Must be one of the - [`DenominatorType`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/type/v3/percent.proto#L34) - values: - - `HUNDRED` (default) - - `TEN_THOUSAND` - - `MILLION` -- All other fields are ignored. - -The following fields will be ignored by gRPC: +#### Config: Bucket Matchers -- `runtime_key`: gRPC does not have Envoy's concept of runtime settings. +RPCs are matched into buckets using the [Unified Matcher API] — an adaptable +framework for xDS components requiring matching features. For details on general +Unified Matcher proto parsing and validation, see [Unified Matcher API Support]. -#### Config: `HeaderValueOption` +The `bucket_matchers` field in the filter config will contain a Unified Matcher +restricted to the protocol-specific types, packed as a [`TypedExtensionConfig`]. -We will support the following fields in the -[`HeaderValueOption`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/config/core/v3/base.proto#L429) -proto: +Evaluating the tree against RPC metadata yields +[Config: `RateLimitQuotaBucketSettings`], which contains the information needed +to associate the RPC with `bucket_id` and the default rate limiting +configuration. -- [`header`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/config/core/v3/base.proto#L458): - Must be present. - - [`key`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/config/core/v3/base.proto#L404): - Value length must be in the range `[1, 16384)`. Must be a valid HTTP/2 - header name. - - [`value`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/config/core/v3/base.proto#L415): - Specifies the header value. Must be shorter than 16384 bytes. Must be a - valid HTTP/2 header value. Not used if `key` ends in `-bin` and - `raw_value` is set. - - [`raw_value`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/config/core/v3/base.proto#L422): - Used only if `key` ends in `-bin`. Must be shorter than 16384 bytes. - Will be base64-encoded on the wire, unless the pure binary metadata - extension from [gRFC G1: True Binary Metadata][G1] is used. -- [`append_action`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/config/core/v3/base.proto#L476): - Must be of the - [`HeaderAppendAction`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/config/core/v3/base.proto#L434) - values: - - `APPEND_IF_EXISTS_OR_ADD` (default) - - `ADD_IF_ABSENT` - - `OVERWRITE_IF_EXISTS_OR_ADD` - - `OVERWRITE_IF_EXISTS` -- [`keep_empty_value`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/config/core/v3/base.proto#L480) -- All other fields are ignored. +Filter-specific Unified Matcher configuration per +[Unified Matcher: Filter Integration]: -The following fields will be ignored by gRPC: +- Supported protocol-specific actions: + 1. [Config: `RateLimitQuotaBucketSettings`]. +- Supported [Unified Matcher: Input Extensions]: + 1. [Unified Matcher: `HttpRequestHeaderMatchInput`]. + 2. [Unified Matcher: `HttpAttributesCelMatchInput`]. +- Supported [Unified Matcher: Matching Extensions]. + 1. [Unified Matcher: `StringMatcher`], input effectively restricted to + `HttpRequestHeaderMatchInput`. + 1. [Unified Matcher: `CelMatcher`], input effectively restricted to + `HttpAttributesCelMatchInput`. +- Filter-specific default no-match behavior: + - The RPC is allowed by default and not reported to the RLQS server. -- `append`: Deprecated in favor of `append_action`. +Any other types are considered invalid and will result in gRPC NACKing the xDS +resource. #### Config: `RateLimitQuotaBucketSettings` @@ -311,38 +285,64 @@ message: In addition to the specification, gRPC will reject values less than 100ms second. -#### Config: Bucket Matchers +#### Config: `HeaderValueOption` -RPCs are matched into buckets using the [Unified Matcher API] — an adaptable -framework for xDS components requiring matching features. For details on general -Unified Matcher proto parsing and validation, see [Unified Matcher API Support]. +We will support the following fields in the +[`HeaderValueOption`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/config/core/v3/base.proto#L429) +proto: -The `bucket_matchers` field in the filter config will contain a Unified Matcher -restricted to the protocol-specific types, packed as a [`TypedExtensionConfig`]. +- [`header`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/config/core/v3/base.proto#L458): + Must be present. + - [`key`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/config/core/v3/base.proto#L404): + Value length must be in the range `[1, 16384)`. Must be a valid HTTP/2 + header name. + - [`value`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/config/core/v3/base.proto#L415): + Specifies the header value. Must be shorter than 16384 bytes. Must be a + valid HTTP/2 header value. Not used if `key` ends in `-bin` and + `raw_value` is set. + - [`raw_value`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/config/core/v3/base.proto#L422): + Used only if `key` ends in `-bin`. Must be shorter than 16384 bytes. + Will be base64-encoded on the wire, unless the pure binary metadata + extension from [gRFC G1: True Binary Metadata][G1] is used. +- [`append_action`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/config/core/v3/base.proto#L476): + Must be of the + [`HeaderAppendAction`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/config/core/v3/base.proto#L434) + values: + - `APPEND_IF_EXISTS_OR_ADD` (default) + - `ADD_IF_ABSENT` + - `OVERWRITE_IF_EXISTS_OR_ADD` + - `OVERWRITE_IF_EXISTS` +- [`keep_empty_value`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/config/core/v3/base.proto#L480) +- All other fields are ignored. -Evaluating the tree against RPC metadata yields -[Config: `RateLimitQuotaBucketSettings`], which contains the information needed -to associate the RPC with `bucket_id` and the default rate limiting -configuration. +The following fields will be ignored by gRPC: -Filter-specific Unified Matcher configuration per -[Unified Matcher: Filter Integration]: +- `append`: Deprecated in favor of `append_action`. -- Supported protocol-specific actions: - 1. [Config: `RateLimitQuotaBucketSettings`]. -- Supported [Unified Matcher: Input Extensions]: - 1. [Unified Matcher: `HttpRequestHeaderMatchInput`]. - 2. [Unified Matcher: `HttpAttributesCelMatchInput`]. -- Supported [Unified Matcher: Matching Extensions]. - 1. [Unified Matcher: `StringMatcher`], input effectively restricted to - `HttpRequestHeaderMatchInput`. - 1. [Unified Matcher: `CelMatcher`], input effectively restricted to - `HttpAttributesCelMatchInput`. -- Filter-specific default no-match behavior: - - The RPC is allowed by default and not reported to the RLQS server. +#### Config: `RuntimeFractionalPercent` -Any other types are considered invalid and will result in gRPC NACKing the xDS -resource. +We will support the following fields in the +[RuntimeFractionalPercent](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/config/core/v3/base.proto#L643) +proto: + +- [`default_value`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/config/core/v3/base.proto#L648): + This field must be present. If the denominator specified is less than the + numerator, the final fractional percentage is capped at 1 (100%). The + fraction specified with: + - [`numerator`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/type/v3/percent.proto#L52): + Non-negative integer, 0 by default. + - [`denominator`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/type/v3/percent.proto#L56): + Must be one of the + [`DenominatorType`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/type/v3/percent.proto#L34) + values: + - `HUNDRED` (default) + - `TEN_THOUSAND` + - `MILLION` +- All other fields are ignored. + +The following fields will be ignored by gRPC: + +- `runtime_key`: gRPC does not have Envoy's concept of runtime settings. ### RLQS Components @@ -351,7 +351,6 @@ resource. The diagram below shows the conceptual components of the RLQS Filter. Note that the actual implementation may vary depending on the language. - ```mermaid --- config: From 3f91227ecd85261a992bf8c3b86df77b54407ec2 Mon Sep 17 00:00:00 2001 From: Sergii Tkachenko Date: Mon, 27 Oct 2025 16:48:39 -0700 Subject: [PATCH 48/53] proofreading --- A77-xds-rate-limiting-rlqs.md | 21 ++++++++++++--------- 1 file changed, 12 insertions(+), 9 deletions(-) diff --git a/A77-xds-rate-limiting-rlqs.md b/A77-xds-rate-limiting-rlqs.md index 78b706bb8..760afd02e 100644 --- a/A77-xds-rate-limiting-rlqs.md +++ b/A77-xds-rate-limiting-rlqs.md @@ -70,6 +70,7 @@ which are covered in the proposal: [RLQS xDS HTTP Filter: Channel Level]: #rlqs-xds-http-filter-channel-level [RLQS Buckets and Multithreading]: #rlqs-buckets-and-multithreading +[Connecting to RLQS Control Plane]: #connecting-to-rlqs-control-plane [Unified Matcher API]: https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/advanced/matching/matching_api.html [Unified Matcher API Support]: #unified-matcher-api-support @@ -119,7 +120,7 @@ message: - [`rlqs_server`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L39): This field must be present. Inside of it, GrpcService as described in - [A102]. + [Connecting to RLQS Control Plane] and [A102]. - [`domain`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L44): This field must be present and non-empty. - [`bucket_matchers`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L115): @@ -145,17 +146,19 @@ message: - [`domain`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L44): If non-empty, overrides the domain value provided on the less specific - definition. + definition. This value overrides the `domain` specified in the channel-level + filter configuration. - [`bucket_matchers`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L115): - If present, must be a valid matchers structure as described in - [Config: Bucket Matchers], and fully overrides the matchers provided on the - less specific definition. + If present, this field must contain a valid matchers structure as described + in [Config: Bucket Matchers]. It fully overrides the `bucket_matchers` + provided of the less specific definition. #### Config: Bucket Matchers RPCs are matched into buckets using the [Unified Matcher API] — an adaptable framework for xDS components requiring matching features. For details on general -Unified Matcher proto parsing and validation, see [Unified Matcher API Support]. +parsing and validation of the Unified Matcher, see +[Unified Matcher API Support]. The `bucket_matchers` field in the filter config will contain a Unified Matcher restricted to the protocol-specific types, packed as a [`TypedExtensionConfig`]. @@ -203,9 +206,9 @@ message: [`string_value`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L272) or a dynamic [`custom_value`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L276). - For `custom_value`, we support [`TypedExtensionConfig`] containing - [Unified Matcher: `HttpRequestHeaderMatchInput`] in the initial - implementation. + For `custom_value`, the initial implementation will support a + [`TypedExtensionConfig`] containing a + [Unified Matcher: `HttpRequestHeaderMatchInput`]. - [`reporting_interval`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L398): Must be present. A [`google.protobuf.Duration`] specifying the interval for reporting quota usage. Must be greater than 100ms. Note that gRPC will apply From 15fb708933007b25a6efe005a0275d62e259afd8 Mon Sep 17 00:00:00 2001 From: Sergii Tkachenko Date: Mon, 27 Oct 2025 17:22:35 -0700 Subject: [PATCH 49/53] add todos --- A77-xds-rate-limiting-rlqs.md | 22 ++++++++++++++-------- 1 file changed, 14 insertions(+), 8 deletions(-) diff --git a/A77-xds-rate-limiting-rlqs.md b/A77-xds-rate-limiting-rlqs.md index 760afd02e..af8cf1715 100644 --- a/A77-xds-rate-limiting-rlqs.md +++ b/A77-xds-rate-limiting-rlqs.md @@ -522,12 +522,8 @@ RLQS Filter State object will include the following data members: - RLQS Client: Accessed when we get the first data plane RPC for a given bucket and when a report timer fires. Notifies RLQS Filter State of responses received from the RLQS server. -- Report Timers Map: Initialized at instantiation. Used to track discovered - reporting intervals and their execution handlers. - - Entries inserted we get the first data plane RPC for a given bucket and - there's no key for bucket reporting interval. - - Entries deleted when there's no more buckets with given reporting - interval in RLQS Bucket Map. +- Report Timers: Initialized at instantiation. Used to schedule periodic + bucket usage reports to the RLQS server. See [On Report Timers] for details. Pseudo-code for RLQS Filter State RPC rate limiting: @@ -689,6 +685,8 @@ counter. #### On RLQS Server Response +TODO(sergiitk): Add detailed response parsing and validation + When receiving an RLQS Server Response, the RLQS Client passes parsed response to the RLQS Filter State. The RLQS Filter State iterates through the bucket assignments in the response, and updates the corresponding buckets in the RLQS @@ -705,6 +703,13 @@ Buckets marked to be abandoned are purged from the cache as described in #### On Report Timers +- TODO(sergiitk): implementation-focused description so that other impl + approaches allowed +- TODO(sergiitk): add details on scalability requirements +- TODO(sergiitk): Are there requirements in terms of when exactly we + reschedule the timer, and how that interacts with flow control on the RLQS + stream? + When a report timer fires, the RLQS Filter State retrieves all buckets with the corresponding reporting interval from the RLQS Bucket Map. For each bucket, the filter snapshots the current usage counters, and resets them as described in @@ -834,8 +839,9 @@ When implementing Unified Matcher API, a filter must define the following: ##### Unified Matcher: `Matcher` -Unified Matcher API allows to build matcher trees of unrestricted depth. gRPC -will reject any matcher definition with +While the Unified Matcher API allows for matcher trees of arbitrary depth, gRPC +will reject any matcher definition with a tree depth greater than 100, NACKing +the xDS resource. We will support the following fields in the [`xds.type.matcher.v3.Matcher`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L22) From 3375cf45d87a1b11afcf3a1cf115adde8da5e6df Mon Sep 17 00:00:00 2001 From: Sergii Tkachenko Date: Mon, 13 Oct 2025 14:01:30 -0700 Subject: [PATCH 50/53] Add filter config parsing, including Unified Matcher API --- A77-xds-rate-limiting-rlqs.md | 700 +++++++++++++++++++++++++++++----- 1 file changed, 601 insertions(+), 99 deletions(-) diff --git a/A77-xds-rate-limiting-rlqs.md b/A77-xds-rate-limiting-rlqs.md index 36ac08090..af8cf1715 100644 --- a/A77-xds-rate-limiting-rlqs.md +++ b/A77-xds-rate-limiting-rlqs.md @@ -36,7 +36,7 @@ which are covered in the proposal: 1. xDS Control Plane will provide RLQS connection details in [the filter config][RateLimitQuotaFilterConfig] via GrpcService message as described in [A102]. -2. Quota assignments will be configured via [TokenBucket] message. +2. Quota assignments will be configured via [`TokenBucket`] message. 3. RPCs will be matched into buckets using [Unified Matcher API]. 4. One of the matching mechanisms will be [CEL](https://cel.dev/) (Common Expression Language). @@ -62,16 +62,46 @@ which are covered in the proposal: [RateLimitQuotaFilterConfig]: #ratelimitquotafilterconfig [RateLimitQuotaOverride]: #ratelimitquotaoverride -[Config: RuntimeFractionalPercent]: #config-runtimefractionalpercent -[Config: HeaderValueOption]: #config-headervalueoption [Config: Bucket Matchers]: #config-bucket-matchers +[Config: `RateLimitQuotaBucketSettings`]: #config-ratelimitquotabucketsettings +[Config: `RateLimitStrategy`]: #config-ratelimitstrategy +[Config: `HeaderValueOption`]: #config-headervalueoption +[Config: `RuntimeFractionalPercent`]: #config-runtimefractionalpercent + [RLQS xDS HTTP Filter: Channel Level]: #rlqs-xds-http-filter-channel-level +[RLQS Buckets and Multithreading]: #rlqs-buckets-and-multithreading +[Connecting to RLQS Control Plane]: #connecting-to-rlqs-control-plane [Unified Matcher API]: https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/advanced/matching/matching_api.html -[Envoy CEL environment]: https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/advanced/attributes - -[TokenBucket]: https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/type/v3/token_bucket.proto -[GrpcService.GoogleGrpc]: https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/config/core/v3/grpc_service.proto#L68 +[Unified Matcher API Support]: #unified-matcher-api-support +[Unified Matcher: Filter Integration]: #unified-matcher-filter-integration +[Unified Matcher: Input Extensions]: #unified-matcher-input-extensions +[Unified Matcher: Matching Extensions]: #unified-matcher-matching-extensions +[Unified Matcher: `Matcher`]: #unified-matcher-matcher +[Unified Matcher: `OnMatch`]: #unified-matcher-onmatch +[Unified Matcher: `MatcherList`]: #unified-matcher-matcherlist +[Unified Matcher: `MatcherTree`]: #unified-matcher-matchertree +[Unified Matcher: `HttpRequestHeaderMatchInput`]: #unified-matcher-httprequestheadermatchinput +[Unified Matcher: `HttpAttributesCelMatchInput`]: #unified-matcher-httpattributescelmatchinput +[Unified Matcher: `StringMatcher`]: #unified-matcher-stringmatcher +[Unified Matcher: `CelMatcher`]: #unified-matcher-celmatcher + +[`cel.expr.CheckedExpr`]: https://github.com/google/cel-spec/blob/master/proto/cel/expr/checked.proto +[CEL Integration]: #cel-integration +[CEL Runtime Restrictions]: #cel-runtime-restrictions +[Supported CEL Variables]: #supported-cel-variables + +[On Data Plane RPC]: #on-data-plane-rpc +[On RLQS Server Response]: #on-rlqs-server-response +[On Report Timers]: #on-report-timers +[On Sending Usage Reports]: #on-sending-usage-reports + +[`TokenBucket`]: https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/type/v3/token_bucket.proto +[`TypedExtensionConfig`]: https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/core/v3/extension.proto#L14 + +[gRPC Status Codes]: https://grpc.github.io/grpc/core/md_doc_statuscodes.html +[`google.rpc.Status`]: https://github.com/googleapis/googleapis/blob/7a87bf05880470b360f42e2b7f9ff5b28fa6cbe0/google/rpc/status.proto +[`google.protobuf.Duration`]: https://developers.google.com/protocol-buffers/docs/reference/google.protobuf#google.protobuf.Duration [rlqs_proto]: https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/service/rate_limit_quota/v3/rlqs.proto @@ -82,108 +112,240 @@ which are covered in the proposal: The RLQS Filter API is defined in [rate_limit_quota.proto](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto). -#### RateLimitQuotaFilterConfig +#### `RateLimitQuotaFilterConfig` We will support the following fields in the -[RateLimitQuotaFilterConfig](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L37) +[`RateLimitQuotaFilterConfig`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L37) message: -- [rlqs_server](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L39): +- [`rlqs_server`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L39): This field must be present. Inside of it, GrpcService as described in - [A102]. -- [domain](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L44): + [Connecting to RLQS Control Plane] and [A102]. +- [`domain`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L44): This field must be present and non-empty. -- [bucket_matchers](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L115): +- [`bucket_matchers`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L115): This field must be present. Inside of it, there must be a valid matchers structure as described in [Config: Bucket Matchers]. -- [filter_enabled](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L121): +- [`filter_enabled`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L121): Specifies the fraction of requests for which the rate limiting is enabled. When not present, the filter is enabled for all requests (100%). Otherwise, - the fraction is determined from [Config: RuntimeFractionalPercent]. -- [filter_enforced](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L132): + the fraction is determined from [Config: `RuntimeFractionalPercent`]. +- [`filter_enforced`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L132): Specifies the fraction of requests for which the rate limiting is enforced. When not present, the filter is enforced for all requests (100%). Otherwise, - the fraction is determined from [Config: RuntimeFractionalPercent]. -- [request_headers_to_add_when_not_enforced](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L137): - If present, must be a list of [Config: HeaderValueOption]. Must not contain - more than 10 items. + the fraction is determined from [Config: `RuntimeFractionalPercent`]. +- [`request_headers_to_add_when_not_enforced`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L137): + If present, must be a list of [Config: `HeaderValueOption`]. Must not + contain more than 10 items. -#### RateLimitQuotaOverride +#### `RateLimitQuotaOverride` We will support the following fields in the -[RateLimitQuotaFilterConfig](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L143) +[`RateLimitQuotaOverride`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L143) message: -- [domain](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L44): +- [`domain`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L44): If non-empty, overrides the domain value provided on the less specific - definition. -- [bucket_matchers](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L115): - If present, must be a valid matchers structure as described in - [Config: Bucket Matchers], and fully overrides the matchers provided on the - less specific definition. + definition. This value overrides the `domain` specified in the channel-level + filter configuration. +- [`bucket_matchers`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L115): + If present, this field must contain a valid matchers structure as described + in [Config: Bucket Matchers]. It fully overrides the `bucket_matchers` + provided of the less specific definition. + +#### Config: Bucket Matchers + +RPCs are matched into buckets using the [Unified Matcher API] — an adaptable +framework for xDS components requiring matching features. For details on general +parsing and validation of the Unified Matcher, see +[Unified Matcher API Support]. + +The `bucket_matchers` field in the filter config will contain a Unified Matcher +restricted to the protocol-specific types, packed as a [`TypedExtensionConfig`]. + +Evaluating the tree against RPC metadata yields +[Config: `RateLimitQuotaBucketSettings`], which contains the information needed +to associate the RPC with `bucket_id` and the default rate limiting +configuration. + +Filter-specific Unified Matcher configuration per +[Unified Matcher: Filter Integration]: -#### Config: RuntimeFractionalPercent +- Supported protocol-specific actions: + 1. [Config: `RateLimitQuotaBucketSettings`]. +- Supported [Unified Matcher: Input Extensions]: + 1. [Unified Matcher: `HttpRequestHeaderMatchInput`]. + 2. [Unified Matcher: `HttpAttributesCelMatchInput`]. +- Supported [Unified Matcher: Matching Extensions]. + 1. [Unified Matcher: `StringMatcher`], input effectively restricted to + `HttpRequestHeaderMatchInput`. + 1. [Unified Matcher: `CelMatcher`], input effectively restricted to + `HttpAttributesCelMatchInput`. +- Filter-specific default no-match behavior: + - The RPC is allowed by default and not reported to the RLQS server. + +Any other types are considered invalid and will result in gRPC NACKing the xDS +resource. + +#### Config: `RateLimitQuotaBucketSettings` We will support the following fields in the -[RuntimeFractionalPercent](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/config/core/v3/base.proto#L643) -proto: +[`RateLimitQuotaBucketSettings`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L169) +message: -- [default_value](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/config/core/v3/base.proto#L648): - This field must be present. If the denominator specified is less than the - numerator, the final fractional percentage is capped at 1 (100%). The - fraction specified with: - - [numerator](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/type/v3/percent.proto#L52): - Non-negative integer, 0 by default. - - [denominator](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/type/v3/percent.proto#L56): - Must be one of the - [DenominatorType](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/type/v3/percent.proto#L34) - values: - - `HUNDRED` (default) - - `TEN_THOUSAND` - - `MILLION` -- All other fields are ignored. +- [`bucket_id_builder`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L348): + If present, must be a valid `BucketIdBuilder` message. This is used to + construct a `BucketId` for each request, which may involve dynamic values + from request metadata. If not set, requests are not reported to the RLQS + server and are handled according to `no_assignment_behavior`. + - `bucket_id_builder`: A map from string to + [`ValueBuilder`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L265). + Must contain at least one entry. In addition to the specification, gRPC + will restrict the total number of key-value pairs to 30. + - `ValueBuilder` can be a static + [`string_value`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L272) + or a dynamic + [`custom_value`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L276). + For `custom_value`, the initial implementation will support a + [`TypedExtensionConfig`] containing a + [Unified Matcher: `HttpRequestHeaderMatchInput`]. +- [`reporting_interval`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L398): + Must be present. A [`google.protobuf.Duration`] specifying the interval for + reporting quota usage. Must be greater than 100ms. Note that gRPC will apply + a best-effort approach to report at the configured interval, and may report + earlier or later in certain situations. See [On Report Timers] for details. +- [`deny_response_settings`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L407): + If not set, RPCs will be denied as specified in the default value of + `grpc_status`. If present, must be a valid + [`DenyResponseSettings`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L227) + message: + - `grpc_status`: A [`google.rpc.Status`] for + [gRPC responses][gRPC Status Codes]. Defaults to `UNAVAILABLE`. + - `response_headers_to_add`: A list of up to 10 + [Config: `HeaderValueOption`] to add to the deny response. +- [`no_assignment_behavior`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L413): + If not set, all requests are allowed. If set, must be a valid + [`NoAssignmentBehavior`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L172) + message. + - `fallback_rate_limit`: Must be present and valid + [Config: `RateLimitStrategy`] to apply. +- [`expired_assignment_behavior`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L422): + Specifies behavior when a quota assignment expires and cannot be refreshed. + If not set, the bucket is abandoned when its quota expires. + - [`expired_assignment_behavior_timeout`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L206): + A [`google.protobuf.Duration`] that limits how long this behavior is + applied. Must be a positive duration if set. If not set, it defaults to + zero and the bucket is abandoned immediately. + - One of the following must be set: + - [`fallback_rate_limit`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L216): + A [Config: `RateLimitStrategy`] to apply. + - [`reuse_last_assignment`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L222): + Reuses the rate limit strategy specified in the last known quota + assignment, effectively extending it by + `expired_assignment_behavior_timeout`. If a quota has never been + assigned to te bucket, the bucket is abandoned immediately. The following fields will be ignored by gRPC: -- runtime_key: gRPC does not have Envoy's concept of runtime settings. +- `DenyResponseSettings.http_status` +- `DenyResponseSettings.http_body` -#### Config: HeaderValueOption +#### Config: `RateLimitStrategy` -We will support the following fields in the [HeaderValueOption](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/config/core/v3/base.proto#L429) proto: +We will support the following fields in the +[`RateLimitStrategy`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/type/v3/ratelimit_strategy.proto#L22) +message: + +- `strategy`: One of the following must be present and valid: + - [`blanket_rule`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/type/v3/ratelimit_strategy.proto#L67): + An + [`BlanketRule`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/type/v3/ratelimit_strategy.proto#L24) + enum that can be `ALLOW_ALL` or `DENY_ALL`. Defaults to `ALLOW_ALL` when + the field is defined, but its value is not specified. + - [`requests_per_time_unit`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/type/v3/ratelimit_strategy.proto#L72): + This field allows to specify the rate limit without mandating a specific + algorithm. gRPC language implementation may choose an algorithm best + suitable to their language, or use a [`TokenBucket`] of an equivalent + rate. The rate limit parameters are specified in a + [`RequestsPerTimeUnit`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/type/v3/ratelimit_strategy.proto#L43) + message: + - [`requests_per_time_unit`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/type/v3/ratelimit_strategy.proto#L54): + Must be present. If set to `0`, all requests are denied. + - [`time_unit`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/type/v3/ratelimit_strategy.proto#L52): + Must be a valid + [`RateLimitUnit`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/type/v3/ratelimit_unit.proto#L16) + enum value. Ignored if `requests_per_time_unit` is `0`. + - [`token_bucket`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/type/v3/ratelimit_strategy.proto#L37): + A [`TokenBucket`] message containing configuration for a + [Token Bucket](https://en.wikipedia.org/wiki/Token_bucket) rate-limiting + algorithm: + - [`max_tokens`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/type/v3/token_bucket.proto#L26): + Must be present and greater than 0. + - [`tokens_per_fill`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/type/v3/token_bucket.proto#L30): + Must be greater than 0. Defaults to 1. + - [`fill_interval`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/type/v3/token_bucket.proto#L35): + Must be present and contain a positive [`google.protobuf.Duration`]. + In addition to the specification, gRPC will reject values less than + 100ms second. + +#### Config: `HeaderValueOption` -- [header](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/config/core/v3/base.proto#L458): +We will support the following fields in the +[`HeaderValueOption`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/config/core/v3/base.proto#L429) +proto: + +- [`header`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/config/core/v3/base.proto#L458): Must be present. - - [key](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/config/core/v3/base.proto#L404): + - [`key`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/config/core/v3/base.proto#L404): Value length must be in the range `[1, 16384)`. Must be a valid HTTP/2 header name. - - [value](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/config/core/v3/base.proto#L415): + - [`value`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/config/core/v3/base.proto#L415): Specifies the header value. Must be shorter than 16384 bytes. Must be a valid HTTP/2 header value. Not used if `key` ends in `-bin` and `raw_value` is set. - - [raw_value](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/config/core/v3/base.proto#L422): + - [`raw_value`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/config/core/v3/base.proto#L422): Used only if `key` ends in `-bin`. Must be shorter than 16384 bytes. Will be base64-encoded on the wire, unless the pure binary metadata extension from [gRFC G1: True Binary Metadata][G1] is used. -- [append_action](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/config/core/v3/base.proto#L476): +- [`append_action`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/config/core/v3/base.proto#L476): Must be of the - [HeaderAppendAction](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/config/core/v3/base.proto#L434) + [`HeaderAppendAction`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/config/core/v3/base.proto#L434) values: - `APPEND_IF_EXISTS_OR_ADD` (default) - `ADD_IF_ABSENT` - `OVERWRITE_IF_EXISTS_OR_ADD` - `OVERWRITE_IF_EXISTS` -- [keep_empty_value](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/config/core/v3/base.proto#L480) +- [`keep_empty_value`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/config/core/v3/base.proto#L480) - All other fields are ignored. The following fields will be ignored by gRPC: -- append: Deprecated in favor of `append_action`. +- `append`: Deprecated in favor of `append_action`. -#### Config: Bucket Matchers +#### Config: `RuntimeFractionalPercent` + +We will support the following fields in the +[RuntimeFractionalPercent](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/config/core/v3/base.proto#L643) +proto: + +- [`default_value`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/config/core/v3/base.proto#L648): + This field must be present. If the denominator specified is less than the + numerator, the final fractional percentage is capped at 1 (100%). The + fraction specified with: + - [`numerator`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/type/v3/percent.proto#L52): + Non-negative integer, 0 by default. + - [`denominator`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/type/v3/percent.proto#L56): + Must be one of the + [`DenominatorType`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/type/v3/percent.proto#L34) + values: + - `HUNDRED` (default) + - `TEN_THOUSAND` + - `MILLION` +- All other fields are ignored. -TODO(sergiitk): add details +The following fields will be ignored by gRPC: ---- +- `runtime_key`: gRPC does not have Envoy's concept of runtime settings. ### RLQS Components @@ -270,7 +432,7 @@ final class RlqsFilter implements Filter { // Merge with per-route overrides if provided. if (overrideConfig instanceof RlqsConfigOverride rlqsConfigOverride) { - // Only domain and matchers can be overriden. + // Only domain and matchers can be overridden. if (!rlqsConfigOverride.domain().isEmpty()) { rlqsConfigBuilder.domain(rlqsConfigOverride.domain()); } @@ -289,7 +451,7 @@ final class RlqsFilter implements Filter { } ``` -##### Future considerations +###### Future considerations This proposal uses the entire RLQS Filter Config to identify the corresponding unique Filter State. As a result, a Filter State will be recreated even if only @@ -360,12 +522,8 @@ RLQS Filter State object will include the following data members: - RLQS Client: Accessed when we get the first data plane RPC for a given bucket and when a report timer fires. Notifies RLQS Filter State of responses received from the RLQS server. -- Report Timers Map: Initialized at instantiation. Used to track discovered - reporting intervals and their execution handlers. - - Entries inserted we get the first data plane RPC for a given bucket and - there's no key for bucket reporting interval. - - Entries deleted when there's no more buckets with given reporting - interval in RLQS Bucket Map. +- Report Timers: Initialized at instantiation. Used to schedule periodic + bucket usage reports to the RLQS server. See [On Report Timers] for details. Pseudo-code for RLQS Filter State RPC rate limiting: @@ -517,8 +675,9 @@ it doesn't already exist. If a new bucket has been created, the RPC is rate-limited according to bucket's default configuration. Then the filter schedules a report on the RLQS stream -informing the server of the bucket's creation, and registers the report timers -for bucket's reporting interval. +informing the server of the bucket's creation as described in +[On Sending Usage Reports], and registers the report timers for bucket's +reporting interval. If the bucket exists, the RPC is rate-limited according to its active quota assignment and usage counters. The bucket increments corresponding bucket usage @@ -526,64 +685,406 @@ counter. #### On RLQS Server Response +TODO(sergiitk): Add detailed response parsing and validation + When receiving an RLQS Server Response, the RLQS Client passes parsed response to the RLQS Filter State. The RLQS Filter State iterates through the bucket assignments in the response, and updates the corresponding buckets in the RLQS -Bucket Map. Buckets marked to be abandoned are purged from the cache. +Bucket Map as described in +[`RateLimitQuotaResponse.QuotaAssignmentAction`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/service/rate_limit_quota/v3/rlqs.proto#L106-L139). If a bucket has a new rate limit assignment, the bucket's active assignment is -updated and bucket usage counters are reset. Otherwise, only the assignment -expiration is updated. +updated and an immediate bucket usage report is sent, see +[On Sending Usage Reports]. Otherwise, only the assignment expiration is +updated. + +Buckets marked to be abandoned are purged from the cache as described in +[`RateLimitQuotaResponse.AbandonAction`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/service/rate_limit_quota/v3/rlqs.proto#L169-L199). #### On Report Timers +- TODO(sergiitk): implementation-focused description so that other impl + approaches allowed +- TODO(sergiitk): add details on scalability requirements +- TODO(sergiitk): Are there requirements in terms of when exactly we + reschedule the timer, and how that interacts with flow control on the RLQS + stream? + When a report timer fires, the RLQS Filter State retrieves all buckets with the corresponding reporting interval from the RLQS Bucket Map. For each bucket, the -filter snapshots the current usage counters, and resets them. The filter then -sends the snapshot to RLQS server using RLQS Client. +filter snapshots the current usage counters, and resets them as described in +[On Sending Usage Reports]. The filter then sends the snapshot to RLQS server +using RLQS Client. If a bucket's active assignment has expired, the bucket is removed from the cache. If there are no more buckets with the given reporting interval, the corresponding timer is removed. +#### On Sending Usage Reports + +When preparing bucket reports, the implementer should keep in mind that bucket +usage counters may be updated concurrently by other threads, see +[RLQS Buckets and Multithreading]. + +One potential approach is preserving the current state of the bucket in a +snapshot and immediately resetting the usage counters. The snapshot would +contain time delta between the new and previous snapshot creation times, and the +number of allowed/denied requests. The pseudo-code for this approach may look +something like this: + +```java + private final AtomicLong numRequestsAllowed = new AtomicLong(); + private final AtomicLong numRequestsDenied = new AtomicLong(); + private final AtomicLong lastSnapshotTimeNanos = new AtomicLong(-1); + + public RlqsBucketUsage snapshotAndResetUsage() { + long snapAllowed = numRequestsAllowed.get(); + long snapDenied = numRequestsDenied.get(); + long snapTime = nanoTimeNow(); + + // Reset stats. + numRequestsAllowed.addAndGet(-snapAllowed); + numRequestsDenied.addAndGet(-snapDenied); + + long lastSnapTime = lastSnapshotTimeNanos.getAndSet(snapTime); + // First snapshot. + if (lastSnapTime < 0) { + lastSnapTime = snapTime; + } + // RlqsBucketUsage snapshots the current bucket state, and will later be transformed + // to the report sent to RLQS Server. + return RlqsBucketUsage.create(bucketId, snapAllowed, snapDenied, snapTime - lastSnapTime); + } +``` + +The `RateLimitQuotaUsageReports` message is sent to the RLQS server via the +`StreamRateLimitQuotas` RPC defined in [rlqs.proto][rlqs_proto]. Each message +contains usage reports for one or more buckets. + +The following fields will be populated in the +[`RateLimitQuotaUsageReports`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/service/rate_limit_quota/v3/rlqs.proto#L66): + +- [`domain`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/service/rate_limit_quota/v3/rlqs.proto#L96): + Populated from the `domain` field in the [RateLimitQuotaFilterConfig]. This + field is only sent in the first `RateLimitQuotaUsageReports` message on a + new gRPC stream to the RLQS server. Subsequent messages on the same stream + will omit this field. +- [`bucket_quota_usages`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/service/rate_limit_quota/v3/rlqs.proto#L100): + A list of + [`BucketQuotaUsage`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/service/rate_limit_quota/v3/rlqs.proto#L72) + messages, each representing the usage report for a specific bucket. Each + `BucketQuotaUsage` message will have the following fields populated: + - [`bucket_id`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/service/rate_limit_quota/v3/rlqs.proto#L74): + Populated from the `bucket_id` of the `RlqsBucket`. This identifies the + bucket for which the usage is being reported. + - [`time_elapsed`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/service/rate_limit_quota/v3/rlqs.proto#L77): + A [`google.protobuf.Duration`] representing the time difference between + the current time and the last usage report was sent for this specific + `bucket_id`. + - [`num_requests_allowed`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/service/rate_limit_quota/v3/rlqs.proto#L83): + The number of requests allowed for this bucket since the last report. + This comes from the `Request Counters` in the `RlqsBucket`. + - [`num_requests_denied`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/service/rate_limit_quota/v3/rlqs.proto#L86): + The number of requests denied for this bucket since the last report. + This also comes from the `Request Counters` in the `RlqsBucket`. + +Usage reports are sent in following scenarios: + +1. **On New Bucket Creation**: When the first RPC for a new `bucket_id` is + processed, an immediate report is scheduled to inform the RLQS server of the + new bucket subscription, see [On Data Plane RPC]. +2. **On Report Timers**: When a report timer fires, see [On Report Timers]. +3. **Replacing the assignment**: When bucket's active assignment is replaced, + as described in [On RLQS Server Response]. + ### Integrations #### Connecting to RLQS Control Plane -xDS Control Plane provides RLQS connection details in [GrpcService.GoogleGrpc] +xDS Control Plane provides RLQS connection details in +[`GrpcService.GoogleGrpc`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/config/core/v3/grpc_service.proto#L68) message as specified in [A102]. -#### Unified Matcher API +#### Unified Matcher API Support -RPCs will be matched into buckets using [Unified Matcher API] — an adaptable -framework that can be used in any xDS component that needs matching features. +[Unified Matcher API] is an adaptable framework that can be used in any xDS +component that needs matching features. -Envoy provides two syntactically equivalent Unified Matcher -definitions: [`envoy.config.common.matcher.v3.Matcher`](https://github.com/envoyproxy/envoy/blob/e3da7ebb16ad01c2ac7662758a75dba5cdc024ce/api/envoy/config/common/matcher/v3/matcher.proto) -and [`xds.type.matcher.v3.Matcher`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto). -We will only support the latter, which is the preferred version for all new APIs -using Unified Matcher. - -For RLQS, Unified Matcher tree will be provided in the filter config. Evaluating -the tree against RPC metadata will yield `RateLimitQuotaBucketSettings`, which -contains the information needed to associate the RPC with `bucket_id` and the -default rate limiting configuration. +Envoy provides two syntactically equivalent Unified Matcher definitions: +[`envoy.config.common.matcher.v3.Matcher`](https://github.com/envoyproxy/envoy/blob/e3da7ebb16ad01c2ac7662758a75dba5cdc024ce/api/envoy/config/common/matcher/v3/matcher.proto) +and +[`xds.type.matcher.v3.Matcher`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto), +which is the preferred version for all new APIs using Unified Matcher. If +`envoy.config.common.matcher.v3.Matcher` is provided, we will interpret it as is +`xds.type.matcher.v3.Matcher`. In this iteration the following Unified Mather extensions will be supported: -1. Inputs: - 1. [`HttpRequestHeaderMatchInput`](https://www.envoyproxy.io/docs/envoy/latest/api-v3/type/matcher/v3/http_inputs.proto#type-matcher-v3-httprequestheadermatchinput) - 2. [`HttpAttributesCelMatchInput`](https://www.envoyproxy.io/docs/envoy/latest/xds/type/matcher/v3/http_inputs.proto#envoy-v3-api-msg-xds-type-matcher-v3-httpattributescelmatchinput) -2. Custom Matchers: - 1. [`CelMatcher`](https://www.envoyproxy.io/docs/envoy/latest/xds/type/matcher/v3/cel.proto.html) +1. Inputs: + 1. [Unified Matcher: `HttpRequestHeaderMatchInput`] + 2. [Unified Matcher: `HttpAttributesCelMatchInput`] +2. Matchers: + 1. [Unified Matcher: `StringMatcher`] (standard matcher) + 1. [Unified Matcher: `CelMatcher`] + +#### Unified Matcher: Filter Integration + +When implementing Unified Matcher API, a filter must define the following: + +- Supported protocol-specific actions (see [Unified Matcher: `OnMatch`]). +- Supported [Unified Matcher: Input Extensions]. +- Supported [Unified Matcher: Matching Extensions], including any additional + limitations on their inputs. +- Filter-specific default no-match behavior (f.e. xDS resource NACK). + +##### Unified Matcher: `Matcher` + +While the Unified Matcher API allows for matcher trees of arbitrary depth, gRPC +will reject any matcher definition with a tree depth greater than 100, NACKing +the xDS resource. + +We will support the following fields in the +[`xds.type.matcher.v3.Matcher`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L22) +message: + +- `matcher_type`: One of the following must be present and valid: + - [`matcher_list`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L126): + A valid [Unified Matcher: `MatcherList`] message. + - [`matcher_tree`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L129): + A valid [Unified Matcher: `MatcherTree`] message. +- [`on_no_match`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L135): + Specifies the action executed if no match is found in the `matcher_list` or + `matcher_tree`. If set, must be a valid [Unified Matcher: `OnMatch`]. + If not set, refer to filter's default no-match behavior. + +##### Unified Matcher: `OnMatch` + +We will support the following fields in the +[`xds.type.matcher.v3.Matcher.OnMatch`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L24) +message: + +- `on_match`: One of the following must be present and valid: + - [`matcher`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L33): + A nested [Unified Matcher: `Matcher`] for more complex, tree-like + matching logic. + - [`action`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L36): + A [`TypedExtensionConfig`] containing a protocol-specific action to + take. + +The following fields will be ignored by gRPC: + +- `keep_matching`: Not supported in the initial implementation, may be added + later. + +##### Unified Matcher: `MatcherList` + +We will support the following fields in the +[`xds.type.matcher.v3.Matcher.MatcherList`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L43) +message: + +- [`matchers`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L96): + A list of + [`Matcher.MatcherList.FieldMatcher`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L87) + messages. Must contain at least 1 item. + - [`predicate`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L89): + Must be set and contain a valid + [`Matcher.MatcherList.Predicate`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L45) + message. + - `match_type`: One of the following must be present and valid: + - [`single_predicate`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L73): + A + [`Matcher.MatcherList.Predicate.SinglePredicate`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L47) + message. The return type of the input must match the input type + of the matcher. + - [`input`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L50): + A valid [`TypedExtensionConfig`]. Must be present and + contain one of the input extensions + [supported by the filter][Unified Matcher: Filter Integration]. + Must have return type compatible with the `matcher`. + - `matcher`: One of the following must be present and valid: + - [`value_match`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L56): + A valid [Unified Matcher: `StringMatcher`]. Only + compatible with `input` that returns a string. + - [`custom_match`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L60): + A valid [`TypedExtensionConfig`] containing one of the + matching extensions + [supported by the filter][Unified Matcher: Filter Integration]. + Must have input type compatible with the `input`. Must + return a boolean indicating the status of the match. + - [`or_matcher`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L76): + A + [`Matcher.MatcherList.Predicate.PredicateList`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L65) + message. + - [`predicate`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L66): + A list of `Matcher.MatcherList.Predicate` messages. Must + contain at least 2 items. Returns true if any of them are + true. + - [`and_matcher`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L79): + A + [`Matcher.MatcherList.Predicate.PredicateList`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L65) + message. + - [`predicate`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L66): + A list of `Matcher.MatcherList.Predicate` messages. Must + contain at least 2 items. Returns true if all of them are + true. + - [`not_matcher`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L82): + A nested `Matcher.MatcherList.Predicate` message. Returns the + inverted result of predicate evaluation. + - [`on_match`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L92): + Must be set and contain a valid [Unified Matcher: `OnMatch`] message. + +##### Unified Matcher: `MatcherTree` + +We will support the following fields in the +[`xds.type.matcher.v3.Matcher.MatcherTree`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L99) +message: + +- [`input`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L106): + A valid [`TypedExtensionConfig`]. Must be present and contain one of the + input extensions + [supported by the filter][Unified Matcher: Filter Integration]. Must have + return type compatible with the `matcher`. +- `tree_type`: One of the following must be present and valid: + - [`exact_match_map`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L114): + A + [`Matcher.MatcherTree.MatchMap`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L101) + message. Only compatible with `input` that returns a string. + - [`map`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L102): + A map from a string to a valid [Unified Matcher: `OnMatch`] message. + Must contain at least 1 pair. + - [`prefix_match_map`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L117): + A + [`Matcher.MatcherTree.MatchMap`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L101) + message. Only compatible with `input` that returns a string. + - [`map`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L102): + A map from a string to a valid [Unified Matcher: `OnMatch`] message. + Must contain at least 1 pair. + - [`custom_match`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L120): + A valid [`TypedExtensionConfig`] containing one of the matching + extensions + [supported by the filter][Unified Matcher: Filter Integration]. Must + have input type compatible with the `input`. Must return a boolean + indicating the status of the match. + +##### Unified Matcher: Input Extensions + +###### Unified Matcher: `HttpRequestHeaderMatchInput` + +Returns a `string` containing the value of the header with name specified in +`header_name`. + +We will support the following fields in the +[`envoy.type.matcher.v3.HttpRequestHeaderMatchInput`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/type/matcher/v3/http_inputs.proto#L22) +message: + +- [`header_name`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/matcher.proto#L106): + Must be present. Value length must be in the range `[1, 16384)`. Must be a + valid HTTP/2 header name. + +###### Unified Matcher: `HttpAttributesCelMatchInput` + +Returns a language-specific interface that allows to access request RPC metadata +as defined in [Supported CEL Variables]. + +We will support the following fields in the +[`xds.type.matcher.v3.HttpAttributesCelMatchInput`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/http_inputs.proto#L22) +message: + +- no fields. + +##### Unified Matcher: Matching Extensions + +###### Unified Matcher: `StringMatcher` + +Compatible with [Unified Matcher: Input Extensions] that return a `string`. + +We will support the following fields in the +[`xds.type.matcher.v3.StringMatcher`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/string.proto#L19) +message: + +- `match_pattern`: One of the following must be present and valid: + - [`exact`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/string.proto#L28): + The input string must match exactly. An empty string is a valid value. + - [`prefix`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/string.proto#L36): + The input string must have this prefix. Must be non-empty. + - [`suffix`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/string.proto#L44): + The input string must have this suffix. Must be non-empty. + - [`contains`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/string.proto#L55): + The input string must contain this substring. Must be non-empty. +- [`ignore_case`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/string.proto#L65): + If `true`, the matching is case-insensitive. + +The following are not supported by gRPC in the initial implementation and will +result in xDS resource NACK: + +- `safe_regex` +- `custom` + +###### Unified Matcher: `CelMatcher` + +Compatible with [Unified Matcher: `HttpAttributesCelMatchInput`]. + +Performs a match by evaluating a Common Expression Language (CEL) expression. +See [CEL Integration] for details. + +We will support the following fields in the +[`xds.type.matcher.v3.CelMatcher`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/cel.proto#L30) +message: + +- [`expr_match`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/cel.proto#L32): + Must be present and contain a valid + [`xds.type.v3.CelExpression`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/v3/cel.proto#L26) + message. + - [`cel_expr_checked`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/v3/cel.proto#L49): + Must be present and contain a valid [`cel.expr.CheckedExpr`] message. + This message will be converted into a native CEL Abstract Syntax Tree + (AST) using the language-specific CEL library. The AST's output (return) + type must be boolean. The resulting CEL program must also be validated + to conform to [CEL Runtime Restrictions]. If any of these conversion or + validation steps fail, gRPC will NACK the xDS resource. +- [`description`](https://github.com/cncf/xds/blob/b4127c9b8d78b77423fd25169f05b7476b6ea932/xds/type/matcher/v3/cel.proto#L36): + An optional string. May be ignored or used for testing/debugging. + +The following fields will be ignored by gRPC: + +- `CelExpression.parsed_expr` - deprecated, only Canonical CEL is supported. +- `CelExpression.checked_expr` - deprecated, only Canonical CEL is supported. +- `CelExpression.cel_expr_parsed` - only Checked CEL expressions are + supported. #### CEL Integration We will support request metadata matching via CEL expressions. Only Canonical -CEL and only checked expressions will be supported (`cel.expr.CheckedExpr`). +CEL and only checked expressions will be supported [`cel.expr.CheckedExpr`]. CEL evaluation environment is a set of available variables and extension -functions in a CEL program. We will match [Envoy CEL environment]. +functions in a CEL program. We will match +[Envoy CEL environment](https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/advanced/attributes) +and CEL interpreter configuration. + +##### CEL Runtime Restrictions + +Certain CEL features can lead to superlinear time complexity or memory +exhaustion. To ensure consistent behavior with Envoy and maintain security, +gRPC will configure the CEL runtime +[similar to Envoy](https://github.com/envoyproxy/envoy/blob/c57801c2afbe26dd6fad7f5ce764f267d07fbd04/source/extensions/filters/common/expr/evaluator.cc#L17-L23): + +```c +// Disables comprehension expressions, e.g. exists(), all(). +options.enable_comprehension = false; + +// Limits the maximum program size for RE2 regex to 100. +options.regex_max_program_size = 100; + +// Disables string() overloads. +options.enable_string_conversion = false; + +// Disables string concatenation overload. +options.enable_string_concat = false; + +// Disables list concatenation overload. +options.enable_list_concat = false; +``` ##### Supported CEL Functions @@ -609,8 +1110,9 @@ except comprehension-style macros. ##### Supported CEL Variables -For RLQS, only the `request` variable is supported in CEL expressions. We will -adapt [Envoy's Request Attributes](https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/advanced/attributes#request-attributes) +In the initial implementation only the `request` variable is supported in CEL +expressions. We will adapt +[Envoy's Request Attributes](https://www.envoyproxy.io/docs/envoy/latest/intro/arch_overview/advanced/attributes#request-attributes) for gRPC. | Attribute | Type | gRPC source | Envoy Description | @@ -637,7 +1139,7 @@ denies requests for all other method types. **2 `request.headers`**\ As defined in [A41], "header" field. -##### Implementation +###### CEL Variable Implementation Details For performance reasons, CEL variables should be resolved on demand. CEL Runtime provides the different variable resolving approaches based on the language: @@ -646,7 +1148,7 @@ provides the different variable resolving approaches based on the language: * Go: [`Activation.ResolveName(string)`](https://github.com/google/cel-go/blob/3f12ecad39e2eb662bcd82b6391cfd0cb4cb1c5e/interpreter/activation.go#L30) * Java: [`CelVariableResolver`](https://javadoc.io/doc/dev.cel/runtime/0.6.0/dev/cel/runtime/CelVariableResolver.html) -### Temporary environment variable protection +### Temporary Environment Variable Protection During initial development, this feature will be enabled via the `GRPC_EXPERIMENTAL_XDS_ENABLE_RLQS` environment variable. This environment From 85695fb27b496b2603695ebc1a79aa068e821635 Mon Sep 17 00:00:00 2001 From: Sergii Tkachenko Date: Mon, 27 Oct 2025 18:23:31 -0700 Subject: [PATCH 51/53] minor: add missing BucketIdBuilder links --- A77-xds-rate-limiting-rlqs.md | 15 +++++++++------ 1 file changed, 9 insertions(+), 6 deletions(-) diff --git a/A77-xds-rate-limiting-rlqs.md b/A77-xds-rate-limiting-rlqs.md index af8cf1715..0278889ef 100644 --- a/A77-xds-rate-limiting-rlqs.md +++ b/A77-xds-rate-limiting-rlqs.md @@ -5,7 +5,7 @@ A77: xDS Server-Side Rate Limiting * Approver: Mark Roth (@markdroth) * Status: In Review * Implemented in: -* Last updated: 2025-10-03 +* Last updated: 2025-10-27 * Discussion at: - [ ] TODO(sergiitk): insert google group thread @@ -194,11 +194,14 @@ We will support the following fields in the message: - [`bucket_id_builder`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L348): - If present, must be a valid `BucketIdBuilder` message. This is used to - construct a `BucketId` for each request, which may involve dynamic values - from request metadata. If not set, requests are not reported to the RLQS - server and are handled according to `no_assignment_behavior`. - - `bucket_id_builder`: A map from string to + If present, must be a valid + [`BucketIdBuilder`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L262) + message. This is used to construct a `BucketId` for each request, which may + involve dynamic values from request metadata. If not set, requests are not + reported to the RLQS server and are handled according to + `no_assignment_behavior`. + - [`bucket_id_builder`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L293): + A map from string to [`ValueBuilder`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L265). Must contain at least one entry. In addition to the specification, gRPC will restrict the total number of key-value pairs to 30. From 55c8557375679873823a0b13203ee4f63a706a88 Mon Sep 17 00:00:00 2001 From: Sergii Tkachenko Date: Mon, 27 Oct 2025 18:23:31 -0700 Subject: [PATCH 52/53] minor: add missing BucketIdBuilder links --- A77-xds-rate-limiting-rlqs.md | 15 +++++++++------ 1 file changed, 9 insertions(+), 6 deletions(-) diff --git a/A77-xds-rate-limiting-rlqs.md b/A77-xds-rate-limiting-rlqs.md index af8cf1715..0278889ef 100644 --- a/A77-xds-rate-limiting-rlqs.md +++ b/A77-xds-rate-limiting-rlqs.md @@ -5,7 +5,7 @@ A77: xDS Server-Side Rate Limiting * Approver: Mark Roth (@markdroth) * Status: In Review * Implemented in: -* Last updated: 2025-10-03 +* Last updated: 2025-10-27 * Discussion at: - [ ] TODO(sergiitk): insert google group thread @@ -194,11 +194,14 @@ We will support the following fields in the message: - [`bucket_id_builder`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L348): - If present, must be a valid `BucketIdBuilder` message. This is used to - construct a `BucketId` for each request, which may involve dynamic values - from request metadata. If not set, requests are not reported to the RLQS - server and are handled according to `no_assignment_behavior`. - - `bucket_id_builder`: A map from string to + If present, must be a valid + [`BucketIdBuilder`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L262) + message. This is used to construct a `BucketId` for each request, which may + involve dynamic values from request metadata. If not set, requests are not + reported to the RLQS server and are handled according to + `no_assignment_behavior`. + - [`bucket_id_builder`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L293): + A map from string to [`ValueBuilder`](https://github.com/envoyproxy/envoy/blob/7ebdf6da0a49240778fd6fed42670157fde371db/api/envoy/extensions/filters/http/rate_limit_quota/v3/rate_limit_quota.proto#L265). Must contain at least one entry. In addition to the specification, gRPC will restrict the total number of key-value pairs to 30. From 7750a54ecba1d91c8941f5771baed3e578fdbc2c Mon Sep 17 00:00:00 2001 From: Sergii Tkachenko Date: Mon, 27 Oct 2025 18:28:05 -0700 Subject: [PATCH 53/53] --- synced with origin/a77-rlqs ---