Skip to content

Add fixed-window counter rate limiting#30

Open
rapind wants to merge 15 commits intonootr:mainfrom
pairshaped:sliding-window
Open

Add fixed-window counter rate limiting#30
rapind wants to merge 15 commits intonootr:mainfrom
pairshaped:sliding-window

Conversation

@rapind
Copy link
Copy Markdown
Contributor

@rapind rapind commented Mar 13, 2026

Summary

  • Adds glimit/window module with fixed-window counters and layered windows
  • Uses ETS with atomic update_counter for lock-free, concurrent operation
  • Unlike the token bucket (which smoothly refills tokens), fixed-window counters
    divide time into discrete windows and count requests. Ideal for:
    • Login/verification attempt limiting (e.g. 5 attempts per 15 minutes)
    • API rate limiting with clear reset boundaries
    • Layered limits (e.g. 1/min + 3/15min + 10/hour + 20/day)
  • 11 new tests, README updated with documentation

Note on storage

The window module uses its own dedicated ETS table with atomic update_counter
operations. It does not go through the pluggable bucket.Store interface. This
is intentional since fixed-window counters are typically a single-node pattern
and benefit from the atomicity guarantees of ETS. A pluggable store interface
for window counters could be added in the future if there's demand for
distributed fixed-window rate limiting.

Usage

let limiter = window.new()

let windows = [
  window.Window(window_seconds: 60, max_count: 1),
  window.Window(window_seconds: 900, max_count: 3),
  window.Window(window_seconds: 3600, max_count: 10),
]

case window.check(limiter, email, windows, now_seconds) {
  Ok(Nil) -> // allowed
  Error(window.Denied(retry_after)) -> // denied
}

Depends on

Test plan

  • All 11 new window tests pass
  • All 88 tests pass (existing + ETS store + window)

rapind added 4 commits March 13, 2026 08:52
ETS provides lower-latency rate limiting by using atomic table operations
directly, avoiding the overhead of OTP actor messages. Suitable for
single-node deployments where low latency is important.

- ets_store.gleam: EtsStore type implementing bucket.Store interface
- ets_store_ffi.erl: Erlang FFI for ETS operations (new/get/set/delete/sweep/size)
- glimit.ets_store() builder for easy integration
- Periodic sweep timer for cleaning up full and idle buckets
- 10 new tests covering all ETS store functionality
Document the new ETS-backed storage backend with usage example,
performance characteristics, and comparison to in-memory mode.
Replace the OTP actor-based in-memory store with ETS as the default.
ETS provides lower-latency, lock-free rate limiting without actor
message overhead. Remove memory_store module entirely — custom stores
can still be plugged in via glimit.store().
Introduces glimit/window module with layered fixed-window counters
using ETS atomic update_counter for lock-free, concurrent operation.

Unlike the token bucket algorithm (which smoothly refills tokens),
fixed-window counters divide time into discrete windows and count
requests within each. Useful for login attempts, verification codes,
and API rate limiting with clear reset boundaries.

- window.gleam: WindowLimiter type with check/reset/cleanup/get_count
- window_ffi.erl: Erlang FFI using ETS select_delete for cleanup
- Supports layered windows (e.g. 1/min + 3/15min + 10/hour + 20/day)
- 11 tests covering all window functionality
@rapind
Copy link
Copy Markdown
Contributor Author

rapind commented Mar 15, 2026

Updated to reflect that ETS is the new default.

@nootr
Copy link
Copy Markdown
Owner

nootr commented Mar 20, 2026

Hi @rapind, thanks for making this change. I really like the idea of adding an option for fixed-window rate limiting, but I am curious if it would be possible to have the public API closer to what it already is. Could we expand the builder pattern with just one extra method?

Maybe something like this?

let limiter =
  glimit.new()
  |> glimit.window(seconds: 60, max: 3)
  |> glimit.window(seconds: 900, max: 10)
  |> glimit.window(seconds: 3600, max: 20)
  |> glimit.identifier(fn(req) { req.email })
  |> glimit.on_limit_exceeded(fn(_) { response(429) })

Then users could use the regular apply() or build()+hit().

Maybe build() should then validate: either per_second is set, or at least one window is set.

I see that you've also updated the README with a paragraph about the fixed-window limiter, but maybe the token bucket and fixed window algorithms should be documented as a whole, so the user could see clearly what their options are. I hope I make sense, English is not my native tongue.

What do you think?

Also, I think there might be a small bug with how multiple windows are updated.

For example: with layered windows [1/min, 3/15min], the check function processes windows left to right. For each window it calls ets:update_counter which atomically increments the counter first, then checks if the count exceeds the max.

Say a user has already used 3 of their 15-min budget:

  1. check processes the first window (1/min): calls update_counter, count goes 0 -> 1, passes
  2. check processes the second window (3/15min): calls update_counter, count goes 3 -> 4, exceeds max, returns Error(Denied)

The request was denied, but the per-minute counter was already incremented to 1. That count stays. So now even if the 15-min window resets, the per-minute window has a phantom count from a request that never actually went through.

Over time, denied requests eat into shorter windows' budgets, causing users to get denied earlier than they should.

I think the fix would be to check all window counts first (read-only), and only increment if all windows allow the request. But that loses the atomicity of update_counter. You'd need a two-phase approach or accept a small race window on the increment step (like we do with the in-memory token bucket).

rapind added 7 commits March 21, 2026 22:06
When a later window denied a request, earlier windows that had already
passed kept their incremented counters. Over time this caused users to
hit shorter window limits earlier than they should.

Uses increment-then-rollback: if any window denies, all previously
incremented windows are decremented. The brief race window (microseconds
between increment and rollback) can only cause a false denial, never a
false admission.
Integrates fixed-window counters into the main builder pattern via
new_window(). Phantom types on the builder prevent invalid combinations
at compile time (e.g. per_second on a window builder). Both strategies
share build/hit/apply. RateLimited now includes retry_after seconds.
Presents token bucket and fixed-window as two strategies within a
single builder pattern. Documents compile-time safety via phantom
types, retry_after in error responses, and the standalone window API.
- Validate window seconds > 0 and max > 0 in builder (prevents
  division by zero in FFI)
- Add defensive catch in FFI for race between insert_new and
  update_counter (key could be deleted by reset/cleanup)
- Document non-atomic layered window check behavior
- Document cleanup guard math in FFI
- Add tests for invalid window parameter validation
- on_limit_exceeded is no longer required by build(). It is only
  validated at runtime when apply() or apply_built() is called. This
  supports the build() + hit() pattern without boilerplate.
- Add glimit.cleanup() for removing expired window entries. Token
  bucket is a no-op (sweep runs automatically).
- Remove standalone glimit/window API section from README.
- Add tests for cleanup, build-without-handler, and apply panics.
@rapind
Copy link
Copy Markdown
Contributor Author

rapind commented Mar 22, 2026

All concerns addressed, but here's the route I took:

  1. Counter bug: Fixed using increment-then-rollback. This avoids the race condition in a peek-then-increment approach, where two concurrent requests could both see room and both increment past the limit. Each window is still incremented atomically via update_counter, but if a later window denies, all previously incremented windows are decremented. The race window is microseconds and can only cause a false denial (conservative), never a false admission.
  2. Builder syntax: Rather than a single new() with runtime validation in build() for incompatible options (e.g., store() + window()), I used phantom types to catch these at compile time. Two entry points, new() (token bucket) and new_window() (fixed-window), return builders with different phantom type parameters. Calling per_second on a window builder or window on a token bucket builder is a compile error. Both share build(), hit(), apply(), get_count(), remove(), and cleanup(). Also added retry_after: Int to RateLimited so both strategies return how long until the limit resets.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants