Conversation
ETS provides lower-latency rate limiting by using atomic table operations directly, avoiding the overhead of OTP actor messages. Suitable for single-node deployments where low latency is important. - ets_store.gleam: EtsStore type implementing bucket.Store interface - ets_store_ffi.erl: Erlang FFI for ETS operations (new/get/set/delete/sweep/size) - glimit.ets_store() builder for easy integration - Periodic sweep timer for cleaning up full and idle buckets - 10 new tests covering all ETS store functionality
Document the new ETS-backed storage backend with usage example, performance characteristics, and comparison to in-memory mode.
Replace the OTP actor-based in-memory store with ETS as the default. ETS provides lower-latency, lock-free rate limiting without actor message overhead. Remove memory_store module entirely — custom stores can still be plugged in via glimit.store().
Introduces glimit/window module with layered fixed-window counters using ETS atomic update_counter for lock-free, concurrent operation. Unlike the token bucket algorithm (which smoothly refills tokens), fixed-window counters divide time into discrete windows and count requests within each. Useful for login attempts, verification codes, and API rate limiting with clear reset boundaries. - window.gleam: WindowLimiter type with check/reset/cleanup/get_count - window_ffi.erl: Erlang FFI using ETS select_delete for cleanup - Supports layered windows (e.g. 1/min + 3/15min + 10/hour + 20/day) - 11 tests covering all window functionality
|
Updated to reflect that ETS is the new default. |
|
Hi @rapind, thanks for making this change. I really like the idea of adding an option for fixed-window rate limiting, but I am curious if it would be possible to have the public API closer to what it already is. Could we expand the builder pattern with just one extra method? Maybe something like this? let limiter =
glimit.new()
|> glimit.window(seconds: 60, max: 3)
|> glimit.window(seconds: 900, max: 10)
|> glimit.window(seconds: 3600, max: 20)
|> glimit.identifier(fn(req) { req.email })
|> glimit.on_limit_exceeded(fn(_) { response(429) })Then users could use the regular Maybe I see that you've also updated the README with a paragraph about the fixed-window limiter, but maybe the token bucket and fixed window algorithms should be documented as a whole, so the user could see clearly what their options are. I hope I make sense, English is not my native tongue. What do you think? Also, I think there might be a small bug with how multiple windows are updated. For example: with layered windows [1/min, 3/15min], the check function processes windows left to right. For each window it calls Say a user has already used 3 of their 15-min budget:
The request was denied, but the per-minute counter was already incremented to 1. That count stays. So now even if the 15-min window resets, the per-minute window has a phantom count from a request that never actually went through. Over time, denied requests eat into shorter windows' budgets, causing users to get denied earlier than they should. I think the fix would be to check all window counts first (read-only), and only increment if all windows allow the request. But that loses the atomicity of update_counter. You'd need a two-phase approach or accept a small race window on the increment step (like we do with the in-memory token bucket). |
When a later window denied a request, earlier windows that had already passed kept their incremented counters. Over time this caused users to hit shorter window limits earlier than they should. Uses increment-then-rollback: if any window denies, all previously incremented windows are decremented. The brief race window (microseconds between increment and rollback) can only cause a false denial, never a false admission.
Integrates fixed-window counters into the main builder pattern via new_window(). Phantom types on the builder prevent invalid combinations at compile time (e.g. per_second on a window builder). Both strategies share build/hit/apply. RateLimited now includes retry_after seconds.
Presents token bucket and fixed-window as two strategies within a single builder pattern. Documents compile-time safety via phantom types, retry_after in error responses, and the standalone window API.
- Validate window seconds > 0 and max > 0 in builder (prevents division by zero in FFI) - Add defensive catch in FFI for race between insert_new and update_counter (key could be deleted by reset/cleanup) - Document non-atomic layered window check behavior - Document cleanup guard math in FFI - Add tests for invalid window parameter validation
- on_limit_exceeded is no longer required by build(). It is only validated at runtime when apply() or apply_built() is called. This supports the build() + hit() pattern without boilerplate. - Add glimit.cleanup() for removing expired window entries. Token bucket is a no-op (sweep runs automatically). - Remove standalone glimit/window API section from README. - Add tests for cleanup, build-without-handler, and apply panics.
|
All concerns addressed, but here's the route I took:
|
Summary
glimit/windowmodule with fixed-window counters and layered windowsupdate_counterfor lock-free, concurrent operationdivide time into discrete windows and count requests. Ideal for:
Note on storage
The window module uses its own dedicated ETS table with atomic
update_counteroperations. It does not go through the pluggable
bucket.Storeinterface. Thisis intentional since fixed-window counters are typically a single-node pattern
and benefit from the atomicity guarantees of ETS. A pluggable store interface
for window counters could be added in the future if there's demand for
distributed fixed-window rate limiting.
Usage
Depends on
Test plan