feat(metrics): add souin_shared_response_counter for request coalescing visibility - #835
Open
sgoroshko wants to merge 6 commits into
Open
feat(metrics): add souin_shared_response_counter for request coalescing visibility#835sgoroshko wants to merge 6 commits into
sgoroshko wants to merge 6 commits into
Conversation
Records the design for a new Prometheus counter that tracks how many concurrent requests reuse a singleflight leader's response during a cache-miss, so this is visible next to the existing hit/miss graphs. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Task-by-task plan covering metric registration, the executedUpstream follower-detection fix in Upstream(), test coverage, and README docs. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…ed followers Track whether the singleflight closure actually executed the upstream call (executedUpstream), since singleflight's shared flag is also true for the leader when at least one follower joined. Only followers now increment the new counter. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
…rs only Fires 3 concurrent requests at the same cache-miss key and asserts the counter increases by exactly N-1 (followers only, not the leader). Uses a per-run unique URL since the default storer lives in a process-wide registry and would otherwise turn repeated runs into cache hits. Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
Co-Authored-By: Claude Sonnet 5 <noreply@anthropic.com>
✅ Deploy Preview for teal-sprinkles-4c7f14 canceled.
|
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Why
We rely on Souin's singleflight-based request coalescing to protect our upstream
during concurrent cache-miss stampedes (one leader request fetches, the group
shares the response). This is an important reliability mechanism for us, but it
was invisible on our dashboards — we only had cache hit/miss counters, with no
way to see how many concurrent requests were actually deduplicated.
What
Adds a new Prometheus counter,
souin_shared_response_counter, incremented onceper "follower" request that reused a leader's response instead of calling the
upstream itself.
pkg/api/prometheus/prometheus.goSouinBaseHandler.Upstream()(cache-miss path). A localexecutedUpstreamflag distinguishes the leader from followers, sincesingleflight's
sharedreturn value is alsotruefor the leader when atleast one follower joined — a naive
if sharedwould over-count by one percoalesced group.
Set-Cookieresponse, orVarymismatch) is correctly not counted.Revalidate()) is intentionally out of scope —can be a follow-up if needed.
Testing
the counter increases by exactly N-1.
go vetpass.Docs
Added the metric to the Prometheus API table in
README.md.