Skip to content

Add Redis backend guide for HA metrics aggregation#113

Open
noalimoy wants to merge 1 commit into
llm-d:mainfrom
noalimoy:config/redis-ha-metrics-aggregation
Open

Add Redis backend guide for HA metrics aggregation#113
noalimoy wants to merge 1 commit into
llm-d:mainfrom
noalimoy:config/redis-ha-metrics-aggregation

Conversation

@noalimoy
Copy link
Copy Markdown
Contributor

What does this PR do?

Add config/redis/README.md — a deployment, configuration, and verification guide for the Redis-backed DataStore, enabling HA metrics aggregation across multiple IPP replicas.

Why is this change needed?

With a single replica, the in-memory DataStore holds the complete picture. Under HA (multiple replicas), each sees only a subset of requests — Scorers make routing decisions on partial data (#79, #85).

This guide documents the Redis backend approach — per-replica keys with TTL, heartbeat/refresh goroutines, and zero network calls on the hot path.

How was this tested?

  • Unit tests added/updated
  • Integration/e2e tests added/updated
  • Manual testing performed

Documentation-only change. Verified that Redis YAML manifests are valid, code references (AttributeMap, InflightRequestsCount, Get()/Put()) match merged code on main, and Helm values align with existing values.yaml conventions.

Checklist

  • Commits are signed off (git commit -s) per DCO
  • Code follows project contributing guidelines
  • Tests pass locally (make test)
  • Linters pass (make lint)
  • Documentation updated (if applicable)

Related Issues

Refs: #79
Refs: #85

@github-actions github-actions Bot added the size/L Denotes a PR that changes 100-499 lines, ignoring generated files. label May 19, 2026
@noalimoy noalimoy force-pushed the config/redis-ha-metrics-aggregation branch from bf57680 to 0c0f743 Compare May 19, 2026 10:24
Comment thread config/redis/README.md Outdated
…A metrics aggregation

Signed-off-by: noalimoy <nlimoy@redhat.com>
@noalimoy noalimoy force-pushed the config/redis-ha-metrics-aggregation branch from 0c0f743 to 0e1396d Compare May 19, 2026 21:58
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/L Denotes a PR that changes 100-499 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants