Request per-request GUID correlation ID forwarded to backend (replace or supplement sequential x-S7P-ID)

[AlejandroValenciaTR]

Summary

We need a globally unique, per-request correlation identifier generated by SimpleL7Proxy and forwarded to backend services for end-to-end tracing. The current x-S7P-ID format is useful for operational debugging but is not suitable as a durable correlation ID in multi-replica deployments (e.g. ECS).

---

Current behavior

On each inbound request, the proxy assigns:

Identifier	Format	Forwarded to backend?
MID → x-S7P-ID	{RequestIDPrefix}-{replicaId}-{sequentialCounter} (e.g. S7P-abc12345-847)	Yes
Guid	Guid.NewGuid() (internal)	No (telemetry/async only)

• replicaId = CONTAINER_APP_REPLICA_NAME if set, else Hostname
• Counter is sequential per process/task and resets to 1 on restart/redeploy

Relevant code paths:

• MID generation: server.cs — IDStr + counter
• Header enrichment: EventDataBuilder.EnrichRequestHeaders() — sets x-S7P-ID
• Backend forward: ProxyWorker → ProxyHelperUtils.CopyHeaders()

---

Problem

1. Uniqueness / counter reset

After an ECS task restart, the counter resets:

Before restart:  S7P-abc12345-847
After restart:   S7P-abc12345-1

The same x-S7P-ID value can appear again over time for the same Hostname, which breaks long-lived or persisted correlation in downstream systems.

2. Collision risk across replicas

If multiple tasks share the same Hostname, they can emit identical IDs in parallel (e.g. two tasks both sending S7P-llm-proxy-prod-1).

3. User profiles do not address this

UserProfileHeader / user profile JSON provides static per-user configuration. It does not generate a dynamic per-request correlation ID.

---

Expected behavior (baseline request)

Provide a GUID-based correlation ID per request that is:

1. Generated by the proxy when the request is accepted (or when missing from the client)
2. Globally unique across replicas, restarts, and time
3. Forwarded to the backend on every proxied request
4. Included in proxy telemetry/logs (CloudWatch / App Insights) for cross-system lookup

Suggested approach (open to contractor design)

• Use the existing internal Guid (rd.Guid) or generate a new UUID per request
• Expose it on a standard header forwarded to backend, e.g.:
  • X-Correlation-ID, and/or
  • x-ms-client-request-id (already referenced in APIM policies)
• Optionally retain x-S7P-ID for infrastructure traceability (replica + local sequence), but correlation for backend persistence should be the GUID

---

Deployment context

• Platform: AWS ECS (Fargate), multiple tasks behind ALB
• Add-in: AWS 2.2.11.2
• Identity segment today: Hostname in task definition (or container HOSTNAME fallback)
• Not available on ECS: CONTAINER_APP_REPLICA_NAME (ACA-specific)

---

ECS examples (current x-S7P-ID behavior)

Good configuration — unique Hostname per task

ECS Task	Hostname	Request #	x-S7P-ID
Task A	a1b2c3d4e5f6	1	S7P-a1b2c3d4e5f6-1
Task A	a1b2c3d4e5f6	847	S7P-a1b2c3d4e5f6-847
Task B	f6e5d4c3b2a1	1	S7P-f6e5d4c3b2a1-1
Task C	9z8y7x6w5v4u	1	S7P-9z8y7x6w5v4u-1

Misconfigured — same Hostname on all tasks

Task	Request #	x-S7P-ID
Task A	1	S7P-llm-proxy-prod-1
Task B	1	S7P-llm-proxy-prod-1 (collision)
Task C	15	S7P-llm-proxy-prod-15

---

Acceptance criteria

• Each proxied request has a unique GUID correlation ID for its lifetime
• ID is stable for that request through backend retries within the same proxy handling
• ID is forwarded to backend on the agreed header name(s)
• ID appears in proxy observability (logs/events) and can be joined with backend logs
• Behavior is documented (config, header name, interaction with existing x-S7P-ID and ParentId)
• No reliance on user profile JSON for correlation ID generation

---

References

• Internal analysis: sequential x-S7P-ID on ECS and uniqueness limits (counter reset, shared Hostname collisions)
• Docs: docs/REQUEST_VALIDATION.md (RequiredHeaders=X-Correlation-ID as client-side pattern only today)
• APIM policy maps x-S7P-id → x-ms-client-request-id (Azure path)