feat(platform): self-hosted Langfuse v3 — infra, LLM tracing, ops entry point (#320) - #328
Merged
Merged
Conversation
…320) Stage 1 of #320: the infrastructure only. No application code is instrumented yet — tracing the model router (#320 stage 2) and the recall pipeline (stage 3) land separately, so this PR is reviewable on its own and changes NO runtime behaviour. Langfuse v3 is not one container. It needs ClickHouse for analytics, Redis for queueing, S3-compatible blob storage for event payloads, and separate web and worker processes. That is far too much to impose on every `pnpm infra:up`, so it all sits behind a `langfuse` compose profile — the same opt-in pattern `metabase` already uses. Verified: `docker compose config --services` still reports exactly `nats postgres redis`, and the Langfuse services appear only with `--profile langfuse`. - `langfuse-db-init` creates a dedicated `langfuse` database on the EXISTING Postgres, mirroring `metabase-db-init` (idempotent, safe on every up) rather than adding a second Postgres. - Redis is reused, not duplicated. NOTE: this makes CLAUDE.md §1's "Redis, NATS and MinIO are reserved, not used" no longer true for Redis — the manual needs updating; flagged in #320. - MinIO is added. It was documented as a reserved dependency but was never actually in this file; this is its first real use. - Host ports: 3000 is admin and 3001 is metabase, so Langfuse takes 3002. Every port is `${VAR:-default}` so it can be moved on a machine that already has something bound — the same lesson as #315/#316. - `TELEMETRY_ENABLED=false`: this is a privacy tool, so it must not itself phone home. SELF-HOSTED ONLY, and that is the point rather than a preference: traces carry prompts and completions, which carry memory text. Langfuse Cloud and LangSmith are rejected because shipping that to a third party violates invariant §2.2. The ADR recording this is still owed (#320 stage 1). The three Langfuse secrets (NEXTAUTH_SECRET, SALT, ENCRYPTION_KEY) are wired as env with placeholder defaults so a local stack boots, and documented in .env.example with the generation command. ENCRYPTION_KEY must be exactly 64 hex chars. No real secret is committed. Verification: `docker compose --profile langfuse config` parses and resolves (the shared YAML anchor for the web/worker env expands correctly), and the default service list is unchanged. The stack has NOT yet been booted end-to-end — the images are a multi-GB pull — so "all services healthy" and the first-run Langfuse migration are UNPROVEN here and must be confirmed with `pnpm langfuse:up` before this is relied on. Refs #320 (stage 1 of 5; does not close it). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
The stage-1 commit stated the stack had NOT been booted end-to-end (the images
were a multi-GB pull still in flight) and that "all services healthy" plus the
first-run migration were unproven. They have now been run — recording the
evidence so the branch does not carry a stale caveat.
docker compose --profile langfuse ps
clickhouse healthy
minio healthy
langfuse-worker running
langfuse (web) running
curl http://localhost:3002 -> 200
curl http://localhost:3002/api/public/health -> 200
langfuse database created on the EXISTING postgres -> present
first-run migration -> 72 tables in public
`langfuse-db-init` created the database, the worker applied the Postgres and
ClickHouse migrations, and the web process serves. The pre-existing containers
were NOT disturbed: postgres/redis/nats kept their 3-4h uptimes across the
profile coming up, confirming the opt-in profile composes with a running dev
stack rather than restarting it.
Still stage 1 of 5 — no application code is instrumented, and tracing must not
be enabled until the stage-2 privacy gating (loopback-only payloads,
unconditional redaction for isPrivate, fail-open) exists.
Refs #320.
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Stage 2 of #320. Every call through the model router now emits a Langfuse trace + generation with model, class, vendor, token-derived cost and latency. `complete()` is the only instrumentation point needed: ADR-0002 already makes it the chokepoint every LLM call passes through, so one seam covers ingest and recall without touching a single call site. Three rules govern `platform/observability/langfuse.ts`, in priority order: 1. OFF BY DEFAULT — with no keys the client is undefined and every export is a no-op, so dev/CI/tests neither emit nor need a server (§2.5 parity). 2. PAYLOADS STAY ON-BOX — trace input/output ARE prompts and completions, i.e. memory text. They are sent only to a loopback host, reusing the same `isLoopbackUrl` gate as the Ollama provider rather than re-deriving the rule. A remote host degrades to metadata-only unless an operator sets LANGFUSE_ALLOW_REMOTE_PAYLOADS explicitly, so moving a URL in `.env` can never start exfiltrating content (M9). `isPrivate` content is redacted UNCONDITIONALLY — even on loopback: a trace is another store, and a private memory must not be duplicated into one. 3. FAIL-OPEN — every emit is wrapped; a Langfuse outage, bad key or serialisation error is swallowed and logged, never surfacing to a user. Same contract as `fireCoactivationLog`. Also fixes two stage-1 compose defects that green healthchecks hid. The stack reported every container healthy, the UI served 200 and 72 tables migrated — and could still persist ZERO traces: - The `langfuse` MinIO bucket was never created (Langfuse does not create it), so every event died with "Failed to upload events to blob storage, aborting event processing". Added an idempotent `langfuse-minio-init` one-shot mirroring `langfuse-db-init`; both langfuse services now gate on it. - Redis auth: passing REDIS_HOST/REDIS_PORT made Langfuse send an AUTH command to a password-less Redis ("ERR AUTH called without any password configured"). Switched to REDIS_CONNECTION_STRING. Trace input/output are set on the TRACE ROOT as well as the child generation. Setting only the generation left the UI showing "this trace didn't receive an input or output" while the payload sat one level down — the trace root is what the UI opens on. Verified end-to-end against the running stack (Ollama gemma3:4b): GENERATION model=gemma3:4b class=strong vendor=ollama costUsd=0 latencyMs=1808.5 trace input : "What is Priya allergic to?" + retrieved context output: "peanuts" type-check, lint green; 77 platform tests pass unchanged. NOT done, so the #320 privacy AC stays open: the two tests proving private memory text never reaches the client, and that a non-loopback host degrades to metadata-only. The gating code exists and logged `payloads: full` correctly on loopback, but it is not test-covered — and stage 1 already showed what happens when only the happy path is exercised. Refs #320 (stage 2 of 5). Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
Langfuse had no entry point — an operator had to know it existed and type
localhost:3002. It now sits in the /ops nav beside "Manage users" and
"Audit trail", where operators already look.
- An external <a>, not a next/link: Langfuse is a SEPARATE self-hosted app on
its own port, not a route in this Next app.
- target=_blank + rel=noopener so the console is not lost on the way out.
- The URL is NEXT_PUBLIC_LANGFUSE_URL with a :3002 default (3000 = admin,
3001 = metabase), so a deployment that moves it does not patch a component.
- Shown unconditionally. The `langfuse` compose profile is opt-in, so the
service is often down; a link that explains itself on hover ("opt-in: pnpm
langfuse:up") beats an entry point that silently vanishes.
Also corrects the Redis service card, which still read "Reserved — not in use
this phase". Langfuse queues on Redis as of stage 1, so that was now false on
the very page an operator consults to see what is running.
Verified in the running admin as a signed-in operator (the page is
client-rendered behind auth, so an unauthenticated fetch only returns the
shell and cannot confirm this): the link renders as "Observability ↗" with
href http://localhost:3002 and target=_blank, and the Redis card shows the
corrected text. type-check and lint green.
Refs #320 (part of stage 5).
Co-Authored-By: Claude Opus 5 <noreply@anthropic.com>
This file contains hidden or bidirectional Unicode text that may be interpreted or compiled differently than what appears below. To review, open the file in an editor that reveals hidden Unicode characters.
Learn more about bidirectional Unicode characters
Sign up for free
to join this conversation on GitHub.
Already have an account?
Sign in to comment
1 participant
Add this suggestion to a batch that can be applied as a single commit.This suggestion is invalid because no changes were made to the code.Suggestions cannot be applied while the pull request is closed.Suggestions cannot be applied while viewing a subset of changes.Only one suggestion per line can be applied in a batch.Add this suggestion to a batch that can be applied as a single commit.Applying suggestions on deleted lines is not supported.You must change the existing code in this line in order to create a valid suggestion.Outdated suggestions cannot be applied.This suggestion has been applied or marked resolved.Suggestions cannot be applied from pending reviews.Suggestions cannot be applied on multi-line comments.Suggestions cannot be applied while the pull request is queued to merge.Suggestion cannot be applied right now. Please check back later.
Stages 1, 2 and the entry-point slice of stage 5 from #320. Self-hosted Langfuse v3 for LLM tracing, cost and latency analytics.
Why: the recall pipeline was a black box. The #319 query-embedding bug took manual pgvector cosine measurements to find; a trace would have shown it instantly.
What lands
langfusecompose profile (web, worker, ClickHouse, MinIO) — opt-in likemetabase, sopnpm infra:upstays exactlynats postgres redis. Reuses the existing Postgres (dedicatedlangfuseDB, mirroringmetabase-db-init) and the existing Redis.platform/observability/langfuse.ts— off by default (no keys → no-op), loopback-only payloads reusingisLoopbackUrl, unconditional redaction forisPrivate, fail-open emits.traceGenerationon bothcomplete()paths — one seam covers every LLM call, since ADR-0002 makes the router the chokepoint.Observability ↗link in the Operations Console; corrects the Redis card, which claimed "not in use" while Langfuse queues on it.Two stage-1 defects green healthchecks hid — all containers healthy, UI 200, 72 tables migrated, and zero traces could persist: the MinIO bucket was never created (Langfuse doesn't create it), and
REDIS_HOST/REDIS_PORTmade Langfuse send AUTH to a password-less Redis. Fixed with an idempotentlangfuse-minio-initandREDIS_CONNECTION_STRING.Verified end-to-end against the running stack (Ollama
gemma3:4b):pnpm langfuse:up→ all services healthy, UI +/api/public/health200, 72 tables migrateddocker compose config --servicesunchanged (nats postgres redis)Refs #320 (stages 3–4 remain). Does not close it.
🤖 Generated with Claude Code