test(platform): prove the Langfuse privacy gating (#325)

[jayesh-keychain]

What and why

#320 stage 2 shipped three privacy guarantees for Langfuse tracing with zero test
coverage. This adds the coverage the issue asked for — and writing it found that
two of the three guarantees did not actually hold.

That is the issue's own thesis proving itself: green healthchecks proved nothing
about stage 1's behaviour, and untested gating was the same trap. Trace input and
output are prompts and completions, i.e. memory text, so these are §2.2 holes.

The two holes (both fixed here)

1 · LANGFUSE_ALLOW_REMOTE_PAYLOADS=false meant true

The flag was z.coerce.boolean(), which is Boolean(value). An env var is always a
string, so every non-empty value is truthy:

"false" -> true      "0"  -> true
"no"    -> true      "on" -> true

An operator writing LANGFUSE_ALLOW_REMOTE_PAYLOADS=false — the most natural way to
express "keep memory text on-box" — would have enabled full payloads to a remote
Langfuse. The flag is the §2.2 escape hatch; inverting it silently is the whole of
failure mode M9.

Replaced with a strict envBoolean accepting only 1/true/yes/on and
0/false/no/off/"", and hard-failing on anything else. Failing closed and loud at
boot beats a security gate whose state nobody can predict.

Not currently triggered on this machine (base URL is loopback and the flag is
unset), but it was one .env edit away.

2 · The isPrivate redaction was unreachable

safePayload redacts isPrivate content unconditionally — the module's strongest
rule. But CompleteArgs had no isPrivate field and traceGeneration never passed
one, so no caller could ever reach that branch. The guarantee was a comment.

Threaded the flag through CompleteArgs → traceGeneration → safePayload.

Worth being precise about the live risk: recall does not leak today, because
memory.service.ts filters private memories out of contexts upstream
(publicHits) before the router sees them. The fence held by accident of ordering,
one layer up. Any future routed call that handles private content would have had no
working fence at all — now it does, and isPrivate: true is the documented way to
ask for it.

Plus a hermeticity gap (AC5)

vitest.config.ts pins every AI provider to its fake so the suite stays offline —
but not Langfuse, which keys off the presence of two env vars. On any machine with
Langfuse keys in .env (including this one), pnpm test built a real client and
emitted traces of test data into it. Both keys are now blanked under test.

Design note

The gating functions now take an injectable TracingConfig defaulting to env,
mirroring resolve(class, keys) in the model router — the repo's existing posture of
injection over mocking. Production call sites are unchanged; the suite drives every
host/flag/private combination without touching global env or the network.

setLangfuseClient() is a test seam shaped exactly like the router's existing
setBudgetTracker(). It is needed because tracing is correctly off in tests, so
without it there is no way to assert what would have been sent.

Acceptance criteria

• A test fails if safePayload stops redacting isPrivate content.
  Mutation-checked: deleting the isPrivate branch turns 5 tests red across two files. Covered on loopback, on a remote host, and with the remote opt-in set — the redaction must win in all three.
• A test fails if a non-loopback host starts sending payloads without the explicit opt-in.
  Mutation-checked: making payloadsAllowed return true turns 5 tests red. Reverting envBoolean to z.coerce.boolean() turns 4 more red. Also covers the localhost.evil.com near-miss and an unparseable URL (both must fail closed).
• A spy-provider test proves private memory text never reaches the Langfuse client on a routed call.
  langfuse-trace.test.ts injects a spy client, drives a real complete() with isPrivate: true, and asserts the private string appears nowhere in anything handed to it. It also asserts the spy was called and that non-private content does get through — so the assertion can't pass vacuously. Mutation-checked: removing the threading turns 2 tests red.
• A throwing Langfuse client does not fail or reject the LLM call.
  Spy stubbed to throw on trace(): complete() still resolves with a non-empty answer, and the error is logged (langfuse.emit-failed) not propagated. Also covered for the no-client case.
• With no keys set, no test touches the network.
  Required the vitest.config.ts fix above. isTracingEnabled() and getLangfuse() are asserted against the real process env inside the suite, so the harness itself is under test — that assertion fails if someone unblanks the keys.

Verification summary

pnpm type-check   7 successful, 7 total
pnpm lint         7 successful, 7 total   (0 errors, 0 warnings)
pnpm build        5 successful, 5 total
pnpm test         5 successful, 5 total  —  144 files, 1018 tests passed, 0 failed
                    @thebrain/api     131 files / 871 tests   (was 128 / 842)
                    @thebrain/core      6 files /  90 tests
                    @thebrain/admin     7 files /  57 tests

29 new tests, all written red-first. Every AC was mutation-checked rather than
just asserted — for each guarantee I reverted the code and confirmed the suite goes
red, then restored it. A test that cannot fail proves nothing, which is precisely the
gap this issue existed to close.

Scope note

The issue was framed as test-only ("the code exists; nothing proves it works"). Two
of its ACs could not be satisfied without the fixes above — a spy test asserting
private text never reaches Langfuse would have failed, and one asserting the remote
gate would have passed for the wrong reason. Fixing them is ~20 lines and squarely
the point of a type: security ticket, so I did rather than filing a follow-up.
Flagging it because it makes this a test + fix PR rather than pure coverage.

Invariant check

• No new queries, no migrations, no new external deps.
• No new off-box call — this narrows an existing one.
• isPrivate is optional and defaults to undefined, so every existing complete()
  caller is unaffected (871 API tests pass unchanged).
• Tests are hermetic: keys: {} forces the fake provider; the Langfuse client is a
  local spy; no new Date() around time-dependent logic.

Closes #325

Co-authored-by: Claude noreply@anthropic.com