Skip to content

feat(lasso): intent double-duty on the classify call - #67

Open
orgersh92 wants to merge 2 commits into
litellm_internal_stagingfrom
feat/intent-double-duty
Open

feat(lasso): intent double-duty on the classify call#67
orgersh92 wants to merge 2 commits into
litellm_internal_stagingfrom
feat/intent-double-duty

Conversation

@orgersh92

Copy link
Copy Markdown

What

Adds intent double-duty to the LiteLLM Lasso custom guardrail. When an app funnels its LLM calls through LiteLLM and seeds per-turn intent headers (via the lasso-sdk GatewayIntent primitive), the plugin's existing /classify call now also carries the per-turn intent signals — so content-safety and intent ride one call, with no separate exporter or second Lasso API key in the app.

Inert when no x-lasso-trace-id is present (a plain guardrail call), so it ships dark until the app starts seeding.

Change

On each classify call the plugin reads the app-seeded headers off the proxy request and stamps the classify payload's messages with intent ids:

Header Meaning
x-lasso-trace-id per-turn trace id (a ULID); groups the turn's events
x-session-id conversation id → sessionId
x-lasso-application-intent seeds sessionInformation.applicationIntent
x-lasso-application-name seeds sessionInformation.agenticAppName

(x-lasso-encoding: pct → percent-decode the baseline values.)

The server derives the intent signal from role × content-kind, so the plugin sends blocks, not signals — reusing the guardrail's existing tool translation (tool_use under model, tool_result under developer, input as an object). Model reasoning is harvested from reasoning_content / thinking_blocks into a reasoning block. Also now sends source={type:"litellm"} for the "Used By" attribution badge.

Ordering + idempotency (the interesting part)

The gateway re-sends the whole conversation history on every turn, so the server dedups on eventId. Ids must therefore be deterministic — derived from (traceId, eventIndex) — so a re-send at the same position collapses instead of duplicating. Reproducible events (text / tool_use / tool_result) are indexed on a stride (p × 10); COMPLETION-only reasoning (never re-sent in history) fills just above the previous reproducible event, off the stride grid, so it sorts before the answer it explains without shifting the re-sendable events. The PROMPT phase caches its reproducible-event count (keyed by litellm_call_id) so the COMPLETION phase continues after it. This mirrors the Kong ai-lasso-guardrail plugin (the reference intent implementation).

Verification

  • Pure id/index logic (stride ordering, deterministic ids, cross-turn idempotency, reasoning gap-slotting, unslottable-leading-reasoning drop) verified against the identical scheme used by the Kong plugin and the server-side LiteLLM shim.
  • Unit tests added in tests/guardrails_tests/test_lasso_guardrails.py (inert-without-header, placeholder-ignored, stride stamping + deterministic ULID ids, pct-decoded session information, completion index continuation + reasoning harvest). Run in CI.

🤖 Generated with Claude Code

Stamp per-turn traceId/eventIndex/eventId + sessionInformation from app-seeded
x-lasso-* headers so the existing /classify call also feeds Lasso's intent pipeline.
Stride-indexed, deterministic ids (idempotent re-sends), reasoning harvested from
reasoning_content/thinking_blocks. Inert without a seeded trace id. Also sends
source={type:litellm}.
@coderabbitai

coderabbitai Bot commented Jul 13, 2026

Copy link
Copy Markdown

Important

Review skipped

Auto reviews are disabled on base/target branches other than the default branch.

Please check the settings in the CodeRabbit UI or the .coderabbit.yaml file in this repository. To trigger a single review, invoke the @coderabbitai review command.

⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro

Run ID: de07c1fc-ad3b-4461-86cc-3957d141552a

You can disable this status message by setting the reviews.review_status to false in the CodeRabbit configuration file.

Use the checkbox below for a quick retry:

  • 🔍 Trigger review
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests
  • Commit unit tests in branch feat/intent-double-duty

Comment @coderabbitai help to get the list of available commands.

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant