Skip to content

Conversation

@RMartires
Copy link
Contributor

@RMartires RMartires commented Jan 18, 2026

Issue Link: #262

sets up trace logging in postHog

  • set up strix scans trace logging in postHog controlled via POSTHOG_LLM_API_KEY and POSTHOG_LLM_HOST env
  • this will send llm req/res directly from LLMLite to postHog
  • this will improve visibility for users to debug any flow
  • added support for 2 postHog accounts: the hardcoded account still gets the events (non llm data), and the env provided account gets the traces (llm data)
image

@greptile-apps
Copy link
Contributor

greptile-apps bot commented Jan 18, 2026

Greptile Summary

This PR adds PostHog tracing for LLM requests/responses to improve debugging visibility. The implementation introduces a dual-account system where the hardcoded primary account receives custom events (non-LLM telemetry) and an optional environment-configured account receives LLM traces.

Key changes:

  • Added configure_litellm_posthog() to set up LiteLLM callbacks for PostHog trace collection
  • Integrated trace metadata ($ai_trace_id, user_id, agent_id, agent_name) across all LLM call sites (llm.py, dedupe.py, memory_compressor.py)
  • Renamed constants to distinguish between primary and LLM PostHog accounts
  • Improved streaming logic in llm.py to continue yielding content after function calls instead of breaking early
  • Minor formatting improvement in state.py

Issues found:

  • Critical logic bug in posthog.py:38 - uses or instead of and for env var validation, which will cause runtime error if only one env var is set

Confidence Score: 2/5

  • This PR has a critical logic bug that will cause runtime errors in production
  • The logic error in env var validation (or vs and on line 38) will cause configure_litellm_posthog() to attempt setting environment variables to None values when only one of the two required env vars is set. This will likely cause LiteLLM integration to fail or behave unexpectedly. The rest of the implementation is solid with proper error handling and metadata propagation.
  • strix/telemetry/posthog.py requires immediate attention to fix the logic bug before merge

Important Files Changed

Filename Overview
strix/telemetry/posthog.py Added dual PostHog account support with LLM tracing configuration, but has logic bug in env var validation that will cause runtime error
strix/llm/llm.py Added PostHog tracing metadata to LLM calls and improved streaming logic to continue after function calls instead of breaking early
strix/llm/dedupe.py Added trace metadata to deduplication LLM calls for PostHog tracking
strix/llm/memory_compressor.py Added trace metadata to memory compression LLM calls for PostHog tracking
strix/agents/state.py Minor formatting change to method signature for better readability

Copy link
Contributor

@greptile-apps greptile-apps bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

5 files reviewed, 1 comment

Edit Code Review Agent Settings | Greptile

rohit.martires and others added 3 commits January 20, 2026 11:20
Co-authored-by: greptile-apps[bot] <165735046+greptile-apps[bot]@users.noreply.github.com>
@RMartires RMartires force-pushed the feat/add-tracing-in-posthog branch from 06fe4ed to 99eb921 Compare January 20, 2026 05:50
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant