Skip to content

feat: attribute web signups to the CLI via UTM parameters - #234

Open
eze-githits wants to merge 6 commits into
mainfrom
feat/cli-utm-attribution
Open

feat: attribute web signups to the CLI via UTM parameters#234
eze-githits wants to merge 6 commits into
mainfrom
feat/cli-utm-attribution

Conversation

@eze-githits

Copy link
Copy Markdown
Collaborator

Tags the OAuth authorization URL the CLI opens with the five standard utm_* parameters, so PostHog can attribute web signups to githits login and githits init.

No new dependencies, no network calls from the CLI, no analytics SDK bundled.

Parameters

Parameter Value
utm_source githits-cli
utm_medium cli
utm_campaign cli-login or cli-init
utm_content CLI version

Only the five standard slots are used, because that is the exact set posthog-js auto-captures (CAMPAIGN_PARAMS) under the current frontend config. A custom name would need custom_campaign_params on the frontend and would capture nothing until that shipped.

Scope — please read before interpreting any number

This measures auth-page arrival → OAuth completion. It does not measure command execution → completion.

A UTM only exists once a browser requests the URL, so anyone who runs the command and never reaches the auth page produces no signal and is absent from the denominator. Numbers from this are a lower bound, not a conversion rate.

Signup vs sign-in does not come from the UTM. The URL is built before the user authenticates, so at tagging time it is not yet known whether an account will be created. The UTM carries origin only; the outcome comes from the frontend's own signup event, joined on the person at query time.

The frontend runs posthog-js with cookieless_mode: "on_reject", under which users who reject cookies have no $anon_distinct_id to link to their signup, and users who never answer the banner have their events dropped entirely. Treat the result as directional and comparative (cli-init vs cli-login, movement across releases) rather than absolute.

Full analysis, with citations to posthog-js source: docs/superpowers/specs/2026-07-27-cli-utm-attribution-design.md.

Notes on the implementation

  • The URL is tagged once, immediately after buildAuthUrl, and the tagged value is used in all three places it is consumed: the browser open, the --no-browser printout, and the "browser did not open" fallback. Tagging only the browser path would drop attribution for SSH users and for anyone whose browser failed to launch — the segments most likely to abandon, which would bias the funnel upward.
  • Tagging sits above buildAuthUrl rather than inside it. OAuth parameters are correctness-critical while attribution is best-effort; they should not share a failure mode. withUtm never throws and returns the URL unchanged if it does not parse.
  • The CLI hardcodes no login URL. It discovers the authorization endpoint via OAuth metadata, so the parameters are appended to whatever discovery returns.

Blocking verification before this measures anything

The CLI side is complete but inert until confirmed, and none of it is verifiable from this repo:

  1. The authorization endpoint preserves the query string through its redirect chain.
  2. PostHog is initialised on a page the flow actually reaches.
  3. The frontend signup event fires with identify() so the anonymous visit links to the account.

Two-minute check, no deploy needed: run githits login --no-browser, open the printed URL, and watch the address bar. If it passes through a frontend page with the utm_* still present, client-side capture works. If it jumps straight to github.com/login/oauth/authorize, the attribution has to be recorded server-side instead.

Worth inspecting first: the frontend already detects initial CLI install. If that works off a URL parameter, the UTMs should ride alongside it rather than duplicate it.

Testing

  • src/shared/utm.test.ts — 5 unit tests (idempotency, unparseable input, parameter preservation, optional content)
  • src/commands/login.test.ts — 4 tests covering both the opened and the printed URL, and that OAuth parameters survive tagging
  • src/commands/init/init.test.ts — 1 test asserting cli-init on the printed URL

bun test on the touched suites: 186 pass. bun run build, bun run typecheck and bun run smoke:cli pass.

One unrelated pre-existing failure remains on main: exec-service.test.ts > rejects with error for non-existent command, verified to fail with this branch stashed.

🤖 Generated with Claude Code

eze-githits and others added 6 commits July 27, 2026 12:19
Records what the UTM approach can and cannot measure: auth-page arrival
through OAuth completion, not command execution through completion.

Documents two constraints found while verifying against posthog-js.
Signup versus sign-in cannot ride on the UTM, because the URL is built
before the user authenticates and the outcome does not exist yet; it has
to be joined from the frontend signup event. And cookieless_mode
on_reject drops events entirely before a consent decision and leaves no
anon_distinct_id after a rejection, so the funnel is a lower bound on a
consenting subset rather than a conversion rate.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Pure URL tagging with the five standard utm_* parameters, which is the
exact set posthog-js auto-captures in CAMPAIGN_PARAMS. A custom parameter
name would need explicit handling on the web side, so the helper is
confined to those slots.

Idempotent so pre-existing attribution is never overwritten, and
fail-safe on unparseable input so attribution can never break sign-in.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Applies utm_source/medium/campaign/content to the authorization URL once,
before it is opened or printed, so the SSH and browser-failure paths carry
the same attribution as the happy path. Those are the segments most likely
to abandon, so tagging only the browser call would bias the funnel upward.

Tagging sits above buildAuthUrl rather than inside it: OAuth parameters
are correctness-critical while attribution is best-effort, and the two
should not share a failure mode.

The CLI discovers its authorization endpoint via OAuth metadata rather
than hardcoding a login URL, so the parameters are appended to whatever
the discovery returns.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
Passes entrypoint: init so utm_campaign distinguishes the two commands.
The populations behave differently: someone running login already knows
what they want, while someone reaching auth through init is mid-IDE-setup
and far likelier to abandon. Without the split the funnel averages two
unlike groups into an unactionable number.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
States plainly that the funnel measures auth-page arrival rather than
command execution, that signup versus sign-in cannot come from the UTM
because the URL is built before the user authenticates, and that
cookieless_mode on_reject makes the result a directional lower bound.
Without these the numbers read as a conversion rate, which they are not.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
The previous wording said a custom parameter name would not be captured,
which is only true under the default configuration. posthog-js exposes
custom_campaign_params to extend the captured list.

Records why the design still uses the five standard slots: a custom name
needs a coordinated frontend config change and captures nothing until that
ships, so standard slots keep the CLI side independent of a web deploy.

Co-Authored-By: Claude Opus 5 (1M context) <noreply@anthropic.com>
@eze-githits eze-githits added the enhancement New feature or request label Jul 27, 2026
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

enhancement New feature or request

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant