Skip to content

feat: surface a coding-agent handoff prompt on wizard exit#595

Merged
gewenyu99 merged 2 commits into
PostHog:mainfrom
ethangui:feat/handoff-outro
Jun 15, 2026
Merged

feat: surface a coding-agent handoff prompt on wizard exit#595
gewenyu99 merged 2 commits into
PostHog:mainfrom
ethangui:feat/handoff-outro

Conversation

@ethangui

@ethangui ethangui commented Jun 8, 2026

Copy link
Copy Markdown
Contributor

Problem

Refs #447. The wizard finishes by writing posthog-setup-report.md, but there's a gap between "wizard done" and "PostHog merged" — the developer (or their coding agent) has to rediscover the remaining verification + glue steps.

This is the wizard half; the verification content lives in context-mill (PostHog/context-mill#174). Supersedes #448 (reworked per @gewenyu99's review — content moved to context-mill, wizard slimmed to the handoff).

Changes

  • OutroData.handoffPrompt, set per-program in buildOutroData, built by buildCodingAgentPrompt.
  • Printed to the terminal's main buffer on exit (getExitLine, extracted to exit-line.ts).

Builds on main's recently-merged getUI().setOutroData() outro delivery — the built OutroData (including handoffPrompt) reaches the store through that path, so no extra plumbing is needed here.

Why this shape:

  • Printed to scrollback on exit, not rendered in the TUI. The TUI runs in the terminal's alternate screen buffer, wiped on exit — an in-TUI handoff vanishes when the user continues, and framing it in a box makes it un-copyable (selection grabs the border chars). A plain, unframed line in the main buffer after releaseTerminal() is the only thing that survives in scrollback and triple-click-selects cleanly. An in-TUI version was tried first and reverted for this reason.
  • The prompt asks for explicit consent before changes, but does not prescribe a workflow. "investigate each item, then list the changes you'd make and get my approval before applying any of them." Some checklist actions have real implications (e.g. source-map upload exposes source to PostHog), so the operator's agent should surface them for approval, not apply silently. It stops short of dictating how changes land (no "open a PR", no edit-style rules) — that's the operator's domain, in their own agent.
  • handoffPrompt is a narrow, program-set OutroData field — product copy stays in the program's buildOutroData; infra (exit-line) stays product-agnostic and renders an optional string.
  • No second file, no deterministic per-integration map — the verification content is integration knowledge, so it lives in context-mill markdown, not wizard code.

Sequencing: depends on PostHog/context-mill#174 merging + releasing first — the prompt references the "Verify before merging" checklist the context-mill skill writes.

Test plan

  • Unit / static: pnpm build, pnpm test (819 passing), pnpm lint all green. New tests cover the exit-line handoff rendering (exit-line) and the prompt wording/consent (handoff).
  • Review: multi-agent code review + a codex second-opinion pass on both repos. The one finding — a dev-only skills-URL env override that widened a trust boundary — was dropped from this PR (kept local-only).
  • Live end-to-end (real OAuth + agent + a real PostHog project, against a Next.js 15 App Router SaaS app):
    • the handoff prints to scrollback and triple-click-selects cleanly;
    • basic run → checklist contains env-var mirroring, source-maps, and returning-visitor identify; $ai_generation correctly absent;
    • LLM-Analytics run → agent set up OpenTelemetry @posthog/ai instrumentation and the checklist correctly gained the $ai_generation item.

LLM context

Co-authored with Claude Code (Opus 4.8). Reviewed via the multi-agent + codex passes noted above before opening. Follows this repo's PR template and the PostHog CONTRIBUTING / AI policy.

cc @PostHog/team-docs-wizard

🤖 Generated with Claude Code

@chatgpt-codex-connector chatgpt-codex-connector Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

💡 Codex Review

Here are some automated review suggestions for this pull request.

Reviewed commit: d7f5fbafc8

ℹ️ About Codex in GitHub

Your team has set up Codex to review pull requests in this repo. Reviews are triggered when you

  • Open a pull request for review
  • Mark a draft as ready
  • Comment "@codex review".

If Codex has suggestions, it will comment; otherwise it will react with 👍.

Codex can also answer questions or update the PR. Try commenting "@codex address that feedback".

Comment thread src/lib/agent/agent-runner.ts Outdated
@ethangui

ethangui commented Jun 8, 2026

Copy link
Copy Markdown
Contributor Author

Hey @gewenyu99 — the reworked split is up per your review: this PR (#595, the wizard handoff) and PostHog/context-mill#174 (the "Verify before merging" checklist content).

One thing: I added cc @PostHog/team-docs-wizard to both, but since I'm an external contributor opening from a fork, GitHub won't link or notify a team mention from a non-member — so it won't actually reach them. Could you loop the team in for review on both? (You noted these sometimes get missed.) Thanks!

@gewenyu99 gewenyu99 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

:stamp: I am going to stamp the vibe of this code and promise to test this locally soon. Thanks for making these changes!

Comment thread src/ui/tui/start-tui.ts
@codecov-commenter

codecov-commenter commented Jun 8, 2026

Copy link
Copy Markdown

⚠️ Please install the 'codecov app svg image' to ensure uploads and comments are reliably processed by Codecov.

Codecov Report

✅ All modified and coverable lines are covered by tests.
⚠️ Please upload report for BASE (main@84b7aad). Learn more about missing BASE report.
❗ Your organization needs to install the Codecov GitHub app to enable full functionality.

Additional details and impacted files
@@           Coverage Diff           @@
##             main     #595   +/-   ##
=======================================
  Coverage        ?   49.25%           
=======================================
  Files           ?      215           
  Lines           ?     8665           
  Branches        ?     2043           
=======================================
  Hits            ?     4268           
  Misses          ?     4394           
  Partials        ?        3           
Flag Coverage Δ
unit-tests 49.25% <100.00%> (?)

Flags with carried forward coverage won't be shown. Click here to find out more.

☔ View full report in Codecov by Harness.
📢 Have feedback on the report? Share it here.

🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

@ethangui ethangui force-pushed the feat/handoff-outro branch from c3aca99 to b363c53 Compare June 8, 2026 16:27
@ethangui

ethangui commented Jun 8, 2026

Copy link
Copy Markdown
Contributor Author

Thanks @gewenyu99! Two notes for your local test:

Sequencing — this depends on PostHog/context-mill#174 landing/releasing first. The handoff points at the "Verify before merging" checklist the context-mill skill writes, so against the currently-published skill that section won't exist yet.

How I tested it (there's probably a better/standard way — happy to switch): I ran context-mill's dev server (pnpm dev, serves skills on localhost:8765) and pointed the wizard's skill source there while leaving the MCP on production. --local-mcp would also redirect the MCP, so I used a small local-only override on getSkillsBaseUrl (not in this PR) to get local-skills + real-MCP. The report then picked up the checklist and the handoff printed on exit.

@ethangui ethangui force-pushed the feat/handoff-outro branch from b363c53 to 24ceb6f Compare June 9, 2026 23:10
@gewenyu99

Copy link
Copy Markdown
Collaborator

Will get to this I :pinky-promise: soon

@ethangui ethangui force-pushed the feat/handoff-outro branch 2 times, most recently from 14620c5 to e182bf3 Compare June 11, 2026 18:34

@gewenyu99 gewenyu99 left a comment

Copy link
Copy Markdown
Collaborator

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Screenshot 2026-06-12 at 5 22 13 PM Screenshot 2026-06-12 at 5 25 18 PM

Great place to start. I'll merge these for ya.

I like where this is going and I'll spend some time next quarter really improving both handoff and reports here.

@gewenyu99

Copy link
Copy Markdown
Collaborator

@ethangui Last thingm can you sign your commits? 🙏
Screenshot 2026-06-12 at 5 35 11 PM

Add OutroData.handoffPrompt, set per-program in buildOutroData and
printed to the terminal's main buffer on exit (getExitLine). Rides on
main's getUI().setOutroData() delivery, so the prompt reaches the outro
data without any extra plumbing. Verification content lives in
context-mill.

Refs PostHog#447.

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
@ethangui ethangui force-pushed the feat/handoff-outro branch from 9125d3e to 0fb7660 Compare June 12, 2026 22:10
@ethangui

Copy link
Copy Markdown
Contributor Author

@ethangui Last thingm can you sign your commits? 🙏 Screenshot 2026-06-12 at 5 35 11 PM

commits signed 👍

@gewenyu99 gewenyu99 merged commit 9762ff3 into PostHog:main Jun 15, 2026
13 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants