Skip to content

feat: add onboarding skill and command#19

Merged
vahidlazio merged 27 commits into
mainfrom
ai-onboarding
Jun 15, 2026
Merged

feat: add onboarding skill and command#19
vahidlazio merged 27 commits into
mainfrom
ai-onboarding

Conversation

@vahidlazio

@vahidlazio vahidlazio commented Jun 9, 2026

Copy link
Copy Markdown
Collaborator

Summary

  • Add onboard-confidence skill for creating Confidence accounts, inviting users, setting up SDK clients, configuring warehouses, and learning experimentation concepts
  • Add /confidence:onboard-confidence slash command
  • Update CLAUDE.md with onboarding command and skill references

Multi-client plugin support (Cursor, Codex, Gemini CLI) has been moved to #20.

Test plan

  • Run /confidence:onboard-confidence status to verify skill loads
  • Run /confidence:onboard-confidence create-account flow
  • Run /confidence:onboard-confidence setup-wizard flow
  • Verify CLAUDE.md instructions are picked up by Claude Code

🤖 Generated with Claude Code

vahidlazio and others added 18 commits June 9, 2026 14:04
Expand the plugin to support Cursor, Codex, and Gemini CLI alongside
Claude Code. Add onboard-confidence and migrate-optimizely skills
and commands.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Keep this branch focused on onboarding skill and multi-client support.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Add Snowflake, Databricks, and Redshift query support in the
data pipeline verification step. Previously only BigQuery was covered.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Report exact error messages from the warehouse validation API instead
of optimistically splitting them into "works" + "but". Adds
Snowflake-specific remediation guidance for auth and permission errors.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…t table APIs

- Warehouse create: body is the object directly, not wrapped in dataWarehouse key
- Connectors: body is the object directly, not wrapped in flagAppliedConnection/eventConnection
- Assignment table: body is the object directly, not wrapped in assignmentTable
- Snowflake connectors require database and schema fields in snowflakeConfig
- Pipeline verify step now lists clients and lets user pick before creating credentials
- Snowflake remediation generates SQL and copies to clipboard

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Users don't know what a crypto key reference is. The skill now:
- Creates the crypto key via IAM API automatically
- Extracts the public key and generates ALTER USER SQL
- Copies the SQL to clipboard for the user to run in Snowflake
- Generates CREATE DATABASE/SCHEMA SQL if needed

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Each config field now explains what it is, where to find it in the
warehouse UI, and what to do if it doesn't exist yet. Covers
BigQuery, Snowflake, Databricks, and Redshift.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Note that validate endpoint only supports BigQuery and Snowflake
- Databricks connectors need connectionConfig wrapper + batchFileConfig
- Skip validation for Databricks/Redshift and go straight to create

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…me issue

Tested every field name variation — the validate endpoint genuinely
only supports BigQuery and Snowflake. Skill now explains this honestly
rather than silently skipping.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Connector returns 500 even with valid credentials (verified OAuth
directly). Note as known issue and suggest contacting support.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Root cause found in logs: IllegalArgumentException "S3BucketConfig needs
to be set". Databricks connectors batch-write to S3 then load into
Databricks. Added s3BucketConfig (bucket, region, roleArn) to connector
format and Step 2 config collection.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Explains upfront that Databricks requires AWS S3 staging bucket,
Databricks admin access, and a schema. Walks through each step with
exact UI paths, commands, and explains why each piece is needed.
Covers trust policy errors and Unity Catalog vs hive_metastore.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Skill can now create S3 bucket, IAM role, and trust policy
automatically via aws CLI. Falls back to manual console steps
if aws CLI not available.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Three bugs found during Databricks testing:
1. accounts.google.com:email rejected by AWS ("requires application id")
2. accounts.google.com:sub with email string accepted but fails at
   runtime ("Not authorized to perform sts:AssumeRoleWithWebIdentity")
3. Only accounts.google.com:sub with the numeric unique ID works

Skill now fetches numeric ID via gcloud and documents the pitfall.
Also: write aws credentials directly instead of interactive configure,
install awscli via brew if missing, open console for login.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Data flows via GCS not S3 (S3 is fallback), fixed flow diagram
- Added ~5 min batch delay warning
- Databricks verification now uses SQL Statement API directly
  instead of just showing queries for the user to run
- Removed stale "known backend issue" — pipeline works end-to-end

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
- Ask one question at a time, confirm before moving on
- Explain every field in plain language with exact UI paths
- Service principal: full step-by-step with fallback for non-admins
- S3 staging: explain the "mailbox" metaphor for why it's needed
- Schema: detect Unity Catalog vs hive_metastore, copy SQL to clipboard
- Accurate data flow explanation (GCS staging, 5-min batches)

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Verified S3 bucket has parquet files: assignments and events staged
there before Databricks COPY INTO. Actual flow is:
Confidence → S3 bucket (customer) → Databricks tables.
GCS is internal only. Removed contradictory "GCS primary" claim.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Move Cursor, Codex, and Gemini CLI plugin configs to the dedicated
feat/multi-client-plugins branch for a separate PR.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@vahidlazio vahidlazio changed the title feat: add multi-client support and onboarding skills feat: add onboarding skill and command Jun 10, 2026
vahidlazio and others added 7 commits June 10, 2026 13:44
Findings from end-to-end Redshift testing:
- Validate endpoint DOES support Redshift (fixed incorrect note)
- One IAM role must be trusted by both Google OIDC and Redshift service
- Role must be attached to cluster for COPY command to work
- GRANT USAGE ON SCHEMA TO PUBLIC required or validation returns "not found"
- Redshift Serverless not supported (needs provisioned cluster identifier)
- S3 staging bucket required (same pattern as Databricks)
- Full aws CLI automation for creating cluster, role, bucket, schema

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
…rackers

Redshift:
- Full step-by-step for someone who's never used AWS or Redshift
- Explains what Redshift is, why S3 is needed, what IAM roles do
- One question at a time, confirm before moving on
- Auto-creates cluster, bucket, role, schema via aws CLI
- Manual AWS Console fallback path
- Explains GRANT USAGE requirement
- Explains why Serverless won't work

Step trackers:
- Each warehouse type gets its own tracker showing actual sub-steps
- BigQuery: 10 steps, Snowflake: 12, Databricks: 13, Redshift: 13
- Tracker updates after every step so user sees progress

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Three changes:

1. Split setup-warehouse into per-type skills for LLM efficiency:
   - setup-warehouse/ (thin dispatcher, ~190 lines)
   - setup-warehouse-bigquery/ (~700 lines)
   - setup-warehouse-snowflake/ (~750 lines)
   - setup-warehouse-databricks/ (~860 lines)
   - setup-warehouse-redshift/ (~870 lines)
   Each skill is self-contained with auth, API refs, and step tracker.
   Onboard-confidence SKILL.md reduced from 2254 to 1152 lines.

2. Remove token persistence — tokens kept in session only, never
   written to disk. Browser login on every new session or expiry.

3. Add dry-run skill (~1380 lines) for testing UX without real APIs.
   Simulates all sub-commands with mock data, [DRY RUN] prefix,
   same step trackers and questions as real skills.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Dual-layer telemetry:
- Layer 1: Skill-embedded events (works on all AI clients)
- Layer 2: Claude Code hooks (PostToolUse + SessionEnd)

New files:
- skills/telemetry/TELEMETRY.md: shared reference with event catalog,
  privacy rules, emit helper, step name registry, sentiment detection,
  identity linking, feedback prompt, session abandonment
- hooks/hooks.json: PostToolUse for API call capture, SessionEnd for
  abandonment detection
- hooks/telemetry_hook.py: detects curl to confidence.dev, emits
  tool.api_call events
- hooks/session_end_telemetry.py: emits session_abandoned if no
  session_completed

Event types: session_started, step_started/completed/failed,
user_choice, session_completed, identity_linked, sentiment,
session_sentiment, feedback, feedback_text, session_abandoned,
warehouse.*, tool.api_call

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Move auth.py into the skill directory so it ships with the plugin
instead of being written to temp files at runtime. Update SKILL.md
to go straight to the setup wizard when triggered without a
sub-command, and refine auth flow to use the bundled script.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Sandboxed Bash calls use /tmp/claude-501/ while dangerouslyDisableSandbox
calls use the system TMPDIR. Tokens saved in sandboxed calls were invisible
to non-sandboxed curl calls, causing "Invalid JWT" errors.

All token writes/reads now require dangerouslyDisableSandbox: true.
Test resolve now passes context fields and tests all targeting cases.

Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@vahidlazio vahidlazio marked this pull request as ready for review June 12, 2026 14:47
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
Co-Authored-By: Claude Opus 4.6 (1M context) <noreply@anthropic.com>
@vahidlazio vahidlazio merged commit 2da1d97 into main Jun 15, 2026
1 check passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

2 participants