Skip to content

feat(settings): auto-fix CI failures with chat launcher hook#770

Merged
jamesbrink merged 12 commits into
mainfrom
doomspork/settings-ci-failure-prompt
May 18, 2026
Merged

feat(settings): auto-fix CI failures with chat launcher hook#770
jamesbrink merged 12 commits into
mainfrom
doomspork/settings-ci-failure-prompt

Conversation

@doomspork
Copy link
Copy Markdown
Member

@doomspork doomspork commented May 12, 2026

Summary

Automatically starts a CI auto-fix chat session when SCM polling observes a workspace branch transition into failed CI.

The implementation uses a split ownership model:

  • Rust SCM polling owns detection, per-repo/global settings, cooldown/dedup state, SCM plugin log fetching, prompt formatting, and DB session creation.
  • Frontend useCiAutoFixSession() hook owns the Tauri event subscription, adding/selecting the new session tab, and calling sendChatMessage.
sequenceDiagram
    participant Poll as Rust SCM polling loop
    participant DB as SQLite/app_settings
    participant Plugin as SCM plugin
    participant Hook as useCiAutoFixSession()
    participant Chat as Chat send path

    Poll->>Poll: derive_overall_ci_status(checks)
    Poll->>Poll: Detect observed transition to Failure
    Poll->>DB: Read global/per-repo settings + cooldown
    Poll->>Plugin: ci_failure_logs(branch, failed_checks)
    Plugin-->>Poll: Failure logs when supported
    Poll->>DB: create_chat_session(workspace_id)
    Poll->>Hook: emit ci-auto-fix-session-created
    Hook->>Hook: get session, add tab, select session
    Hook->>Chat: sendChatMessage(prompt, model/backend)
Loading

Key behavior

  • Adds a global CI auto-fix setting with model, prompt template, and cooldown controls. Cooldown is clamped to 60–3600 s on both the UI and the Rust side (parse_ci_auto_fix_cooldown_seconds).
  • Adds per-repository overrides for enablement and prompt template (repo:<id>:ci_auto_fix_enabled, repo:<id>:ci_auto_fix_prompt), prefetched once per poll cycle into an in-memory snapshot.
  • Triggers only on an observed non-failure → failure transition. The first poll after startup with no entry in ci_last_status establishes a baseline without firing (is_ci_failure_transition(None, _) == false).
  • The auto-fix model picker uses useModelRegistry(), so it applies the same codexEnabled / Pi-SDK / Claude-OAuth filters as every other chat-side selector and cannot save a model the Rust resolver would later reject.
  • When the auto-fix model is left at Use default model, the snapshot reads default_model + default_agent_backend as a pair, so a user whose global default is a non-Anthropic backend (Codex Native / Pi / OpenAI) gets the matching harness on the auto-fix session.
  • Fetches failure logs through SCM plugins when supported:
    • GitHub: gh run view --log-failed. Matches by check/job name first; falls back to the most recent failed runs when exact-name matching misses, since gh pr checks returns job names while gh run list returns workflow names. Capped at 3 runs to bound per-tick wall time.
    • GitLab: glab ci trace per failed job.
    • Both plugins check result.code before reading stdout and log stderr on failure.
  • Embeds log text in a Markdown fence sized to the longest backtick run found in the content (markdown_code_fence_for), so logs containing ``` don't break prompt formatting.
  • Degrades cleanly when a plugin does not support log fetching (OperationNotSupported ⇒ check names + URLs only).
  • Auto-fix sessions intentionally use backend defaults for permission level / thinking / plan / fast — a background-triggered session has no per-tab toolbar state to inherit. Only model, backend_id, and disable1mContext are passed through. The hook documents this in-line.

Reliability & dedup

  • Cooldown / dedup uses an in-memory ci_last_status: HashMap<workspace_id, CiTransitionState> on AppState.
  • After the polling loop attempts an auto-fix, next_ci_transition_state always records the observed overall_status — including when the attempt fails (DB open, create_chat_session, or Tauri event emit). This prevents a persistent failure from firing every 30 s. The cooldown stamp (last_auto_fix_triggered) only advances on success, so a genuine failure can still be retried on the next observed transition.
  • If handle.emit(...) fails after a session was created in SQLite, the just-created session is archived to avoid orphan tabs. The path still counts as "handled" so a transient emit failure doesn't spam retries.

Settings UI

Settings → Git → CI Automation:

  • Auto-fix CI failures toggle
  • Model (drops to default_model + default_agent_backend when unset)
  • Prompt template (blank uses the built-in default; placeholders: {{failed_checks}}, {{failure_logs}}, {{branch}}, {{pr_title}}, {{pr_url}}, {{pr_number}}, {{all_checks}})
  • Cooldown (seconds) with revert-on-failure
  • Per-repo overrides on Settings → Repositories → <repo>

Translations added under the settings namespace for all five shipped locales (en, es, ja, pt-BR, zh-CN).

Docs

  • site/src/content/docs/features/settings.mdx — rows in the Git settings table.
  • site/src/content/docs/features/scm-providers.mdx — cross-reference to the feature.
  • site/src/content/docs/features/per-repo-settings.mdx — per-repo override row.

Verification

  • nix develop -c cargo fmt --all --check
  • nix develop -c cargo clippy -p claudette -p claudette-server -p claudette-cli --all-targets --all-features (CI scope; -Dwarnings)
  • nix develop -c cargo test -p claudette-tauri commands::scm::ci_auto_fix_tests — 12 tests pass (prompt formatting, transition logic, cooldown clamping, fence-sizing, and the new next_ci_transition_state_* cases pinning the unhandled-status-record behavior).
  • nix develop -c bash -lc 'cd src/ui && bunx tsc -b'
  • nix develop -c bash -lc 'cd src/ui && bun run lint' — passes with existing warnings in unrelated files.
  • nix develop -c bash -lc 'cd src/ui && bun run test' — 2517 frontend tests pass.
  • Manual UAT completed end-to-end against a real CI failure.

Checklist

  • Rust tests added/updated for CI status, transition state, prompt formatting, fence sizing, and cooldown clamping
  • Frontend type-check / test / lint run
  • User-facing settings docs updated (settings.mdx, scm-providers.mdx, per-repo-settings.mdx)
  • Translations added for all five shipped locales
  • Copilot review threads addressed and resolved
  • Rebased on latest origin/main

Copilot AI review requested due to automatic review settings May 12, 2026 16:10
@jamesbrink jamesbrink self-assigned this May 12, 2026
@codecov
Copy link
Copy Markdown

codecov Bot commented May 12, 2026

Codecov Report

✅ All modified and coverable lines are covered by tests.
✅ Project coverage is 79.70%. Comparing base (f23ec66) to head (3f511d2).
⚠️ Report is 1 commits behind head on main.

Additional details and impacted files

Impacted file tree graph

@@            Coverage Diff             @@
##             main     #770      +/-   ##
==========================================
+ Coverage   79.65%   79.70%   +0.05%     
==========================================
  Files         115      115              
  Lines       39741    39847     +106     
==========================================
+ Hits        31657    31762     +105     
- Misses       8084     8085       +1     
🚀 New features to boost your workflow:
  • ❄️ Test Analytics: Detect flaky tests, report on failures, and find test suite problems.
  • 📦 JS Bundle Analysis: Save yourself from yourself by tracking and limiting bundle sizes in JS merges.

Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Adds a CI-automation feature that detects CI failures during SCM polling and auto-creates a new chat session preloaded with a failure-analysis prompt (optionally including CI logs fetched via SCM plugins), plus Settings/Repo Settings controls to configure the behavior.

Changes:

  • Backend: detect CI status transitions to failure during SCM polling, apply cooldown/dedup, create a new chat session, and emit a ci-auto-fix-session-created event with the formatted prompt.
  • Frontend: add Settings + Repo Settings controls for CI auto-fix enablement/model/prompt/cooldown, and wire an App-level listener to open the new session and send the prompt as the first message.
  • SCM plugins: introduce a new ci_failure_logs operation for GitHub/GitLab to fetch failed-check log output (best-effort, truncated).

Reviewed changes

Copilot reviewed 12 out of 12 changed files in this pull request and generated 12 comments.

Show a summary per file
File Description
src/ui/src/types/plugin.ts Adds a CiFailureLog type for plugin log payloads.
src/ui/src/components/settings/Settings.module.css Adds styling for a new settings subsection heading.
src/ui/src/components/settings/sections/RepoSettings.tsx Adds per-repo CI auto-fix override + optional prompt override UI, persisted via app settings.
src/ui/src/components/settings/sections/GitSettings.tsx Adds global CI automation settings (toggle/model/prompt/cooldown), persisted via app settings.
src/ui/src/App.tsx Listens for ci-auto-fix-session-created, opens/selects the session, and sends the generated prompt.
src/scm/types.rs Adds CiFailureLog + derive_overall_ci_status helper and tests.
src-tauri/src/state.rs Adds in-memory per-workspace CI transition/cooldown tracking state.
src-tauri/src/commands/scm.rs Implements CI failure transition detection, prompt formatting, session creation, event emission, and formatting tests.
plugins/scm-gitlab/plugin.json Declares new ci_failure_logs operation for GitLab SCM plugin.
plugins/scm-gitlab/init.lua Implements ci_failure_logs via glab to return failed job traces.
plugins/scm-github/plugin.json Declares new ci_failure_logs operation for GitHub SCM plugin.
plugins/scm-github/init.lua Implements ci_failure_logs via gh to return failed run logs.

Comment thread src-tauri/src/commands/scm.rs Outdated
Comment thread src-tauri/src/commands/scm.rs
Comment thread src-tauri/src/commands/scm.rs Outdated
Comment thread src-tauri/src/commands/scm.rs Outdated
Comment thread src-tauri/src/commands/scm.rs Outdated
Comment thread src/ui/src/components/settings/sections/GitSettings.tsx Outdated
Comment thread src/ui/src/components/settings/sections/RepoSettings.tsx
Comment thread plugins/scm-github/init.lua Outdated
Comment thread src-tauri/src/commands/scm.rs
Comment thread src/ui/src/components/settings/sections/GitSettings.tsx Outdated
Copilot AI review requested due to automatic review settings May 12, 2026 17:09
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 20 out of 20 changed files in this pull request and generated 1 comment.

Comment thread src-tauri/src/commands/scm.rs Outdated
Copilot AI review requested due to automatic review settings May 13, 2026 02:33
@jamesbrink jamesbrink changed the title feat(settings): auto-create session on CI failure feat(settings): auto-fix CI failures with chat launcher hook May 13, 2026
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 21 out of 21 changed files in this pull request and generated 5 comments.

Comment thread src-tauri/src/commands/scm.rs Outdated
Comment thread src-tauri/src/commands/scm.rs Outdated
Comment thread src-tauri/src/commands/scm.rs Outdated
Comment thread src-tauri/src/commands/scm.rs Outdated
Comment thread src-tauri/src/commands/scm.rs Outdated
Copilot AI review requested due to automatic review settings May 13, 2026 15:24
@jamesbrink jamesbrink force-pushed the doomspork/settings-ci-failure-prompt branch from 1ea999d to d930c88 Compare May 13, 2026 15:24
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 21 out of 21 changed files in this pull request and generated 3 comments.

Comment thread src/ui/src/components/settings/sections/GitSettings.tsx
Comment thread plugins/scm-github/init.lua Outdated
Comment thread plugins/scm-gitlab/init.lua Outdated
@jamesbrink jamesbrink force-pushed the doomspork/settings-ci-failure-prompt branch from d930c88 to 9b14900 Compare May 14, 2026 17:08
Copilot AI review requested due to automatic review settings May 14, 2026 17:10
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 21 out of 21 changed files in this pull request and generated no new comments.

doomspork and others added 7 commits May 17, 2026 20:38
When CI transitions to a failed state, automatically create a new chat
session with a configurable prompt containing failure details and logs.

- Add ci_failure_logs plugin operation to GitHub and GitLab plugins
- Add CI transition detection in the SCM polling loop with cooldown
- Add CI Automation settings (toggle, model, prompt template, cooldown)
- Add per-repo override for CI auto-fix in repository settings
- Add CiFailureLog type and derive_overall_ci_status helper with tests
- Wire ci-auto-fix-session-created event through to sendChatMessage
@jamesbrink jamesbrink force-pushed the doomspork/settings-ci-failure-prompt branch from 326e6cb to 9ae1afb Compare May 18, 2026 03:47
`buildModelRegistry(alternativeBackendsEnabled, agentBackends)` skips
the `codexEnabled` flag and the Claude-OAuth Pi-anthropic filter that
`useModelRegistry()` applies elsewhere, so the CI auto-fix model
picker could save a model the Rust resolver later rejects at send
time (or hide valid Codex Native models). Switch to the hook so this
picker matches every other chat-side selector.
`gh pr checks` returns job/check names while `gh run list` returns
workflow names, so exact-name matching almost always misses (a failed
job named "Lint" inside a workflow named "CI" produces `failed_checks
= ["Lint"]` and `run.name = "CI"`). When that happens, fall back to
fetching logs from the most recent failed runs as a best-effort
signal so the prompt still has something useful to work with.

Also cap fetched logs at 3 runs. Each `gh run view --log-failed` is a
sequential network call and the SCM polling semaphore caps workspace
concurrency, not per-workspace fan-out — without a cap, a noisy repo
could stretch a single poll tick well past the 30 s cadence.
When `auto_create_ci_fix_session` returned `false` (DB open / lookup /
session-create failure), the polling loop left `ci_last_status` at
the *prior* non-failure value. The next 30 s tick re-evaluated the
same transition as still happening and tried again — under a
persistent DB error that meant an auto-fix attempt every 30 s
indefinitely.

Always record the observed `overall_status`. The cooldown stamp
(`last_auto_fix_triggered`) stays gated on success so a genuine
failure can still be retried on the next observed transition.

Extracted the decision into `next_ci_transition_state` so the
bookkeeping rules can be unit-tested without an `AppState`.
Copilot AI review requested due to automatic review settings May 18, 2026 04:04
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 21 out of 21 changed files in this pull request and generated 1 comment.

Comment thread src-tauri/src/commands/scm.rs Outdated
When `ci_auto_fix_model` is unset, the model already falls back to
`default_model`, but the provider stayed at the (empty)
`ci_auto_fix_model_provider` setting. For a user with a non-Anthropic
default (Codex Native, Pi, OpenAI), that routed auto-fix sessions
through the default Anthropic gateway with the wrong model name —
silently using the wrong harness.

Fall back as a *pair*: when there's no explicit CI auto-fix model,
read both `default_model` and `default_agent_backend` together.
Explicit-model selections still take their explicit provider.

Also documents the intentional parameter omissions in
`useCiAutoFixSession` — auto-fix sessions use backend defaults for
permission/thinking/plan/fast flags rather than inheriting toolbar
state, because there is no per-tab UI state to inherit on a
background-triggered session.
@jamesbrink jamesbrink marked this pull request as ready for review May 18, 2026 04:35
@jamesbrink jamesbrink requested a review from a team as a code owner May 18, 2026 04:35
Copilot AI review requested due to automatic review settings May 18, 2026 04:35
Copy link
Copy Markdown

Copilot AI left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Pull request overview

Copilot reviewed 21 out of 21 changed files in this pull request and generated no new comments.

`Database::open` returns an owned value and the local `db` is only
used through `&db` / `db.method(...)` afterwards, so the `mut` was
unused and produced an `unused_mut` warning on dev builds. CI doesn't
clippy `claudette-tauri`, so the warning was visible only when running
the app locally — drop it.
@jamesbrink jamesbrink merged commit 9dd5942 into main May 18, 2026
17 checks passed
@jamesbrink jamesbrink deleted the doomspork/settings-ci-failure-prompt branch May 18, 2026 04:40
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

3 participants