Skip to content

feat: support OpenAI-compatible transcription and independent chat/transcription defaults - #1007

Open
Yiipu wants to merge 19 commits into
team-reflect:masterfrom
Yiipu:draft/openai-compatible-transcription-v2
Open

feat: support OpenAI-compatible transcription and independent chat/transcription defaults#1007
Yiipu wants to merge 19 commits into
team-reflect:masterfrom
Yiipu:draft/openai-compatible-transcription-v2

Conversation

@Yiipu

@Yiipu Yiipu commented Jul 31, 2026

Copy link
Copy Markdown

Problem

#996 added openai-compatible providers for chat, editor AI transforms, capture enrichment, asset descriptions, and audio memo title generation. That change did not include transcription. The /v1/audio/transcriptions endpoint is separate from /v1/chat/completions. The model types also differ (speech-to-text versus chat).

Users with OpenAI-compatible transcription endpoints want to route transcription through the same provider entry. These endpoints include local Whisper servers, Groq, and API proxies that expose /v1/audio/transcriptions. The base URL stays the same. Only the model differs.

This PR supersedes #995. The scope grew beyond transcription. It now adds independent chat and transcription defaults. It also adds a disabled sentinel for openai-compatible entries and the full desktop and mobile settings UI.

The original PR body no longer matches the diff. The bot review also landed on a stale description. This PR starts from the same commits with a clean review surface.

Changes

Schema and catalog

  • openai-compatible entries gain an optional transcriptionModel field. An empty string (the default) and the 'disabled' sentinel both mean the entry does not transcribe.
  • A new defaultTranscriptionProviderId setting is independent of the chat default. Chat and transcription can use different providers.
  • AiProviderInfo gains supportsTranscription. This is a compile-time constant for hosted providers. OpenAI and Google set it to true. Anthropic and OpenRouter set it to false. For openai-compatible, the catalog declares false. The user opts in per entry via transcriptionModel.
  • aiProviderSupportsTranscription() merges the catalog constant with the per-entry transcriptionModel.
  • aiProviderSupportsChat() lets an openai-compatible entry opt out of chat by setting model to 'disabled' (a Whisper-only endpoint).
  • The catalog offers a DISABLED_OPENAI_COMPATIBLE_MODEL sentinel as an option so it round-trips through the settings schema.

Provider config

  • This PR replaces the hardcoded TRANSCRIPTION_PROVIDERS constant with transcriptionProviders(). This function derives eligible providers from the configured state.
  • resolveTranscriptionTarget and pickTranscriptionConfig work with any provider that has transcription capability.
  • defaultTranscriptionProvider() resolves the transcription default the same way defaultAiProvider() resolves the chat default.
  • TranscriptionProvider widens to include 'openai-compatible'. This PR removes TranscriptionConfig and uses AiProviderConfig instead.

Transcription client

  • transcribeAudio accepts optional baseUrl and model params. When the provider is openai-compatible, the client posts to {baseUrl}/audio/transcriptions with the user-supplied model.
  • Hosted OpenAI transcription keeps its hardcoded model and fallback retry. The fallback only runs when the caller did not supply a model.
  • httpError reports the actual provider, not a hardcoded 'openai'.

Audio memo pipeline

  • reconcileAudioMemos resolves the key through aiApiKeyForConfig (shared with chat) instead of a direct keychain read. This lets openai-compatible entries with an optional key work.
  • transcribeSessionParts forwards baseUrl and transcriptionModel when the provider is openai-compatible.

Chat model options

  • chatModelOptions() filters out entries whose chat model is the 'disabled' sentinel. The picker cannot switch to a chat-disabled entry.

Settings UI (desktop and mobile)

  • The add-provider dialog and the mobile add-provider sheet show a second model picker for transcription when the provider is openai-compatible. Hosted transcription-capable providers show their fixed model as a disabled combo.
  • Both "Use as default for chat" and "Use as default for transcription" checkboxes appear when the provider supports the feature.
  • The provider row on desktop shows a second model combo and a second default control for transcription. The mobile settings list badges both defaults.
  • The mobile provider-actions drawer shows the transcription model section and the transcription default action.
  • ModelCombobox gains a disabled prop for the fixed-model display.

Screenshots

image image

Checklist

  • pnpm typecheck passes
  • pnpm lint passes (no new warnings)
  • Targeted tests pass: provider-config, provider-catalog, transcribe, secrets, model-options, schema, audio-memo, ai-providers-section, transcription-reconciler, add-ai-provider-drawer, ai-provider-actions-drawer (196 tests total)
  • No any types, zod at boundaries, kebab-case filenames
  • Public APIs documented with doc comments
  • iOS UI test (blocked: the base-ui Select inside the add-provider Dialog has a pre-existing interaction bug. The dropdown flashes open then becomes unresponsive on the next click. You cannot manually verify the new transcription model picker on iOS. This PR did not introduce the bug. It affects the existing provider picker too.)

Relation to #995

#995 is the same work at an earlier stage. This PR replaces it. The commits are identical. Only the PR body and review surface are new.

Summary by CodeRabbit

  • New Features
    • Configure separate chat and transcription providers and default models.
    • OpenAI-compatible providers support custom transcription models and endpoints.
    • Choose local, hosted, or disabled models through model selectors.
    • Mobile and desktop settings show transcription capabilities and defaults.
  • Bug Fixes
    • Providers without chat support are no longer offered as chat defaults.
    • Audio memo availability and setup guidance recognize all transcription-capable models.
    • Improved handling of unavailable or explicitly selected transcription models.

@coderabbitai

coderabbitai Bot commented Jul 31, 2026

Copy link
Copy Markdown

Review Change Stack

No actionable comments were generated in the recent review. 🎉

ℹ️ Recent review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 99fff473-da56-4160-b96d-9cba8cdbd39e

📥 Commits

Reviewing files that changed from the base of the PR and between ff115dd and 29d4e22.

📒 Files selected for processing (3)
  • apps/desktop/src-tauri/build.rs
  • apps/desktop/src/components/settings-screen.test.tsx
  • apps/desktop/src/components/settings/describe-assets-field.test.tsx

Walkthrough

The change adds an independent transcription provider setting. It supports OpenAI-compatible transcription models and endpoints, updates provider resolution and persistence, and adds desktop and mobile controls for transcription models and defaults.

Changes

Transcription provider support

Layer / File(s) Summary
Provider capabilities and defaults
packages/core/src/ai/*, packages/core/src/settings/schema.*, packages/core/src/exports/ai-actions.ts
Provider schemas, capability checks, disabled model handling, transcription-provider ordering, and separate defaults now support OpenAI-compatible transcription.
Transcription execution
packages/core/src/actions/audio-memo*.ts, packages/core/src/ai/transcribe*.ts
Transcription requests can use provider-specific endpoints and models. Key resolution uses full provider configurations.
Desktop settings
apps/desktop/src/components/settings/*, apps/desktop/src/hooks/use-ai-providers.ts
Desktop settings configure transcription models and transcription defaults. Unsupported chat and transcription actions are disabled.
Mobile settings
apps/desktop/src/mobile/*settings*, apps/desktop/src/mobile/*provider*drawer*
Mobile settings expose transcription model controls, transcription-default actions, and separate provider badges.
Pipeline and persisted-state integration
apps/desktop/src/providers/*, apps/desktop/src/hooks/use-audio-memo-pipeline.ts, apps/desktop/src/mobile/recording-drawer.tsx, apps/desktop/src-tauri/build.rs
Provider state, persisted settings, iOS linking, and audio-memo guidance use the dedicated transcription default and generic transcription-capability terminology.

Estimated code review effort: 4 (Complex) | ~60 minutes

Sequence Diagram(s)

sequenceDiagram
  participant SettingsUI
  participant UseAiProviders
  participant ProviderConfig
  participant AudioMemoPipeline
  participant TranscribeAudio

  SettingsUI->>UseAiProviders: set transcription model or default
  UseAiProviders->>ProviderConfig: persist provider state
  AudioMemoPipeline->>ProviderConfig: resolve transcription target
  ProviderConfig-->>AudioMemoPipeline: return provider configuration
  AudioMemoPipeline->>TranscribeAudio: send endpoint and model overrides
  TranscribeAudio-->>AudioMemoPipeline: return transcription result
Loading

Possibly related PRs

Suggested labels: codex

🚥 Pre-merge checks | ✅ 4 | ❌ 1

❌ Failed checks (1 warning)

Check name Status Explanation Resolution
Docstring Coverage ⚠️ Warning Docstring coverage is 63.46% which is insufficient. The required threshold is 80.00%. Write docstrings for the functions missing them to satisfy the coverage threshold.
✅ Passed checks (4 passed)
Check name Status Explanation
Description Check ✅ Passed Check skipped - CodeRabbit’s high-level summary is enabled.
Title check ✅ Passed The title clearly and concisely summarizes OpenAI-compatible transcription support and independent chat and transcription defaults.
Linked Issues check ✅ Passed Check skipped because no linked issues were found for this pull request.
Out of Scope Changes check ✅ Passed Check skipped because no linked issues were found for this pull request.
✨ Finishing Touches
🧪 Generate unit tests (beta)
  • Create PR with unit tests

Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out.

❤️ Share

Comment @coderabbitai help to get the list of available commands.

@Yiipu
Yiipu marked this pull request as ready for review July 31, 2026 08:34

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 5

Caution

Some comments are outside the diff and can’t be posted inline due to platform limitations.

⚠️ Outside diff range comments (1)
apps/desktop/src/mobile/add-ai-provider-drawer.tsx (1)

124-132: 🗄️ Data Integrity & Integration | 🟡 Minor | ⚡ Quick win

Reset the transcription-default flag when switching to a provider without transcription support.

Provider selection only clears transcriptionModel for openai-compatible; isTranscriptionDefault stays true for Anthropic, OpenRouter, and similar providers. submitDraft sends that flag, so withAiProviderAdded stores defaultTranscriptionProviderId for a provider without a transcription model. Reset the flag in onValueChange when aiProviderSupportsTranscription(next) is false.

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@apps/desktop/src/mobile/add-ai-provider-drawer.tsx` around lines 124 - 132,
Update the provider-switching logic in onValueChange to reset
isTranscriptionDefault whenever aiProviderSupportsTranscription(next) is false,
alongside clearing transcriptionModel for unsupported providers. Preserve the
existing transcription-default state when the selected provider supports
transcription, and ensure submitDraft cannot receive a stale true flag.
🧹 Nitpick comments (5)
apps/desktop/src/components/settings/ai-provider-row.tsx (1)

122-137: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Avoid recomputing transcriptionModelLabel(config) three times.

transcriptionModelLabel(config) is called three times in this branch for the same config. Compute it once into a local variable and reuse it.

♻️ Proposed fix
-          ) : (
-            <ModelCombobox
-              value={transcriptionModelLabel(config)}
-              provider={config.provider}
-              models={[
-                {
-                  id: transcriptionModelLabel(config),
-                  label: transcriptionModelLabel(config),
-                  contextWindow: DEFAULT_CONTEXT_WINDOW,
-                },
-              ]}
+          ) : (
+            (() => {
+              const fixedModel = transcriptionModelLabel(config)
+              return (
+                <ModelCombobox
+                  value={fixedModel}
+                  provider={config.provider}
+                  models={[{ id: fixedModel, label: fixedModel, contextWindow: DEFAULT_CONTEXT_WINDOW }]}
+                  onChange={() => {}}
+                  ariaLabel={`Transcription model for ${providerLabel}`}
+                  disabled
+                />
+              )
+            })()
           )
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@apps/desktop/src/components/settings/ai-provider-row.tsx` around lines 122 -
137, In the disabled ModelCombobox branch, compute
transcriptionModelLabel(config) once in a local variable and reuse it for the
value, model id, and model label fields. Keep the existing ModelCombobox
behavior and configuration unchanged.
apps/desktop/src/components/settings/add-ai-provider-dialog.tsx (1)

251-260: 🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick win

Disable the transcription-default checkbox for a disabled transcription model.

The checkbox at Line 251-260 has no disabled state. A user can select the disabled sentinel in the transcription model combobox and still check "Use as default for transcription." ai-provider-row.tsx disables its equivalent "Make transcription default" button with disabled={!supportsTranscription} for this exact case. Apply the same guard here for consistency, so the checkbox does not silently no-op.

♻️ Proposed fix
           {provider.supportsTranscription || isOpenAICompatible ? (
             <label className="flex items-center gap-2">
               <input
                 type="checkbox"
                 className="accent-accent"
+                disabled={isOpenAICompatible && transcriptionModelValue === DISABLED_OPENAI_COMPATIBLE_MODEL}
                 {...register('isTranscriptionDefault')}
               />
               <span className="text-sm text-text">Use as default for transcription</span>
             </label>
           ) : null}
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@apps/desktop/src/components/settings/add-ai-provider-dialog.tsx` around lines
251 - 260, Update the transcription-default checkbox in the add-provider
dialog’s provider supportsTranscription/isOpenAICompatible branch to be disabled
when the selected transcription model is the disabled sentinel, matching the
guard used by ai-provider-row.tsx. Reuse the existing selected-model state or
helper that identifies the disabled model, while preserving the current
rendering conditions.
packages/core/src/ai/provider-config.ts (2)

126-141: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low value

Sort comparator uses single-character parameter names.

(a, b) => {...} at line 134 uses single-character variable names. Rename them to something descriptive, for example left/right or providerA/providerB.

♻️ Proposed rename
-  return candidates.sort((a, b) => {
-    const groupDiff = groupOrder(a.provider) - groupOrder(b.provider)
+  return candidates.sort((left, right) => {
+    const groupDiff = groupOrder(left.provider) - groupOrder(right.provider)
     if (groupDiff !== 0) return groupDiff
-    const aDefault = a.id === state.defaultTranscriptionProviderId ? 0 : 1
-    const bDefault = b.id === state.defaultTranscriptionProviderId ? 0 : 1
-    return aDefault - bDefault
+    const leftDefault = left.id === state.defaultTranscriptionProviderId ? 0 : 1
+    const rightDefault = right.id === state.defaultTranscriptionProviderId ? 0 : 1
+    return leftDefault - rightDefault
   })

As per coding guidelines, "Never use single-character variable names."

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/core/src/ai/provider-config.ts` around lines 126 - 141, Rename the
transcriptionProviders sort comparator parameters from single-character names to
descriptive names such as left and right, and update all references within the
comparator consistently without changing sorting behavior.

Source: Coding guidelines


143-153: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Doc comment describes behavior pickTranscriptionConfig does not perform.

The doc comment says this returns "the first transcription-capable provider whose keychain key resolves" and that "a keyless entry is skipped rather than stopping the pass." The implementation is synchronous and only returns candidates[0] — it never checks the keychain. That description matches resolveTranscriptionTarget (lines 197-203), not this function. Both callers (audio-memo.ts for an error message, use-audio-memo-pipeline.ts for an existence check) only need "the first configured transcription-capable provider," so the implementation itself is fine — only the doc is misleading.

📝 Proposed doc fix
 /**
- * The configured entry audio transcription should run on: the first
- * transcription-capable provider whose keychain key resolves.  Providers
- * are tried in {`@link` transcriptionProviders} order; a keyless entry is
- * skipped rather than stopping the pass.  `null` means no capable provider
- * is configured — the feature is unavailable.
+ * The first configured transcription-capable provider, in
+ * {`@link` transcriptionProviders} order.  This does not check the keychain —
+ * use {`@link` resolveTranscriptionTarget} to resolve a usable key.  `null`
+ * means no capable provider is configured — the feature is unavailable.
  */
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/core/src/ai/provider-config.ts` around lines 143 - 153, Correct the
doc comment for pickTranscriptionConfig to describe only its actual synchronous
behavior: return the first configured transcription-capable provider, or null
when none are configured. Remove references to keychain resolution, provider
skipping, and runtime availability; leave the implementation unchanged.
packages/core/src/actions/audio-memo.ts (1)

576-583: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick win

Replace the as TranscriptionProvider assertion with a type guard.

config.provider as TranscriptionProvider asserts a narrowing that TypeScript cannot verify. It is only safe because resolveTranscriptionTarget's candidates are pre-filtered by transcriptionProviders to openai/google/openai-compatible — but nothing at this call site enforces that invariant at the type level. This assertion became necessary because TranscriptionTarget.config was broadened to AiProviderConfig in provider-config.ts (previously narrower).

Export a type guard (for example isTranscriptionProvider(provider): provider is TranscriptionProvider, backed by TRANSCRIPTION_PROVIDERS.includes(...)) from provider-config.ts and use it here instead of asserting.

♻️ Proposed refactor
+export function isTranscriptionProvider(provider: AiProviderId): provider is TranscriptionProvider {
+  return (TRANSCRIPTION_PROVIDERS as readonly string[]).includes(provider)
+}
-      const parts = await transcribeSessionParts({
-        session,
-        provider: config.provider as TranscriptionProvider,
+      if (!isTranscriptionProvider(config.provider)) {
+        throw new ReflectError('config', `unexpected transcription provider: ${config.provider}`)
+      }
+      const parts = await transcribeSessionParts({
+        session,
+        provider: config.provider,

As per coding guidelines, "Avoid unnecessary type assertions" and "Use discriminated unions and type guards for variant data; export helper predicates when they clarify a public contract."

🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

In `@packages/core/src/actions/audio-memo.ts` around lines 576 - 583, Replace the
`config.provider as TranscriptionProvider` assertion in the transcription
configuration flow with an exported type guard from `provider-config.ts`, such
as `isTranscriptionProvider`, implemented using the existing
`TRANSCRIPTION_PROVIDERS` membership check. Use that guard before constructing
the provider payload so `TranscriptionProvider` is narrowed by runtime
validation rather than assertion, while preserving the existing
supported-provider behavior.

Source: Coding guidelines

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@apps/desktop/src/components/settings/ai-provider-row.tsx`:
- Around line 104-139: Update the ariaLabel values on the default and
transcription ModelCombobox instances in the provider row to include a stable
per-entry identifier such as config.keyHint or config.baseUrl alongside
providerLabel, ensuring duplicate provider types receive distinct accessible
names while preserving the existing model-role descriptions.

In `@apps/desktop/src/mobile/add-ai-provider-drawer.tsx`:
- Around line 149-170: Update the model controls around the Select using
provider.models and the corresponding second control at the noted alternate
range so openai-compatible providers support editable custom model IDs,
including a separate custom transcription model during creation. Preserve
catalog-only Select behavior for fixed providers, and keep the existing model
state and resetUnverified handling intact.

In `@apps/desktop/src/mobile/ai-provider-actions-drawer.tsx`:
- Around line 159-162: Update the TranscriptionModelInput usage in the provider
drawer to reset its local transcription-model draft whenever the managed
provider changes, preferably by keying the component with provider.id; preserve
the existing provider and onSetTranscriptionModel props.

In `@packages/core/src/ai/chat/model-options.ts`:
- Around line 29-48: Update chatModelOptions to exclude the 'disabled' model
option before mapping provider catalog entries into picker options, while
preserving custom active model entries and all other catalog models. Ensure
ChatModelDrawer cannot receive 'disabled' through the returned ChatModelOption
list.

In `@packages/core/src/ai/provider-config.ts`:
- Around line 37-53: Update withAiProviderAdded so the implicit isDefault
fallback only selects a transcription default when
aiProviderSupportsTranscription(entry) is true, matching the existing isFirst
capability check. Preserve explicit isTranscriptionDefault behavior and add a
regression test covering a non-transcription-capable default entry with the
transcription flag omitted.

---

Outside diff comments:
In `@apps/desktop/src/mobile/add-ai-provider-drawer.tsx`:
- Around line 124-132: Update the provider-switching logic in onValueChange to
reset isTranscriptionDefault whenever aiProviderSupportsTranscription(next) is
false, alongside clearing transcriptionModel for unsupported providers. Preserve
the existing transcription-default state when the selected provider supports
transcription, and ensure submitDraft cannot receive a stale true flag.

---

Nitpick comments:
In `@apps/desktop/src/components/settings/add-ai-provider-dialog.tsx`:
- Around line 251-260: Update the transcription-default checkbox in the
add-provider dialog’s provider supportsTranscription/isOpenAICompatible branch
to be disabled when the selected transcription model is the disabled sentinel,
matching the guard used by ai-provider-row.tsx. Reuse the existing
selected-model state or helper that identifies the disabled model, while
preserving the current rendering conditions.

In `@apps/desktop/src/components/settings/ai-provider-row.tsx`:
- Around line 122-137: In the disabled ModelCombobox branch, compute
transcriptionModelLabel(config) once in a local variable and reuse it for the
value, model id, and model label fields. Keep the existing ModelCombobox
behavior and configuration unchanged.

In `@packages/core/src/actions/audio-memo.ts`:
- Around line 576-583: Replace the `config.provider as TranscriptionProvider`
assertion in the transcription configuration flow with an exported type guard
from `provider-config.ts`, such as `isTranscriptionProvider`, implemented using
the existing `TRANSCRIPTION_PROVIDERS` membership check. Use that guard before
constructing the provider payload so `TranscriptionProvider` is narrowed by
runtime validation rather than assertion, while preserving the existing
supported-provider behavior.

In `@packages/core/src/ai/provider-config.ts`:
- Around line 126-141: Rename the transcriptionProviders sort comparator
parameters from single-character names to descriptive names such as left and
right, and update all references within the comparator consistently without
changing sorting behavior.
- Around line 143-153: Correct the doc comment for pickTranscriptionConfig to
describe only its actual synchronous behavior: return the first configured
transcription-capable provider, or null when none are configured. Remove
references to keychain resolution, provider skipping, and runtime availability;
leave the implementation unchanged.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: b557b08d-0f1f-4003-ac9f-ef66725e38a1

📥 Commits

Reviewing files that changed from the base of the PR and between 0491123 and 322c4cd.

📒 Files selected for processing (51)
  • apps/desktop/src/components/audio-memo/audio-memo-button.test.tsx
  • apps/desktop/src/components/audio-memo/audio-memo-button.tsx
  • apps/desktop/src/components/settings/add-ai-provider-dialog.tsx
  • apps/desktop/src/components/settings/ai-provider-row.tsx
  • apps/desktop/src/components/settings/ai-providers-section.test.tsx
  • apps/desktop/src/components/settings/ai-providers-section.tsx
  • apps/desktop/src/components/settings/describe-assets-field.tsx
  • apps/desktop/src/components/settings/model-combobox.tsx
  • apps/desktop/src/components/settings/section.tsx
  • apps/desktop/src/components/sidebar/sidebar.test.tsx
  • apps/desktop/src/hooks/use-ai-providers.ts
  • apps/desktop/src/hooks/use-audio-memo-pipeline.ts
  • apps/desktop/src/lib/asset-describe-controller.test.tsx
  • apps/desktop/src/lib/capture-controller.test.tsx
  • apps/desktop/src/lib/transcription-reconciler.test.tsx
  • apps/desktop/src/mobile/add-ai-provider-drawer.test.tsx
  • apps/desktop/src/mobile/add-ai-provider-drawer.tsx
  • apps/desktop/src/mobile/ai-provider-actions-drawer.test.tsx
  • apps/desktop/src/mobile/ai-provider-actions-drawer.tsx
  • apps/desktop/src/mobile/audio-memo-fab.tsx
  • apps/desktop/src/mobile/audio-memo-provider.test.tsx
  • apps/desktop/src/mobile/audio-memo-provider.tsx
  • apps/desktop/src/mobile/recording-drawer.test.tsx
  • apps/desktop/src/mobile/recording-drawer.tsx
  • apps/desktop/src/mobile/screens/settings.tsx
  • apps/desktop/src/providers/asset-describe-provider.tsx
  • apps/desktop/src/providers/audio-memo-provider.test.tsx
  • apps/desktop/src/providers/audio-memo-provider.tsx
  • apps/desktop/src/providers/capture-provider.tsx
  • apps/desktop/src/providers/chat-provider.tsx
  • apps/desktop/src/providers/settings-provider.test.tsx
  • packages/core/src/actions/asset-description.test.ts
  • packages/core/src/actions/audio-memo-session.ts
  • packages/core/src/actions/audio-memo.test.ts
  • packages/core/src/actions/audio-memo.ts
  • packages/core/src/actions/capture-harness.ts
  • packages/core/src/ai/audio-memo-title.test.ts
  • packages/core/src/ai/chat/model-options.test.ts
  • packages/core/src/ai/chat/model-options.ts
  • packages/core/src/ai/language-model.test.ts
  • packages/core/src/ai/openai-compatible.ts
  • packages/core/src/ai/provider-catalog.test.ts
  • packages/core/src/ai/provider-catalog.ts
  • packages/core/src/ai/provider-config.test.ts
  • packages/core/src/ai/provider-config.ts
  • packages/core/src/ai/secrets.test.ts
  • packages/core/src/ai/transcribe.test.ts
  • packages/core/src/ai/transcribe.ts
  • packages/core/src/exports/ai-actions.ts
  • packages/core/src/settings/schema.test.ts
  • packages/core/src/settings/schema.ts

Comment thread apps/desktop/src/components/settings/ai-provider-row.tsx
Comment thread apps/desktop/src/mobile/add-ai-provider-drawer.tsx Outdated
Comment thread apps/desktop/src/mobile/ai-provider-actions-drawer.tsx
Comment thread packages/core/src/ai/chat/model-options.ts
Comment thread packages/core/src/ai/provider-config.ts
Yiipu and others added 16 commits August 2, 2026 15:22
- Add optional transcriptionModel field to openai-compatible provider config
- Add defaultTranscriptionProviderId to settings schema (independent of chat default)
- Add supportsTranscription flag to AiProviderInfo catalog
- Add aiProviderSupportsTranscription() helper that merges catalog constants
  with per-entry user configuration for openai-compatible providers

Transcription eligibility is now derived: hosted providers are known at
compile time (OpenAI/Google: true, Anthropic/OpenRouter: false), while
openai-compatible entries become eligible when the user supplies a
non-empty transcriptionModel.
Replace the compile-time TRANSCRIPTION_PROVIDERS constant with a dynamic
transcriptionProviders() function that derives eligible providers from
the configured state. Changes:

- transcriptionProviders(): returns all providers where
  aiProviderSupportsTranscription() is true, sorted OpenAI > Google >
  openai-compatible, with the transcription default first within each group
- pickTranscriptionConfig(): returns the first candidate (no longer
  hardcoded to only OpenAI/Google)
- resolveTranscriptionTarget(): iterates candidates, skips keyless entries
- defaultTranscriptionProvider(): resolves the transcription default,
  falling back to the first transcription-capable entry
- AiProvidersState gains defaultTranscriptionProviderId
- withAiProviderAdded/Removed maintain the transcription default
  alongside the chat default

The TranscriptionProvider type is widened to include 'openai-compatible'
alongside 'openai' and 'google'.
Extend transcribeAudio and transcribeWithOpenAi to accept optional
baseUrl and model parameters. When provided (openai-compatible
providers), the client posts to {baseUrl}/audio/transcriptions instead
of the hardcoded api.openai.com endpoint, and uses the user-supplied
model instead of the default with fallback retry.

The provider routing is simplified: anything that isn't 'google' now
flows through transcribeWithOpenAi, since openai-compatible providers
use the same /v1/audio/transcriptions API shape.
…ipeline

memoNoteBody now accepts AiProviderConfig (instead of the removed
TranscriptionConfig) and forwards baseUrl + transcriptionModel to
transcribeAudio when the provider is openai-compatible.

The error message for missing transcription config is updated from
'No OpenAI or Gemini model is configured' to the generic
'No transcription provider is configured'.

Exports are updated to include the new public API surface:
aiProviderSupportsTranscription, defaultTranscriptionProvider,
resolveTranscriptionTarget, transcriptionProviders.
…scriptionProviderId

Every AiProvidersState construction site now includes the new
defaultTranscriptionProviderId field.  useAiProviders gains a
makeTranscriptionDefault callback that writes the transcription
default independently of the chat default.

New openai-compatible entries are initialized with an empty
transcriptionModel (transcription off by default).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Remove duplicate TranscriptionTarget, TranscriptionMiss, and
resolveTranscriptionTarget definitions left over from a merge
between the master-base version and the current-branch version.

The retained version uses the dynamic transcriptionProviders()
function with the correct defaultTranscriptionProviderId.

Also:
- Add TRANSCRIPTION_PROVIDERS constant for desktop consent UI
- Remove unused memoNoteBody function (dead code)
- Fix type narrowing with as TranscriptionProvider assertion
- Fix test snapshots missing defaultTranscriptionProviderId

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…lt provider

- Split default checkbox into chat and transcription options in add forms

- Add transcriptionModel field to add-provider forms (openai-compatible)

- Extend AiProviderRow with second row for transcription (model + default)

- Add setTranscriptionModel hook and transcriptionProvider to useAiProviders

- Update AiProviderActionsDrawer with transcription model editing

- Update NO_PROVIDER_REASON to include OpenAI-compatible providers

- Export transcription model constants from @reflect/core

Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The openai-compatible transcription provider was effectively
non-functional: two wiring gaps stopped it from ever reaching the
user's local endpoint.

1. transcribeSessionParts dropped baseUrl and transcriptionModel when
   calling transcribeAudio, so audio was sent to OpenAI's hosted
   endpoint with gpt-4o-mini-transcribe regardless of the configured
   local server.

2. resolveTranscriptionTarget resolved keys via a raw getSecret, which
   bypassed aiApiKeyForConfig's no-key handling - a keyless
   openai-compatible entry (a local server with no API key) always
   read null from the keychain and was skipped as 'no-key', silently
   stopping every pass before any transcription call.

The reconciler now forwards config.baseUrl and config.transcriptionModel
for openai-compatible providers, and resolveTranscriptionTarget receives
the full AiProviderConfig so callers can use aiApiKeyForConfig for key
resolution.
The model combobox trigger (w-full inside an auto grid column) could
outgrow the section card, clipping the Default badge at the card edge;
the row also showed a noisy 'Transcription disabled (no model set)'
hint under the base URL.

- Drop the transcription-disabled hint; the left column shows only the
  key hint and base URL.
- Give badges their own grid column and center them on the combobox
  rows (h-8), and center the remove button vertically.
- Clip the settings card (overflow-hidden) and floor the combobox
  popover width at 20rem so narrow triggers still get a readable list.
OpenAI/Google rows render their fixed transcription model as an inert
ModelCombobox (same shape as the editable pickers, never opens) instead
of a bare muted string, so the two model rows read consistently. The
model remains a compile-time constant; the single-item curated list is
only there to satisfy the combobox contract.
The openai-compatible catalog now offers two model options — local-model
and a Disabled sentinel — for both the chat and the transcription slot.
Picking Disabled opts the entry out of that feature:

- core: aiProviderSupportsChat predicate; chatModelOptions and
  defaultAiProvider skip chat-disabled entries; the transcription
  predicate treats 'disabled' like the legacy unset ''.
- desktop: both model pickers on a settings row are catalog-driven; the
  transcription combo stays visible when disabled, with Make default /
  Make transcription default greyed out for an ineligible slot. The add
  dialog's transcription free-text input becomes the same combobox.
- ios: the add sheet offers the catalog list as a Select; the manage
  sheet keeps the transcription section visible while disabled and
  greys the default actions the same way.

Transcription model ids remain free-form (custom ids via the combobox
on desktop, the text field on iOS); the sentinel only names the two
states the feature toggles between.
The settings list value now appends '· Transcription' alongside the
chat '· Default' marker, so an entry that only carries the
transcription default is still distinguishable from a plain row.
…s too

The add-provider dialog/sheet offered a free-text input for the chat
model of an openai-compatible endpoint while every other model field
had moved to the catalog pickers. Both now use the same controls as
the transcription slot: the desktop dialog gets the ModelCombobox
(catalog options, custom ids by typing + Enter), the mobile sheet the
catalog Select.
Add JSDoc to the previously-undocumented exports touched by PR team-reflect#1007
across the transcription surface: the openai-compatible constants and
URL helpers, the fixed OpenAI/Gemini transcription model ids, the
SendOptions interface, and the capture/reconcile/session input-output
types and functions in actions/audio-memo*.
- provider-config: guard isDefault transcription fallback with capability check
- provider-config: export isTranscriptionProvider type guard, drop assertion
- provider-config: rename sort comparator params, fix pickTranscriptionConfig doc
- model-options: exclude disabled sentinel from chat picker
- audio-memo: use type guard instead of provider type assertion
- mobile add-provider: allow free-form model entry for openai-compatible
- mobile add-provider: reset isTranscriptionDefault when provider lacks capability
- mobile actions-drawer: key TranscriptionModelInput by provider id
- desktop provider-row: disambiguate combobox ariaLabel per entry
- desktop provider-row: cache transcriptionModelLabel in disabled branch
- desktop add-dialog: disable transcription-default checkbox for disabled model
@Yiipu
Yiipu force-pushed the draft/openai-compatible-transcription-v2 branch from 503b63d to aa090de Compare August 2, 2026 07:23

@coderabbitai coderabbitai Bot left a comment

Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Actionable comments posted: 1

🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.

Inline comments:
In `@apps/desktop/src/mobile/ai-provider-actions-drawer.test.tsx`:
- Around line 123-149: Add an unsaved transcription model edit to the first
input before rerendering the provider, then blur the newly rendered input after
the provider change. Keep the assertion that onSetTranscriptionModel remains
unused, so the test exercises and verifies the stale-draft save path in the
provider-change case.
🪄 Autofix (Beta)

Fix all unresolved CodeRabbit comments on this PR:

  • Push a commit to this branch (recommended)
  • Create a new PR with the fixes

ℹ️ Review info
⚙️ Run configuration

Configuration used: Organization UI

Review profile: CHILL

Plan: Pro Plus

Run ID: 35a43017-8810-490a-b02b-f91f665c476f

📥 Commits

Reviewing files that changed from the base of the PR and between 322c4cd and aa090de.

📒 Files selected for processing (52)
  • apps/desktop/src/components/audio-memo/audio-memo-button.test.tsx
  • apps/desktop/src/components/audio-memo/audio-memo-button.tsx
  • apps/desktop/src/components/settings/add-ai-provider-dialog.tsx
  • apps/desktop/src/components/settings/ai-provider-row.tsx
  • apps/desktop/src/components/settings/ai-providers-section.test.tsx
  • apps/desktop/src/components/settings/ai-providers-section.tsx
  • apps/desktop/src/components/settings/describe-assets-field.tsx
  • apps/desktop/src/components/settings/model-combobox.tsx
  • apps/desktop/src/components/settings/section.tsx
  • apps/desktop/src/components/sidebar/sidebar.test.tsx
  • apps/desktop/src/hooks/use-ai-providers.ts
  • apps/desktop/src/hooks/use-audio-memo-pipeline.ts
  • apps/desktop/src/lib/asset-describe-controller.test.tsx
  • apps/desktop/src/lib/capture-controller.test.tsx
  • apps/desktop/src/lib/transcription-reconciler.test.tsx
  • apps/desktop/src/mobile/add-ai-provider-drawer.test.tsx
  • apps/desktop/src/mobile/add-ai-provider-drawer.tsx
  • apps/desktop/src/mobile/ai-provider-actions-drawer.test.tsx
  • apps/desktop/src/mobile/ai-provider-actions-drawer.tsx
  • apps/desktop/src/mobile/audio-memo-fab.tsx
  • apps/desktop/src/mobile/audio-memo-provider.test.tsx
  • apps/desktop/src/mobile/audio-memo-provider.tsx
  • apps/desktop/src/mobile/recording-drawer.test.tsx
  • apps/desktop/src/mobile/recording-drawer.tsx
  • apps/desktop/src/mobile/screens/settings.tsx
  • apps/desktop/src/providers/asset-describe-provider.tsx
  • apps/desktop/src/providers/audio-memo-provider.test.tsx
  • apps/desktop/src/providers/audio-memo-provider.tsx
  • apps/desktop/src/providers/capture-provider.tsx
  • apps/desktop/src/providers/chat-provider.tsx
  • apps/desktop/src/providers/settings-provider.test.tsx
  • packages/core/src/actions/asset-description.test.ts
  • packages/core/src/actions/audio-memo-session.ts
  • packages/core/src/actions/audio-memo.test.ts
  • packages/core/src/actions/audio-memo.ts
  • packages/core/src/actions/capture-harness.ts
  • packages/core/src/ai/audio-memo-title.test.ts
  • packages/core/src/ai/chat/model-options.test.ts
  • packages/core/src/ai/chat/model-options.ts
  • packages/core/src/ai/language-model.test.ts
  • packages/core/src/ai/openai-compatible.ts
  • packages/core/src/ai/provider-catalog.test.ts
  • packages/core/src/ai/provider-catalog.ts
  • packages/core/src/ai/provider-config.test.ts
  • packages/core/src/ai/provider-config.ts
  • packages/core/src/ai/secrets.test.ts
  • packages/core/src/ai/transcribe-http.ts
  • packages/core/src/ai/transcribe.test.ts
  • packages/core/src/ai/transcribe.ts
  • packages/core/src/exports/ai-actions.ts
  • packages/core/src/settings/schema.test.ts
  • packages/core/src/settings/schema.ts
🚧 Files skipped from review as they are similar to previous changes (49)
  • apps/desktop/src/mobile/recording-drawer.test.tsx
  • apps/desktop/src/providers/audio-memo-provider.test.tsx
  • packages/core/src/ai/language-model.test.ts
  • apps/desktop/src/components/sidebar/sidebar.test.tsx
  • apps/desktop/src/components/audio-memo/audio-memo-button.tsx
  • apps/desktop/src/mobile/audio-memo-fab.tsx
  • apps/desktop/src/components/settings/describe-assets-field.tsx
  • apps/desktop/src/mobile/audio-memo-provider.test.tsx
  • apps/desktop/src/mobile/audio-memo-provider.tsx
  • apps/desktop/src/components/audio-memo/audio-memo-button.test.tsx
  • apps/desktop/src/providers/asset-describe-provider.tsx
  • packages/core/src/ai/secrets.test.ts
  • apps/desktop/src/lib/asset-describe-controller.test.tsx
  • packages/core/src/actions/asset-description.test.ts
  • apps/desktop/src/providers/capture-provider.tsx
  • packages/core/src/ai/chat/model-options.ts
  • apps/desktop/src/mobile/recording-drawer.tsx
  • apps/desktop/src/providers/audio-memo-provider.tsx
  • packages/core/src/ai/provider-catalog.test.ts
  • packages/core/src/ai/audio-memo-title.test.ts
  • apps/desktop/src/components/settings/ai-providers-section.tsx
  • packages/core/src/actions/audio-memo.test.ts
  • apps/desktop/src/hooks/use-audio-memo-pipeline.ts
  • packages/core/src/ai/chat/model-options.test.ts
  • packages/core/src/settings/schema.test.ts
  • apps/desktop/src/providers/settings-provider.test.tsx
  • packages/core/src/ai/openai-compatible.ts
  • apps/desktop/src/components/settings/ai-providers-section.test.tsx
  • apps/desktop/src/mobile/screens/settings.tsx
  • packages/core/src/exports/ai-actions.ts
  • apps/desktop/src/providers/chat-provider.tsx
  • apps/desktop/src/components/settings/model-combobox.tsx
  • apps/desktop/src/lib/transcription-reconciler.test.tsx
  • packages/core/src/actions/audio-memo.ts
  • apps/desktop/src/components/settings/section.tsx
  • apps/desktop/src/lib/capture-controller.test.tsx
  • apps/desktop/src/mobile/add-ai-provider-drawer.tsx
  • packages/core/src/actions/capture-harness.ts
  • packages/core/src/settings/schema.ts
  • apps/desktop/src/hooks/use-ai-providers.ts
  • apps/desktop/src/components/settings/ai-provider-row.tsx
  • packages/core/src/ai/provider-catalog.ts
  • packages/core/src/actions/audio-memo-session.ts
  • packages/core/src/ai/transcribe.ts
  • apps/desktop/src/components/settings/add-ai-provider-dialog.tsx
  • packages/core/src/ai/provider-config.test.ts
  • apps/desktop/src/mobile/ai-provider-actions-drawer.tsx
  • packages/core/src/ai/provider-config.ts
  • packages/core/src/ai/transcribe.test.ts

Comment thread apps/desktop/src/mobile/ai-provider-actions-drawer.test.tsx
Yiipu added 3 commits August 2, 2026 15:33
CodeRabbit noted the transcription-model reset test only checked the
value after rerender; it did not verify the stale-draft save path is
dead. Fill the input with an unsaved edit before rerendering, then
blur the remounted input and assert onSetTranscriptionModel stays
uncalled - proving the key-driven remount discards the old draft.
The iOS cargo build compiles all three crate-types (staticlib, cdylib,
rlib) before Xcode compiles NativeDiagnostics.swift. The staticlib
(libapp.a, what iOS actually links) tolerates the unresolved
reflect_start_native_diagnostics reference; the cdylib does not, and
its link step failed before Xcode ran. Pass -undefined dynamic_lookup
to the cdylib link only - the cdylib is not shipped on iOS, so this
just unblocks the cargo step the Xcode build phase runs.
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant