feat: support OpenAI-compatible transcription and independent chat/transcription defaults - #1007
feat: support OpenAI-compatible transcription and independent chat/transcription defaults#1007Yiipu wants to merge 19 commits into
Conversation
|
No actionable comments were generated in the recent review. 🎉 ℹ️ Recent review info⚙️ Run configurationConfiguration used: Organization UI Review profile: CHILL Plan: Pro Plus Run ID: 📒 Files selected for processing (3)
WalkthroughThe change adds an independent transcription provider setting. It supports OpenAI-compatible transcription models and endpoints, updates provider resolution and persistence, and adds desktop and mobile controls for transcription models and defaults. ChangesTranscription provider support
Estimated code review effort: 4 (Complex) | ~60 minutes Sequence Diagram(s)sequenceDiagram
participant SettingsUI
participant UseAiProviders
participant ProviderConfig
participant AudioMemoPipeline
participant TranscribeAudio
SettingsUI->>UseAiProviders: set transcription model or default
UseAiProviders->>ProviderConfig: persist provider state
AudioMemoPipeline->>ProviderConfig: resolve transcription target
ProviderConfig-->>AudioMemoPipeline: return provider configuration
AudioMemoPipeline->>TranscribeAudio: send endpoint and model overrides
TranscribeAudio-->>AudioMemoPipeline: return transcription result
Possibly related PRs
Suggested labels: 🚥 Pre-merge checks | ✅ 4 | ❌ 1❌ Failed checks (1 warning)
✅ Passed checks (4 passed)
✨ Finishing Touches🧪 Generate unit tests (beta)
Thanks for using CodeRabbit! It's free for OSS, and your support helps us grow. If you like it, consider giving us a shout-out. Comment |
There was a problem hiding this comment.
Actionable comments posted: 5
Caution
Some comments are outside the diff and can’t be posted inline due to platform limitations.
⚠️ Outside diff range comments (1)
apps/desktop/src/mobile/add-ai-provider-drawer.tsx (1)
124-132: 🗄️ Data Integrity & Integration | 🟡 Minor | ⚡ Quick winReset the transcription-default flag when switching to a provider without transcription support.
Provider selection only clears
transcriptionModelforopenai-compatible;isTranscriptionDefaultstaystruefor Anthropic, OpenRouter, and similar providers.submitDraftsends that flag, sowithAiProviderAddedstoresdefaultTranscriptionProviderIdfor a provider without a transcription model. Reset the flag inonValueChangewhenaiProviderSupportsTranscription(next)is false.🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@apps/desktop/src/mobile/add-ai-provider-drawer.tsx` around lines 124 - 132, Update the provider-switching logic in onValueChange to reset isTranscriptionDefault whenever aiProviderSupportsTranscription(next) is false, alongside clearing transcriptionModel for unsupported providers. Preserve the existing transcription-default state when the selected provider supports transcription, and ensure submitDraft cannot receive a stale true flag.
🧹 Nitpick comments (5)
apps/desktop/src/components/settings/ai-provider-row.tsx (1)
122-137: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueAvoid recomputing
transcriptionModelLabel(config)three times.
transcriptionModelLabel(config)is called three times in this branch for the sameconfig. Compute it once into a local variable and reuse it.♻️ Proposed fix
- ) : ( - <ModelCombobox - value={transcriptionModelLabel(config)} - provider={config.provider} - models={[ - { - id: transcriptionModelLabel(config), - label: transcriptionModelLabel(config), - contextWindow: DEFAULT_CONTEXT_WINDOW, - }, - ]} + ) : ( + (() => { + const fixedModel = transcriptionModelLabel(config) + return ( + <ModelCombobox + value={fixedModel} + provider={config.provider} + models={[{ id: fixedModel, label: fixedModel, contextWindow: DEFAULT_CONTEXT_WINDOW }]} + onChange={() => {}} + ariaLabel={`Transcription model for ${providerLabel}`} + disabled + /> + ) + })() )🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@apps/desktop/src/components/settings/ai-provider-row.tsx` around lines 122 - 137, In the disabled ModelCombobox branch, compute transcriptionModelLabel(config) once in a local variable and reuse it for the value, model id, and model label fields. Keep the existing ModelCombobox behavior and configuration unchanged.apps/desktop/src/components/settings/add-ai-provider-dialog.tsx (1)
251-260: 🎯 Functional Correctness | 🔵 Trivial | ⚡ Quick winDisable the transcription-default checkbox for a disabled transcription model.
The checkbox at Line 251-260 has no disabled state. A user can select the disabled sentinel in the transcription model combobox and still check "Use as default for transcription."
ai-provider-row.tsxdisables its equivalent "Make transcription default" button withdisabled={!supportsTranscription}for this exact case. Apply the same guard here for consistency, so the checkbox does not silently no-op.♻️ Proposed fix
{provider.supportsTranscription || isOpenAICompatible ? ( <label className="flex items-center gap-2"> <input type="checkbox" className="accent-accent" + disabled={isOpenAICompatible && transcriptionModelValue === DISABLED_OPENAI_COMPATIBLE_MODEL} {...register('isTranscriptionDefault')} /> <span className="text-sm text-text">Use as default for transcription</span> </label> ) : null}🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@apps/desktop/src/components/settings/add-ai-provider-dialog.tsx` around lines 251 - 260, Update the transcription-default checkbox in the add-provider dialog’s provider supportsTranscription/isOpenAICompatible branch to be disabled when the selected transcription model is the disabled sentinel, matching the guard used by ai-provider-row.tsx. Reuse the existing selected-model state or helper that identifies the disabled model, while preserving the current rendering conditions.packages/core/src/ai/provider-config.ts (2)
126-141: 📐 Maintainability & Code Quality | 🔵 Trivial | 💤 Low valueSort comparator uses single-character parameter names.
(a, b) => {...}at line 134 uses single-character variable names. Rename them to something descriptive, for exampleleft/rightorproviderA/providerB.♻️ Proposed rename
- return candidates.sort((a, b) => { - const groupDiff = groupOrder(a.provider) - groupOrder(b.provider) + return candidates.sort((left, right) => { + const groupDiff = groupOrder(left.provider) - groupOrder(right.provider) if (groupDiff !== 0) return groupDiff - const aDefault = a.id === state.defaultTranscriptionProviderId ? 0 : 1 - const bDefault = b.id === state.defaultTranscriptionProviderId ? 0 : 1 - return aDefault - bDefault + const leftDefault = left.id === state.defaultTranscriptionProviderId ? 0 : 1 + const rightDefault = right.id === state.defaultTranscriptionProviderId ? 0 : 1 + return leftDefault - rightDefault })As per coding guidelines, "Never use single-character variable names."
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/core/src/ai/provider-config.ts` around lines 126 - 141, Rename the transcriptionProviders sort comparator parameters from single-character names to descriptive names such as left and right, and update all references within the comparator consistently without changing sorting behavior.Source: Coding guidelines
143-153: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winDoc comment describes behavior
pickTranscriptionConfigdoes not perform.The doc comment says this returns "the first transcription-capable provider whose keychain key resolves" and that "a keyless entry is skipped rather than stopping the pass." The implementation is synchronous and only returns
candidates[0]— it never checks the keychain. That description matchesresolveTranscriptionTarget(lines 197-203), not this function. Both callers (audio-memo.tsfor an error message,use-audio-memo-pipeline.tsfor an existence check) only need "the first configured transcription-capable provider," so the implementation itself is fine — only the doc is misleading.📝 Proposed doc fix
/** - * The configured entry audio transcription should run on: the first - * transcription-capable provider whose keychain key resolves. Providers - * are tried in {`@link` transcriptionProviders} order; a keyless entry is - * skipped rather than stopping the pass. `null` means no capable provider - * is configured — the feature is unavailable. + * The first configured transcription-capable provider, in + * {`@link` transcriptionProviders} order. This does not check the keychain — + * use {`@link` resolveTranscriptionTarget} to resolve a usable key. `null` + * means no capable provider is configured — the feature is unavailable. */🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/core/src/ai/provider-config.ts` around lines 143 - 153, Correct the doc comment for pickTranscriptionConfig to describe only its actual synchronous behavior: return the first configured transcription-capable provider, or null when none are configured. Remove references to keychain resolution, provider skipping, and runtime availability; leave the implementation unchanged.packages/core/src/actions/audio-memo.ts (1)
576-583: 📐 Maintainability & Code Quality | 🔵 Trivial | ⚡ Quick winReplace the
as TranscriptionProviderassertion with a type guard.
config.provider as TranscriptionProviderasserts a narrowing that TypeScript cannot verify. It is only safe becauseresolveTranscriptionTarget's candidates are pre-filtered bytranscriptionProvidersto openai/google/openai-compatible — but nothing at this call site enforces that invariant at the type level. This assertion became necessary becauseTranscriptionTarget.configwas broadened toAiProviderConfiginprovider-config.ts(previously narrower).Export a type guard (for example
isTranscriptionProvider(provider): provider is TranscriptionProvider, backed byTRANSCRIPTION_PROVIDERS.includes(...)) fromprovider-config.tsand use it here instead of asserting.♻️ Proposed refactor
+export function isTranscriptionProvider(provider: AiProviderId): provider is TranscriptionProvider { + return (TRANSCRIPTION_PROVIDERS as readonly string[]).includes(provider) +}- const parts = await transcribeSessionParts({ - session, - provider: config.provider as TranscriptionProvider, + if (!isTranscriptionProvider(config.provider)) { + throw new ReflectError('config', `unexpected transcription provider: ${config.provider}`) + } + const parts = await transcribeSessionParts({ + session, + provider: config.provider,As per coding guidelines, "Avoid unnecessary type assertions" and "Use discriminated unions and type guards for variant data; export helper predicates when they clarify a public contract."
🤖 Prompt for AI Agents
Verify each finding against current code. Fix only still-valid issues, skip the rest with a brief reason, keep changes minimal, and validate. In `@packages/core/src/actions/audio-memo.ts` around lines 576 - 583, Replace the `config.provider as TranscriptionProvider` assertion in the transcription configuration flow with an exported type guard from `provider-config.ts`, such as `isTranscriptionProvider`, implemented using the existing `TRANSCRIPTION_PROVIDERS` membership check. Use that guard before constructing the provider payload so `TranscriptionProvider` is narrowed by runtime validation rather than assertion, while preserving the existing supported-provider behavior.Source: Coding guidelines
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@apps/desktop/src/components/settings/ai-provider-row.tsx`:
- Around line 104-139: Update the ariaLabel values on the default and
transcription ModelCombobox instances in the provider row to include a stable
per-entry identifier such as config.keyHint or config.baseUrl alongside
providerLabel, ensuring duplicate provider types receive distinct accessible
names while preserving the existing model-role descriptions.
In `@apps/desktop/src/mobile/add-ai-provider-drawer.tsx`:
- Around line 149-170: Update the model controls around the Select using
provider.models and the corresponding second control at the noted alternate
range so openai-compatible providers support editable custom model IDs,
including a separate custom transcription model during creation. Preserve
catalog-only Select behavior for fixed providers, and keep the existing model
state and resetUnverified handling intact.
In `@apps/desktop/src/mobile/ai-provider-actions-drawer.tsx`:
- Around line 159-162: Update the TranscriptionModelInput usage in the provider
drawer to reset its local transcription-model draft whenever the managed
provider changes, preferably by keying the component with provider.id; preserve
the existing provider and onSetTranscriptionModel props.
In `@packages/core/src/ai/chat/model-options.ts`:
- Around line 29-48: Update chatModelOptions to exclude the 'disabled' model
option before mapping provider catalog entries into picker options, while
preserving custom active model entries and all other catalog models. Ensure
ChatModelDrawer cannot receive 'disabled' through the returned ChatModelOption
list.
In `@packages/core/src/ai/provider-config.ts`:
- Around line 37-53: Update withAiProviderAdded so the implicit isDefault
fallback only selects a transcription default when
aiProviderSupportsTranscription(entry) is true, matching the existing isFirst
capability check. Preserve explicit isTranscriptionDefault behavior and add a
regression test covering a non-transcription-capable default entry with the
transcription flag omitted.
---
Outside diff comments:
In `@apps/desktop/src/mobile/add-ai-provider-drawer.tsx`:
- Around line 124-132: Update the provider-switching logic in onValueChange to
reset isTranscriptionDefault whenever aiProviderSupportsTranscription(next) is
false, alongside clearing transcriptionModel for unsupported providers. Preserve
the existing transcription-default state when the selected provider supports
transcription, and ensure submitDraft cannot receive a stale true flag.
---
Nitpick comments:
In `@apps/desktop/src/components/settings/add-ai-provider-dialog.tsx`:
- Around line 251-260: Update the transcription-default checkbox in the
add-provider dialog’s provider supportsTranscription/isOpenAICompatible branch
to be disabled when the selected transcription model is the disabled sentinel,
matching the guard used by ai-provider-row.tsx. Reuse the existing
selected-model state or helper that identifies the disabled model, while
preserving the current rendering conditions.
In `@apps/desktop/src/components/settings/ai-provider-row.tsx`:
- Around line 122-137: In the disabled ModelCombobox branch, compute
transcriptionModelLabel(config) once in a local variable and reuse it for the
value, model id, and model label fields. Keep the existing ModelCombobox
behavior and configuration unchanged.
In `@packages/core/src/actions/audio-memo.ts`:
- Around line 576-583: Replace the `config.provider as TranscriptionProvider`
assertion in the transcription configuration flow with an exported type guard
from `provider-config.ts`, such as `isTranscriptionProvider`, implemented using
the existing `TRANSCRIPTION_PROVIDERS` membership check. Use that guard before
constructing the provider payload so `TranscriptionProvider` is narrowed by
runtime validation rather than assertion, while preserving the existing
supported-provider behavior.
In `@packages/core/src/ai/provider-config.ts`:
- Around line 126-141: Rename the transcriptionProviders sort comparator
parameters from single-character names to descriptive names such as left and
right, and update all references within the comparator consistently without
changing sorting behavior.
- Around line 143-153: Correct the doc comment for pickTranscriptionConfig to
describe only its actual synchronous behavior: return the first configured
transcription-capable provider, or null when none are configured. Remove
references to keychain resolution, provider skipping, and runtime availability;
leave the implementation unchanged.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: b557b08d-0f1f-4003-ac9f-ef66725e38a1
📒 Files selected for processing (51)
apps/desktop/src/components/audio-memo/audio-memo-button.test.tsxapps/desktop/src/components/audio-memo/audio-memo-button.tsxapps/desktop/src/components/settings/add-ai-provider-dialog.tsxapps/desktop/src/components/settings/ai-provider-row.tsxapps/desktop/src/components/settings/ai-providers-section.test.tsxapps/desktop/src/components/settings/ai-providers-section.tsxapps/desktop/src/components/settings/describe-assets-field.tsxapps/desktop/src/components/settings/model-combobox.tsxapps/desktop/src/components/settings/section.tsxapps/desktop/src/components/sidebar/sidebar.test.tsxapps/desktop/src/hooks/use-ai-providers.tsapps/desktop/src/hooks/use-audio-memo-pipeline.tsapps/desktop/src/lib/asset-describe-controller.test.tsxapps/desktop/src/lib/capture-controller.test.tsxapps/desktop/src/lib/transcription-reconciler.test.tsxapps/desktop/src/mobile/add-ai-provider-drawer.test.tsxapps/desktop/src/mobile/add-ai-provider-drawer.tsxapps/desktop/src/mobile/ai-provider-actions-drawer.test.tsxapps/desktop/src/mobile/ai-provider-actions-drawer.tsxapps/desktop/src/mobile/audio-memo-fab.tsxapps/desktop/src/mobile/audio-memo-provider.test.tsxapps/desktop/src/mobile/audio-memo-provider.tsxapps/desktop/src/mobile/recording-drawer.test.tsxapps/desktop/src/mobile/recording-drawer.tsxapps/desktop/src/mobile/screens/settings.tsxapps/desktop/src/providers/asset-describe-provider.tsxapps/desktop/src/providers/audio-memo-provider.test.tsxapps/desktop/src/providers/audio-memo-provider.tsxapps/desktop/src/providers/capture-provider.tsxapps/desktop/src/providers/chat-provider.tsxapps/desktop/src/providers/settings-provider.test.tsxpackages/core/src/actions/asset-description.test.tspackages/core/src/actions/audio-memo-session.tspackages/core/src/actions/audio-memo.test.tspackages/core/src/actions/audio-memo.tspackages/core/src/actions/capture-harness.tspackages/core/src/ai/audio-memo-title.test.tspackages/core/src/ai/chat/model-options.test.tspackages/core/src/ai/chat/model-options.tspackages/core/src/ai/language-model.test.tspackages/core/src/ai/openai-compatible.tspackages/core/src/ai/provider-catalog.test.tspackages/core/src/ai/provider-catalog.tspackages/core/src/ai/provider-config.test.tspackages/core/src/ai/provider-config.tspackages/core/src/ai/secrets.test.tspackages/core/src/ai/transcribe.test.tspackages/core/src/ai/transcribe.tspackages/core/src/exports/ai-actions.tspackages/core/src/settings/schema.test.tspackages/core/src/settings/schema.ts
- Add optional transcriptionModel field to openai-compatible provider config - Add defaultTranscriptionProviderId to settings schema (independent of chat default) - Add supportsTranscription flag to AiProviderInfo catalog - Add aiProviderSupportsTranscription() helper that merges catalog constants with per-entry user configuration for openai-compatible providers Transcription eligibility is now derived: hosted providers are known at compile time (OpenAI/Google: true, Anthropic/OpenRouter: false), while openai-compatible entries become eligible when the user supplies a non-empty transcriptionModel.
Replace the compile-time TRANSCRIPTION_PROVIDERS constant with a dynamic transcriptionProviders() function that derives eligible providers from the configured state. Changes: - transcriptionProviders(): returns all providers where aiProviderSupportsTranscription() is true, sorted OpenAI > Google > openai-compatible, with the transcription default first within each group - pickTranscriptionConfig(): returns the first candidate (no longer hardcoded to only OpenAI/Google) - resolveTranscriptionTarget(): iterates candidates, skips keyless entries - defaultTranscriptionProvider(): resolves the transcription default, falling back to the first transcription-capable entry - AiProvidersState gains defaultTranscriptionProviderId - withAiProviderAdded/Removed maintain the transcription default alongside the chat default The TranscriptionProvider type is widened to include 'openai-compatible' alongside 'openai' and 'google'.
Extend transcribeAudio and transcribeWithOpenAi to accept optional
baseUrl and model parameters. When provided (openai-compatible
providers), the client posts to {baseUrl}/audio/transcriptions instead
of the hardcoded api.openai.com endpoint, and uses the user-supplied
model instead of the default with fallback retry.
The provider routing is simplified: anything that isn't 'google' now
flows through transcribeWithOpenAi, since openai-compatible providers
use the same /v1/audio/transcriptions API shape.
…ipeline memoNoteBody now accepts AiProviderConfig (instead of the removed TranscriptionConfig) and forwards baseUrl + transcriptionModel to transcribeAudio when the provider is openai-compatible. The error message for missing transcription config is updated from 'No OpenAI or Gemini model is configured' to the generic 'No transcription provider is configured'. Exports are updated to include the new public API surface: aiProviderSupportsTranscription, defaultTranscriptionProvider, resolveTranscriptionTarget, transcriptionProviders.
…scriptionProviderId Every AiProvidersState construction site now includes the new defaultTranscriptionProviderId field. useAiProviders gains a makeTranscriptionDefault callback that writes the transcription default independently of the chat default. New openai-compatible entries are initialized with an empty transcriptionModel (transcription off by default).
Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
Remove duplicate TranscriptionTarget, TranscriptionMiss, and resolveTranscriptionTarget definitions left over from a merge between the master-base version and the current-branch version. The retained version uses the dynamic transcriptionProviders() function with the correct defaultTranscriptionProviderId. Also: - Add TRANSCRIPTION_PROVIDERS constant for desktop consent UI - Remove unused memoNoteBody function (dead code) - Fix type narrowing with as TranscriptionProvider assertion - Fix test snapshots missing defaultTranscriptionProviderId Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
…lt provider - Split default checkbox into chat and transcription options in add forms - Add transcriptionModel field to add-provider forms (openai-compatible) - Extend AiProviderRow with second row for transcription (model + default) - Add setTranscriptionModel hook and transcriptionProvider to useAiProviders - Update AiProviderActionsDrawer with transcription model editing - Update NO_PROVIDER_REASON to include OpenAI-compatible providers - Export transcription model constants from @reflect/core Co-Authored-By: Claude Opus 4.8 (1M context) <noreply@anthropic.com>
The openai-compatible transcription provider was effectively non-functional: two wiring gaps stopped it from ever reaching the user's local endpoint. 1. transcribeSessionParts dropped baseUrl and transcriptionModel when calling transcribeAudio, so audio was sent to OpenAI's hosted endpoint with gpt-4o-mini-transcribe regardless of the configured local server. 2. resolveTranscriptionTarget resolved keys via a raw getSecret, which bypassed aiApiKeyForConfig's no-key handling - a keyless openai-compatible entry (a local server with no API key) always read null from the keychain and was skipped as 'no-key', silently stopping every pass before any transcription call. The reconciler now forwards config.baseUrl and config.transcriptionModel for openai-compatible providers, and resolveTranscriptionTarget receives the full AiProviderConfig so callers can use aiApiKeyForConfig for key resolution.
The model combobox trigger (w-full inside an auto grid column) could outgrow the section card, clipping the Default badge at the card edge; the row also showed a noisy 'Transcription disabled (no model set)' hint under the base URL. - Drop the transcription-disabled hint; the left column shows only the key hint and base URL. - Give badges their own grid column and center them on the combobox rows (h-8), and center the remove button vertically. - Clip the settings card (overflow-hidden) and floor the combobox popover width at 20rem so narrow triggers still get a readable list.
OpenAI/Google rows render their fixed transcription model as an inert ModelCombobox (same shape as the editable pickers, never opens) instead of a bare muted string, so the two model rows read consistently. The model remains a compile-time constant; the single-item curated list is only there to satisfy the combobox contract.
The openai-compatible catalog now offers two model options — local-model and a Disabled sentinel — for both the chat and the transcription slot. Picking Disabled opts the entry out of that feature: - core: aiProviderSupportsChat predicate; chatModelOptions and defaultAiProvider skip chat-disabled entries; the transcription predicate treats 'disabled' like the legacy unset ''. - desktop: both model pickers on a settings row are catalog-driven; the transcription combo stays visible when disabled, with Make default / Make transcription default greyed out for an ineligible slot. The add dialog's transcription free-text input becomes the same combobox. - ios: the add sheet offers the catalog list as a Select; the manage sheet keeps the transcription section visible while disabled and greys the default actions the same way. Transcription model ids remain free-form (custom ids via the combobox on desktop, the text field on iOS); the sentinel only names the two states the feature toggles between.
The settings list value now appends '· Transcription' alongside the chat '· Default' marker, so an entry that only carries the transcription default is still distinguishable from a plain row.
…s too The add-provider dialog/sheet offered a free-text input for the chat model of an openai-compatible endpoint while every other model field had moved to the catalog pickers. Both now use the same controls as the transcription slot: the desktop dialog gets the ModelCombobox (catalog options, custom ids by typing + Enter), the mobile sheet the catalog Select.
Add JSDoc to the previously-undocumented exports touched by PR team-reflect#1007 across the transcription surface: the openai-compatible constants and URL helpers, the fixed OpenAI/Gemini transcription model ids, the SendOptions interface, and the capture/reconcile/session input-output types and functions in actions/audio-memo*.
- provider-config: guard isDefault transcription fallback with capability check - provider-config: export isTranscriptionProvider type guard, drop assertion - provider-config: rename sort comparator params, fix pickTranscriptionConfig doc - model-options: exclude disabled sentinel from chat picker - audio-memo: use type guard instead of provider type assertion - mobile add-provider: allow free-form model entry for openai-compatible - mobile add-provider: reset isTranscriptionDefault when provider lacks capability - mobile actions-drawer: key TranscriptionModelInput by provider id - desktop provider-row: disambiguate combobox ariaLabel per entry - desktop provider-row: cache transcriptionModelLabel in disabled branch - desktop add-dialog: disable transcription-default checkbox for disabled model
503b63d to
aa090de
Compare
There was a problem hiding this comment.
Actionable comments posted: 1
🤖 Prompt for all review comments with AI agents
Verify each finding against current code. Fix only still-valid issues, skip the
rest with a brief reason, keep changes minimal, and validate.
Inline comments:
In `@apps/desktop/src/mobile/ai-provider-actions-drawer.test.tsx`:
- Around line 123-149: Add an unsaved transcription model edit to the first
input before rerendering the provider, then blur the newly rendered input after
the provider change. Keep the assertion that onSetTranscriptionModel remains
unused, so the test exercises and verifies the stale-draft save path in the
provider-change case.
🪄 Autofix (Beta)
Fix all unresolved CodeRabbit comments on this PR:
- Push a commit to this branch (recommended)
- Create a new PR with the fixes
ℹ️ Review info
⚙️ Run configuration
Configuration used: Organization UI
Review profile: CHILL
Plan: Pro Plus
Run ID: 35a43017-8810-490a-b02b-f91f665c476f
📒 Files selected for processing (52)
apps/desktop/src/components/audio-memo/audio-memo-button.test.tsxapps/desktop/src/components/audio-memo/audio-memo-button.tsxapps/desktop/src/components/settings/add-ai-provider-dialog.tsxapps/desktop/src/components/settings/ai-provider-row.tsxapps/desktop/src/components/settings/ai-providers-section.test.tsxapps/desktop/src/components/settings/ai-providers-section.tsxapps/desktop/src/components/settings/describe-assets-field.tsxapps/desktop/src/components/settings/model-combobox.tsxapps/desktop/src/components/settings/section.tsxapps/desktop/src/components/sidebar/sidebar.test.tsxapps/desktop/src/hooks/use-ai-providers.tsapps/desktop/src/hooks/use-audio-memo-pipeline.tsapps/desktop/src/lib/asset-describe-controller.test.tsxapps/desktop/src/lib/capture-controller.test.tsxapps/desktop/src/lib/transcription-reconciler.test.tsxapps/desktop/src/mobile/add-ai-provider-drawer.test.tsxapps/desktop/src/mobile/add-ai-provider-drawer.tsxapps/desktop/src/mobile/ai-provider-actions-drawer.test.tsxapps/desktop/src/mobile/ai-provider-actions-drawer.tsxapps/desktop/src/mobile/audio-memo-fab.tsxapps/desktop/src/mobile/audio-memo-provider.test.tsxapps/desktop/src/mobile/audio-memo-provider.tsxapps/desktop/src/mobile/recording-drawer.test.tsxapps/desktop/src/mobile/recording-drawer.tsxapps/desktop/src/mobile/screens/settings.tsxapps/desktop/src/providers/asset-describe-provider.tsxapps/desktop/src/providers/audio-memo-provider.test.tsxapps/desktop/src/providers/audio-memo-provider.tsxapps/desktop/src/providers/capture-provider.tsxapps/desktop/src/providers/chat-provider.tsxapps/desktop/src/providers/settings-provider.test.tsxpackages/core/src/actions/asset-description.test.tspackages/core/src/actions/audio-memo-session.tspackages/core/src/actions/audio-memo.test.tspackages/core/src/actions/audio-memo.tspackages/core/src/actions/capture-harness.tspackages/core/src/ai/audio-memo-title.test.tspackages/core/src/ai/chat/model-options.test.tspackages/core/src/ai/chat/model-options.tspackages/core/src/ai/language-model.test.tspackages/core/src/ai/openai-compatible.tspackages/core/src/ai/provider-catalog.test.tspackages/core/src/ai/provider-catalog.tspackages/core/src/ai/provider-config.test.tspackages/core/src/ai/provider-config.tspackages/core/src/ai/secrets.test.tspackages/core/src/ai/transcribe-http.tspackages/core/src/ai/transcribe.test.tspackages/core/src/ai/transcribe.tspackages/core/src/exports/ai-actions.tspackages/core/src/settings/schema.test.tspackages/core/src/settings/schema.ts
🚧 Files skipped from review as they are similar to previous changes (49)
- apps/desktop/src/mobile/recording-drawer.test.tsx
- apps/desktop/src/providers/audio-memo-provider.test.tsx
- packages/core/src/ai/language-model.test.ts
- apps/desktop/src/components/sidebar/sidebar.test.tsx
- apps/desktop/src/components/audio-memo/audio-memo-button.tsx
- apps/desktop/src/mobile/audio-memo-fab.tsx
- apps/desktop/src/components/settings/describe-assets-field.tsx
- apps/desktop/src/mobile/audio-memo-provider.test.tsx
- apps/desktop/src/mobile/audio-memo-provider.tsx
- apps/desktop/src/components/audio-memo/audio-memo-button.test.tsx
- apps/desktop/src/providers/asset-describe-provider.tsx
- packages/core/src/ai/secrets.test.ts
- apps/desktop/src/lib/asset-describe-controller.test.tsx
- packages/core/src/actions/asset-description.test.ts
- apps/desktop/src/providers/capture-provider.tsx
- packages/core/src/ai/chat/model-options.ts
- apps/desktop/src/mobile/recording-drawer.tsx
- apps/desktop/src/providers/audio-memo-provider.tsx
- packages/core/src/ai/provider-catalog.test.ts
- packages/core/src/ai/audio-memo-title.test.ts
- apps/desktop/src/components/settings/ai-providers-section.tsx
- packages/core/src/actions/audio-memo.test.ts
- apps/desktop/src/hooks/use-audio-memo-pipeline.ts
- packages/core/src/ai/chat/model-options.test.ts
- packages/core/src/settings/schema.test.ts
- apps/desktop/src/providers/settings-provider.test.tsx
- packages/core/src/ai/openai-compatible.ts
- apps/desktop/src/components/settings/ai-providers-section.test.tsx
- apps/desktop/src/mobile/screens/settings.tsx
- packages/core/src/exports/ai-actions.ts
- apps/desktop/src/providers/chat-provider.tsx
- apps/desktop/src/components/settings/model-combobox.tsx
- apps/desktop/src/lib/transcription-reconciler.test.tsx
- packages/core/src/actions/audio-memo.ts
- apps/desktop/src/components/settings/section.tsx
- apps/desktop/src/lib/capture-controller.test.tsx
- apps/desktop/src/mobile/add-ai-provider-drawer.tsx
- packages/core/src/actions/capture-harness.ts
- packages/core/src/settings/schema.ts
- apps/desktop/src/hooks/use-ai-providers.ts
- apps/desktop/src/components/settings/ai-provider-row.tsx
- packages/core/src/ai/provider-catalog.ts
- packages/core/src/actions/audio-memo-session.ts
- packages/core/src/ai/transcribe.ts
- apps/desktop/src/components/settings/add-ai-provider-dialog.tsx
- packages/core/src/ai/provider-config.test.ts
- apps/desktop/src/mobile/ai-provider-actions-drawer.tsx
- packages/core/src/ai/provider-config.ts
- packages/core/src/ai/transcribe.test.ts
CodeRabbit noted the transcription-model reset test only checked the value after rerender; it did not verify the stale-draft save path is dead. Fill the input with an unsaved edit before rerendering, then blur the remounted input and assert onSetTranscriptionModel stays uncalled - proving the key-driven remount discards the old draft.
The iOS cargo build compiles all three crate-types (staticlib, cdylib, rlib) before Xcode compiles NativeDiagnostics.swift. The staticlib (libapp.a, what iOS actually links) tolerates the unresolved reflect_start_native_diagnostics reference; the cdylib does not, and its link step failed before Xcode ran. Pass -undefined dynamic_lookup to the cdylib link only - the cdylib is not shipped on iOS, so this just unblocks the cargo step the Xcode build phase runs.
Problem
#996 added
openai-compatibleproviders for chat, editor AI transforms, capture enrichment, asset descriptions, and audio memo title generation. That change did not include transcription. The/v1/audio/transcriptionsendpoint is separate from/v1/chat/completions. The model types also differ (speech-to-text versus chat).Users with OpenAI-compatible transcription endpoints want to route transcription through the same provider entry. These endpoints include local Whisper servers, Groq, and API proxies that expose
/v1/audio/transcriptions. The base URL stays the same. Only the model differs.This PR supersedes #995. The scope grew beyond transcription. It now adds independent chat and transcription defaults. It also adds a
disabledsentinel for openai-compatible entries and the full desktop and mobile settings UI.The original PR body no longer matches the diff. The bot review also landed on a stale description. This PR starts from the same commits with a clean review surface.
Changes
Schema and catalog
openai-compatibleentries gain an optionaltranscriptionModelfield. An empty string (the default) and the'disabled'sentinel both mean the entry does not transcribe.defaultTranscriptionProviderIdsetting is independent of the chat default. Chat and transcription can use different providers.AiProviderInfogainssupportsTranscription. This is a compile-time constant for hosted providers. OpenAI and Google set it to true. Anthropic and OpenRouter set it to false. Foropenai-compatible, the catalog declaresfalse. The user opts in per entry viatranscriptionModel.aiProviderSupportsTranscription()merges the catalog constant with the per-entrytranscriptionModel.aiProviderSupportsChat()lets anopenai-compatibleentry opt out of chat by settingmodelto'disabled'(a Whisper-only endpoint).DISABLED_OPENAI_COMPATIBLE_MODELsentinel as an option so it round-trips through the settings schema.Provider config
TRANSCRIPTION_PROVIDERSconstant withtranscriptionProviders(). This function derives eligible providers from the configured state.resolveTranscriptionTargetandpickTranscriptionConfigwork with any provider that has transcription capability.defaultTranscriptionProvider()resolves the transcription default the same waydefaultAiProvider()resolves the chat default.TranscriptionProviderwidens to include'openai-compatible'. This PR removesTranscriptionConfigand usesAiProviderConfiginstead.Transcription client
transcribeAudioaccepts optionalbaseUrlandmodelparams. When the provider isopenai-compatible, the client posts to{baseUrl}/audio/transcriptionswith the user-supplied model.httpErrorreports the actual provider, not a hardcoded'openai'.Audio memo pipeline
reconcileAudioMemosresolves the key throughaiApiKeyForConfig(shared with chat) instead of a direct keychain read. This lets openai-compatible entries with an optional key work.transcribeSessionPartsforwardsbaseUrlandtranscriptionModelwhen the provider isopenai-compatible.Chat model options
chatModelOptions()filters out entries whose chat model is the'disabled'sentinel. The picker cannot switch to a chat-disabled entry.Settings UI (desktop and mobile)
openai-compatible. Hosted transcription-capable providers show their fixed model as a disabled combo.ModelComboboxgains adisabledprop for the fixed-model display.Screenshots
Checklist
pnpm typecheckpassespnpm lintpasses (no new warnings)provider-config,provider-catalog,transcribe,secrets,model-options,schema,audio-memo,ai-providers-section,transcription-reconciler,add-ai-provider-drawer,ai-provider-actions-drawer(196 tests total)anytypes, zod at boundaries, kebab-case filenamesSelectinside the add-providerDialoghas a pre-existing interaction bug. The dropdown flashes open then becomes unresponsive on the next click. You cannot manually verify the new transcription model picker on iOS. This PR did not introduce the bug. It affects the existing provider picker too.)Relation to #995
#995 is the same work at an earlier stage. This PR replaces it. The commits are identical. Only the PR body and review surface are new.
Summary by CodeRabbit