Skip to content

feat(stt): serve /tasks/audio/transcriptions from the 51-language ASR model#241

Merged
PatrickCmd merged 3 commits into
mainfrom
faster-whisper-asr
Jul 23, 2026
Merged

feat(stt): serve /tasks/audio/transcriptions from the 51-language ASR model#241
PatrickCmd merged 3 commits into
mainfrom
faster-whisper-asr

Conversation

@PatrickCmd

Copy link
Copy Markdown
Contributor

Summary

Replaces the Modal/RunPod dispatch behind POST /tasks/audio/transcriptions with Sunbird's faster-whisper ASR deployment (Sunbird/faster-whisper-51-african-languages, a Whisper large-v3 fine-tune), taking the endpoint from 10 to 51 African languages.

The 51-code list was taken from language_map.json in the converted repo rather than the prose model card. All ten previously supported languages (ach teo eng lug lgg nyn swa kin xog myx) are still served — no language regresses.

The deployment is OpenAI-compatible and maps Sunbird ISO 639-3 codes to Whisper's language-token slots server-side, so codes are forwarded verbatim. Because the fine-tune reuses those slots, auto-detection is meaningless and language stays required.

⚠️ Breaking change

The endpoint no longer accepts platform, adapter, whisper, recognise_speakers, org, or gcs_blob_name — these now return 422. Speaker diarization and the organization workflow remain available on the deprecated /tasks/stt* routes.

New contract:

POST /tasks/audio/transcriptions
  audio      : file            required
  language   : <one of 51>     required
  timestamps : bool = false    -> populates `segments`

⚠️ Deploy requirement

ASR_BASE_URL must be set in Cloud Run before this ships. It has no working default — the fallback is a literal <ENDPOINT_ID> placeholder, so an unconfigured deploy fails at request time, not at boot. It must include the trailing /v1. Auth reuses the existing RUNPOD_API_KEY.

ASR_MODEL_NAME (sunbird-asr-51) and ASR_TIMEOUT_SECONDS (600, generous for cold starts) are optional.

Changes

File Change
app/services/asr_service.py New. ASRService — httpx client for the deployment; maps HTTP/timeout/connection/parse failures to ExternalServiceError.
app/core/config.py Adds asr_base_url, asr_model_name, asr_timeout_seconds; promotes RUNPOD_API_KEY to a real Settings field.
app/schemas/stt.py Adds ASRLanguage (51 codes), ASR_LANGUAGE_NAMES, TranscriptionSegment. STTTranscript gains segments + duration_seconds. SttbLanguage untouched for the legacy routes.
app/routers/audio.py Endpoint rewritten to the contract above.
app/deps.py TranscriptionServiceDepASRServiceDep.
app/services/transcription_service.py Deleted — existed only to dispatch across the removed platforms.
.env.example, app/docs.py, docs/tutorial.md, frontend/src/pages/Tutorial.tsx Full 51-language list, timestamps section, breaking-change notice. Endpoint matrix grew from 44 to 68 rows.

Two behavioural notes: audio_url is a gs:// URI (matching what the legacy RunPod path returned, not a signed HTTPS URL), and the GCS upload is best-effort — if it fails, transcription still succeeds with audio_url/audio_transcription_id as null.

Test plan

  • pytest app/tests/1189 passed. 6 failures in test_config.py / test_inference_service.py reproduce on a clean git stash: pre-existing, caused by a local .env leaking into Settings.
  • New test_asr_service.py (12 tests, mocked httpx transport: request shape, verbose_json/segments, language fallback, HTTP/timeout/connection/malformed-body error mapping, health check).
  • test_audio_transcriptions.py rewritten — new contract, removed-param rejection, segments on/off, 51-code enum, upstream-error → 502.
  • make lint-check — black/isort clean; only pre-existing node_modules/flatted E722 remains.
  • npm run build (incl. tsc) passes.
  • Not run: npm run lint — pre-existing breakage, ESLint 10.5.0 installed against an .eslintrc.* config (v9+ requires eslint.config.js). Unrelated to this PR; no JS dependencies added.
  • Not verified: live transcription against the deployed endpoint — needs ASR_BASE_URL pointed at a running worker.

npm audit reports 8 pre-existing vulnerabilities (3 high, vite/launch-editor); the fix is a breaking vite@8 major, left for a separate PR.

🤖 Generated with Claude Code

PatrickCmd and others added 3 commits July 23, 2026 10:29
Replace the Modal/RunPod dispatch behind POST /tasks/audio/transcriptions
with Sunbird's faster-whisper ASR deployment
(Sunbird/faster-whisper-51-african-languages), expanding coverage from 10
to 51 African languages. All ten previously supported languages remain.

The deployment is OpenAI-compatible and maps Sunbird ISO 639-3 codes to
Whisper's language-token slots server-side, so codes are forwarded
verbatim. Because the fine-tune reuses those slots, auto-detection is
unreliable and `language` stays required.

BREAKING CHANGE: the endpoint no longer accepts `platform`, `adapter`,
`whisper`, `recognise_speakers`, `org`, or `gcs_blob_name`; they now
return 422. Speaker diarization and the organization workflow remain
available on the deprecated /tasks/stt* routes.

- add ASRService (app/services/asr_service.py) wrapping the deployment,
  mapping HTTP/timeout/connection/parse failures to ExternalServiceError
- add ASR_BASE_URL, ASR_MODEL_NAME and ASR_TIMEOUT_SECONDS settings;
  promote RUNPOD_API_KEY to a Settings field and document all four in
  .env.example. ASR_BASE_URL has no working default and must be set.
- add ASRLanguage (51 codes) and TranscriptionSegment; STTTranscript
  gains optional `segments` (via timestamps=true) and `duration_seconds`
- retire TranscriptionService, which existed only to dispatch across the
  removed platforms
- update app/docs.py, docs/tutorial.md and the React Tutorial page with
  the full 51-language list and the breaking-change notice

Co-Authored-By: Claude Opus 4.8 <noreply@anthropic.com>
@PatrickCmd
PatrickCmd merged commit 2379066 into main Jul 23, 2026
2 checks passed
Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

None yet

Projects

None yet

Development

Successfully merging this pull request may close these issues.

1 participant