diff --git a/.agents/AGENTS.md b/.agents/AGENTS.md index 3b34cb5..24e3c35 100644 --- a/.agents/AGENTS.md +++ b/.agents/AGENTS.md @@ -29,6 +29,7 @@ make run-dev # Run with custom port (5092) for development make models # Download int8 models (default, ~670MB) make models-int8 # Download int8 quantized models make models-fp32 # Download full precision models (~2.5GB) +make models-silero-vad # Download + verify the Silero VAD model (MIT, pinned) # Test make test # Run tests @@ -67,6 +68,10 @@ parakeet/ ├── internal/ │ ├── asr/ │ │ ├── transcriber.go # ONNX inference pipeline, TDT decoding +│ │ ├── chunker.go # Long-audio window planning + VAD/mel/midpoint boundaries +│ │ ├── boundary.go # Chunk-boundary oracle cascade (VAD -> mel energy -> midpoint) +│ │ ├── vad.go # Silero VAD ONNX session wrapper (shared, stateful via tensors) +│ │ ├── seam.go # Seam-level token dedup (absolute-timestep based) │ │ ├── mel.go # Mel filterbank feature extraction (FFT, windowing) │ │ ├── audio.go # WAV parsing, magic-byte detection, resampling to 16kHz │ │ ├── ffmpeg.go # Optional ffmpeg-backed converter for non-WAV inputs @@ -76,7 +81,9 @@ parakeet/ │ ├── server.go # HTTP server, route setup, lifecycle management │ ├── handlers.go # API endpoint handlers, response formatting │ └── types.go # Request/response type definitions -├── models/ # ONNX models (downloaded separately) +├── models/ # ONNX models (downloaded separately, incl. silero_vad.onnx) +├── testdata/ +│ └── reference/ # Issue #18 reproduction audio + reference transcripts ├── bin/ # Build output directory ├── Makefile # Build recipes ├── Dockerfile # Multi-stage container build (CPU) @@ -96,7 +103,7 @@ parakeet/ ### `main.go` (Entry Point) -- Parses CLI flags: `-port`, `-models`, `-log-level`, `-log-format`, `-workers`, `-ffmpeg`, `-ffmpeg-path`, `-ffmpeg-timeout`, `-gpu`, `-gpu-device` +- Parses CLI flags: `-port`, `-models`, `-log-level`, `-log-format`, `-workers`, `-ffmpeg`, `-ffmpeg-path`, `-ffmpeg-timeout`, `-gpu`, `-gpu-device`, `-chunk-seconds`, `-chunk-overlap-seconds`, `-long-audio`, `-disable-vad-based-chunking`, `-disable-mel-based-chunking`, `-vad-model-path` - Configures `slog` global logger (text or JSON handler, four log levels) - Runs server in background goroutine, listens for SIGINT/SIGTERM - Graceful shutdown: waits up to 30s for in-flight requests via `http.Server.Shutdown` @@ -107,7 +114,7 @@ parakeet/ #### `server.go` -- `Config` struct: Port, ModelsDir, LogLevel, LogFormat, Workers, FFmpegEnabled, FFmpegPath, FFmpegTimeout, GPUProvider, GPUDeviceID +- `Config` struct: Port, ModelsDir, LogLevel, LogFormat, Workers, FFmpegEnabled, FFmpegPath, FFmpegTimeout, GPUProvider, GPUDeviceID, ChunkSeconds, ChunkOverlapSeconds, LongAudio, DisableVADBasedChunking, DisableMelBasedChunking, VADModelPath - `Server` struct: wraps config, transcriber, `http.Server`, HTTP mux, and API key - `New()` - Parses the GPU provider via `asr.ParseProvider` (fails fast on unknown values), initializes transcriber with worker pool, execution provider, and optional ffmpeg converter, reads `PARAKEET_API_KEY` env var, and sets up routes - `Run()` - Starts HTTP listener (blocks until shutdown or error) @@ -154,6 +161,14 @@ parakeet/ - `tdtDecode()` - TDT greedy decoding loop reusing pooled session and tensors - `tokensToText()` - Token IDs to text with cleanup +#### `chunker.go`, `boundary.go`, `vad.go`, `seam.go` (Long-Audio Chunking) + +- `planForAudio` / `planForAudioWithBoundaries` - Decide single-pass vs overlapping windows (long audio); the latter takes a boundary oracle. +- `planChunks` / `planChunksWithBoundaries` - Lay out overlapping windows and split each overlap's emission ownership; the boundary is chosen by the oracle and clamped so the emit ranges always tile the timeline. +- `boundaryOracle` interface with `vadBoundaryOracle`, `melEnergyBoundaryOracle`, `midpointBoundaryOracle`, chained by `chainBoundaryOracle` (cascade VAD -> mel energy -> midpoint). See DD-014. +- `sileroVAD` - Shared Silero VAD ONNX session; `vadState` carries per-request recurrent state + context so the session is safe to share (runs OUTSIDE the worker pool). +- `dedupSeam` - Drops window i+1's leading tokens that collide (in absolute encoder-frame timestep) with window i's tail; the earlier window wins. Always on, no flag. + #### `ffmpeg.go` - `FFmpegConfig` - Public struct with `Enabled`, `BinaryPath`, `Timeout` @@ -229,6 +244,12 @@ parakeet/ | `PARAKEET_API_KEY` | API key for `/v1/*` endpoint authentication | Empty (auth disabled) | | `PARAKEET_GPU` | Execution provider: `cpu` or `cuda` | `cpu` | | `PARAKEET_GPU_DEVICE` | GPU device index for `cuda` | `0` | +| `PARAKEET_LONG_AUDIO` | Split over-limit audio into overlapping chunks | `false` | +| `PARAKEET_CHUNK_SECONDS` | Sliding-window size in seconds | `300` | +| `PARAKEET_CHUNK_OVERLAP_SECONDS` | Overlap between chunks in seconds | `15` | +| `PARAKEET_DISABLE_VAD_BASED_CHUNKING` | Drop the Silero VAD boundary layer | `false` | +| `PARAKEET_DISABLE_MEL_BASED_CHUNKING` | Drop the mel-energy boundary layer | `false` | +| `PARAKEET_VAD_MODEL_PATH` | Path to silero_vad.onnx | `/silero_vad.onnx` | ## Dependencies @@ -330,7 +351,14 @@ No other external Go dependencies. Standard library used for HTTP, JSON, audio p - `encoder-model.int8.onnx` (~652MB) or `encoder-model.onnx` (~2.5GB) - `decoder_joint-model.int8.onnx` (~18MB) or `decoder_joint-model.onnx` (~72MB) - `config.json`, `vocab.txt`, `nemo128.onnx` -- Download via `make models` or manually from HuggingFace +- `silero_vad.onnx` (~2.3MB, snakers4/silero-vad v6.2.1, MIT) - used only for VAD-aware chunk boundaries in long-audio mode. Missing file is a graceful degrade (warns once, falls back to mel energy), not a fatal error. +- Download via `make models` or manually from HuggingFace (Silero from its GitHub release) + +### Chunk Boundary Selection (long-audio mode) + +- With `-long-audio`, audio over the model's frame limit is split into overlapping windows. Each overlap's emission boundary is placed on silence by a cascade: Silero VAD, then smoothed mel energy, then the arithmetic midpoint (see DD-014). +- A second always-on safety net (`dedupSeam`) removes duplicate/colliding tokens right at each seam, using absolute encoder-frame timesteps; the earlier (warmed-up) window wins collisions. +- Tuning constants (VAD threshold, tolerances, smoothing) are named constants in `boundary.go`/`seam.go`, deliberately NOT flags. Flags only toggle whole layers (`-disable-vad-based-chunking`, `-disable-mel-based-chunking`) and the VAD model path (`-vad-model-path`). ### Tensor Memory Management diff --git a/.agents/DESIGN_DECISIONS.md b/.agents/DESIGN_DECISIONS.md index 17ee89b..e59235e 100644 --- a/.agents/DESIGN_DECISIONS.md +++ b/.agents/DESIGN_DECISIONS.md @@ -218,3 +218,57 @@ Key properties: - CPU images, CPU mode, and the int8/fp32 CPU images are unaffected. - **Known limitation**: the encoder runs in a single pass, so peak memory scales with audio length. On GPU this is bounded by VRAM — very long inputs can exceed device memory and fail with an ORT allocation error. Chunked/streaming encoding is the future fix. - Other accelerators (TensorRT, ROCm, DirectML, OpenVINO) are out of scope; `buildSessionOptions` is the single place to add them. + +## DD-014: VAD-Aware Chunk Boundaries and Seam Token Dedup + +**Context**: Long-audio mode (PR #21) covers audio longer than the model's 5000 encoder-frame limit with overlapping windows and splits ownership of each overlap at its arithmetic MIDPOINT (`chunker.go`). Because the midpoint is blind to content it often lands mid-word. Each window timestamps tokens slightly differently, so a word straddling the boundary is emitted twice (duplicates like "to to") or by neither window (drops like "for., and how"). Issue #18 reproduces this on a ~30 minute file. + +**Decision**: Move the boundary onto silence with a three-layer cascade, and add an always-on seam-level token dedup as a second safety net. Both operate only within the existing overlaps; the mission stays faithful to Parakeet TDT (no global segmentation, no silence skipping). + +### Boundary selection cascade (interface-based oracles) + +A `boundaryOracle` interface (`boundary.go`) picks the mel-frame split inside an overlap. The three implementations are alternative strategies for the same contract, tried in order by `chainBoundaryOracle`: + +1. `vadBoundaryOracle`: runs Silero VAD over the overlap's waveform and picks the CENTER of the LONGEST run of 32 ms windows whose speech probability is below `vadSilenceThreshold` (0.4, a tuned constant in the agreed 0.35-0.5 band, NOT a flag). Silero's state is reset per overlap, so its first ~1 s is warmup (`vadWarmupWindows`); a boundary after the warmup is preferred, falling back to the warmup region only if no later silence exists. Declines (falls through) when nothing is below the threshold. +2. `melEnergyBoundaryOracle`: smooths per-frame energy from the already-extracted mel features (`melSmoothingFrames` ~150 ms) and returns the quietest frame in the overlap. Always decides when it has features, so it is the robust fallback when the VAD is disabled or its model is missing. +3. `midpointBoundaryOracle`: the arithmetic midpoint, the pre-#18 behaviour. Always decides, so the result is never worse than before. + +`planChunksWithBoundaries` takes the oracle and CLAMPS whatever it returns into the legal overlap range, keeping the tiling invariant (`emitEnd[i] == emitStart[i+1]`, first `emitStart` 0, last `emitEnd` total, each emit range inside its window) for ANY oracle output. `planChunks` is now the nil-oracle (pure-midpoint) case, so the existing canary tests are unchanged. + +**Why an interface**: the three layers are genuinely interchangeable implementations of one decision, and the cascade is just "try each until one decides". An interface makes each layer independently unit-testable from synthetic probability/energy arrays and lets the disable flags drop layers by simply not adding them to the chain. + +**Why VAD only on overlaps**: the overlaps are short (15 s each by default), so the VAD cost is bounded and proportional to the number of seams, not the file length. Running Silero over the whole file would be the first step towards VAD segmentation, which this project deliberately rejects. + +### Silero VAD integration + +- Model file `silero_vad.onnx` lives in the `--models` directory like the other models, downloaded by the Makefile and Docker builds (NOT embedded in the binary). Pinned to release **v6.2.1** (MIT) with sha256 `1a153a22f4509e292a94e67d6f9b85e8deb25b4988682b7e174c65279d8788e3`, verified with `sha256sum -c` at download time. NOTE: at v6.2.1 the ONNX file is at `src/silero_vad/data/silero_vad.onnx`; the older `files/silero_vad.onnx` path no longer exists. +- One shared `DynamicAdvancedSession`, same pattern as the encoder (DD-011). Silero is stateful, but its recurrent state travels through the `state`/`stateN` TENSORS per call, not inside the session, so a single session is safe under concurrency as long as each request keeps its own `vadState`. The loop feeds 512-sample (32 ms) windows sequentially, prepending a 64-sample context and carrying the state between calls; v5+ uses the combined `state` tensor plus an `sr` sample-rate input, verified against the pinned model. +- **Missing model is not fatal**: `slog.Warn` once at startup ("VAD model not found, chunk boundaries fall back to mel energy") and degrade to the mel layer. Any other load error is fatal so a corrupt model surfaces loudly. +- **Concurrency caveat**: like the encoder, the VAD session runs OUTSIDE the decoder worker pool, so its concurrency is bounded by the number of in-flight HTTP requests, not by `-workers`. + +### Seam token dedup (always on, no flag) + +Operating on emitted tokens tagged with their ABSOLUTE encoder-frame timesteps (`decodedToken`, `seam.go`), `dedupSeam` drops a leading token of window i+1 when its timestep is within `seamTimestepToleranceFrames` (3 frames, ~240 ms) of any of window i's last `seamMaxTokens` (3) tokens. One rule covers both #18 failure modes: same text at the same position is a duplicate; different text at the same position is a collision that window i WINS (its LSTM reaches the seam fully warmed up while window i+1 is still warming). A duplicate further apart than the tolerance is kept. All tolerances are named constants with comments, not flags. + +**Streaming**: `TranscribeStream` emits deltas as tokens are produced. To let dedup run before window i+1's head reaches the client, `tdtDecode` BUFFERS at most the first `seamMaxTokens` owned tokens of each window after the first, resolves them through the deduper, streams the survivors in order, then streams the rest as it decodes. Buffering is minimal (a handful of tokens per seam) and streaming order is preserved. + +### Rejected alternatives + +- **LCS / sequence stitching** of overlapping transcripts: fragile on repeated words, needs a similarity threshold that is itself a tuning knob, and operates on text after the timing information that actually identifies the seam has been thrown away. Placing the boundary on silence removes the ambiguity at the source; the timestep-based dedup is a cheap, deterministic backstop. +- **Full-file VAD segmentation**: would skip silence and re-segment globally, changing what audio Parakeet sees and breaking faithfulness to the model. Out of scope by mission. + +### Known limitation + +Loud or dynamic music has no quiet frame, so the mel-energy layer degrades to roughly midpoint quality; the VAD layer is the robust answer there because it distinguishes speech from non-speech energy rather than loud from quiet. With the VAD disabled AND music present, boundaries fall back to the midpoint, i.e. the pre-#18 behaviour. + +**Configuration surface** (env vars come free via `applyEnvDefaults`): + +- `--disable-vad-based-chunking` (bool, default false): drops layer 1. +- `--disable-mel-based-chunking` (bool, default false): drops layer 2. +- `--vad-model-path` (string, default `silero_vad.onnx` inside `--models`). + +**Consequences**: + +- New files: `internal/asr/boundary.go` (oracle stack), `internal/asr/vad.go` (Silero session), `internal/asr/seam.go` (dedup). `chunker.go` gains `planChunksWithBoundaries`/`planForAudioWithBoundaries`; `transcriber.go` threads absolute timesteps, the seam buffer, and the per-request oracle chain. +- `silero_vad.onnx` is a new required-for-VAD model file; its absence is a graceful degrade, not a failure. +- Reference reproduction material (the issue #18 MP3 plus `.srt`/`.vtt`/`.txt`) lives in `testdata/reference/`, with a build-tag-gated Go seam inspector (`-tags=seaminspect`) that prints transcribed vs reference text around every seam. No Python, no network. diff --git a/.agents/TODO.md b/.agents/TODO.md index eb9e87f..84030e6 100644 --- a/.agents/TODO.md +++ b/.agents/TODO.md @@ -14,6 +14,7 @@ Pending tasks and improvements for the project. ## Performance +- [x] **VAD-aware chunk boundaries + seam dedup**: Fixes chunk-seam hallucinations (issue #18) in long-audio mode. Overlap ownership is split on silence via a VAD -> mel-energy -> midpoint cascade, and a seam-level token dedup removes duplicated/colliding tokens. Toggle layers with `-disable-vad-based-chunking` / `-disable-mel-based-chunking`; VAD model path via `-vad-model-path`. See DD-014. - [x] **GPU inference** — Implemented via ONNX Runtime execution providers. Opt in with `-gpu cuda` / `-gpu-device N`; a dedicated `*-cuda` Docker image ships the GPU build. CPU remains the default. See DD-013. TensorRT and other accelerators remain out of scope (extend `buildSessionOptions`). - [ ] **Batch inference support** — Current implementation processes one audio file at a time. Batching multiple requests could improve throughput under load. - [ ] **Higher quality resampling** — `audio.go:resample()` uses linear interpolation. Sinc-based or polyphase resampling would improve audio quality. diff --git a/.dockerignore b/.dockerignore index 1af5353..df7aee6 100644 --- a/.dockerignore +++ b/.dockerignore @@ -8,6 +8,7 @@ models/ # Test files test/ +testdata/ *.wav # Git diff --git a/Dockerfile b/Dockerfile index 9442413..dd66c9e 100644 --- a/Dockerfile +++ b/Dockerfile @@ -35,6 +35,12 @@ FROM debian:bookworm-slim # Model precision: "int8" (default, ~670MB) or "fp32" (~2.5GB) ARG MODEL_PRECISION=int8 +# Silero VAD model for chunk-boundary selection (long-audio mode). Pinned tag + +# sha256 so the image build is reproducible and verified. snakers4/silero-vad is +# MIT licensed. At v6.2.1 the ONNX file lives at src/silero_vad/data/. +ARG SILERO_VAD_VERSION=v6.2.1 +ARG SILERO_VAD_SHA256=1a153a22f4509e292a94e67d6f9b85e8deb25b4988682b7e174c65279d8788e3 + # Install ONNX Runtime and ffmpeg (used for non-WAV audio conversion) ARG ONNXRUNTIME_VERSION=1.25.1 ARG TARGETARCH @@ -66,6 +72,8 @@ RUN mkdir -p /models && \ curl -L -o /models/config.json "https://huggingface.co/istupakov/parakeet-tdt-0.6b-v3-onnx/resolve/main/config.json" && \ curl -L -o /models/vocab.txt "https://huggingface.co/istupakov/parakeet-tdt-0.6b-v3-onnx/resolve/main/vocab.txt" && \ curl -L -o /models/nemo128.onnx "https://huggingface.co/istupakov/parakeet-tdt-0.6b-v3-onnx/resolve/main/nemo128.onnx" && \ + curl -L -o /models/silero_vad.onnx "https://github.com/snakers4/silero-vad/raw/${SILERO_VAD_VERSION}/src/silero_vad/data/silero_vad.onnx" && \ + echo "${SILERO_VAD_SHA256} /models/silero_vad.onnx" | sha256sum -c - && \ if [ "$MODEL_PRECISION" = "fp32" ]; then \ curl -L -o /models/encoder-model.onnx "https://huggingface.co/istupakov/parakeet-tdt-0.6b-v3-onnx/resolve/main/encoder-model.onnx" && \ curl -L -o /models/encoder-model.onnx.data "https://huggingface.co/istupakov/parakeet-tdt-0.6b-v3-onnx/resolve/main/encoder-model.onnx.data" && \ diff --git a/Dockerfile.cuda b/Dockerfile.cuda index a9f7cb1..644763d 100644 --- a/Dockerfile.cuda +++ b/Dockerfile.cuda @@ -64,6 +64,12 @@ FROM ${CUDA_RUNTIME_IMAGE} ARG MODEL_PRECISION=fp32 ARG ONNXRUNTIME_VERSION +# Silero VAD model for chunk-boundary selection (long-audio mode). Pinned tag + +# sha256 so the image build is reproducible and verified. snakers4/silero-vad is +# MIT licensed. At v6.2.1 the ONNX file lives at src/silero_vad/data/. +ARG SILERO_VAD_VERSION=v6.2.1 +ARG SILERO_VAD_SHA256=1a153a22f4509e292a94e67d6f9b85e8deb25b4988682b7e174c65279d8788e3 + # Install ONNX Runtime GPU build and ffmpeg (used for non-WAV audio conversion) RUN apt-get update && apt-get install -y --no-install-recommends \ ca-certificates \ @@ -92,6 +98,8 @@ RUN mkdir -p /models && \ curl -L -o /models/config.json "https://huggingface.co/istupakov/parakeet-tdt-0.6b-v3-onnx/resolve/main/config.json" && \ curl -L -o /models/vocab.txt "https://huggingface.co/istupakov/parakeet-tdt-0.6b-v3-onnx/resolve/main/vocab.txt" && \ curl -L -o /models/nemo128.onnx "https://huggingface.co/istupakov/parakeet-tdt-0.6b-v3-onnx/resolve/main/nemo128.onnx" && \ + curl -L -o /models/silero_vad.onnx "https://github.com/snakers4/silero-vad/raw/${SILERO_VAD_VERSION}/src/silero_vad/data/silero_vad.onnx" && \ + echo "${SILERO_VAD_SHA256} /models/silero_vad.onnx" | sha256sum -c - && \ if [ "$MODEL_PRECISION" = "int8" ]; then \ curl -L -o /models/encoder-model.int8.onnx "https://huggingface.co/istupakov/parakeet-tdt-0.6b-v3-onnx/resolve/main/encoder-model.int8.onnx" && \ curl -L -o /models/decoder_joint-model.int8.onnx "https://huggingface.co/istupakov/parakeet-tdt-0.6b-v3-onnx/resolve/main/decoder_joint-model.int8.onnx"; \ diff --git a/Makefile b/Makefile index 8e957d6..9fa11c3 100644 --- a/Makefile +++ b/Makefile @@ -25,7 +25,7 @@ MODELS_DIR := ./models .PHONY: all build clean test fmt vet lint run help .PHONY: docker-build-int8 docker-build-fp32 docker-build-cuda docker-run-int8 docker-run-fp32 docker-run-cuda docker-push -.PHONY: models models-int8 models-fp32 +.PHONY: models models-int8 models-fp32 models-silero-vad .PHONY: release release-linux release-darwin release-windows .PHONY: deps-onnxruntime @@ -116,9 +116,25 @@ deps-onnxruntime: ## Install ONNX Runtime library # Models are downloaded from HuggingFace: https://huggingface.co/istupakov/parakeet-tdt-0.6b-v3-onnx # ONNX conversion by Ivan Googol Stupakov (https://github.com/istupakov) +# Silero VAD model for chunk-boundary selection (long-audio mode). Pinned to a +# specific release tag and sha256 so the download is reproducible and verified. +# snakers4/silero-vad is MIT licensed. NOTE: at v6.2.1 the ONNX file lives at +# src/silero_vad/data/silero_vad.onnx (the older files/silero_vad.onnx path is +# gone), so the URL points there. +SILERO_VAD_VERSION ?= v6.2.1 +SILERO_VAD_SHA256 ?= 1a153a22f4509e292a94e67d6f9b85e8deb25b4988682b7e174c65279d8788e3 +SILERO_VAD_URL := https://github.com/snakers4/silero-vad/raw/$(SILERO_VAD_VERSION)/src/silero_vad/data/silero_vad.onnx + models: models-int8 ## Download models (default: int8) -models-int8: ## Download int8 quantized models (~670MB) +models-silero-vad: ## Download and verify the Silero VAD model (MIT) + @mkdir -p $(MODELS_DIR) + @echo "Downloading Silero VAD $(SILERO_VAD_VERSION) (MIT) ..." + @curl -L -o $(MODELS_DIR)/silero_vad.onnx "$(SILERO_VAD_URL)" + @echo "$(SILERO_VAD_SHA256) $(MODELS_DIR)/silero_vad.onnx" | sha256sum -c - + @echo "Silero VAD model verified" + +models-int8: models-silero-vad ## Download int8 quantized models (~670MB) @mkdir -p $(MODELS_DIR) @echo "Downloading Parakeet TDT int8 models from https://huggingface.co/istupakov/parakeet-tdt-0.6b-v3-onnx ..." @curl -L -o $(MODELS_DIR)/config.json "https://huggingface.co/istupakov/parakeet-tdt-0.6b-v3-onnx/resolve/main/config.json" @@ -128,7 +144,7 @@ models-int8: ## Download int8 quantized models (~670MB) @curl -L -o $(MODELS_DIR)/decoder_joint-model.int8.onnx "https://huggingface.co/istupakov/parakeet-tdt-0.6b-v3-onnx/resolve/main/decoder_joint-model.int8.onnx" @echo "Models downloaded to $(MODELS_DIR)" -models-fp32: ## Download fp32 full precision models (~2.5GB) +models-fp32: models-silero-vad ## Download fp32 full precision models (~2.5GB) @mkdir -p $(MODELS_DIR) @echo "Downloading Parakeet TDT fp32 models from https://huggingface.co/istupakov/parakeet-tdt-0.6b-v3-onnx ..." @curl -L -o $(MODELS_DIR)/config.json "https://huggingface.co/istupakov/parakeet-tdt-0.6b-v3-onnx/resolve/main/config.json" @@ -215,6 +231,7 @@ help: ## Show this help message @echo " \033[36mmodels\033[0m Download models (default: int8)" @echo " \033[36mmodels-int8\033[0m Download int8 quantized models (~670MB)" @echo " \033[36mmodels-fp32\033[0m Download fp32 full precision models (~2.5GB)" + @echo " \033[36mmodels-silero-vad\033[0m Download and verify the Silero VAD model (MIT)" @echo "" @echo "\033[1mDocker:\033[0m" @echo " \033[36mdocker-build-int8\033[0m Build Docker image with int8 models" diff --git a/README.md b/README.md index b8895e7..5a6d91f 100644 --- a/README.md +++ b/README.md @@ -305,6 +305,9 @@ chunk_%03d.wav`) or use a CPU image, which is bounded by system RAM instead. | `-long-audio` | Split audio over the model limit into chunks instead of rejecting it | `false` | `-long-audio` | | `-chunk-seconds` | Sliding-window size for long audio, in seconds | `300` | `-chunk-seconds 240` | | `-chunk-overlap-seconds` | Overlap between consecutive chunks, in seconds | `15` | `-chunk-overlap-seconds 10` | +| `-disable-vad-based-chunking` | Disable the Silero VAD chunk-boundary layer (falls back to mel energy) | `false` | `-disable-vad-based-chunking` | +| `-disable-mel-based-chunking` | Disable the mel-energy chunk-boundary layer (falls back to the midpoint) | `false` | `-disable-mel-based-chunking` | +| `-vad-model-path` | Path to the Silero VAD ONNX model | `/silero_vad.onnx` | `-vad-model-path /opt/silero_vad.onnx` | **Examples:** @@ -337,6 +340,22 @@ here). Pass `-long-audio` to split it into overlapping windows results, dropping the overlap so words at the seams are not duplicated. Files under the chunk size are transcribed in one pass either way. +**How chunk boundaries are chosen.** A blind split in the middle of an overlap +can fall mid-word and make that word show up twice or vanish at the seam. To +avoid this, the overlap is split on silence using a cascade (each layer falls +through to the next when it cannot decide): + +1. **Silero VAD** picks the centre of the longest silence in the overlap. This + needs `silero_vad.onnx` in the models directory (downloaded by `make models`; + see below). A missing model is not fatal: it warns once and falls back to the + next layer. +2. **Mel energy** picks the quietest point of the already-extracted features. +3. **Midpoint** is the final fallback (the original behaviour). + +A second, always-on safety net removes any duplicate or colliding tokens right +at each seam. You can turn off individual layers with +`-disable-vad-based-chunking` / `-disable-mel-based-chunking`. + ### Environment Variables Every command-line flag also reads from an environment variable: take the flag @@ -364,11 +383,15 @@ The following files are required in the models directory: | ------------------------------- | ------ | ------------------------ | | `config.json` | 97 B | Model configuration | | `vocab.txt` | 94 KB | SentencePiece vocabulary | +| `nemo128.onnx` | 140 KB | Preprocessor graph | | `encoder-model.int8.onnx` | 652 MB | Quantized encoder | | `decoder_joint-model.int8.onnx` | 18 MB | Quantized TDT decoder | +| `silero_vad.onnx` | 2.3 MB | Silero VAD (chunk boundaries, long-audio only; optional) | For full precision models, use `encoder-model.onnx` (requires `encoder-model.onnx.data`, 2.5GB total) and `decoder_joint-model.onnx` (72MB). +`silero_vad.onnx` ([snakers4/silero-vad](https://github.com/snakers4/silero-vad), MIT, pinned to release v6.2.1) is downloaded and checksum-verified by `make models`. It is only used to place chunk boundaries on silence in long-audio mode; if it is missing the server logs a warning once and falls back to mel-energy boundaries. + ## API Reference ### Authentication diff --git a/internal/asr/boundary.go b/internal/asr/boundary.go new file mode 100644 index 0000000..f6294df --- /dev/null +++ b/internal/asr/boundary.go @@ -0,0 +1,288 @@ +// SPDX-FileCopyrightText: 2026 Alby Hernández +// SPDX-License-Identifier: Apache-2.0 + +package asr + +import "log/slog" + +// This file implements the chunk-boundary selection stack for long-audio mode. +// +// When long-audio mode splits an utterance into overlapping windows, the shared +// overlap between two neighbours must be split at a single mel-frame index that +// decides which window emits which tokens. A blind arithmetic midpoint can land +// in the middle of a word, so the word is timestamped slightly differently by +// each window and ends up emitted twice or dropped (issue #18). The oracles +// below try to move that split onto silence instead. + +const ( + // vadSilenceThreshold is the Silero speech-probability level below which a + // window counts as silence for boundary selection. Tuned within the agreed + // 0.35-0.5 band: low enough to ignore breaths and room tone, high enough to + // still find gaps between words in continuous speech. Not a flag on purpose. + vadSilenceThreshold float32 = 0.4 + + // vadWarmupWindows is how many leading 32 ms windows of an overlap the VAD + // treats as warmup. Silero's recurrent state is reset at the start of every + // overlap, so its first ~1 s of probabilities are unreliable; we prefer a + // boundary after the warmup and only fall back to the warmup region when no + // silence is found later. 16000 Hz / 512 samples ~= 31 windows per second. + vadWarmupWindows = int(vadSampleRate) / vadWindowSamples + + // melSmoothingFrames is the moving-average width (in mel frames, 10 ms each) + // applied to per-frame energy before picking the quietest point. ~150 ms + // smooths out single-frame dips so the minimum lands in a genuine pause + // rather than on a transient glottal closure inside a word. + melSmoothingFrames = 15 +) + +// overlapRegion describes, in absolute mel-frame coordinates, the region shared +// by window i and window i+1 that a boundaryOracle must split. midpoint is the +// historical arithmetic split, provided so an oracle can bias towards it. +type overlapRegion struct { + start int64 // first mel frame of the overlap (inclusive) + end int64 // one past the last mel frame of the overlap (exclusive) + midpoint int64 // arithmetic midpoint, the pre-issue-#18 boundary +} + +// boundaryOracle is the contract for choosing an emission boundary inside an +// overlap region. The implementations below are alternative strategies for the +// same contract, tried as a cascade (see chainBoundaryOracle): +// +// - vadBoundaryOracle: centre of the longest Silero-detected silence. +// - melEnergyBoundaryOracle: quietest smoothed mel-energy frame. +// - midpointBoundaryOracle: the arithmetic midpoint (always decides; the +// final fallback so the result is never worse than before). +// +// boundary returns the chosen split as an absolute mel-frame index and true when +// the oracle can decide, or (_, false) to fall through to the next layer. The +// returned frame is advisory: planChunksWithBoundaries clamps it into the legal +// overlap range so the tiling invariant always holds. +type boundaryOracle interface { + boundary(r overlapRegion) (int64, bool) + name() string +} + +// midpointBoundaryOracle reproduces the pre-issue-#18 behaviour: split at the +// arithmetic midpoint of the overlap. It always decides, so it is the terminal +// layer of the cascade and guarantees a boundary is always chosen. +type midpointBoundaryOracle struct{} + +func (midpointBoundaryOracle) name() string { return "midpoint" } + +func (midpointBoundaryOracle) boundary(r overlapRegion) (int64, bool) { + return r.midpoint, true +} + +// melEnergyBoundaryOracle splits the overlap at the quietest point of the +// already-extracted mel features. It smooths per-frame energy once for the whole +// utterance, then returns the minimum inside each overlap. It always yields a +// value when it has features, so it is the robust fallback when the VAD is +// disabled or unavailable. Loud, dynamic music defeats the energy heuristic +// (there is no quiet frame), in which case this degrades to roughly midpoint +// quality; the VAD layer is the robust answer there, since it was trained to +// tell speech from sound rather than measuring loudness. +type melEnergyBoundaryOracle struct { + smoothed []float64 +} + +func newMelEnergyBoundaryOracle(features [][]float32) *melEnergyBoundaryOracle { + return &melEnergyBoundaryOracle{ + smoothed: smoothEnergies(frameEnergies(features), melSmoothingFrames), + } +} + +func (o *melEnergyBoundaryOracle) name() string { return "mel-energy" } + +func (o *melEnergyBoundaryOracle) boundary(r overlapRegion) (int64, bool) { + if len(o.smoothed) == 0 || r.end <= r.start { + return 0, false + } + idx := int64(argMinInRange(o.smoothed, int(r.start), int(r.end))) + return idx, true +} + +// vadBoundaryOracle splits the overlap at the centre of the longest run of +// Silero-detected silence. It runs the VAD only over the overlap's waveform (not +// the whole file: this repo stays faithful to Parakeet TDT and never segments or +// skips audio globally), resetting the recurrent state per overlap. When no run +// falls below the silence threshold it falls through to the next layer. +type vadBoundaryOracle struct { + vad *sileroVAD + state *vadState + waveform []float32 + hopLength int64 +} + +func (o *vadBoundaryOracle) name() string { return "vad" } + +func (o *vadBoundaryOracle) boundary(r overlapRegion) (int64, bool) { + if o.vad == nil || r.end <= r.start { + return 0, false + } + + sampleStart := r.start * o.hopLength + sampleEnd := r.end * o.hopLength + if sampleStart < 0 { + sampleStart = 0 + } + if sampleEnd > int64(len(o.waveform)) { + sampleEnd = int64(len(o.waveform)) + } + if sampleEnd-sampleStart < vadWindowSamples { + return 0, false + } + + probs := o.vad.speechProbabilities(o.state, o.waveform[sampleStart:sampleEnd]) + if len(probs) == 0 { + return 0, false + } + + center, ok := longestSubThresholdCenter(probs, vadSilenceThreshold, vadWarmupWindows) + if !ok { + return 0, false + } + + // Map the chosen window back to the centre sample, then to a mel frame, + // clamped defensively into the overlap. + centerSample := sampleStart + int64(center)*vadWindowSamples + vadWindowSamples/2 + frame := centerSample / o.hopLength + if frame < r.start { + frame = r.start + } + if frame >= r.end { + frame = r.end - 1 + } + return frame, true +} + +// chainBoundaryOracle tries each oracle in order and returns the first decision, +// implementing the VAD -> mel-energy -> midpoint cascade. A nil or empty chain +// decides nothing, which makes planChunksWithBoundaries fall back to the +// midpoint itself. +type chainBoundaryOracle struct { + oracles []boundaryOracle +} + +func (c chainBoundaryOracle) name() string { return "chain" } + +func (c chainBoundaryOracle) boundary(r overlapRegion) (int64, bool) { + for _, o := range c.oracles { + if frame, ok := o.boundary(r); ok { + if DebugMode { + slog.Debug("chunk boundary chosen", + "oracle", o.name(), + "frame", frame, + "overlapStart", r.start, + "overlapEnd", r.end, + "midpoint", r.midpoint, + ) + } + return frame, true + } + } + return 0, false +} + +// longestSubThresholdCenter returns the window index at the centre of the +// longest run of consecutive windows whose probability is below threshold. It +// prefers a run that starts at or after warmupWindows (Silero's unreliable +// warmup) and only considers the warmup region when no later silence exists. It +// returns false when no window is below threshold at all. Ties on run length +// keep the earliest run, so the selection is deterministic (a canary property). +func longestSubThresholdCenter(probs []float32, threshold float32, warmupWindows int) (int, bool) { + if c, ok := longestRunCenter(probs, threshold, warmupWindows); ok { + return c, true + } + return longestRunCenter(probs, threshold, 0) +} + +func longestRunCenter(probs []float32, threshold float32, from int) (int, bool) { + if from < 0 { + from = 0 + } + bestStart, bestLen := -1, 0 + runStart := -1 + for i := from; i < len(probs); i++ { + if probs[i] < threshold { + if runStart < 0 { + runStart = i + } + if runLen := i - runStart + 1; runLen > bestLen { + bestLen = runLen + bestStart = runStart + } + continue + } + runStart = -1 + } + if bestStart < 0 { + return 0, false + } + return bestStart + (bestLen-1)/2, true +} + +// frameEnergies reduces each mel frame to a single loudness proxy by summing its +// (normalized log-mel) bins. Quieter frames sum lower, so the minimum marks the +// best place to cut. Working off the already-extracted features means the +// mel-energy layer costs nothing extra to compute. +func frameEnergies(features [][]float32) []float64 { + energies := make([]float64, len(features)) + for i, frame := range features { + var sum float64 + for _, v := range frame { + sum += float64(v) + } + energies[i] = sum + } + return energies +} + +// smoothEnergies applies a centred moving average of the given width (in frames) +// using a prefix sum, so single-frame dips inside a word do not masquerade as +// pauses. A width <= 1 returns the input unchanged. +func smoothEnergies(energies []float64, window int) []float64 { + if window <= 1 || len(energies) == 0 { + return energies + } + n := len(energies) + prefix := make([]float64, n+1) + for i := 0; i < n; i++ { + prefix[i+1] = prefix[i] + energies[i] + } + half := window / 2 + out := make([]float64, n) + for i := 0; i < n; i++ { + lo := i - half + if lo < 0 { + lo = 0 + } + hi := i + half + 1 + if hi > n { + hi = n + } + out[i] = (prefix[hi] - prefix[lo]) / float64(hi-lo) + } + return out +} + +// argMinInRange returns the index of the smallest value in values[start:end), +// clamping the range into bounds. Ties keep the earliest index, so selection is +// deterministic. When the range is empty it returns start. +func argMinInRange(values []float64, start, end int) int { + if start < 0 { + start = 0 + } + if end > len(values) { + end = len(values) + } + if start >= end { + return start + } + minIdx := start + for i := start + 1; i < end; i++ { + if values[i] < values[minIdx] { + minIdx = i + } + } + return minIdx +} diff --git a/internal/asr/boundary_test.go b/internal/asr/boundary_test.go new file mode 100644 index 0000000..bcb9353 --- /dev/null +++ b/internal/asr/boundary_test.go @@ -0,0 +1,223 @@ +// SPDX-FileCopyrightText: 2026 Alby Hernández +// SPDX-License-Identifier: Apache-2.0 + +package asr + +import "testing" + +// funcOracle is a test boundaryOracle whose decision is supplied by a closure, +// so tests can drive arbitrary and adversarial oracle behaviour. +type funcOracle struct { + id string + calls *int + decide func(r overlapRegion) (int64, bool) +} + +func (f funcOracle) name() string { return f.id } + +func (f funcOracle) boundary(r overlapRegion) (int64, bool) { + if f.calls != nil { + *f.calls++ + } + return f.decide(r) +} + +// The midpoint oracle always decides and returns the arithmetic midpoint, so it +// can terminate the cascade. +func TestMidpointBoundaryOracle_AlwaysDecidesMidpoint(t *testing.T) { + frame, ok := midpointBoundaryOracle{}.boundary(overlapRegion{start: 30, end: 40, midpoint: 35}) + if !ok || frame != 35 { + t.Fatalf("midpoint oracle = (%d,%v), want (35,true)", frame, ok) + } +} + +// The cascade returns the first oracle that decides and does not consult later +// oracles once one has decided. +func TestChainBoundaryOracle_FallthroughOrder(t *testing.T) { + var c1, c2, c3 int + chain := chainBoundaryOracle{oracles: []boundaryOracle{ + funcOracle{id: "a", calls: &c1, decide: func(overlapRegion) (int64, bool) { return 0, false }}, + funcOracle{id: "b", calls: &c2, decide: func(overlapRegion) (int64, bool) { return 7, true }}, + funcOracle{id: "c", calls: &c3, decide: func(overlapRegion) (int64, bool) { return 9, true }}, + }} + + frame, ok := chain.boundary(overlapRegion{start: 0, end: 20, midpoint: 10}) + if !ok || frame != 7 { + t.Fatalf("chain = (%d,%v), want (7,true)", frame, ok) + } + if c1 != 1 || c2 != 1 { + t.Fatalf("expected first two oracles consulted once, got c1=%d c2=%d", c1, c2) + } + if c3 != 0 { + t.Fatalf("third oracle must not be consulted after a decision, got c3=%d", c3) + } +} + +// A cascade in which every oracle declines returns ok=false, so the planner +// falls back to the midpoint itself. +func TestChainBoundaryOracle_AllDecline(t *testing.T) { + chain := chainBoundaryOracle{oracles: []boundaryOracle{ + funcOracle{id: "a", decide: func(overlapRegion) (int64, bool) { return 0, false }}, + funcOracle{id: "b", decide: func(overlapRegion) (int64, bool) { return 0, false }}, + }} + if _, ok := chain.boundary(overlapRegion{start: 0, end: 10, midpoint: 5}); ok { + t.Fatal("chain must decline when all oracles decline") + } +} + +// Canary: longestSubThresholdCenter must pick the centre of the longest run of +// windows below the threshold, honour the warmup preference, and decline when no +// window is below the threshold. A regression in the selection changes these. +func TestLongestSubThresholdCenter(t *testing.T) { + const th = 0.4 + tests := []struct { + name string + probs []float32 + warmup int + want int + wantOK bool + }{ + { + name: "single run centre", + probs: []float32{0.9, 0.1, 0.1, 0.1, 0.9}, + warmup: 0, + want: 2, // run [1,3], centre 2 + wantOK: true, + }, + { + name: "longest of two runs", + probs: []float32{0.1, 0.1, 0.9, 0.1, 0.1, 0.1, 0.9}, + warmup: 0, + want: 4, // longer run [3,5], centre 4 + wantOK: true, + }, + { + name: "tie keeps earliest run", + probs: []float32{0.1, 0.1, 0.9, 0.1, 0.1}, + warmup: 0, + want: 0, // both runs length 2; earliest [0,1], centre 0 + wantOK: true, + }, + { + name: "warmup skips early run for later one", + probs: []float32{0.1, 0.1, 0.1, 0.9, 0.1, 0.1}, + warmup: 3, + want: 4, // only [4,5] eligible after warmup, centre 4 + wantOK: true, + }, + { + name: "warmup falls back when only silence is in warmup", + probs: []float32{0.1, 0.1, 0.1, 0.9, 0.9, 0.9}, + warmup: 3, + want: 1, // nothing after warmup; fall back to whole region, run [0,2] centre 1 + wantOK: true, + }, + { + name: "no window below threshold", + probs: []float32{0.9, 0.8, 0.7}, + warmup: 0, + want: 0, + wantOK: false, + }, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + got, ok := longestSubThresholdCenter(tt.probs, th, tt.warmup) + if ok != tt.wantOK || (ok && got != tt.want) { + t.Fatalf("longestSubThresholdCenter(%v, warmup=%d) = (%d,%v), want (%d,%v)", + tt.probs, tt.warmup, got, ok, tt.want, tt.wantOK) + } + }) + } +} + +func TestFrameEnergies(t *testing.T) { + features := [][]float32{ + {1, 2, 3}, // 6 + {0, 0, 0}, // 0 + {-1, -1}, // -2 + } + got := frameEnergies(features) + want := []float64{6, 0, -2} + if len(got) != len(want) { + t.Fatalf("len = %d, want %d", len(got), len(want)) + } + for i := range want { + if got[i] != want[i] { + t.Fatalf("frameEnergies[%d] = %v, want %v", i, got[i], want[i]) + } + } +} + +func TestSmoothEnergies(t *testing.T) { + // Window 3, centred moving average with edge clamping. + in := []float64{0, 3, 6, 9} + got := smoothEnergies(in, 3) + want := []float64{ + (0 + 3) / 2.0, // 1.5 + (0 + 3 + 6) / 3.0, // 3 + (3 + 6 + 9) / 3.0, // 6 + (6 + 9) / 2.0, // 7.5 + } + for i := range want { + if got[i] != want[i] { + t.Fatalf("smoothEnergies[%d] = %v, want %v", i, got[i], want[i]) + } + } + + // Window <= 1 returns the input unchanged. + if got := smoothEnergies(in, 1); &got[0] != &in[0] { + t.Fatal("smoothEnergies with window 1 must return the input slice unchanged") + } +} + +func TestArgMinInRange(t *testing.T) { + values := []float64{5, 2, 8, 1, 9, 1} + tests := []struct { + name string + start, end int + want int + }{ + {"full range, first minimum wins", 0, 6, 3}, + {"sub range", 0, 3, 1}, + {"range excludes global min", 0, 3, 1}, + {"out of bounds clamped", 4, 100, 5}, + {"empty range returns start", 2, 2, 2}, + } + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + if got := argMinInRange(values, tt.start, tt.end); got != tt.want { + t.Fatalf("argMinInRange(%d,%d) = %d, want %d", tt.start, tt.end, got, tt.want) + } + }) + } +} + +// The mel-energy oracle must place the boundary inside the quiet valley within +// the overlap, and always decide when it has features. +func TestMelEnergyBoundaryOracle(t *testing.T) { + // 80 frames of loud audio with a quiet valley at frames [40,50). + features := make([][]float32, 80) + for i := range features { + v := float32(10) + if i >= 40 && i < 50 { + v = 0 + } + features[i] = []float32{v} + } + oracle := newMelEnergyBoundaryOracle(features) + + frame, ok := oracle.boundary(overlapRegion{start: 30, end: 60, midpoint: 45}) + if !ok { + t.Fatal("mel-energy oracle must decide when it has features") + } + if frame < 40 || frame >= 50 { + t.Fatalf("boundary %d not inside the quiet valley [40,50)", frame) + } + + // No features: declines so the cascade falls through. + empty := newMelEnergyBoundaryOracle(nil) + if _, ok := empty.boundary(overlapRegion{start: 0, end: 10, midpoint: 5}); ok { + t.Fatal("mel-energy oracle must decline without features") + } +} diff --git a/internal/asr/chunker.go b/internal/asr/chunker.go index 92ec4b2..5dd9813 100644 --- a/internal/asr/chunker.go +++ b/internal/asr/chunker.go @@ -45,18 +45,38 @@ type chunkWindow struct { } // planChunks splits a mel sequence of total frames into overlapping windows no -// larger than chunkFrames, each sharing overlapFrames with its neighbours. +// larger than chunkFrames, each sharing overlapFrames with its neighbours, +// splitting each overlap at its arithmetic midpoint. It is the pure-midpoint +// case of planChunksWithBoundaries and is kept for the tests and callers that +// do not need a smarter boundary. +// +// Callers must pass chunkFrames > overlapFrames >= 0 and total > 0. +func planChunks(total, chunkFrames, overlapFrames int64) []chunkWindow { + return planChunksWithBoundaries(total, chunkFrames, overlapFrames, nil) +} + +// planChunksWithBoundaries splits a mel sequence of total frames into overlapping +// windows no larger than chunkFrames, each sharing overlapFrames with its +// neighbours, and asks oracle where to split the ownership of each interior +// overlap. // // The overlap gives the encoder acoustic context and the decoder LSTM time to // warm up before its owned region begins. Ownership of the shared overlap is -// split at its midpoint: the earlier window emits up to the midpoint, the later -// window from the midpoint on, so every frame is emitted by exactly one window. +// split at a single mel frame: the earlier window emits up to it, the later +// window from it on, so every frame is emitted by exactly one window. A nil +// oracle (or one that declines) falls back to the arithmetic midpoint, which +// reproduces the pre-issue-#18 behaviour. +// +// Whatever the oracle returns, the boundary is clamped so the emit ranges still +// tile [0, total) with no gap or overlap and stay inside each window: the first +// window emits from 0, the last emits to total, emitStart is non-decreasing, and +// emitEnd[i] == emitStart[i+1]. This invariant holds for ANY oracle output. // // When the audio fits in a single window (total <= chunkFrames), it returns one // window covering everything, which reproduces the non-chunked behaviour. // // Callers must pass chunkFrames > overlapFrames >= 0 and total > 0. -func planChunks(total, chunkFrames, overlapFrames int64) []chunkWindow { +func planChunksWithBoundaries(total, chunkFrames, overlapFrames int64, oracle boundaryOracle) []chunkWindow { if total <= chunkFrames { return []chunkWindow{{start: 0, end: total, emitStart: 0, emitEnd: total}} } @@ -76,21 +96,50 @@ func planChunks(total, chunkFrames, overlapFrames int64) []chunkWindow { } // Assign emit ranges. The first window owns from 0, the last owns to the - // end, and every interior boundary sits at the midpoint of the overlap - // shared by the two windows straddling it. + // end, and every interior boundary is chosen by the oracle inside the + // overlap shared by the two windows straddling it, then clamped so the + // tiling invariant holds regardless of what the oracle returns. + prevEmitEnd := int64(0) for i := range windows { - if i == 0 { - windows[i].emitStart = 0 - } else { - windows[i].emitStart = windows[i-1].emitEnd - } + windows[i].emitStart = prevEmitEnd if i == len(windows)-1 { windows[i].emitEnd = total - } else { - // Midpoint of the overlap between window i and i+1. - windows[i].emitEnd = (windows[i+1].start + windows[i].end) / 2 + prevEmitEnd = total + continue } + + overlapStart := windows[i+1].start + overlapEnd := windows[i].end + midpoint := (overlapStart + overlapEnd) / 2 + + boundary := midpoint + if oracle != nil { + if frame, ok := oracle.boundary(overlapRegion{ + start: overlapStart, + end: overlapEnd, + midpoint: midpoint, + }); ok { + boundary = frame + } + } + + // Clamp into the legal overlap range, then keep emitStart monotonic so a + // pathological oracle can never make emitStart exceed emitEnd. Because + // window ends are non-decreasing, prevEmitEnd <= overlapEnd always, so + // the final boundary stays inside [emitStart, windows[i].end]. + if boundary < overlapStart { + boundary = overlapStart + } + if boundary > overlapEnd { + boundary = overlapEnd + } + if boundary < windows[i].emitStart { + boundary = windows[i].emitStart + } + + windows[i].emitEnd = boundary + prevEmitEnd = boundary } return windows @@ -112,8 +161,16 @@ func melToEncoderFrame(melOffset, subsampling int64) int64 { // overrun the model's single-pass limit, so the caller fails cleanly instead of // letting the encoder crash on an out-of-range positional-encoding slice. func planForAudio(total, chunkFrames, overlapFrames, subsampling int64, longAudio bool) ([]chunkWindow, error) { + return planForAudioWithBoundaries(total, chunkFrames, overlapFrames, subsampling, longAudio, nil) +} + +// planForAudioWithBoundaries is planForAudio with a boundary oracle: when long +// audio is on it uses the oracle to place each interior overlap boundary +// (falling back to the midpoint when the oracle declines or is nil). The +// single-window and ErrAudioTooLong paths are unchanged. +func planForAudioWithBoundaries(total, chunkFrames, overlapFrames, subsampling int64, longAudio bool, oracle boundaryOracle) ([]chunkWindow, error) { if longAudio { - return planChunks(total, chunkFrames, overlapFrames), nil + return planChunksWithBoundaries(total, chunkFrames, overlapFrames, oracle), nil } if melToEncoderFrame(total, subsampling) > modelMaxEncoderFrames { return nil, ErrAudioTooLong diff --git a/internal/asr/chunker_test.go b/internal/asr/chunker_test.go index e0c4273..17882a6 100644 --- a/internal/asr/chunker_test.go +++ b/internal/asr/chunker_test.go @@ -184,6 +184,103 @@ func TestPlanForAudio(t *testing.T) { }) } +// planChunksWithBoundaries must honour an oracle's boundary when it is inside +// the overlap and clamp it into the legal range otherwise, while keeping the +// tiling invariant for ANY oracle output. +func TestPlanChunksWithBoundaries_HonoursAndClampsOracle(t *testing.T) { + // Windows for planChunks(100,40,10): starts 0,30,60 ends 40,70,100. + // Overlaps: [30,40) mid 35, and [60,70) mid 65. + t.Run("in-range boundaries honoured", func(t *testing.T) { + oracle := funcOracle{decide: func(r overlapRegion) (int64, bool) { + if r.start == 30 { + return 32, true + } + return 68, true + }} + got := planChunksWithBoundaries(100, 40, 10, oracle) + want := []chunkWindow{ + {start: 0, end: 40, emitStart: 0, emitEnd: 32}, + {start: 30, end: 70, emitStart: 32, emitEnd: 68}, + {start: 60, end: 100, emitStart: 68, emitEnd: 100}, + } + if !reflect.DeepEqual(got, want) { + t.Fatalf("got %+v, want %+v", got, want) + } + }) + + t.Run("out-of-range boundaries clamped into overlap", func(t *testing.T) { + oracle := funcOracle{decide: func(r overlapRegion) (int64, bool) { + if r.start == 30 { + return -1000, true // below overlapStart -> clamp to 30 + } + return 1000, true // above overlapEnd -> clamp to 70 + }} + got := planChunksWithBoundaries(100, 40, 10, oracle) + want := []chunkWindow{ + {start: 0, end: 40, emitStart: 0, emitEnd: 30}, + {start: 30, end: 70, emitStart: 30, emitEnd: 70}, + {start: 60, end: 100, emitStart: 70, emitEnd: 100}, + } + if !reflect.DeepEqual(got, want) { + t.Fatalf("got %+v, want %+v", got, want) + } + }) +} + +// The tiling invariant must hold for arbitrary and adversarial oracles: the emit +// ranges tile [0,total) with no gap or overlap and stay inside their windows, +// no matter what the oracle returns. +func TestPlanChunksWithBoundaries_TilingInvariantArbitraryOracle(t *testing.T) { + oracles := map[string]boundaryOracle{ + "nil (midpoint)": nil, + "always zero": funcOracle{decide: func(overlapRegion) (int64, bool) { return 0, true }}, + "always huge": funcOracle{decide: func(overlapRegion) (int64, bool) { return 1 << 40, true }}, + "always negative": funcOracle{decide: func(overlapRegion) (int64, bool) { return -1 << 40, true }}, + "always decline": funcOracle{decide: func(overlapRegion) (int64, bool) { return 0, false }}, + "overlap start-ish": funcOracle{decide: func(r overlapRegion) (int64, bool) { return r.start + 1, true }}, + "overlap end-ish": funcOracle{decide: func(r overlapRegion) (int64, bool) { return r.end - 1, true }}, + } + cases := []struct { + name string + total int64 + chunkFrames int64 + overlapFrames int64 + }{ + {"even multiple", 90, 40, 10}, + {"one over chunk", 4001, 4000, 500}, + {"large overlap", 100, 40, 35}, + {"no overlap", 100, 40, 0}, + {"many windows", 100000, 30000, 1500}, + } + for oname, oracle := range oracles { + for _, tc := range cases { + t.Run(oname+"/"+tc.name, func(t *testing.T) { + windows := planChunksWithBoundaries(tc.total, tc.chunkFrames, tc.overlapFrames, oracle) + if len(windows) == 0 { + t.Fatal("no windows produced") + } + if windows[0].emitStart != 0 { + t.Errorf("first emitStart = %d, want 0", windows[0].emitStart) + } + if last := windows[len(windows)-1]; last.emitEnd != tc.total { + t.Errorf("last emitEnd = %d, want %d", last.emitEnd, tc.total) + } + for i, w := range windows { + if w.emitStart > w.emitEnd { + t.Errorf("window %d has emitStart %d > emitEnd %d", i, w.emitStart, w.emitEnd) + } + if w.start > w.emitStart || w.emitEnd > w.end { + t.Errorf("window %d emit range [%d,%d) not inside window [%d,%d)", i, w.emitStart, w.emitEnd, w.start, w.end) + } + if i > 0 && windows[i-1].emitEnd != w.emitStart { + t.Errorf("gap/overlap between window %d emitEnd %d and window %d emitStart %d", i-1, windows[i-1].emitEnd, i, w.emitStart) + } + } + }) + } + } +} + func TestMelToEncoderFrame(t *testing.T) { tests := []struct { name string diff --git a/internal/asr/mel.go b/internal/asr/mel.go index 23b0872..2809649 100644 --- a/internal/asr/mel.go +++ b/internal/asr/mel.go @@ -92,6 +92,13 @@ func (m *MelFilterbank) FramesPerSecond() int { return m.sampleRate / m.hopLength } +// HopLength returns the hop between mel frames in samples. It maps a mel-frame +// index to a sample offset (frame * hopLength), which the VAD boundary oracle +// uses to slice the waveform for an overlap region. +func (m *MelFilterbank) HopLength() int { + return m.hopLength +} + // Extract computes mel filterbank features from audio samples func (m *MelFilterbank) Extract(samples []float32) [][]float32 { numFrames := (len(samples)-m.winLength)/m.hopLength + 1 diff --git a/internal/asr/seam.go b/internal/asr/seam.go new file mode 100644 index 0000000..3032c93 --- /dev/null +++ b/internal/asr/seam.go @@ -0,0 +1,84 @@ +// SPDX-FileCopyrightText: 2026 Alby Hernández +// SPDX-License-Identifier: Apache-2.0 + +package asr + +// This file implements seam-level token deduplication for long-audio chunking. +// Even with the boundary placed on silence, the two windows straddling a seam +// timestamp a word landing near the boundary slightly differently, so a token +// can still be emitted twice (duplicate) or, less often, as two different +// guesses at the same position. Dedup runs on the emitted tokens tagged with +// their ABSOLUTE encoder-frame timesteps and is always on (no flag). + +const ( + // seamMaxTokens caps how many leading tokens of a window are held and + // compared against the previous window's tail. k <= 3 keeps the streaming + // buffer tiny: only the first few tokens of each window (after the first) + // wait for the seam check, everything else streams as it is decoded. + seamMaxTokens = 3 + + // seamTimestepToleranceFrames is how close two tokens must be, in absolute + // encoder frames (80 ms each), to count as the same acoustic position at a + // seam. 3 frames is ~240 ms, wide enough to absorb the per-window timestamp + // jitter that makes a boundary-straddling word land a frame or two apart in + // each window, narrow enough not to swallow genuinely distinct neighbours. + seamTimestepToleranceFrames = 3 +) + +// decodedToken is a token emitted by the TDT decoder tagged with its ABSOLUTE +// encoder-frame timestep. Absolute timesteps (as opposed to per-window local +// ones) let dedupSeam line up tokens emitted by two different windows that cover +// the same audio around a seam. +type decodedToken struct { + id int + timestep int64 +} + +// dedupSeam decides which of window i+1's leading tokens (head) survive when +// compared against window i's trailing tokens (prevTail). It returns the +// survivors in order. +// +// A head token is dropped when its timestep is within seamTimestepToleranceFrames +// of any of the previous window's last seamMaxTokens tokens. That single rule +// covers the two failure modes from issue #18: +// +// - Same text at (nearly) the same timestep: a duplicate ("to to"); drop it. +// - Different text at (nearly) the same timestep: a collision; window i wins +// because its LSTM reaches the seam fully warmed up while window i+1 is still +// warming up, so drop window i+1's guess. +// +// A duplicate whose timesteps are further apart than the tolerance is kept, as +// are tokens that do not collide with the previous tail at all. +func dedupSeam(prevTail, head []decodedToken) []decodedToken { + if len(prevTail) == 0 || len(head) == 0 { + return head + } + + // Only the tail-most tokens of the previous window can sit at the seam. + tail := prevTail + if len(tail) > seamMaxTokens { + tail = tail[len(tail)-seamMaxTokens:] + } + + survivors := make([]decodedToken, 0, len(head)) + for _, h := range head { + drop := false + for _, p := range tail { + if absDiffInt64(h.timestep, p.timestep) <= seamTimestepToleranceFrames { + drop = true + break + } + } + if !drop { + survivors = append(survivors, h) + } + } + return survivors +} + +func absDiffInt64(a, b int64) int64 { + if a > b { + return a - b + } + return b - a +} diff --git a/internal/asr/seam_test.go b/internal/asr/seam_test.go new file mode 100644 index 0000000..18a57c0 --- /dev/null +++ b/internal/asr/seam_test.go @@ -0,0 +1,89 @@ +// SPDX-FileCopyrightText: 2026 Alby Hernández +// SPDX-License-Identifier: Apache-2.0 + +package asr + +import ( + "reflect" + "testing" +) + +// dedupSeam is the canary for the seam deduplication rules from issue #18. Each +// case pins one behaviour: exact duplicate within tolerance dropped, duplicate +// outside tolerance kept, text-mismatch collision resolved in favour of the +// earlier window, no collision kept, and the k boundary cases. +func TestDedupSeam(t *testing.T) { + tests := []struct { + name string + prevTail []decodedToken + head []decodedToken + want []decodedToken + }{ + { + name: "exact duplicate within tolerance is dropped", + prevTail: []decodedToken{{id: 5, timestep: 100}}, + head: []decodedToken{{id: 5, timestep: 101}}, + want: []decodedToken{}, + }, + { + name: "duplicate outside tolerance is kept", + prevTail: []decodedToken{{id: 5, timestep: 100}}, + head: []decodedToken{{id: 5, timestep: 110}}, + want: []decodedToken{{id: 5, timestep: 110}}, + }, + { + name: "text mismatch collision keeps the earlier window", + prevTail: []decodedToken{{id: 5, timestep: 100}}, + head: []decodedToken{{id: 7, timestep: 101}}, + want: []decodedToken{}, + }, + { + name: "no collision is kept", + prevTail: []decodedToken{{id: 5, timestep: 100}}, + head: []decodedToken{{id: 7, timestep: 200}}, + want: []decodedToken{{id: 7, timestep: 200}}, + }, + { + name: "empty previous tail keeps the whole head", + prevTail: nil, + head: []decodedToken{{id: 5, timestep: 100}}, + want: []decodedToken{{id: 5, timestep: 100}}, + }, + { + name: "empty head stays empty", + prevTail: []decodedToken{{id: 5, timestep: 100}}, + head: nil, + want: nil, + }, + { + name: "mixed head drops colliding tokens and keeps the rest", + prevTail: []decodedToken{{id: 1, timestep: 50}, {id: 2, timestep: 100}}, + head: []decodedToken{{id: 2, timestep: 101}, {id: 3, timestep: 102}, {id: 9, timestep: 300}}, + want: []decodedToken{{id: 9, timestep: 300}}, + }, + { + name: "only the last seamMaxTokens of the tail are compared", + prevTail: []decodedToken{ + {id: 1, timestep: 10}, {id: 2, timestep: 20}, + {id: 3, timestep: 30}, {id: 4, timestep: 40}, + }, + // timestep 11 collides with the dropped-from-comparison {1,10} only, + // so the head token survives. + head: []decodedToken{{id: 9, timestep: 11}}, + want: []decodedToken{{id: 9, timestep: 11}}, + }, + } + + for _, tt := range tests { + t.Run(tt.name, func(t *testing.T) { + got := dedupSeam(tt.prevTail, tt.head) + // Normalize nil vs empty for the "empty head" case. + if len(got) == 0 && len(tt.want) == 0 { + return + } + if !reflect.DeepEqual(got, tt.want) { + t.Fatalf("dedupSeam(%v, %v) = %v, want %v", tt.prevTail, tt.head, got, tt.want) + } + }) + } +} diff --git a/internal/asr/seaminspect_test.go b/internal/asr/seaminspect_test.go new file mode 100644 index 0000000..c6e06d2 --- /dev/null +++ b/internal/asr/seaminspect_test.go @@ -0,0 +1,242 @@ +// SPDX-FileCopyrightText: 2026 Alby Hernández +// SPDX-License-Identifier: Apache-2.0 + +//go:build seaminspect + +// Package asr's seam inspector. Build-tag gated so it never runs in the normal +// test suite (it needs ONNX Runtime, the models, and the reference audio). No +// Python anywhere in this repo: this is the Go equivalent of a scratch script. +// +// It transcribes the reference MP3 through the full long-audio pipeline, logs +// the chosen chunk-boundary positions (enable debug to see which oracle layer +// decided each one), and prints the transcribed words around every seam next to +// the reference .srt text for the same time range, so a human can eyeball each +// seam for the duplicated/dropped words from issue #18. It needs no network. +// +// Usage: +// +// PARAKEET_MODELS=./models \ +// PARAKEET_SEAM_AUDIO=./testdata/reference/learn-case-interviews.mp3 \ +// PARAKEET_SEAM_SRT=./testdata/reference/learn-case-interviews.srt \ +// go test -tags=seaminspect -run TestSeamInspection -v ./internal/asr/ +package asr + +import ( + "bufio" + "context" + "log/slog" + "os" + "sort" + "strconv" + "strings" + "testing" + "time" +) + +// seamInspectWindowSeconds is how far either side of a seam we quote text. +const seamInspectWindowSeconds = 5.0 + +func TestSeamInspection(t *testing.T) { + audioPath := os.Getenv("PARAKEET_SEAM_AUDIO") + if audioPath == "" { + audioPath = "../../testdata/reference/learn-case-interviews.mp3" + } + srtPath := os.Getenv("PARAKEET_SEAM_SRT") + if srtPath == "" { + srtPath = "../../testdata/reference/learn-case-interviews.srt" + } + modelsDir := os.Getenv("PARAKEET_MODELS") + if modelsDir == "" { + modelsDir = "../../models" + } + + if _, err := os.Stat(audioPath); err != nil { + t.Skipf("reference audio not found (%v); nothing to inspect", err) + } + + // Show which oracle decided each boundary. + slog.SetDefault(slog.New(slog.NewTextHandler(os.Stdout, &slog.HandlerOptions{Level: slog.LevelDebug}))) + DebugMode = true + + tr, err := NewTranscriber(modelsDir, 2, Options{ + FFmpeg: FFmpegConfig{Enabled: true, Timeout: 120 * time.Second}, + Chunk: ChunkConfig{Enabled: true, Seconds: DefaultChunkSeconds, OverlapSeconds: DefaultChunkOverlapSeconds}, + }) + if err != nil { + t.Skipf("could not initialize transcriber (needs ONNX Runtime + models): %v", err) + } + defer tr.Close() + + data, err := os.ReadFile(audioPath) + if err != nil { + t.Fatalf("read audio: %v", err) + } + waveform, err := tr.loadAudio(data, "mp3") + if err != nil { + t.Fatalf("decode audio (needs ffmpeg): %v", err) + } + features := tr.mel.Extract(waveform) + if len(features) == 0 { + t.Fatal("no mel features extracted") + } + + subsampling := int64(tr.config.SubsamplingFactor) + fps := float64(tr.mel.FramesPerSecond()) + // Absolute encoder frame -> seconds (one encoder frame = subsampling mel frames). + frameSeconds := float64(subsampling) / fps + + oracle := tr.newBoundaryOracle(features, waveform) + plan, err := planForAudioWithBoundaries(int64(len(features)), tr.chunkFrames, tr.overlapFrames, subsampling, true, oracle) + if err != nil { + t.Fatalf("plan: %v", err) + } + t.Logf("planned %d windows over %.1fs of audio", len(plan), float64(len(features))/fps) + + // Decode every window, sharing the exact seam-dedup logic the server uses. + ctx := context.Background() + var all []decodedToken + var prevTail []decodedToken + for i, win := range plan { + frameOffset := melToEncoderFrame(win.start, subsampling) + emitStart := melToEncoderFrame(win.emitStart-win.start, subsampling) + emitEnd := melToEncoderFrame(win.emitEnd-win.start, subsampling) + + holdFirst := 0 + var resolveSeam func(head []decodedToken) []decodedToken + if i > 0 { + holdFirst = seamMaxTokens + tail := prevTail + resolveSeam = func(head []decodedToken) []decodedToken { return dedupSeam(tail, head) } + } + + wt, err := tr.runInference(ctx, features[win.start:win.end], emitStart, emitEnd, frameOffset, holdFirst, resolveSeam, nil) + if err != nil { + t.Fatalf("window %d inference: %v", i, err) + } + all = append(all, wt...) + prevTail = wt + } + + cues := parseSRT(t, srtPath) + + // One report per interior seam. + for i := 0; i < len(plan)-1; i++ { + seamSec := float64(plan[i].emitEnd) / fps + lo := seamSec - seamInspectWindowSeconds + hi := seamSec + seamInspectWindowSeconds + + t.Logf("========================================================") + t.Logf("SEAM %d at %s (mel frame %d)", i+1, fmtSeconds(seamSec), plan[i].emitEnd) + t.Logf("-- transcribed [%s .. %s] --", fmtSeconds(lo), fmtSeconds(hi)) + t.Logf(" %s", tokensTextInRange(tr, all, lo, hi, frameSeconds)) + t.Logf("-- reference SRT [%s .. %s] --", fmtSeconds(lo), fmtSeconds(hi)) + t.Logf(" %s", srtTextInRange(cues, lo, hi)) + } +} + +// tokensTextInRange renders the transcribed tokens whose absolute timestep falls +// within [lo, hi] seconds. +func tokensTextInRange(tr *Transcriber, tokens []decodedToken, lo, hi, frameSeconds float64) string { + var b strings.Builder + for _, tok := range tokens { + sec := float64(tok.timestep) * frameSeconds + if sec < lo || sec > hi { + continue + } + b.WriteString(tr.tokenText(tok.id)) + } + return strings.TrimSpace(strings.Join(strings.Fields(b.String()), " ")) +} + +type srtCue struct { + start, end float64 + text string +} + +// srtTextInRange concatenates the reference cue text overlapping [lo, hi]. +func srtTextInRange(cues []srtCue, lo, hi float64) string { + var parts []string + for _, c := range cues { + if c.end < lo || c.start > hi { + continue + } + parts = append(parts, c.text) + } + return strings.TrimSpace(strings.Join(parts, " ")) +} + +// parseSRT reads a SubRip file into time-ordered cues. It is intentionally +// small: enough to line reference text up with seam times, not a full parser. +func parseSRT(t *testing.T, path string) []srtCue { + f, err := os.Open(path) + if err != nil { + t.Logf("no SRT reference (%v); reference column will be empty", err) + return nil + } + defer f.Close() + + var cues []srtCue + var cur *srtCue + scanner := bufio.NewScanner(f) + for scanner.Scan() { + line := strings.TrimSpace(scanner.Text()) + switch { + case line == "": + if cur != nil { + cur.text = strings.TrimSpace(cur.text) + cues = append(cues, *cur) + cur = nil + } + case strings.Contains(line, "-->"): + parts := strings.Split(line, "-->") + if len(parts) != 2 { + continue + } + cur = &srtCue{ + start: parseSRTTime(strings.TrimSpace(parts[0])), + end: parseSRTTime(strings.TrimSpace(parts[1])), + } + case isSRTIndex(line): + // Skip the numeric index line unless we are mid-cue text. + if cur != nil { + cur.text += " " + line + } + default: + if cur != nil { + cur.text += " " + line + } + } + } + if cur != nil { + cur.text = strings.TrimSpace(cur.text) + cues = append(cues, *cur) + } + sort.Slice(cues, func(i, j int) bool { return cues[i].start < cues[j].start }) + return cues +} + +func isSRTIndex(line string) bool { + _, err := strconv.Atoi(line) + return err == nil +} + +// parseSRTTime parses "HH:MM:SS,mmm" into seconds. +func parseSRTTime(s string) float64 { + s = strings.ReplaceAll(s, ",", ".") + parts := strings.Split(s, ":") + if len(parts) != 3 { + return 0 + } + h, _ := strconv.ParseFloat(parts[0], 64) + m, _ := strconv.ParseFloat(parts[1], 64) + sec, _ := strconv.ParseFloat(parts[2], 64) + return h*3600 + m*60 + sec +} + +func fmtSeconds(sec float64) string { + if sec < 0 { + sec = 0 + } + d := time.Duration(sec * float64(time.Second)) + return d.Truncate(time.Millisecond).String() +} diff --git a/internal/asr/transcriber.go b/internal/asr/transcriber.go index daf2cf7..3fa08ee 100644 --- a/internal/asr/transcriber.go +++ b/internal/asr/transcriber.go @@ -181,27 +181,31 @@ type GPUConfig struct { } type Transcriber struct { - config Config - vocab map[int]string - vocabSize int - blankIdx int - maxTokensPerStep int - chunkFrames int64 - overlapFrames int64 - longAudio bool - mel *MelFilterbank - encoder *ort.DynamicAdvancedSession - decoderPool chan *decoderWorker - ffmpeg *ffmpegConverter + config Config + vocab map[int]string + vocabSize int + blankIdx int + maxTokensPerStep int + chunkFrames int64 + overlapFrames int64 + longAudio bool + disableVADChunking bool + disableMelChunking bool + mel *MelFilterbank + encoder *ort.DynamicAdvancedSession + vad *sileroVAD + decoderPool chan *decoderWorker + ffmpeg *ffmpegConverter } // Options groups optional knobs passed to NewTranscriber. Zero values keep // the previous behavior: WAV-only input, no ffmpeg conversion, CPU inference, -// default chunk sizes. +// default chunk sizes, and the full boundary stack (VAD then mel then midpoint). type Options struct { - FFmpeg FFmpegConfig - GPU GPUConfig - Chunk ChunkConfig + FFmpeg FFmpegConfig + GPU GPUConfig + Chunk ChunkConfig + Boundary BoundaryConfig } // ChunkConfig sets the sliding-window sizes that keep long audio within the @@ -213,6 +217,17 @@ type ChunkConfig struct { OverlapSeconds int } +// BoundaryConfig tunes how the emission boundary inside each chunk overlap is +// chosen. By default the cascade is VAD -> mel energy -> midpoint; +// the disable flags drop the earlier layers so the cascade falls through to the +// next one. VADModelPath points at the Silero VAD ONNX file; when empty the +// caller resolves it to silero_vad.onnx inside the models directory. +type BoundaryConfig struct { + DisableVAD bool + DisableMel bool + VADModelPath string +} + // buildSessionOptions returns the ONNX Runtime session options for the // configured execution provider. It returns (nil, nil) for the CPU provider so // sessions are created with default CPU behavior, identical to the pre-GPU code @@ -315,6 +330,8 @@ func NewTranscriber(modelsDir string, workers int, opts Options) (*Transcriber, t.chunkFrames = int64(chunkSeconds) * fps t.overlapFrames = int64(overlapSeconds) * fps t.longAudio = opts.Chunk.Enabled + t.disableVADChunking = opts.Boundary.DisableVAD + t.disableMelChunking = opts.Boundary.DisableMel if t.longAudio { if err := validateChunking(t.chunkFrames, t.overlapFrames, int64(t.config.SubsamplingFactor)); err != nil { return nil, fmt.Errorf("invalid chunk configuration: %w", err) @@ -407,12 +424,36 @@ func NewTranscriber(modelsDir string, workers int, opts Options) (*Transcriber, t.decoderPool <- w } + // Load the Silero VAD model for chunk-boundary selection. It is only useful + // when long-audio windowing is on, and only when the VAD layer is enabled. + // A missing model file is not fatal: warn once and let the boundary stack + // fall back to mel energy. Any other load error is fatal so a + // corrupt model surfaces loudly at startup. + if t.longAudio && !t.disableVADChunking { + vadPath := opts.Boundary.VADModelPath + if vadPath == "" { + vadPath = filepath.Join(modelsDir, "silero_vad.onnx") + } + vad, err := newSileroVAD(vadPath, sessOpts) + switch { + case err == nil: + t.vad = vad + case os.IsNotExist(err): + slog.Warn("VAD model not found, chunk boundaries fall back to mel energy", + "path", vadPath) + default: + t.Close() + return nil, fmt.Errorf("failed to load Silero VAD model: %w", err) + } + } + slog.Info("transcriber initialized", "workers", workers, "provider", string(provider(opts.GPU)), "encoder", filepath.Base(encoderPath), "decoder", filepath.Base(decoderPath), "vocabSize", t.vocabSize, + "vad", t.vad != nil, ) return t, nil @@ -468,6 +509,10 @@ func (t *Transcriber) Close() { t.encoder.Destroy() t.encoder = nil } + if t.vad != nil { + t.vad.destroy() + t.vad = nil + } if t.decoderPool != nil { close(t.decoderPool) for w := range t.decoderPool { @@ -528,30 +573,12 @@ func (t *Transcriber) transcribe(ctx context.Context, audioData []byte, format, slog.Debug("mel features extracted", "frames", len(features), "featuresPerFrame", len(features[0])) } - // Build a token-level callback that converts each emitted token into a - // printable text delta, skipping special <...> tokens. Streaming is - // purely additive: callers get the same text whether or not emit is set. - var onToken func(tok int) - if emit != nil { - onToken = func(tok int) { - text, ok := t.vocab[tok] - if !ok { - return - } - if strings.HasPrefix(text, "<") && strings.HasSuffix(text, ">") { - return - } - // vocab values already have word-boundary marks (U+2581) translated - // to spaces at load time (see loadVocab), so the token text is emitted - // as-is. Clients accumulate deltas to render text live. - if text != "" { - emit(text) - } - } - } - subsampling := int64(t.config.SubsamplingFactor) - plan, err := planForAudio(int64(len(features)), t.chunkFrames, t.overlapFrames, subsampling, t.longAudio) + // Build the boundary oracle cascade (VAD -> mel energy -> midpoint) over this + // request's data and plan the chunk windows with it. When long-audio is off + // the oracle is unused (single window or ErrAudioTooLong). + oracle := t.newBoundaryOracle(features, waveform) + plan, err := planForAudioWithBoundaries(int64(len(features)), t.chunkFrames, t.overlapFrames, subsampling, t.longAudio, oracle) if err != nil { slog.Warn("audio exceeds the single-pass model limit; enable --long-audio to transcribe long files in overlapping chunks", "seconds", float64(len(features))/float64(t.mel.FramesPerSecond()), @@ -563,27 +590,68 @@ func (t *Transcriber) transcribe(ctx context.Context, audioData []byte, format, slog.Debug("chunk plan", "windows", len(plan), "melFrames", len(features), "longAudio", t.longAudio) } - var tokens []int - for _, win := range plan { + // Decode window by window. Adjacent windows share an overlap, so window i+1's + // first few tokens are held and compared against window i's tail before they + // are emitted, dropping seam duplicates and letting the earlier (warmed-up) + // window win text collisions. Held tokens are released in order + // before the rest of the window streams, so streaming order is preserved. + var tokens []decodedToken + var prevTail []decodedToken + for i, win := range plan { // Emit bounds are the window's owned region expressed in the window's // local encoder frames, so tdtDecode drops the overlap it does not own. emitStart := melToEncoderFrame(win.emitStart-win.start, subsampling) emitEnd := melToEncoderFrame(win.emitEnd-win.start, subsampling) + // frameOffset turns per-window local timesteps into absolute encoder + // frames so the seam deduper can align tokens across windows. + frameOffset := melToEncoderFrame(win.start, subsampling) + + holdFirst := 0 + var resolveSeam func(head []decodedToken) []decodedToken + if i > 0 { + holdFirst = seamMaxTokens + tail := prevTail + resolveSeam = func(head []decodedToken) []decodedToken { + return dedupSeam(tail, head) + } + } - windowTokens, err := t.runInference(ctx, features[win.start:win.end], emitStart, emitEnd, onToken) + windowTokens, err := t.runInference(ctx, features[win.start:win.end], emitStart, emitEnd, frameOffset, holdFirst, resolveSeam, emit) if err != nil { return "", fmt.Errorf("inference failed: %w", err) } tokens = append(tokens, windowTokens...) + prevTail = windowTokens } if DebugMode { - slog.Debug("tokens decoded", "count", len(tokens), "tokens", tokens) + slog.Debug("tokens decoded", "count", len(tokens)) } return t.tokensToText(tokens), nil } +// newBoundaryOracle builds the per-request chunk-boundary cascade over this +// request's mel features and waveform: Silero VAD first (when enabled and the +// model loaded), then smoothed mel energy (when enabled), then the arithmetic +// midpoint as the always-decides fallback. +func (t *Transcriber) newBoundaryOracle(features [][]float32, waveform []float32) boundaryOracle { + var oracles []boundaryOracle + if !t.disableVADChunking && t.vad != nil { + oracles = append(oracles, &vadBoundaryOracle{ + vad: t.vad, + state: &vadState{}, + waveform: waveform, + hopLength: int64(t.mel.HopLength()), + }) + } + if !t.disableMelChunking { + oracles = append(oracles, newMelEnergyBoundaryOracle(features)) + } + oracles = append(oracles, midpointBoundaryOracle{}) + return chainBoundaryOracle{oracles: oracles} +} + // loadAudio decodes raw request bytes into mono 16 kHz float32 samples. // // Detection is done by content, not by filename extension: an OpenAI client @@ -619,7 +687,7 @@ func (t *Transcriber) loadAudio(data []byte, format string) ([]float32, error) { return parseWAV(wavData) } -func (t *Transcriber) runInference(ctx context.Context, features [][]float32, emitStart, emitEnd int64, onToken func(tok int)) ([]int, error) { +func (t *Transcriber) runInference(ctx context.Context, features [][]float32, emitStart, emitEnd, frameOffset int64, holdFirst int, resolveSeam func(head []decodedToken) []decodedToken, emit func(delta string)) ([]decodedToken, error) { batchSize := int64(1) numFeatures := int64(t.config.FeaturesSize) numFrames := int64(len(features)) @@ -676,7 +744,7 @@ func (t *Transcriber) runInference(ctx context.Context, features [][]float32, em // Decoder tensors (encoderOut) must remain alive during tdtDecode. // The defers above fire after tdtDecode returns, so this is safe. - return t.tdtDecode(ctx, encoderOut, actualEncodedLen, emitStart, emitEnd, onToken) + return t.tdtDecode(ctx, encoderOut, actualEncodedLen, emitStart, emitEnd, frameOffset, holdFirst, resolveSeam, emit) } // tdtDecode greedily decodes the encoder output for one window. It decodes the @@ -684,7 +752,14 @@ func (t *Transcriber) runInference(ctx context.Context, features [][]float32, em // only collects and streams tokens whose timestep falls in [emitStart, emitEnd); // this drops the overlap region owned by an adjacent window. Pass emitStart=0 // and emitEnd=encodedLen to keep everything. -func (t *Transcriber) tdtDecode(ctx context.Context, encoderOut []float32, encodedLen, emitStart, emitEnd int64, onToken func(tok int)) ([]int, error) { +// +// Owned tokens are tagged with an absolute encoder-frame timestep (local +// timestep + frameOffset). When holdFirst > 0 the first holdFirst owned tokens +// are buffered and passed to resolveSeam (the seam deduper) before being +// emitted; the survivors are streamed in order, then the rest of the window +// streams as it is decoded. This keeps streaming order correct while buffering +// only a handful of tokens per seam. +func (t *Transcriber) tdtDecode(ctx context.Context, encoderOut []float32, encodedLen, emitStart, emitEnd, frameOffset int64, holdFirst int, resolveSeam func(head []decodedToken) []decodedToken, emit func(delta string)) ([]decodedToken, error) { // Acquire a pre-initialized worker. Honor cancellation so a client that // disconnects while all workers are busy does not leak a goroutine. var w *decoderWorker @@ -714,11 +789,37 @@ func (t *Transcriber) tdtDecode(ctx context.Context, encoderOut []float32, encod s2[i] = 0 } - var tokens []int + var result []decodedToken + var head []decodedToken + resolved := holdFirst <= 0 timestep := int64(0) emittedTokens := 0 prevToken := t.blankIdx + // emitText streams one token's printable text, skipping special <...> tokens. + emitText := func(id int) { + if emit == nil { + return + } + if text := t.tokenText(id); text != "" { + emit(text) + } + } + // flushHead resolves the buffered seam head through the deduper and streams + // the survivors in order, then marks the seam resolved. + flushHead := func() { + survivors := head + if resolveSeam != nil { + survivors = resolveSeam(head) + } + for _, s := range survivors { + result = append(result, s) + emitText(s.id) + } + head = nil + resolved = true + } + encOutData := w.encOut.GetData() for timestep < encodedLen { @@ -765,9 +866,17 @@ func (t *Transcriber) tdtDecode(ctx context.Context, encoderOut []float32, encod // Collect and stream only tokens this window owns; the rest belong // to an adjacent window's overlap and would duplicate speech. if timestep >= emitStart && timestep < emitEnd { - tokens = append(tokens, token) - if onToken != nil { - onToken(token) + dt := decodedToken{id: token, timestep: frameOffset + timestep} + if resolved { + result = append(result, dt) + emitText(dt.id) + } else { + // Hold the window's leading tokens for the seam deduper. Once + // holdFirst are buffered, resolve and start streaming again. + head = append(head, dt) + if len(head) >= holdFirst { + flushHead() + } } } } @@ -776,7 +885,10 @@ func (t *Transcriber) tdtDecode(ctx context.Context, encoderOut []float32, encod // or an expired deadline frees the worker promptly. select { case <-ctx.Done(): - return tokens, ctx.Err() + if !resolved { + flushHead() + } + return result, ctx.Err() default: } @@ -789,7 +901,13 @@ func (t *Transcriber) tdtDecode(ctx context.Context, encoderOut []float32, encod } } - return tokens, nil + // The window ended before holdFirst tokens were seen: resolve whatever the + // seam head holds (possibly empty) so nothing is left buffered. + if !resolved { + flushHead() + } + + return result, nil } func argmax(data []float32) int { @@ -807,13 +925,25 @@ func argmax(data []float32) int { return maxIdx } -func (t *Transcriber) tokensToText(tokens []int) string { +// tokenText returns the printable text for a token id, or "" for unknown tokens +// and special <...> markers. vocab values already have word-boundary marks +// (U+2581) translated to spaces at load time (see loadVocab), so the text is +// returned as-is. +func (t *Transcriber) tokenText(id int) string { + text, ok := t.vocab[id] + if !ok { + return "" + } + if strings.HasPrefix(text, "<") && strings.HasSuffix(text, ">") { + return "" + } + return text +} + +func (t *Transcriber) tokensToText(tokens []decodedToken) string { var parts []string for _, tok := range tokens { - if text, ok := t.vocab[tok]; ok { - if strings.HasPrefix(text, "<") && strings.HasSuffix(text, ">") { - continue - } + if text := t.tokenText(tok.id); text != "" { parts = append(parts, text) } } diff --git a/internal/asr/vad.go b/internal/asr/vad.go new file mode 100644 index 0000000..d0cf5af --- /dev/null +++ b/internal/asr/vad.go @@ -0,0 +1,190 @@ +// SPDX-FileCopyrightText: 2026 Alby Hernández +// SPDX-License-Identifier: Apache-2.0 + +package asr + +import ( + "fmt" + "log/slog" + "os" + + ort "github.com/yalue/onnxruntime_go" +) + +// Silero VAD constants. These describe the pinned v6.2.1 ONNX contract +// (snakers4/silero-vad, MIT). The model is stateful: its recurrent state travels +// through the "state"/"stateN" tensors per call rather than living inside the +// session, so a single session can be shared across concurrent requests as long +// as every caller keeps its own vadState. +const ( + // vadWindowSamples is the fixed analysis window Silero v5+ expects at 16 kHz: + // 512 samples, i.e. 32 ms per inference. + vadWindowSamples = 512 + + // vadContextSamples is the left-context Silero v5+ prepends to each window. + // The 16 kHz model consumes vadContextSamples+vadWindowSamples (576) samples + // per call; the trailing vadContextSamples of one window become the context + // of the next. The first window uses zero context. + vadContextSamples = 64 + + // vadStateElements is the flattened size of the recurrent state tensor, + // shaped [2, 1, 128]. + vadStateElements = 2 * 1 * 128 + + // vadSampleRate is the only sample rate this integration feeds the model. + // The rest of the pipeline is already 16 kHz mono (see audio.go). + vadSampleRate int64 = 16000 +) + +// vadState carries one request's Silero recurrent state and left-context between +// sequential window inferences. Each transcription owns its own vadState, so the +// shared session stays safe under concurrency. Reset zeroes it, which the +// boundary oracle does at the start of every overlap region. +type vadState struct { + state [vadStateElements]float32 + ctx [vadContextSamples]float32 +} + +// reset clears the recurrent state and left-context so the next window starts a +// fresh analysis. The boundary oracle resets before each overlap region. +func (s *vadState) reset() { + for i := range s.state { + s.state[i] = 0 + } + for i := range s.ctx { + s.ctx[i] = 0 + } +} + +// sileroVAD wraps the shared Silero VAD ONNX session. Like the encoder, +// it is a single long-lived DynamicAdvancedSession reused across requests and +// runs OUTSIDE the decoder worker pool, so its concurrency is bounded by the +// number of in-flight HTTP requests rather than by -workers. +type sileroVAD struct { + session *ort.DynamicAdvancedSession + srData []int64 +} + +// newSileroVAD loads the Silero VAD model from path and creates the shared +// session. A missing file is NOT fatal: the caller logs a warning once and the +// boundary stack degrades to mel energy. Any other error (corrupt model, ORT +// failure) is returned so it surfaces loudly at startup. +func newSileroVAD(path string, sessOpts *ort.SessionOptions) (*sileroVAD, error) { + if _, err := os.Stat(path); os.IsNotExist(err) { + return nil, os.ErrNotExist + } + + session, err := ort.NewDynamicAdvancedSession( + path, + []string{"input", "state", "sr"}, + []string{"output", "stateN"}, + sessOpts, + ) + if err != nil { + return nil, fmt.Errorf("create Silero VAD session: %w", err) + } + + return &sileroVAD{ + session: session, + srData: []int64{vadSampleRate}, + }, nil +} + +// destroy releases the underlying ONNX session. +func (v *sileroVAD) destroy() { + if v != nil && v.session != nil { + v.session.Destroy() + v.session = nil + } +} + +// infer runs one window through the model and returns the speech probability in +// [0, 1]. It prepends st.ctx as left context, updates st.ctx with the trailing +// samples of window, and advances the recurrent state in st.state. window must +// be exactly vadWindowSamples long. +func (v *sileroVAD) infer(st *vadState, window []float32) (float32, error) { + if len(window) != vadWindowSamples { + return 0, fmt.Errorf("silero window must be %d samples, got %d", vadWindowSamples, len(window)) + } + + // Assemble context + window into the model input. + input := make([]float32, vadContextSamples+vadWindowSamples) + copy(input, st.ctx[:]) + copy(input[vadContextSamples:], window) + // Save the trailing samples as context for the next window. + copy(st.ctx[:], window[vadWindowSamples-vadContextSamples:]) + + inputTensor, err := ort.NewTensor(ort.NewShape(1, int64(len(input))), input) + if err != nil { + return 0, fmt.Errorf("create VAD input tensor: %w", err) + } + defer inputTensor.Destroy() + + // The state tensor is written in place across calls, so give the model a + // copy of the current state and read the fresh state from the output. + stateData := make([]float32, vadStateElements) + copy(stateData, st.state[:]) + stateTensor, err := ort.NewTensor(ort.NewShape(2, 1, 128), stateData) + if err != nil { + return 0, fmt.Errorf("create VAD state tensor: %w", err) + } + defer stateTensor.Destroy() + + srTensor, err := ort.NewTensor(ort.NewShape(1), v.srData) + if err != nil { + return 0, fmt.Errorf("create VAD sr tensor: %w", err) + } + defer srTensor.Destroy() + + outputTensor, err := ort.NewEmptyTensor[float32](ort.NewShape(1, 1)) + if err != nil { + return 0, fmt.Errorf("create VAD output tensor: %w", err) + } + defer outputTensor.Destroy() + + stateOutTensor, err := ort.NewEmptyTensor[float32](ort.NewShape(2, 1, 128)) + if err != nil { + return 0, fmt.Errorf("create VAD state output tensor: %w", err) + } + defer stateOutTensor.Destroy() + + if err := v.session.Run( + []ort.Value{inputTensor, stateTensor, srTensor}, + []ort.Value{outputTensor, stateOutTensor}, + ); err != nil { + return 0, fmt.Errorf("VAD run failed: %w", err) + } + + copy(st.state[:], stateOutTensor.GetData()) + + prob := outputTensor.GetData()[0] + return prob, nil +} + +// speechProbabilities runs the model over samples in sequential windows and +// returns one speech probability per full window. It resets st first so each +// overlap region is analysed independently. Trailing samples that do not fill a +// full window are ignored. On any inference error it logs and returns what it +// has, so a VAD hiccup degrades to fewer probabilities rather than failing the +// whole transcription. +func (v *sileroVAD) speechProbabilities(st *vadState, samples []float32) []float32 { + st.reset() + + numWindows := len(samples) / vadWindowSamples + if numWindows == 0 { + return nil + } + + probs := make([]float32, 0, numWindows) + for i := 0; i < numWindows; i++ { + window := samples[i*vadWindowSamples : (i+1)*vadWindowSamples] + prob, err := v.infer(st, window) + if err != nil { + slog.Warn("VAD inference failed mid-overlap, using partial probabilities", + "window", i, "error", err) + break + } + probs = append(probs, prob) + } + return probs +} diff --git a/internal/server/server.go b/internal/server/server.go index 568a09f..c7e4aa7 100644 --- a/internal/server/server.go +++ b/internal/server/server.go @@ -51,6 +51,15 @@ type Config struct { ChunkSeconds int ChunkOverlapSeconds int LongAudio bool + + // DisableVADBasedChunking and DisableMelBasedChunking turn off the first two + // layers of the chunk-boundary cascade (Silero VAD, then mel energy). The + // arithmetic midpoint is always the final fallback. VADModelPath overrides + // where the Silero VAD model is loaded from; empty means silero_vad.onnx + // inside the models directory. + DisableVADBasedChunking bool + DisableMelBasedChunking bool + VADModelPath string } // Server represents the HTTP server for the ASR service @@ -88,6 +97,11 @@ func New(cfg Config) (*Server, error) { Seconds: cfg.ChunkSeconds, OverlapSeconds: cfg.ChunkOverlapSeconds, }, + Boundary: asr.BoundaryConfig{ + DisableVAD: cfg.DisableVADBasedChunking, + DisableMel: cfg.DisableMelBasedChunking, + VADModelPath: cfg.VADModelPath, + }, }) if err != nil { return nil, fmt.Errorf("failed to initialize transcriber: %w", err) diff --git a/main.go b/main.go index 99aa902..f0451c1 100644 --- a/main.go +++ b/main.go @@ -35,6 +35,9 @@ func main() { flag.IntVar(&cfg.ChunkSeconds, "chunk-seconds", 300, "Sliding-window size in seconds for long audio (must stay under the model limit)") flag.IntVar(&cfg.ChunkOverlapSeconds, "chunk-overlap-seconds", 15, "Overlap in seconds between consecutive chunks") flag.BoolVar(&cfg.LongAudio, "long-audio", false, "Split audio longer than the model limit into overlapping chunks instead of rejecting it") + flag.BoolVar(&cfg.DisableVADBasedChunking, "disable-vad-based-chunking", false, "Disable the Silero VAD layer of the chunk-boundary cascade (falls back to mel energy)") + flag.BoolVar(&cfg.DisableMelBasedChunking, "disable-mel-based-chunking", false, "Disable the mel-energy layer of the chunk-boundary cascade (falls back to the midpoint)") + flag.StringVar(&cfg.VADModelPath, "vad-model-path", "", "Path to the Silero VAD ONNX model (default: silero_vad.onnx inside the models dir)") flag.Parse() // Any flag not set on the command line falls back to its matching env var, diff --git a/testdata/reference/README.md b/testdata/reference/README.md new file mode 100644 index 0000000..8384020 --- /dev/null +++ b/testdata/reference/README.md @@ -0,0 +1,44 @@ +# Reference audio for chunk-seam validation + +This directory holds the exact audio and reference transcripts used to reproduce +and validate the fix for issue #18 (duplicated or dropped words at the seams +between overlapping windows in long-audio mode). + +## Provenance + +- Source: the YouTube video "Learn Case Interviews in Under 30 minutes", the + exact file attached to issue #18. +- Original filenames (renamed here to shell-friendly ones): + - `Learn Case Interviews in Under 30 minutes.mp3` -> `learn-case-interviews.mp3` + - `Learn Case Interviews in Under 30 minutes.srt` -> `learn-case-interviews.srt` + - `Learn Case Interviews in Under 30 minutes.vtt` -> `learn-case-interviews.vtt` + - `Learn Case Interviews in Under 30 minutes.txt` -> `learn-case-interviews.txt` + +The MP3 (~41 MB) is committed to the repository on purpose: reproducing the +seam behaviour needs the real long-audio input, and a stable checked-in copy +means the validation does not depend on any network access. + +## Files + +- `learn-case-interviews.mp3` - the input audio (~30 minutes, needs `-long-audio`). +- `learn-case-interviews.srt` / `.vtt` - timed reference transcripts, used to + print the expected words around each seam for a side-by-side comparison. +- `learn-case-interviews.txt` - the plain reference transcript. + +## Usage + +The seam inspector is a build-tag-gated Go test (no Python in this repo). It +transcribes the MP3 through the full pipeline with long-audio enabled, logs the +chosen boundary positions, and prints the transcribed text around each seam next +to the reference `.srt` text for the same time range, so a human can eyeball +every seam quickly. It needs ONNX Runtime and the models locally but no network. + +```bash +# Requires the models (make models-int8) and ONNX Runtime on the host. +PARAKEET_MODELS=./models \ +PARAKEET_SEAM_AUDIO=./testdata/reference/learn-case-interviews.mp3 \ +PARAKEET_SEAM_SRT=./testdata/reference/learn-case-interviews.srt \ +go test -tags=seaminspect -run TestSeamInspection -v ./internal/asr/ +``` + +See `.agents/DESIGN_DECISIONS.md` (DD-014) for the boundary-selection design. diff --git a/testdata/reference/learn-case-interviews.mp3 b/testdata/reference/learn-case-interviews.mp3 new file mode 100644 index 0000000..56b3eb5 Binary files /dev/null and b/testdata/reference/learn-case-interviews.mp3 differ diff --git a/testdata/reference/learn-case-interviews.srt b/testdata/reference/learn-case-interviews.srt new file mode 100644 index 0000000..7385e3d --- /dev/null +++ b/testdata/reference/learn-case-interviews.srt @@ -0,0 +1,542 @@ +1 +00:00:01,000 --> 00:00:32,000 +In this video, I'm going to teach you fundamental strategies and frameworks for tackling case interviews in roughly 30 minutes. +We'll cover each step of the case interview from beginning to end, and go through a full case interview together. +By the end of this video, you will not only fully understand what case interviews are, +but you will learn effective strategies to use on interview day to make you feel more confident in yourself. +We have a lot of material to cover, so please give this video a quick like to support the channel and we'll get started + +2 +00:00:33,000 --> 00:01:03,000 +right away. +So, what is a case interview? +As you may already know, case interviews are a special type of interview question that is commonly used by management consulting companies such as McKinsey, +BCG, and Bain. + +A case interview or case is a 30 to 45 minute exercise in which you and the interviewer will work together to develop a recommendation +to answer or solve a business problem. +These business problems can be anything that real companies face. + +3 +00:01:04,000 --> 00:01:35,000 +A few examples are, how can Coca-Cola increase its profitability? +What should Netflix do to increase customer retention? +How should Apple price its new iPhone? +And where should Disney open its next Disneyland theme park? + +At the beginning of the interview, the interviewer will give you background information and the business problem. +And in the end, you'll present a recommendation and provide two to three reasons that support your recommendation. +These two to three reasons will likely be both quantitative and qualitative. + +4 +00:01:36,000 --> 00:02:06,000 +You may need to make estimations or assumptions, perform calculations, interpret charts and graphs, brainstorm ideas, and have qualitative discussions. +There is usually no single right answer for a case. +As long as you have logical reasons and evidence that support your recommendation, you will do well. +Although there are many business problems you could be asked to solve, almost all case interviews follow the same format or structure. +So, the good news is that you can use the same strategies for every single case interview. + +5 +00:02:07,000 --> 00:02:38,000 +There are seven main parts of a case interview. +Understanding the case background information, asking clarifying questions, structuring a framework, starting the case, solving quantitative problems, answering qualitative questions, and delivering a recommendation. +To best illustrate exactly how a case interview looks like, we'll go through each of these parts step by step. +Let's start with the first part, understanding the case background. +The case interview will begin with the interviewer giving you the case background information. + +6 +00:02:39,000 --> 00:03:13,000 +Let's say that the interviewer reads you the following. +Our client is a large manufacturer and retailer of non-alcoholic beverages such as sodas, juices, sports drinks, and teas. +They have annual revenues of roughly$30 billion and an operating margin of roughly 30%. +Our client is looking to grow and is considering entering the beer market in the United States. + +Should they enter? +As the interviewer reads this, take notes. +It is important to understand what the objective of the case is and keep track of information. + +7 +00:03:14,000 --> 00:03:48,000 +One strategy for taking notes effectively is to turn your paper landscape and draw a vertical line to divide your paper into two sections. +The first section should be roughly two thirds of the page, while the second section will be roughly one third of the page. +Take notes in the second section of your page. + +After the interviewer finishes giving the case background information, confirm that you understand the situation and objective. +Provide a concise synthesis like the following. +To make sure I understand correctly, our client is a large manufacturer and retailer of non-alcoholic beverages. + +8 +00:03:49,000 --> 00:04:20,000 +They are looking to grow, and our objective is to determine whether or not they should enter the U.S. +Beer market. +Make sure your synthesis is concise. +You do not want to regurgitate verbatim everything that the interviewer has said. +Only mention the most important pieces of information. + +You should also make sure that you verify the objective of the case. +Answering or solving the wrong case objective is the quickest way to fail a case interview. +Next, we'll move on to the next step, asking clarifying questions. + +9 +00:04:21,000 --> 00:04:55,000 +Now you'll have the opportunity to ask questions before you begin thinking about how to solve the case. +At this point, only ask questions that are absolutely critical for you to fully understand the case background and objective. +You'll be able to ask more questions later. +There are four types of questions you could ask. + +One, asking for a definition of a term you're unfamiliar with. +Two, asking for information that strengthens your understanding of the company or situation. +Three, asking questions that clarify the objective of the case. + +10 +00:04:55,000 --> 00:05:28,000 +Four, asking to repeat information you may have missed. +So for this case, you might ask the following questions. +Is our client looking to specifically grow revenues or profits? +Answer, our client wants to grow profits. + +Is there a particular financial goal or metric our client is trying to reach within a certain timeframe? +Answer, they're looking to grow annual profits by$5 billion within five years. +Now we'll move on to the third step of the case interview, structuring a framework. + +11 +00:05:29,000 --> 00:06:01,000 +After you understand the case background and objective, lay out a framework of what areas you would want to look into to answer or solve +the case. +A framework is simply a tool that helps you structure and break down complex problems into simpler, smaller components. + +Think of a framework as brainstorming different ideas and organizing them into different categories. +When creating a framework, it is completely acceptable to ask the interviewer for a few minutes of silence to write it out. +You might say something like this. + +12 +00:06:02,000 --> 00:06:34,000 +Would you mind if I take a few minutes to structure my thoughts and develop a framework to tackle this case? +Most of the time, the interviewer will gladly give you a couple of minutes to gather your thoughts. +For this case example, what do you need to know in order to help the client decide whether or not they should enter the beer +market? +You might brainstorm the following questions. + +Does our client know how to produce beer? +Would people buy beer made by our client? +Where would our client sell its beer? +How much would it cost to enter the beer market? + +13 +00:06:34,000 --> 00:07:08,000 +Will our client be profitable from doing this? +How can our client out-compete competitors? +And what is the size of the beer market? +This is not a very structured way of tackling the case. +Instead, you should organize your ideas into a framework that has three to four broad areas, also known as buckets, + +that you want to investigate. +An easy way to develop these buckets is to ask yourself, +what are the three to four things that absolutely must be true for you to 100% recommend that the client should enter the beer + +14 +00:07:08,000 --> 00:07:19,000 +market? +In an ideal world, these four things would need to be true. +1. The beer market is an attractive market with high profit margins. + +15 +00:07:19,000 --> 00:07:19,000 +2. + +16 +00:07:19,000 --> 00:07:25,000 +Competitors are weak and our client will be able to capture significant market share. + +17 +00:07:25,000 --> 00:07:25,000 +3. + +18 +00:07:25,000 --> 00:07:29,000 +Our client has the capabilities to produce an outstanding beer product. + +19 +00:07:31,000 --> 00:07:31,000 +4. + +20 +00:07:31,000 --> 00:08:03,000 +Our client will be extremely profitable. +You can rephrase these points to be the broad categories in your framework. +You can write your framework in the first section of your paper. + +Notice that the four buckets in the framework correspond to the four things that we believe need to be true to be confident in recommending +that our client should enter the beer market. +Next, let's add a few bullet points under each category to give more details on exactly what information we need to know to decide + +21 +00:08:03,000 --> 00:08:34,000 +whether our client should enter the beer market. +Notice that each bullet in a bucket is a question that helps answer the overall question of the bucket. +For example, in the first bucket, knowing the market size, the market growth rate, + +and average profit margins helps us determine whether the beer market is an attractive market. +This entire process of brainstorming ideas and developing a structured framework should only take a few minutes. +So how do you come up with a framework so quickly? + +22 +00:08:34,000 --> 00:09:12,000 +Most candidates make the mistake of either using a single memorized framework for every case, +or memorizing a different framework for every type of case. +The issue with memorized frameworks is that they aren't tailored to the specific case you are solving for. + +When given an atypical business problem, your framework elements will not be entirely relevant. +Interviewers can easily tell that you are regurgitating memorized information and not thinking critically. +So instead of memorizing frameworks, I recommend memorizing a list of eight to 10 broad business elements, such as the following. + +23 +00:09:13,000 --> 00:09:48,000 +Market attractiveness, competitive landscape, company capabilities or attractiveness, customer segments or needs, profitability or financials, strategic alternatives, risks and mitigations, +and creating your own bucket. +When given a case, mentally run through this list and pick the three to four elements that are most relevant to the case. + +This will be your framework. +If the list does not give you enough elements, brainstorm and add your own unique elements to your framework. +This strategy guarantees that your framework elements are relevant to the case. + +24 +00:09:49,000 --> 00:10:21,000 +It also demonstrates that you can create unique tailored frameworks for every business problem or situation. +Using this strategy for this case, you would run through your list of memorized business elements and select the following. +Market attractiveness, competitive landscape, company capabilities, and profitability. +As you can see, this strategy is a shortcut for creating unique tailored frameworks for every business problem. +You do not need to develop a framework entirely from scratch each time. + +25 +00:10:22,000 --> 00:10:56,000 +Now that you have your framework, turn your paper to face the interviewer and walk them through it. +It might sound something like this. +To decide whether or not our client should enter the market, I want to look into four main areas. +One, I want to look into the beer market attractiveness. +Is this an attractive market to enter? + +I'd want to look into areas such as the market size, growth rate, and profit margins. +Two, I want to look into the beer competitive landscape. +Is this market competitive, and will our client be able to capture meaningful market share? + +26 +00:10:57,000 --> 00:11:27,000 +To do this, I want to look into questions such as the number of competitors, how much market share each competitor has, +and whether competitors have any competitive advantages. +Three, I want to look into our clients'capabilities. +Do they have the capabilities to succeed in the beer market? + +I want to look into things such as whether they have the expertise to produce beer, +whether they have the distribution channels to sell that beer, and whether there's any existing synergies they can leverage. +4. I want to look into expected profitability. + +27 +00:11:27,000 --> 00:11:58,000 +Will our client be profitable from entering the beer market? +I'd like to look into areas such as expected revenues, expected costs, and how long it would take to break even. +At this point, the interviewer might ask a few questions on your framework, + +but will otherwise indicate whether they agree or disagree with your approach. +Moving on to the next step of the case interview, starting the case. +There are two different styles of case interviews, interviewer-led cases and candidate-led cases. + +28 +00:11:59,000 --> 00:12:32,000 +If this is an interviewer-led case, the interviewer will propose which area of your framework they would like to dive deeper into. +They might say something like the following. +Your framework makes sense to me. +Why don't we start by estimating the size of the US beer market? + +If this is a candidate-led case, you will be expected to propose an area to look into. +There is no right or wrong answer as to which area to start first. +So, feel free to propose any area of your framework as long as you have a reason for it. + +29 +00:12:32,000 --> 00:13:05,000 +You could say something like the following. +To start, I'd like to look into the beer market attractiveness. +I'd like to first understand the market size to determine if the beer market is an attractive market. +If you end up picking an area that the interviewer does not want you to explore, + +they will redirect you to an area that they do want you to explore. +So, these two styles of case interviews are nearly identical. +The only difference is whether or not you have to proactively propose what area to explore first and what area you want to explore + +30 +00:13:05,000 --> 00:13:42,000 +next. +So, once the case has been kicked off by either you or the interviewer, you'll have to solve quantitative problems and answer qualitative questions, +and in this section, we'll focus on solving quantitative problems first. +There are three different types of quantitative problems you may get asked to solve. + +Market sizing or estimation questions, profit or profitability questions, and charts and graphs questions. +Let's start with the first type, market sizing or estimation questions. +These types of questions ask you to estimate the size of a particular market or calculate a particular figure. + +31 +00:13:42,000 --> 00:14:16,000 +Let's say that the interviewer asks you the following, what is the market size of beer in the US? +Most candidates jump right into the math, stating the US population and then performing various calculations. +Doing math without laying out a structure often leads to making unnecessary calculations or reaching a dead end. +Instead, lay out an upfront approach to help avoid these mistakes and demonstrate that you are a logical, structured thinker. +For this market sizing problem, you could structure your approach in the following way. + +32 +00:14:16,000 --> 00:14:46,000 +Start with the US population, estimate the percentage that are legally allowed to drink alcohol, then estimate the percentage that drink beer, +then estimate the frequency in which people drink beer, and then finally estimate the average price per can or bottle of beer. +Multiplying all of these steps together will give you the answer. +By laying out an approach upfront, the interviewer can easily understand how you are thinking about the problem. +With the right structure, the rest of the problem is just simple arithmetic. + +33 +00:14:47,000 --> 00:15:25,000 +Sometimes the interviewer will give you numbers to use for these calculations, but other times you'll be expected to make assumptions or estimates. +When performing your calculations, make sure to do them on a separate sheet of paper. +Calculations often get messy, and you want to keep your original paper clean and organized. + +A sample answer to this question could sound like the following. +Following my approach, I'll assume the US population is 320 million people. +Assuming the average life expectancy is 80 years old and an even distribution of ages, roughly 75% of the population can legally drink alcohol. + +34 +00:15:26,000 --> 00:16:03,000 +This gives us 240 million people. +Of these, let's assume 75% of people drink beer. +That gives us 180 million beer drinkers. + +On average, a person drinks perhaps 5 beers a week, or roughly 250 beers per year, if we're assuming roughly 50 weeks per year. +So, that gives us 180 million people times 250 beers per year, or 45 billion cans or bottles of beer. +Assuming the average can or bottle costs$2, this gives us a market size of$90 billion. + +35 +00:16:04,000 --> 00:16:41,000 +Now that you have your answer, you should not only just answer the question, but tie your answer back to the case objective. +In other words, how does knowing the US market size of beer help you decide whether or not our client should enter the beer +market? +You could say something like the following. + +Given that our client has annual revenues of$30 billion, a$90 billion beer market represents a massive opportunity. +The market size makes the beer market look attractive, +but I'd like to understand if beer margins are typically high and determine how much market share our client could realistically capture. + +36 +00:16:42,000 --> 00:17:15,000 +The second type of quantitative question you could be asked is to calculate profit or profitability. +To answer profit or profitability questions, you only really need to know the basic formulas for profit. +These are that profit equals revenue minus costs. +Revenue equals quantity times price. + +Costs equal total variable costs plus total fixed costs. +And finally, total variable costs equal quantity times variable cost per unit. +Putting these equations together, we can rewrite them in the following simplified way. + +37 +00:17:16,000 --> 00:17:50,000 +Profit equals price minus variable costs times quantity minus total fixed costs. +You should also know that profit margin equals revenue minus costs, all divided by revenue. +And you should also know what a break-even point is. + +A break-even point occurs when a company sells enough product such that it has exactly recouped all of its costs. +In other words, break-even occurs when profit equals zero. +To solve for the break-even point, you would simply set profit equal to zero and solve for the unknown variable in the equation. + +38 +00:17:51,000 --> 00:18:24,000 +As an example of a profit or profitability problem, let's say that the interviewer asks you the following. +Assume that a 12-ounce can of beer sells for$2 on average. +To produce a keg of beer, it costs$100 for raw materials,$95 for labor, and$75 for storage. +If a keg of beer holds 1,800 ounces of beer, what is the profit margin for beer? +As with any other type of quantitative problem, make sure to structure your approach before you begin doing any calculations. + +39 +00:18:25,000 --> 00:18:59,000 +A sample answer could look like the following. +To calculate the average margin for beer, I will first calculate the total costs to produce a keg of beer. +Next, I will divide the total volume of a keg by the total volume of a can to determine how many cans a keg +of beer produces. + +Afterwards, I will divide the total cost of producing a keg of beer by the number of cans to determine cost per can. +Finally, I can use the price and cost per can of beer to calculate the average margin of beer. +Following this approach, the total cost of a keg of beer is$100 plus$95 plus$75, or$270. + +40 +00:19:05,000 --> 00:19:38,000 +The number of cans of beer in a keg is 1,800 ounces divided by 12 ounces, or 150 cans. +Therefore, the cost per can of beer is$270 divided by 150 cans, or$1.80. +Since the average price of beer is$2 per can, the profit is$0.20 per can. +This makes the margin$0.20 divided by$2, or a 10% profit margin. +Now that you have your answer, remember to tie your answer back to the overall case objective. + +41 +00:19:39,000 --> 00:20:12,000 +Compared to our client's overall operating margin of 30%, the beer market profit margin of 10% is significantly lower. +Although the market size for beer is large, the low margin makes the beer market less attractive. +The third type of quantitative question you could get asked is interpreting charts and graphs. +The types of charts and graphs you should be familiar with include bar charts, pie charts, line graphs, bubble charts, and waterfall charts. +Returning to our case example, the interviewer may show you the following. + +42 +00:20:13,000 --> 00:20:44,000 +A helpful strategy is to start your analysis by explaining what the axes of the chart show. +This will help you understand the chart better. +Don't just read what numbers the chart shows, but instead interpret what those numbers mean for the case objective. + +A sample answer might sound like the following. +For this chart, we have market share on the y-axis and different categories of beer on the x-axis. +For each category, we see that market share is concentrated among a few large players. + +43 +00:20:45,000 --> 00:21:17,000 +This implies a highly competitive market with high barriers to entry. +Because of this, the beer market does not look attractive because it is so competitive. +In addition to asking quantitative questions, the interviewer will also ask qualitative questions. +There are two types of qualitative questions, brainstorming questions and business opinion questions. +Looking at brainstorming questions first, the interviewer might ask, what are the barriers to entry in the beer market? + +44 +00:21:18,000 --> 00:21:52,000 +Most candidates answer by listing ideas that immediately come to mind, such as brewing equipment, beer production expertise, distribution channels, or brand name. +This is a highly unstructured way of answering the question. +Make sure to use a simple structure to organize your thoughts. + +A simple structure, such as thinking about barriers to entry as either economic barriers or non-economic barriers, +helps facilitate brainstorming and demonstrates logic and structure. +With this structure, you might come up with the following answer. + +45 +00:21:52,000 --> 00:22:27,000 +For economic barriers, you have equipment, raw materials, and other capital. +For non-economic barriers, you have beer brewing expertise, brand name, and distribution channels. +Examples of other simple structures that you can use include the following. + +We have internal, external, short-term, long-term, economic, non-economic, quantitative and qualitative, direct and indirect, supply side and demand side, upside and downside, +and benefits and costs. +Additionally, take your answer and connect it back to the case objective. + +46 +00:22:28,000 --> 00:23:00,000 +In this example, are these barriers to entry high or low in your opinion? +Do you think our client can overcome these obstacles to enter the beer market? +You might answer this question in the following way. +I'm thinking of barriers to entry as economic barriers and non-economic barriers. + +Economic barriers include things such as equipment, raw material, and other capital. +Non-economic barriers include beer brewing expertise, brand name, and distribution channels. +Looking at these barriers, I think it will take our client a lot of work to overcome these barriers. + +47 +00:23:01,000 --> 00:23:34,000 +While our client does have a brand name and distribution channels, +they lack beer brewing expertise and would have to buy a lot of expensive equipment and machinery. +These barriers make entering the beer market difficult. +The second type of qualitative question are business opinion questions. + +The interviewer may ask you something like the following. +Do you think there are significant production synergies in producing non-alcoholic beverages and producing beer? +As always, structure your answer and connect your answer to the case objective. + +48 +00:23:35,000 --> 00:24:08,000 +Here is a sample answer. +Production involves equipment, raw materials, and labor. +There is likely some overlap in equipment, such as using the same bottling machines, +but our client will likely need new equipment for brewing beer. + +Raw materials on the other hand are completely different. +Our client will need to source barley, hops, and yeast, which it currently does not use in its existing beverages. +Finally, the same labor can be used, but employees will need new training since producing beer is fairly different from producing a non-alcoholic drink. + +49 +00:24:09,000 --> 00:24:42,000 +Overall, I think there are only a few production synergies that our client can leverage, which makes entering the market a bit more difficult. +You've done a ton of work, and now it is time to put everything you've done together into a recommendation. +Throughout the interview, you should have been making notes of key takeaways after each question that you answer. + +Take a look at the key takeaways you've accumulated so far and decide whether you want to recommend entering the beer market or not +entering the beer market. +Throughout the case, you may have identified the following key takeaways. + +50 +00:24:43,000 --> 00:25:19,000 +The US beer market size is$90 billion compared to our client's annual revenue of$30 billion. +The beer market profit margins are 10%, compared to our client's average margin of 30%. +The beer market is highly concentrated across all categories. + +Barriers to entry are moderate, and there are some synergies with existing production. +Remember, there is no right or wrong recommendation, as long as you support your recommendation with reasons and evidence. +Secondly, regardless of what stance you take, make sure you have a firm recommendation. + +51 +00:25:19,000 --> 00:25:52,000 +You do not want to be flimsy and switch back and forth between recommending entering the market and not entering the market. +Third, make sure your recommendation is clear and concise. +And finally, include next steps in your recommendation. +You can use the following structure when delivering your recommendation. + +First, clearly state what your recommendation is. +Then, follow that with two to three reasons that support your recommendation. +And finally, state what potential next steps would be to further validate your recommendation. + +52 +00:25:53,000 --> 00:26:27,000 +Next steps can include a number of different things. +You could include buckets in your framework that you were not able to cover. +You could also include open questions that you still have regarding the case. +Finally, you could name potential risks relating to your recommendation that you would like to explore. +Typically, the interviewer will prompt you for a final recommendation. + +They might say something like, let's say that you bump into the CEO in the elevator. +He asks what your preliminary recommendation is. +What would you say? +The conclusion of this case might sound like the following. + +53 +00:26:28,000 --> 00:27:00,000 +I recommend that our client should not enter the beer market for the following three reasons. +One, although the market size is fairly large at$90 billion, the margins for beer are just 10%, +significantly less than our client's overall operating margin of 30%. + +Two, the beer market is very competitive. +In all beer segments, market share is concentrated among a few players, which implies high barriers to entry. +Our client lacks beer brewing expertise to produce a great product to compete with incumbents. + +54 +00:27:01,000 --> 00:27:34,000 +Three, there are not that many production synergies that a client can leverage with its existing products. +Our client would need to buy new equipment, source new raw materials, and provide new training to employees, which would be time-consuming and costly. +For next steps, I want to look into our client's annual expected profits if they were to enter the U.S. + +Beer market. +I hypothesize they will be unable to achieve an increase in annual profits of$2 billion within five years, +but I'd like to confirm this through further analysis. + +55 +00:27:36,000 --> 00:28:09,000 +And with that, we have made our way through an entire case interview. +That's all there is to it. +To recap, all case interviews roughly follow the same format or structure. +The industry you are working with may be different, the quantitative problems may be different, and the qualitative questions may be different, + +but the overall strategy and approach remain the same. +If you follow the fundamental strategies covered in this video and continue practicing case interviews, you should be in great shape for your interview. +We've covered a lot of material in this video, but of course, + +56 +00:28:10,000 --> 00:28:43,000 +there is many more case interview strategies to learn and cases to be practiced. +This video is a short preview of the full Hacking the Case Interview online course, +which covers each step of the case interview in even more detail. + +Through over 50 video lessons and 20 full-length practice cases, +this 15-25 hour online course is designed to get you a prestigious and lucrative job offer at a top-tier consulting firm. +Second, if you found this video helpful, please take a second to give this video a like to help support the channel and encourage + +57 +00:28:43,000 --> 00:28:59,000 +more content. +Finally, the only way to get better at case interviews is to practice, +so make sure to do as many mock case interviews as you can. +Check out some of the other videos on my channel where I walk through many full-length mock case interviews. + diff --git a/testdata/reference/learn-case-interviews.txt b/testdata/reference/learn-case-interviews.txt new file mode 100644 index 0000000..533803b --- /dev/null +++ b/testdata/reference/learn-case-interviews.txt @@ -0,0 +1,485 @@ +[00:01]->[00:32] +In this video, I'm going to teach you fundamental strategies and frameworks for tackling case interviews in roughly 30 minutes. +We'll cover each step of the case interview from beginning to end, and go through a full case interview together. +By the end of this video, you will not only fully understand what case interviews are, +but you will learn effective strategies to use on interview day to make you feel more confident in yourself. +We have a lot of material to cover, so please give this video a quick like to support the channel and we'll get started + +[00:33]->[01:03] +right away. +So, what is a case interview? +As you may already know, case interviews are a special type of interview question that is commonly used by management consulting companies such as McKinsey, +BCG, and Bain. + +A case interview or case is a 30 to 45 minute exercise in which you and the interviewer will work together to develop a recommendation +to answer or solve a business problem. +These business problems can be anything that real companies face. + +[01:04]->[01:35] +A few examples are, how can Coca-Cola increase its profitability? +What should Netflix do to increase customer retention? +How should Apple price its new iPhone? +And where should Disney open its next Disneyland theme park? + +At the beginning of the interview, the interviewer will give you background information and the business problem. +And in the end, you'll present a recommendation and provide two to three reasons that support your recommendation. +These two to three reasons will likely be both quantitative and qualitative. + +[01:36]->[02:06] +You may need to make estimations or assumptions, perform calculations, interpret charts and graphs, brainstorm ideas, and have qualitative discussions. +There is usually no single right answer for a case. +As long as you have logical reasons and evidence that support your recommendation, you will do well. +Although there are many business problems you could be asked to solve, almost all case interviews follow the same format or structure. +So, the good news is that you can use the same strategies for every single case interview. + +[02:07]->[02:38] +There are seven main parts of a case interview. +Understanding the case background information, asking clarifying questions, structuring a framework, starting the case, solving quantitative problems, answering qualitative questions, and delivering a recommendation. +To best illustrate exactly how a case interview looks like, we'll go through each of these parts step by step. +Let's start with the first part, understanding the case background. +The case interview will begin with the interviewer giving you the case background information. + +[02:39]->[03:13] +Let's say that the interviewer reads you the following. +Our client is a large manufacturer and retailer of non-alcoholic beverages such as sodas, juices, sports drinks, and teas. +They have annual revenues of roughly$30 billion and an operating margin of roughly 30%. +Our client is looking to grow and is considering entering the beer market in the United States. + +Should they enter? +As the interviewer reads this, take notes. +It is important to understand what the objective of the case is and keep track of information. + +[03:14]->[03:48] +One strategy for taking notes effectively is to turn your paper landscape and draw a vertical line to divide your paper into two sections. +The first section should be roughly two thirds of the page, while the second section will be roughly one third of the page. +Take notes in the second section of your page. + +After the interviewer finishes giving the case background information, confirm that you understand the situation and objective. +Provide a concise synthesis like the following. +To make sure I understand correctly, our client is a large manufacturer and retailer of non-alcoholic beverages. + +[03:49]->[04:20] +They are looking to grow, and our objective is to determine whether or not they should enter the U.S. +Beer market. +Make sure your synthesis is concise. +You do not want to regurgitate verbatim everything that the interviewer has said. +Only mention the most important pieces of information. + +You should also make sure that you verify the objective of the case. +Answering or solving the wrong case objective is the quickest way to fail a case interview. +Next, we'll move on to the next step, asking clarifying questions. + +[04:21]->[04:55] +Now you'll have the opportunity to ask questions before you begin thinking about how to solve the case. +At this point, only ask questions that are absolutely critical for you to fully understand the case background and objective. +You'll be able to ask more questions later. +There are four types of questions you could ask. + +One, asking for a definition of a term you're unfamiliar with. +Two, asking for information that strengthens your understanding of the company or situation. +Three, asking questions that clarify the objective of the case. + +[04:55]->[05:28] +Four, asking to repeat information you may have missed. +So for this case, you might ask the following questions. +Is our client looking to specifically grow revenues or profits? +Answer, our client wants to grow profits. + +Is there a particular financial goal or metric our client is trying to reach within a certain timeframe? +Answer, they're looking to grow annual profits by$5 billion within five years. +Now we'll move on to the third step of the case interview, structuring a framework. + +[05:29]->[06:01] +After you understand the case background and objective, lay out a framework of what areas you would want to look into to answer or solve +the case. +A framework is simply a tool that helps you structure and break down complex problems into simpler, smaller components. + +Think of a framework as brainstorming different ideas and organizing them into different categories. +When creating a framework, it is completely acceptable to ask the interviewer for a few minutes of silence to write it out. +You might say something like this. + +[06:02]->[06:34] +Would you mind if I take a few minutes to structure my thoughts and develop a framework to tackle this case? +Most of the time, the interviewer will gladly give you a couple of minutes to gather your thoughts. +For this case example, what do you need to know in order to help the client decide whether or not they should enter the beer +market? +You might brainstorm the following questions. + +Does our client know how to produce beer? +Would people buy beer made by our client? +Where would our client sell its beer? +How much would it cost to enter the beer market? + +[06:34]->[07:08] +Will our client be profitable from doing this? +How can our client out-compete competitors? +And what is the size of the beer market? +This is not a very structured way of tackling the case. +Instead, you should organize your ideas into a framework that has three to four broad areas, also known as buckets, + +that you want to investigate. +An easy way to develop these buckets is to ask yourself, +what are the three to four things that absolutely must be true for you to 100% recommend that the client should enter the beer + +[07:08]->[07:19] +market? +In an ideal world, these four things would need to be true. +1. The beer market is an attractive market with high profit margins. + +[07:19]->[07:19] +2. + +[07:19]->[07:25] +Competitors are weak and our client will be able to capture significant market share. + +[07:25]->[07:25] +3. + +[07:25]->[07:29] +Our client has the capabilities to produce an outstanding beer product. + +[07:31]->[07:31] +4. + +[07:31]->[08:03] +Our client will be extremely profitable. +You can rephrase these points to be the broad categories in your framework. +You can write your framework in the first section of your paper. + +Notice that the four buckets in the framework correspond to the four things that we believe need to be true to be confident in recommending +that our client should enter the beer market. +Next, let's add a few bullet points under each category to give more details on exactly what information we need to know to decide + +[08:03]->[08:34] +whether our client should enter the beer market. +Notice that each bullet in a bucket is a question that helps answer the overall question of the bucket. +For example, in the first bucket, knowing the market size, the market growth rate, + +and average profit margins helps us determine whether the beer market is an attractive market. +This entire process of brainstorming ideas and developing a structured framework should only take a few minutes. +So how do you come up with a framework so quickly? + +[08:34]->[09:12] +Most candidates make the mistake of either using a single memorized framework for every case, +or memorizing a different framework for every type of case. +The issue with memorized frameworks is that they aren't tailored to the specific case you are solving for. + +When given an atypical business problem, your framework elements will not be entirely relevant. +Interviewers can easily tell that you are regurgitating memorized information and not thinking critically. +So instead of memorizing frameworks, I recommend memorizing a list of eight to 10 broad business elements, such as the following. + +[09:13]->[09:48] +Market attractiveness, competitive landscape, company capabilities or attractiveness, customer segments or needs, profitability or financials, strategic alternatives, risks and mitigations, +and creating your own bucket. +When given a case, mentally run through this list and pick the three to four elements that are most relevant to the case. + +This will be your framework. +If the list does not give you enough elements, brainstorm and add your own unique elements to your framework. +This strategy guarantees that your framework elements are relevant to the case. + +[09:49]->[10:21] +It also demonstrates that you can create unique tailored frameworks for every business problem or situation. +Using this strategy for this case, you would run through your list of memorized business elements and select the following. +Market attractiveness, competitive landscape, company capabilities, and profitability. +As you can see, this strategy is a shortcut for creating unique tailored frameworks for every business problem. +You do not need to develop a framework entirely from scratch each time. + +[10:22]->[10:56] +Now that you have your framework, turn your paper to face the interviewer and walk them through it. +It might sound something like this. +To decide whether or not our client should enter the market, I want to look into four main areas. +One, I want to look into the beer market attractiveness. +Is this an attractive market to enter? + +I'd want to look into areas such as the market size, growth rate, and profit margins. +Two, I want to look into the beer competitive landscape. +Is this market competitive, and will our client be able to capture meaningful market share? + +[10:57]->[11:27] +To do this, I want to look into questions such as the number of competitors, how much market share each competitor has, +and whether competitors have any competitive advantages. +Three, I want to look into our clients'capabilities. +Do they have the capabilities to succeed in the beer market? + +I want to look into things such as whether they have the expertise to produce beer, +whether they have the distribution channels to sell that beer, and whether there's any existing synergies they can leverage. +4. I want to look into expected profitability. + +[11:27]->[11:58] +Will our client be profitable from entering the beer market? +I'd like to look into areas such as expected revenues, expected costs, and how long it would take to break even. +At this point, the interviewer might ask a few questions on your framework, + +but will otherwise indicate whether they agree or disagree with your approach. +Moving on to the next step of the case interview, starting the case. +There are two different styles of case interviews, interviewer-led cases and candidate-led cases. + +[11:59]->[12:32] +If this is an interviewer-led case, the interviewer will propose which area of your framework they would like to dive deeper into. +They might say something like the following. +Your framework makes sense to me. +Why don't we start by estimating the size of the US beer market? + +If this is a candidate-led case, you will be expected to propose an area to look into. +There is no right or wrong answer as to which area to start first. +So, feel free to propose any area of your framework as long as you have a reason for it. + +[12:32]->[13:05] +You could say something like the following. +To start, I'd like to look into the beer market attractiveness. +I'd like to first understand the market size to determine if the beer market is an attractive market. +If you end up picking an area that the interviewer does not want you to explore, + +they will redirect you to an area that they do want you to explore. +So, these two styles of case interviews are nearly identical. +The only difference is whether or not you have to proactively propose what area to explore first and what area you want to explore + +[13:05]->[13:42] +next. +So, once the case has been kicked off by either you or the interviewer, you'll have to solve quantitative problems and answer qualitative questions, +and in this section, we'll focus on solving quantitative problems first. +There are three different types of quantitative problems you may get asked to solve. + +Market sizing or estimation questions, profit or profitability questions, and charts and graphs questions. +Let's start with the first type, market sizing or estimation questions. +These types of questions ask you to estimate the size of a particular market or calculate a particular figure. + +[13:42]->[14:16] +Let's say that the interviewer asks you the following, what is the market size of beer in the US? +Most candidates jump right into the math, stating the US population and then performing various calculations. +Doing math without laying out a structure often leads to making unnecessary calculations or reaching a dead end. +Instead, lay out an upfront approach to help avoid these mistakes and demonstrate that you are a logical, structured thinker. +For this market sizing problem, you could structure your approach in the following way. + +[14:16]->[14:46] +Start with the US population, estimate the percentage that are legally allowed to drink alcohol, then estimate the percentage that drink beer, +then estimate the frequency in which people drink beer, and then finally estimate the average price per can or bottle of beer. +Multiplying all of these steps together will give you the answer. +By laying out an approach upfront, the interviewer can easily understand how you are thinking about the problem. +With the right structure, the rest of the problem is just simple arithmetic. + +[14:47]->[15:25] +Sometimes the interviewer will give you numbers to use for these calculations, but other times you'll be expected to make assumptions or estimates. +When performing your calculations, make sure to do them on a separate sheet of paper. +Calculations often get messy, and you want to keep your original paper clean and organized. + +A sample answer to this question could sound like the following. +Following my approach, I'll assume the US population is 320 million people. +Assuming the average life expectancy is 80 years old and an even distribution of ages, roughly 75% of the population can legally drink alcohol. + +[15:26]->[16:03] +This gives us 240 million people. +Of these, let's assume 75% of people drink beer. +That gives us 180 million beer drinkers. + +On average, a person drinks perhaps 5 beers a week, or roughly 250 beers per year, if we're assuming roughly 50 weeks per year. +So, that gives us 180 million people times 250 beers per year, or 45 billion cans or bottles of beer. +Assuming the average can or bottle costs$2, this gives us a market size of$90 billion. + +[16:04]->[16:41] +Now that you have your answer, you should not only just answer the question, but tie your answer back to the case objective. +In other words, how does knowing the US market size of beer help you decide whether or not our client should enter the beer +market? +You could say something like the following. + +Given that our client has annual revenues of$30 billion, a$90 billion beer market represents a massive opportunity. +The market size makes the beer market look attractive, +but I'd like to understand if beer margins are typically high and determine how much market share our client could realistically capture. + +[16:42]->[17:15] +The second type of quantitative question you could be asked is to calculate profit or profitability. +To answer profit or profitability questions, you only really need to know the basic formulas for profit. +These are that profit equals revenue minus costs. +Revenue equals quantity times price. + +Costs equal total variable costs plus total fixed costs. +And finally, total variable costs equal quantity times variable cost per unit. +Putting these equations together, we can rewrite them in the following simplified way. + +[17:16]->[17:50] +Profit equals price minus variable costs times quantity minus total fixed costs. +You should also know that profit margin equals revenue minus costs, all divided by revenue. +And you should also know what a break-even point is. + +A break-even point occurs when a company sells enough product such that it has exactly recouped all of its costs. +In other words, break-even occurs when profit equals zero. +To solve for the break-even point, you would simply set profit equal to zero and solve for the unknown variable in the equation. + +[17:51]->[18:24] +As an example of a profit or profitability problem, let's say that the interviewer asks you the following. +Assume that a 12-ounce can of beer sells for$2 on average. +To produce a keg of beer, it costs$100 for raw materials,$95 for labor, and$75 for storage. +If a keg of beer holds 1,800 ounces of beer, what is the profit margin for beer? +As with any other type of quantitative problem, make sure to structure your approach before you begin doing any calculations. + +[18:25]->[18:59] +A sample answer could look like the following. +To calculate the average margin for beer, I will first calculate the total costs to produce a keg of beer. +Next, I will divide the total volume of a keg by the total volume of a can to determine how many cans a keg +of beer produces. + +Afterwards, I will divide the total cost of producing a keg of beer by the number of cans to determine cost per can. +Finally, I can use the price and cost per can of beer to calculate the average margin of beer. +Following this approach, the total cost of a keg of beer is$100 plus$95 plus$75, or$270. + +[19:05]->[19:38] +The number of cans of beer in a keg is 1,800 ounces divided by 12 ounces, or 150 cans. +Therefore, the cost per can of beer is$270 divided by 150 cans, or$1.80. +Since the average price of beer is$2 per can, the profit is$0.20 per can. +This makes the margin$0.20 divided by$2, or a 10% profit margin. +Now that you have your answer, remember to tie your answer back to the overall case objective. + +[19:39]->[20:12] +Compared to our client's overall operating margin of 30%, the beer market profit margin of 10% is significantly lower. +Although the market size for beer is large, the low margin makes the beer market less attractive. +The third type of quantitative question you could get asked is interpreting charts and graphs. +The types of charts and graphs you should be familiar with include bar charts, pie charts, line graphs, bubble charts, and waterfall charts. +Returning to our case example, the interviewer may show you the following. + +[20:13]->[20:44] +A helpful strategy is to start your analysis by explaining what the axes of the chart show. +This will help you understand the chart better. +Don't just read what numbers the chart shows, but instead interpret what those numbers mean for the case objective. + +A sample answer might sound like the following. +For this chart, we have market share on the y-axis and different categories of beer on the x-axis. +For each category, we see that market share is concentrated among a few large players. + +[20:45]->[21:17] +This implies a highly competitive market with high barriers to entry. +Because of this, the beer market does not look attractive because it is so competitive. +In addition to asking quantitative questions, the interviewer will also ask qualitative questions. +There are two types of qualitative questions, brainstorming questions and business opinion questions. +Looking at brainstorming questions first, the interviewer might ask, what are the barriers to entry in the beer market? + +[21:18]->[21:52] +Most candidates answer by listing ideas that immediately come to mind, such as brewing equipment, beer production expertise, distribution channels, or brand name. +This is a highly unstructured way of answering the question. +Make sure to use a simple structure to organize your thoughts. + +A simple structure, such as thinking about barriers to entry as either economic barriers or non-economic barriers, +helps facilitate brainstorming and demonstrates logic and structure. +With this structure, you might come up with the following answer. + +[21:52]->[22:27] +For economic barriers, you have equipment, raw materials, and other capital. +For non-economic barriers, you have beer brewing expertise, brand name, and distribution channels. +Examples of other simple structures that you can use include the following. + +We have internal, external, short-term, long-term, economic, non-economic, quantitative and qualitative, direct and indirect, supply side and demand side, upside and downside, +and benefits and costs. +Additionally, take your answer and connect it back to the case objective. + +[22:28]->[23:00] +In this example, are these barriers to entry high or low in your opinion? +Do you think our client can overcome these obstacles to enter the beer market? +You might answer this question in the following way. +I'm thinking of barriers to entry as economic barriers and non-economic barriers. + +Economic barriers include things such as equipment, raw material, and other capital. +Non-economic barriers include beer brewing expertise, brand name, and distribution channels. +Looking at these barriers, I think it will take our client a lot of work to overcome these barriers. + +[23:01]->[23:34] +While our client does have a brand name and distribution channels, +they lack beer brewing expertise and would have to buy a lot of expensive equipment and machinery. +These barriers make entering the beer market difficult. +The second type of qualitative question are business opinion questions. + +The interviewer may ask you something like the following. +Do you think there are significant production synergies in producing non-alcoholic beverages and producing beer? +As always, structure your answer and connect your answer to the case objective. + +[23:35]->[24:08] +Here is a sample answer. +Production involves equipment, raw materials, and labor. +There is likely some overlap in equipment, such as using the same bottling machines, +but our client will likely need new equipment for brewing beer. + +Raw materials on the other hand are completely different. +Our client will need to source barley, hops, and yeast, which it currently does not use in its existing beverages. +Finally, the same labor can be used, but employees will need new training since producing beer is fairly different from producing a non-alcoholic drink. + +[24:09]->[24:42] +Overall, I think there are only a few production synergies that our client can leverage, which makes entering the market a bit more difficult. +You've done a ton of work, and now it is time to put everything you've done together into a recommendation. +Throughout the interview, you should have been making notes of key takeaways after each question that you answer. + +Take a look at the key takeaways you've accumulated so far and decide whether you want to recommend entering the beer market or not +entering the beer market. +Throughout the case, you may have identified the following key takeaways. + +[24:43]->[25:19] +The US beer market size is$90 billion compared to our client's annual revenue of$30 billion. +The beer market profit margins are 10%, compared to our client's average margin of 30%. +The beer market is highly concentrated across all categories. + +Barriers to entry are moderate, and there are some synergies with existing production. +Remember, there is no right or wrong recommendation, as long as you support your recommendation with reasons and evidence. +Secondly, regardless of what stance you take, make sure you have a firm recommendation. + +[25:19]->[25:52] +You do not want to be flimsy and switch back and forth between recommending entering the market and not entering the market. +Third, make sure your recommendation is clear and concise. +And finally, include next steps in your recommendation. +You can use the following structure when delivering your recommendation. + +First, clearly state what your recommendation is. +Then, follow that with two to three reasons that support your recommendation. +And finally, state what potential next steps would be to further validate your recommendation. + +[25:53]->[26:27] +Next steps can include a number of different things. +You could include buckets in your framework that you were not able to cover. +You could also include open questions that you still have regarding the case. +Finally, you could name potential risks relating to your recommendation that you would like to explore. +Typically, the interviewer will prompt you for a final recommendation. + +They might say something like, let's say that you bump into the CEO in the elevator. +He asks what your preliminary recommendation is. +What would you say? +The conclusion of this case might sound like the following. + +[26:28]->[27:00] +I recommend that our client should not enter the beer market for the following three reasons. +One, although the market size is fairly large at$90 billion, the margins for beer are just 10%, +significantly less than our client's overall operating margin of 30%. + +Two, the beer market is very competitive. +In all beer segments, market share is concentrated among a few players, which implies high barriers to entry. +Our client lacks beer brewing expertise to produce a great product to compete with incumbents. + +[27:01]->[27:34] +Three, there are not that many production synergies that a client can leverage with its existing products. +Our client would need to buy new equipment, source new raw materials, and provide new training to employees, which would be time-consuming and costly. +For next steps, I want to look into our client's annual expected profits if they were to enter the U.S. + +Beer market. +I hypothesize they will be unable to achieve an increase in annual profits of$2 billion within five years, +but I'd like to confirm this through further analysis. + +[27:36]->[28:09] +And with that, we have made our way through an entire case interview. +That's all there is to it. +To recap, all case interviews roughly follow the same format or structure. +The industry you are working with may be different, the quantitative problems may be different, and the qualitative questions may be different, + +but the overall strategy and approach remain the same. +If you follow the fundamental strategies covered in this video and continue practicing case interviews, you should be in great shape for your interview. +We've covered a lot of material in this video, but of course, + +[28:10]->[28:43] +there is many more case interview strategies to learn and cases to be practiced. +This video is a short preview of the full Hacking the Case Interview online course, +which covers each step of the case interview in even more detail. + +Through over 50 video lessons and 20 full-length practice cases, +this 15-25 hour online course is designed to get you a prestigious and lucrative job offer at a top-tier consulting firm. +Second, if you found this video helpful, please take a second to give this video a like to help support the channel and encourage + +[28:43]->[28:59] +more content. +Finally, the only way to get better at case interviews is to practice, +so make sure to do as many mock case interviews as you can. +Check out some of the other videos on my channel where I walk through many full-length mock case interviews. + diff --git a/testdata/reference/learn-case-interviews.vtt b/testdata/reference/learn-case-interviews.vtt new file mode 100644 index 0000000..a671d4a --- /dev/null +++ b/testdata/reference/learn-case-interviews.vtt @@ -0,0 +1,544 @@ +WEBVTT + +1 +00:00:01.000 --> 00:00:32.000 +In this video, I'm going to teach you fundamental strategies and frameworks for tackling case interviews in roughly 30 minutes. +We'll cover each step of the case interview from beginning to end, and go through a full case interview together. +By the end of this video, you will not only fully understand what case interviews are, +but you will learn effective strategies to use on interview day to make you feel more confident in yourself. +We have a lot of material to cover, so please give this video a quick like to support the channel and we'll get started + +2 +00:00:33.000 --> 00:01:03.000 +right away. +So, what is a case interview? +As you may already know, case interviews are a special type of interview question that is commonly used by management consulting companies such as McKinsey, +BCG, and Bain. + +A case interview or case is a 30 to 45 minute exercise in which you and the interviewer will work together to develop a recommendation +to answer or solve a business problem. +These business problems can be anything that real companies face. + +3 +00:01:04.000 --> 00:01:35.000 +A few examples are, how can Coca-Cola increase its profitability? +What should Netflix do to increase customer retention? +How should Apple price its new iPhone? +And where should Disney open its next Disneyland theme park? + +At the beginning of the interview, the interviewer will give you background information and the business problem. +And in the end, you'll present a recommendation and provide two to three reasons that support your recommendation. +These two to three reasons will likely be both quantitative and qualitative. + +4 +00:01:36.000 --> 00:02:06.000 +You may need to make estimations or assumptions, perform calculations, interpret charts and graphs, brainstorm ideas, and have qualitative discussions. +There is usually no single right answer for a case. +As long as you have logical reasons and evidence that support your recommendation, you will do well. +Although there are many business problems you could be asked to solve, almost all case interviews follow the same format or structure. +So, the good news is that you can use the same strategies for every single case interview. + +5 +00:02:07.000 --> 00:02:38.000 +There are seven main parts of a case interview. +Understanding the case background information, asking clarifying questions, structuring a framework, starting the case, solving quantitative problems, answering qualitative questions, and delivering a recommendation. +To best illustrate exactly how a case interview looks like, we'll go through each of these parts step by step. +Let's start with the first part, understanding the case background. +The case interview will begin with the interviewer giving you the case background information. + +6 +00:02:39.000 --> 00:03:13.000 +Let's say that the interviewer reads you the following. +Our client is a large manufacturer and retailer of non-alcoholic beverages such as sodas, juices, sports drinks, and teas. +They have annual revenues of roughly$30 billion and an operating margin of roughly 30%. +Our client is looking to grow and is considering entering the beer market in the United States. + +Should they enter? +As the interviewer reads this, take notes. +It is important to understand what the objective of the case is and keep track of information. + +7 +00:03:14.000 --> 00:03:48.000 +One strategy for taking notes effectively is to turn your paper landscape and draw a vertical line to divide your paper into two sections. +The first section should be roughly two thirds of the page, while the second section will be roughly one third of the page. +Take notes in the second section of your page. + +After the interviewer finishes giving the case background information, confirm that you understand the situation and objective. +Provide a concise synthesis like the following. +To make sure I understand correctly, our client is a large manufacturer and retailer of non-alcoholic beverages. + +8 +00:03:49.000 --> 00:04:20.000 +They are looking to grow, and our objective is to determine whether or not they should enter the U.S. +Beer market. +Make sure your synthesis is concise. +You do not want to regurgitate verbatim everything that the interviewer has said. +Only mention the most important pieces of information. + +You should also make sure that you verify the objective of the case. +Answering or solving the wrong case objective is the quickest way to fail a case interview. +Next, we'll move on to the next step, asking clarifying questions. + +9 +00:04:21.000 --> 00:04:55.000 +Now you'll have the opportunity to ask questions before you begin thinking about how to solve the case. +At this point, only ask questions that are absolutely critical for you to fully understand the case background and objective. +You'll be able to ask more questions later. +There are four types of questions you could ask. + +One, asking for a definition of a term you're unfamiliar with. +Two, asking for information that strengthens your understanding of the company or situation. +Three, asking questions that clarify the objective of the case. + +10 +00:04:55.000 --> 00:05:28.000 +Four, asking to repeat information you may have missed. +So for this case, you might ask the following questions. +Is our client looking to specifically grow revenues or profits? +Answer, our client wants to grow profits. + +Is there a particular financial goal or metric our client is trying to reach within a certain timeframe? +Answer, they're looking to grow annual profits by$5 billion within five years. +Now we'll move on to the third step of the case interview, structuring a framework. + +11 +00:05:29.000 --> 00:06:01.000 +After you understand the case background and objective, lay out a framework of what areas you would want to look into to answer or solve +the case. +A framework is simply a tool that helps you structure and break down complex problems into simpler, smaller components. + +Think of a framework as brainstorming different ideas and organizing them into different categories. +When creating a framework, it is completely acceptable to ask the interviewer for a few minutes of silence to write it out. +You might say something like this. + +12 +00:06:02.000 --> 00:06:34.000 +Would you mind if I take a few minutes to structure my thoughts and develop a framework to tackle this case? +Most of the time, the interviewer will gladly give you a couple of minutes to gather your thoughts. +For this case example, what do you need to know in order to help the client decide whether or not they should enter the beer +market? +You might brainstorm the following questions. + +Does our client know how to produce beer? +Would people buy beer made by our client? +Where would our client sell its beer? +How much would it cost to enter the beer market? + +13 +00:06:34.000 --> 00:07:08.000 +Will our client be profitable from doing this? +How can our client out-compete competitors? +And what is the size of the beer market? +This is not a very structured way of tackling the case. +Instead, you should organize your ideas into a framework that has three to four broad areas, also known as buckets, + +that you want to investigate. +An easy way to develop these buckets is to ask yourself, +what are the three to four things that absolutely must be true for you to 100% recommend that the client should enter the beer + +14 +00:07:08.000 --> 00:07:19.000 +market? +In an ideal world, these four things would need to be true. +1. The beer market is an attractive market with high profit margins. + +15 +00:07:19.000 --> 00:07:19.000 +2. + +16 +00:07:19.000 --> 00:07:25.000 +Competitors are weak and our client will be able to capture significant market share. + +17 +00:07:25.000 --> 00:07:25.000 +3. + +18 +00:07:25.000 --> 00:07:29.000 +Our client has the capabilities to produce an outstanding beer product. + +19 +00:07:31.000 --> 00:07:31.000 +4. + +20 +00:07:31.000 --> 00:08:03.000 +Our client will be extremely profitable. +You can rephrase these points to be the broad categories in your framework. +You can write your framework in the first section of your paper. + +Notice that the four buckets in the framework correspond to the four things that we believe need to be true to be confident in recommending +that our client should enter the beer market. +Next, let's add a few bullet points under each category to give more details on exactly what information we need to know to decide + +21 +00:08:03.000 --> 00:08:34.000 +whether our client should enter the beer market. +Notice that each bullet in a bucket is a question that helps answer the overall question of the bucket. +For example, in the first bucket, knowing the market size, the market growth rate, + +and average profit margins helps us determine whether the beer market is an attractive market. +This entire process of brainstorming ideas and developing a structured framework should only take a few minutes. +So how do you come up with a framework so quickly? + +22 +00:08:34.000 --> 00:09:12.000 +Most candidates make the mistake of either using a single memorized framework for every case, +or memorizing a different framework for every type of case. +The issue with memorized frameworks is that they aren't tailored to the specific case you are solving for. + +When given an atypical business problem, your framework elements will not be entirely relevant. +Interviewers can easily tell that you are regurgitating memorized information and not thinking critically. +So instead of memorizing frameworks, I recommend memorizing a list of eight to 10 broad business elements, such as the following. + +23 +00:09:13.000 --> 00:09:48.000 +Market attractiveness, competitive landscape, company capabilities or attractiveness, customer segments or needs, profitability or financials, strategic alternatives, risks and mitigations, +and creating your own bucket. +When given a case, mentally run through this list and pick the three to four elements that are most relevant to the case. + +This will be your framework. +If the list does not give you enough elements, brainstorm and add your own unique elements to your framework. +This strategy guarantees that your framework elements are relevant to the case. + +24 +00:09:49.000 --> 00:10:21.000 +It also demonstrates that you can create unique tailored frameworks for every business problem or situation. +Using this strategy for this case, you would run through your list of memorized business elements and select the following. +Market attractiveness, competitive landscape, company capabilities, and profitability. +As you can see, this strategy is a shortcut for creating unique tailored frameworks for every business problem. +You do not need to develop a framework entirely from scratch each time. + +25 +00:10:22.000 --> 00:10:56.000 +Now that you have your framework, turn your paper to face the interviewer and walk them through it. +It might sound something like this. +To decide whether or not our client should enter the market, I want to look into four main areas. +One, I want to look into the beer market attractiveness. +Is this an attractive market to enter? + +I'd want to look into areas such as the market size, growth rate, and profit margins. +Two, I want to look into the beer competitive landscape. +Is this market competitive, and will our client be able to capture meaningful market share? + +26 +00:10:57.000 --> 00:11:27.000 +To do this, I want to look into questions such as the number of competitors, how much market share each competitor has, +and whether competitors have any competitive advantages. +Three, I want to look into our clients'capabilities. +Do they have the capabilities to succeed in the beer market? + +I want to look into things such as whether they have the expertise to produce beer, +whether they have the distribution channels to sell that beer, and whether there's any existing synergies they can leverage. +4. I want to look into expected profitability. + +27 +00:11:27.000 --> 00:11:58.000 +Will our client be profitable from entering the beer market? +I'd like to look into areas such as expected revenues, expected costs, and how long it would take to break even. +At this point, the interviewer might ask a few questions on your framework, + +but will otherwise indicate whether they agree or disagree with your approach. +Moving on to the next step of the case interview, starting the case. +There are two different styles of case interviews, interviewer-led cases and candidate-led cases. + +28 +00:11:59.000 --> 00:12:32.000 +If this is an interviewer-led case, the interviewer will propose which area of your framework they would like to dive deeper into. +They might say something like the following. +Your framework makes sense to me. +Why don't we start by estimating the size of the US beer market? + +If this is a candidate-led case, you will be expected to propose an area to look into. +There is no right or wrong answer as to which area to start first. +So, feel free to propose any area of your framework as long as you have a reason for it. + +29 +00:12:32.000 --> 00:13:05.000 +You could say something like the following. +To start, I'd like to look into the beer market attractiveness. +I'd like to first understand the market size to determine if the beer market is an attractive market. +If you end up picking an area that the interviewer does not want you to explore, + +they will redirect you to an area that they do want you to explore. +So, these two styles of case interviews are nearly identical. +The only difference is whether or not you have to proactively propose what area to explore first and what area you want to explore + +30 +00:13:05.000 --> 00:13:42.000 +next. +So, once the case has been kicked off by either you or the interviewer, you'll have to solve quantitative problems and answer qualitative questions, +and in this section, we'll focus on solving quantitative problems first. +There are three different types of quantitative problems you may get asked to solve. + +Market sizing or estimation questions, profit or profitability questions, and charts and graphs questions. +Let's start with the first type, market sizing or estimation questions. +These types of questions ask you to estimate the size of a particular market or calculate a particular figure. + +31 +00:13:42.000 --> 00:14:16.000 +Let's say that the interviewer asks you the following, what is the market size of beer in the US? +Most candidates jump right into the math, stating the US population and then performing various calculations. +Doing math without laying out a structure often leads to making unnecessary calculations or reaching a dead end. +Instead, lay out an upfront approach to help avoid these mistakes and demonstrate that you are a logical, structured thinker. +For this market sizing problem, you could structure your approach in the following way. + +32 +00:14:16.000 --> 00:14:46.000 +Start with the US population, estimate the percentage that are legally allowed to drink alcohol, then estimate the percentage that drink beer, +then estimate the frequency in which people drink beer, and then finally estimate the average price per can or bottle of beer. +Multiplying all of these steps together will give you the answer. +By laying out an approach upfront, the interviewer can easily understand how you are thinking about the problem. +With the right structure, the rest of the problem is just simple arithmetic. + +33 +00:14:47.000 --> 00:15:25.000 +Sometimes the interviewer will give you numbers to use for these calculations, but other times you'll be expected to make assumptions or estimates. +When performing your calculations, make sure to do them on a separate sheet of paper. +Calculations often get messy, and you want to keep your original paper clean and organized. + +A sample answer to this question could sound like the following. +Following my approach, I'll assume the US population is 320 million people. +Assuming the average life expectancy is 80 years old and an even distribution of ages, roughly 75% of the population can legally drink alcohol. + +34 +00:15:26.000 --> 00:16:03.000 +This gives us 240 million people. +Of these, let's assume 75% of people drink beer. +That gives us 180 million beer drinkers. + +On average, a person drinks perhaps 5 beers a week, or roughly 250 beers per year, if we're assuming roughly 50 weeks per year. +So, that gives us 180 million people times 250 beers per year, or 45 billion cans or bottles of beer. +Assuming the average can or bottle costs$2, this gives us a market size of$90 billion. + +35 +00:16:04.000 --> 00:16:41.000 +Now that you have your answer, you should not only just answer the question, but tie your answer back to the case objective. +In other words, how does knowing the US market size of beer help you decide whether or not our client should enter the beer +market? +You could say something like the following. + +Given that our client has annual revenues of$30 billion, a$90 billion beer market represents a massive opportunity. +The market size makes the beer market look attractive, +but I'd like to understand if beer margins are typically high and determine how much market share our client could realistically capture. + +36 +00:16:42.000 --> 00:17:15.000 +The second type of quantitative question you could be asked is to calculate profit or profitability. +To answer profit or profitability questions, you only really need to know the basic formulas for profit. +These are that profit equals revenue minus costs. +Revenue equals quantity times price. + +Costs equal total variable costs plus total fixed costs. +And finally, total variable costs equal quantity times variable cost per unit. +Putting these equations together, we can rewrite them in the following simplified way. + +37 +00:17:16.000 --> 00:17:50.000 +Profit equals price minus variable costs times quantity minus total fixed costs. +You should also know that profit margin equals revenue minus costs, all divided by revenue. +And you should also know what a break-even point is. + +A break-even point occurs when a company sells enough product such that it has exactly recouped all of its costs. +In other words, break-even occurs when profit equals zero. +To solve for the break-even point, you would simply set profit equal to zero and solve for the unknown variable in the equation. + +38 +00:17:51.000 --> 00:18:24.000 +As an example of a profit or profitability problem, let's say that the interviewer asks you the following. +Assume that a 12-ounce can of beer sells for$2 on average. +To produce a keg of beer, it costs$100 for raw materials,$95 for labor, and$75 for storage. +If a keg of beer holds 1,800 ounces of beer, what is the profit margin for beer? +As with any other type of quantitative problem, make sure to structure your approach before you begin doing any calculations. + +39 +00:18:25.000 --> 00:18:59.000 +A sample answer could look like the following. +To calculate the average margin for beer, I will first calculate the total costs to produce a keg of beer. +Next, I will divide the total volume of a keg by the total volume of a can to determine how many cans a keg +of beer produces. + +Afterwards, I will divide the total cost of producing a keg of beer by the number of cans to determine cost per can. +Finally, I can use the price and cost per can of beer to calculate the average margin of beer. +Following this approach, the total cost of a keg of beer is$100 plus$95 plus$75, or$270. + +40 +00:19:05.000 --> 00:19:38.000 +The number of cans of beer in a keg is 1,800 ounces divided by 12 ounces, or 150 cans. +Therefore, the cost per can of beer is$270 divided by 150 cans, or$1.80. +Since the average price of beer is$2 per can, the profit is$0.20 per can. +This makes the margin$0.20 divided by$2, or a 10% profit margin. +Now that you have your answer, remember to tie your answer back to the overall case objective. + +41 +00:19:39.000 --> 00:20:12.000 +Compared to our client's overall operating margin of 30%, the beer market profit margin of 10% is significantly lower. +Although the market size for beer is large, the low margin makes the beer market less attractive. +The third type of quantitative question you could get asked is interpreting charts and graphs. +The types of charts and graphs you should be familiar with include bar charts, pie charts, line graphs, bubble charts, and waterfall charts. +Returning to our case example, the interviewer may show you the following. + +42 +00:20:13.000 --> 00:20:44.000 +A helpful strategy is to start your analysis by explaining what the axes of the chart show. +This will help you understand the chart better. +Don't just read what numbers the chart shows, but instead interpret what those numbers mean for the case objective. + +A sample answer might sound like the following. +For this chart, we have market share on the y-axis and different categories of beer on the x-axis. +For each category, we see that market share is concentrated among a few large players. + +43 +00:20:45.000 --> 00:21:17.000 +This implies a highly competitive market with high barriers to entry. +Because of this, the beer market does not look attractive because it is so competitive. +In addition to asking quantitative questions, the interviewer will also ask qualitative questions. +There are two types of qualitative questions, brainstorming questions and business opinion questions. +Looking at brainstorming questions first, the interviewer might ask, what are the barriers to entry in the beer market? + +44 +00:21:18.000 --> 00:21:52.000 +Most candidates answer by listing ideas that immediately come to mind, such as brewing equipment, beer production expertise, distribution channels, or brand name. +This is a highly unstructured way of answering the question. +Make sure to use a simple structure to organize your thoughts. + +A simple structure, such as thinking about barriers to entry as either economic barriers or non-economic barriers, +helps facilitate brainstorming and demonstrates logic and structure. +With this structure, you might come up with the following answer. + +45 +00:21:52.000 --> 00:22:27.000 +For economic barriers, you have equipment, raw materials, and other capital. +For non-economic barriers, you have beer brewing expertise, brand name, and distribution channels. +Examples of other simple structures that you can use include the following. + +We have internal, external, short-term, long-term, economic, non-economic, quantitative and qualitative, direct and indirect, supply side and demand side, upside and downside, +and benefits and costs. +Additionally, take your answer and connect it back to the case objective. + +46 +00:22:28.000 --> 00:23:00.000 +In this example, are these barriers to entry high or low in your opinion? +Do you think our client can overcome these obstacles to enter the beer market? +You might answer this question in the following way. +I'm thinking of barriers to entry as economic barriers and non-economic barriers. + +Economic barriers include things such as equipment, raw material, and other capital. +Non-economic barriers include beer brewing expertise, brand name, and distribution channels. +Looking at these barriers, I think it will take our client a lot of work to overcome these barriers. + +47 +00:23:01.000 --> 00:23:34.000 +While our client does have a brand name and distribution channels, +they lack beer brewing expertise and would have to buy a lot of expensive equipment and machinery. +These barriers make entering the beer market difficult. +The second type of qualitative question are business opinion questions. + +The interviewer may ask you something like the following. +Do you think there are significant production synergies in producing non-alcoholic beverages and producing beer? +As always, structure your answer and connect your answer to the case objective. + +48 +00:23:35.000 --> 00:24:08.000 +Here is a sample answer. +Production involves equipment, raw materials, and labor. +There is likely some overlap in equipment, such as using the same bottling machines, +but our client will likely need new equipment for brewing beer. + +Raw materials on the other hand are completely different. +Our client will need to source barley, hops, and yeast, which it currently does not use in its existing beverages. +Finally, the same labor can be used, but employees will need new training since producing beer is fairly different from producing a non-alcoholic drink. + +49 +00:24:09.000 --> 00:24:42.000 +Overall, I think there are only a few production synergies that our client can leverage, which makes entering the market a bit more difficult. +You've done a ton of work, and now it is time to put everything you've done together into a recommendation. +Throughout the interview, you should have been making notes of key takeaways after each question that you answer. + +Take a look at the key takeaways you've accumulated so far and decide whether you want to recommend entering the beer market or not +entering the beer market. +Throughout the case, you may have identified the following key takeaways. + +50 +00:24:43.000 --> 00:25:19.000 +The US beer market size is$90 billion compared to our client's annual revenue of$30 billion. +The beer market profit margins are 10%, compared to our client's average margin of 30%. +The beer market is highly concentrated across all categories. + +Barriers to entry are moderate, and there are some synergies with existing production. +Remember, there is no right or wrong recommendation, as long as you support your recommendation with reasons and evidence. +Secondly, regardless of what stance you take, make sure you have a firm recommendation. + +51 +00:25:19.000 --> 00:25:52.000 +You do not want to be flimsy and switch back and forth between recommending entering the market and not entering the market. +Third, make sure your recommendation is clear and concise. +And finally, include next steps in your recommendation. +You can use the following structure when delivering your recommendation. + +First, clearly state what your recommendation is. +Then, follow that with two to three reasons that support your recommendation. +And finally, state what potential next steps would be to further validate your recommendation. + +52 +00:25:53.000 --> 00:26:27.000 +Next steps can include a number of different things. +You could include buckets in your framework that you were not able to cover. +You could also include open questions that you still have regarding the case. +Finally, you could name potential risks relating to your recommendation that you would like to explore. +Typically, the interviewer will prompt you for a final recommendation. + +They might say something like, let's say that you bump into the CEO in the elevator. +He asks what your preliminary recommendation is. +What would you say? +The conclusion of this case might sound like the following. + +53 +00:26:28.000 --> 00:27:00.000 +I recommend that our client should not enter the beer market for the following three reasons. +One, although the market size is fairly large at$90 billion, the margins for beer are just 10%, +significantly less than our client's overall operating margin of 30%. + +Two, the beer market is very competitive. +In all beer segments, market share is concentrated among a few players, which implies high barriers to entry. +Our client lacks beer brewing expertise to produce a great product to compete with incumbents. + +54 +00:27:01.000 --> 00:27:34.000 +Three, there are not that many production synergies that a client can leverage with its existing products. +Our client would need to buy new equipment, source new raw materials, and provide new training to employees, which would be time-consuming and costly. +For next steps, I want to look into our client's annual expected profits if they were to enter the U.S. + +Beer market. +I hypothesize they will be unable to achieve an increase in annual profits of$2 billion within five years, +but I'd like to confirm this through further analysis. + +55 +00:27:36.000 --> 00:28:09.000 +And with that, we have made our way through an entire case interview. +That's all there is to it. +To recap, all case interviews roughly follow the same format or structure. +The industry you are working with may be different, the quantitative problems may be different, and the qualitative questions may be different, + +but the overall strategy and approach remain the same. +If you follow the fundamental strategies covered in this video and continue practicing case interviews, you should be in great shape for your interview. +We've covered a lot of material in this video, but of course, + +56 +00:28:10.000 --> 00:28:43.000 +there is many more case interview strategies to learn and cases to be practiced. +This video is a short preview of the full Hacking the Case Interview online course, +which covers each step of the case interview in even more detail. + +Through over 50 video lessons and 20 full-length practice cases, +this 15-25 hour online course is designed to get you a prestigious and lucrative job offer at a top-tier consulting firm. +Second, if you found this video helpful, please take a second to give this video a like to help support the channel and encourage + +57 +00:28:43.000 --> 00:28:59.000 +more content. +Finally, the only way to get better at case interviews is to practice, +so make sure to do as many mock case interviews as you can. +Check out some of the other videos on my channel where I walk through many full-length mock case interviews. +