voxcraft

Local YouTube transcript-prep pipeline for Apple Silicon Macs.

The workflow is:

prefer creator-provided subtitles
never use auto-captions
otherwise download audio and transcribe locally
optionally diarize speakers
chunk the transcript
optionally summarize chunks and the full video through a supported agent CLI

Outputs are written per video under data/videos/<upload-date>--<title-slug>--<youtube_id>/ when YouTube exposes an upload date. Older undated workspaces are still reused by video ID.

For the implementation-level overview, see docs/architecture.md.

Status

The current default stack is:

ASR backend: qwen3-asr
Qwen model: mlx-community/Qwen3-ASR-1.7B-8bit
Forced aligner: Qwen/Qwen3-ForcedAligner-0.6B
Diarization model: pyannote/speaker-diarization-community-1
Fallback backend: whisper.cpp
Default summary provider: codex
Default Codex summary model: gpt-5.5
Default Codex thinking level: high

The default Qwen path is invoked through the repo-owned voxcraft-qwen wrapper. That wrapper patches the upstream loader so the hybrid mlx-community/Qwen3-ASR-1.7B-8bit checkpoint works in a fresh environment without hand-editing the venv.

Known Limitations

The default ASR path is Apple Silicon oriented because it depends on MLX.
Summarization is not local. It requires an authenticated supported summary CLI.
Diarization requires Hugging Face access to the gated pyannote model.
Long Qwen runs currently use --quiet --no-progress, so they can appear idle while they are still computing.

Requirements

Supported target:

Apple Silicon macOS
Python 3.11

Required external tool:

ffmpeg

Optional external tools:

codex Optional summary CLI
claude Optional summary CLI
gemini Optional summary CLI
pi Optional summary CLI
whisper-cli Required only for --asr-backend whisper-cpp

Server mode setting:

VOXCRAFT_SERVER_TOKEN Required API token for voxcraft server

Python dependencies are installed with uv sync from pyproject.toml.

Install

Install the system prerequisites:

brew install python@3.11 uv ffmpeg

If you want the fallback backend too:

brew install whisper-cpp

Create the environment and install the project:

uv sync --group dev --python 3.11
source .venv/bin/activate
voxcraft doctor

Auth And Gating

There are two important gated integrations:

Pyannote diarization The diarization model runs locally, but its weights are hosted on Hugging Face and are access-gated.

Before using --diarize:
- accept the model terms for pyannote/speaker-diarization-community-1
- export a token
Example:
```
export PYANNOTE_AUTH_TOKEN=hf_...
```
The code also accepts:
- HF_TOKEN
- HUGGINGFACE_TOKEN
Summary CLI authentication The transcript pipeline is local. Summarization is not.

voxcraft summarize ... and voxcraft process ... --summarize require:
- one supported summary CLI on PATH: codex, claude, gemini, or pi
- an authenticated session for that CLI

Environment Variables

Supported runtime environment variables:

VOXCRAFT_CONFIG Override the runtime config.json path
PYANNOTE_AUTH_TOKEN Hugging Face token for gated pyannote diarization models
HF_TOKEN Alternate token name accepted by the diarization path
HUGGINGFACE_TOKEN Alternate token name accepted by the diarization path
WHISPER_CPP_MODEL Exact whisper.cpp model file path
WHISPER_CPP_MODEL_DIR Directory containing whisper.cpp model files
VOXCRAFT_SERVER_TOKEN API token required by voxcraft server
VOXCRAFT_SERVER_URL Default base URL for remote job client commands such as submit-job and check-job

The CLI does not load a .env file by itself. These variables must already be present in the shell environment.

Runtime Config

The CLI now treats a JSON config as the primary source of runtime defaults.

Lookup order:

voxcraft --config /path/to/config.json ...
$VOXCRAFT_CONFIG
~/.config/voxcraft/config.json

CLI flags still work, but they are now per-run overrides on top of the loaded config.

Example config:

{
  "base_data_dir": "data/videos",
  "language_preference": "en",
  "default_asr_backend": "qwen3-asr",
  "summary_provider": "pi",
  "summary_profiles": {
    "codex": {
      "command": "codex",
      "model": "gpt-5.5",
      "thinking_level": "high"
    },
    "claude": {
      "command": "claude",
      "model": "claude-sonnet-4-5"
    },
    "gemini": {
      "command": "gemini",
      "model": "gemini-2.5-pro"
    },
    "pi": {
      "command": "pi",
      "model": "openai/gpt-5.5",
      "thinking_level": "high"
    }
  }
}

An example file is included at config.example.json.

Quick Start

Probe a video without downloading media:

voxcraft process "https://www.youtube.com/watch?v=..." --dry-run

Run the normal local pipeline:

voxcraft process "https://www.youtube.com/watch?v=..."

Run the pipeline and summarize:

voxcraft process "https://www.youtube.com/watch?v=..." --summarize

Use a specific config file for the whole run:

voxcraft --config ./config.json process "https://www.youtube.com/watch?v=..." --summarize

Temporarily override the configured summary CLI:

voxcraft process "https://www.youtube.com/watch?v=..." \
  --summarize \
  --summary-provider claude

Enable diarization for multi-speaker audio:

voxcraft process "https://www.youtube.com/watch?v=..." --diarize --num-speakers 2

Use the safe fallback backend:

voxcraft process "https://www.youtube.com/watch?v=..." \
  --asr-backend whisper-cpp \
  --whisper-cpp-model ./models/ggml-large-v3.bin

Summarize an already-processed video:

voxcraft summarize <youtube_id>

Summarize with a specific config file:

voxcraft --config ./config.json summarize <youtube_id>

Override the configured Pi model for one run:

voxcraft summarize <youtube_id> \
  --provider pi \
  --model openai/gpt-5.5 \
  --thinking-level high

Commands

Available commands:

voxcraft doctor
voxcraft --config ./config.json doctor
voxcraft process "<youtube-url>" --dry-run
voxcraft process "<youtube-url>"
voxcraft process "<youtube-url>" --summarize
voxcraft --config ./config.json process "<youtube-url>" --summarize
voxcraft process "<youtube-url>" --summarize --summary-provider claude
voxcraft process "<youtube-url>" --summarize --summary-provider pi --summary-model openai/gpt-5.5 --summary-thinking-level high
voxcraft process "<youtube-url>" --diarize
voxcraft summarize <youtube_id>
voxcraft --config ./config.json summarize <youtube_id>
voxcraft summarize <youtube_id> --provider gemini
voxcraft summarize <youtube_id> --provider pi --model openai/gpt-5.5 --thinking-level high
voxcraft rechunk <youtube_id>
VOXCRAFT_SERVER_TOKEN=... voxcraft server --host 127.0.0.1 --port 8765
VOXCRAFT_SERVER_URL=http://<host>:8765 VOXCRAFT_SERVER_TOKEN=... voxcraft submit-job "<youtube-url>" --wait 300 --print-final
voxcraft check-job <job_id>
voxcraft fetch-final <job_id>
voxcraft fetch-final <job_id> --output ./final.md
voxcraft fetch-log <job_id>

Command behavior:

doctor Shows environment status, installed Python packages, auth-related readiness, and the loaded config path
process --dry-run Probes metadata and prints the planned source path without downloading media
process Runs the real subtitle or ASR pipeline
summarize Reuses existing chunk artifacts and generates chunk summaries plus final.md --summary-provider / --provider, --summary-model / --model, and --summary-thinking-level / --thinking-level are per-run overrides on top of the loaded config
rechunk Regenerates chunk files from existing transcript segments Rebuilds the summary payload from existing transcript artifacts
server Runs the authenticated async job API for long-running URL-to-final.md jobs
submit-job, check-job, latest-job, fetch-final, fetch-log Talk to a remote voxcraft server using $VOXCRAFT_SERVER_URL and $VOXCRAFT_SERVER_TOKEN

Server Mode

voxcraft server runs an authenticated async job API for long-running YouTube summary jobs. It uses the same pipeline and writes the same data/videos/.../final.md artifacts as the CLI.

Start it on the Mac mini:

export VOXCRAFT_SERVER_TOKEN="choose-a-long-random-token"
voxcraft server --host 127.0.0.1 --port 8765

For LAN or Tailscale access, bind to the specific LAN/Tailscale IP instead of exposing it publicly:

voxcraft server --host <lan-or-tailscale-ip> --port 8765

Submit one URL:

curl -sS -X POST http://<host>:8765/jobs \
  -H "Authorization: Bearer $VOXCRAFT_SERVER_TOKEN" \
  -H "Content-Type: application/json" \
  -d '{"url":"https://www.youtube.com/watch?v=..."}'

Check status and fetch the final markdown:

curl -sS http://<host>:8765/jobs/<job_id> \
  -H "Authorization: Bearer $VOXCRAFT_SERVER_TOKEN"

curl -sS http://<host>:8765/jobs/<job_id>/final.md \
  -H "Authorization: Bearer $VOXCRAFT_SERVER_TOKEN"

The MVP worker processes one job at a time. Job state is stored in SQLite at data/server/jobs.sqlite3 by default, or at --jobs-db <path> when provided. On startup, the server reconciles jobs that were running during a restart: if the workspace already has final.md, the job is marked done; otherwise the job is marked failed and any discovered logs/pipeline.log path is preserved for debugging.

The CLI can also act as a client for the server. Configure the server URL and token:

export VOXCRAFT_SERVER_URL="http://<host>:8765"
export VOXCRAFT_SERVER_TOKEN="choose-a-long-random-token"

Submit a job and wait up to five minutes for short videos:

voxcraft submit-job "https://www.youtube.com/watch?v=..." --wait 300

If the job is already done within the wait window, print the final markdown directly:

voxcraft submit-job "https://www.youtube.com/watch?v=..." --wait 300 --print-final

Check or fetch later:

voxcraft check-job <job_id>
voxcraft check-job <job_id> --wait 300 --print-final
voxcraft check-job <job_id> --wait 300 --output ./final.md
voxcraft latest-job
voxcraft fetch-final <job_id>
voxcraft fetch-final <job_id> --output ./final.md
voxcraft fetch-log <job_id>

Workflow

At a high level:

YouTube URL
  -> yt-dlp metadata probe
  -> creator subtitles?
     -> yes: download + parse subtitles
     -> no: download source audio -> ffmpeg -> audio.wav -> local ASR
  -> optional pyannote diarization
  -> transcript cleanup
  -> transcript chunks
  -> summary_input/payload.json
  -> optional summary CLI chunk summaries
  -> optional summary CLI final summary

Important policy decisions:

creator-provided subtitles are used when available
auto-captions are always ignored
local ASR consumes the normalized source/audio.wav
diarization is supported only on the Qwen path

Output Layout

Each processed video gets a folder like:

data/videos/<upload-date>--<title-slug>--<youtube_id>/

The upload date uses YYYY-MM-DD, so normal filename sorting groups videos chronologically. If a date is unavailable, the folder falls back to <title-slug>--<youtube_id>/.

Common files:

metadata.json Compact video metadata and subtitle language lists
source/info.json Full yt-dlp probe output
source/audio.<ext>
source/audio.wav
transcript/raw.txt
transcript/clean.txt
transcript/segments.json
transcript/speaker_segments.json
transcript/asr_output.json
transcript/transcript.srt
chunks/chunk-*.txt
chunks/index.json
summary_input/payload.json
summary/manifest.json
final.md
logs/pipeline.log

Sample Output

Example workspace after a successful run:

data/videos/2026-06-11--how-hardware-makes-threads-less-of-a-nightmare--IMceN4_rieo/
├── metadata.json
├── final.md
├── source/
│   ├── info.json
│   ├── audio.webm
│   └── audio.wav
├── transcript/
│   ├── raw.txt
│   ├── clean.txt
│   ├── segments.json
│   ├── asr_output.json
│   └── transcript.srt
├── chunks/
│   ├── chunk-001.txt
│   ├── chunk-002.txt
│   └── index.json
├── summary_input/
│   └── payload.json
├── summary/
│   ├── chunk-001.md
│   ├── chunk-002.md
│   ├── manifest.json
│   └── prompts/
└── logs/
    └── pipeline.log

Typical process result fields shown by the CLI:

video_id: IMceN4_rieo
source_kind: local-asr
subtitle_policy: creator-only
cached: False
chunk_count: 2
transcription_backend: qwen3-asr
transcription_model: mlx-community/Qwen3-ASR-1.7B-8bit
language: en
diarized: False

Typical final outputs to inspect:

final.md
summary/manifest.json
summary_input/payload.json
transcript/segments.json
logs/pipeline.log

Reproducibility Notes

Caching is per video workspace and is not purely keyed by URL. The pipeline checks the requested processing mode before reusing artifacts.

For local ASR runs, cache reuse depends on:

backend
model
language
diarization enabled or disabled
fixed speaker count, if explicitly requested

Use --force to recompute.

Operational Caveats

The primary ASR path is Apple Silicon oriented because it depends on MLX.
First runs can be slow because model weights may need to be downloaded from Hugging Face.
Long Qwen runs currently use --quiet --no-progress, so they can look idle even when they are still computing.
whisper.cpp is the safer fallback path if the Qwen stack is unavailable or unstable on a given machine.
Summary provider, command, model, and thinking level can all be set in config.json; CLI flags override only the selected provider for the current run.
The loaded runtime config comes from --config, then $VOXCRAFT_CONFIG, then ~/.config/voxcraft/config.json.
--summary-thinking-level and --thinking-level are currently applied to providers that expose a comparable headless flag: codex and pi.
The python3.11 and yt-dlp shell commands are convenient but not strictly required after the environment is built, because the runtime uses the active interpreter and the installed yt_dlp Python package.

Code Map

Main modules:

Tests

Run:

.venv/bin/pytest -q

The current repo test suite covers the main planning and artifact-generation paths, including the repo-owned Qwen wrapper patch logic.

License

This project is released under the MIT License. See LICENSE.

Name		Name	Last commit message	Last commit date
Latest commit History 75 Commits
.agents/skills/voxcraft		.agents/skills/voxcraft
data		data
docs		docs
src/voxcraft		src/voxcraft
tests		tests
.gitignore		.gitignore
LICENSE		LICENSE
README.md		README.md
config.example.json		config.example.json
pyproject.toml		pyproject.toml
uv.lock		uv.lock

Navigation Menu

Repository files navigation

voxcraft

Status

Known Limitations

Requirements

Install

Auth And Gating

Environment Variables

Runtime Config

Quick Start

Commands

Server Mode

Workflow

Output Layout

Sample Output

Reproducibility Notes

Operational Caveats

Code Map

Tests

License

About

Releases

Packages

Contributors

Languages

Folders and files

Latest commit

History

Repository files navigation

voxcraft

Status

Known Limitations

Requirements

Install

Auth And Gating

Environment Variables

Runtime Config

Quick Start

Commands

Server Mode

Workflow

Output Layout

Sample Output

Reproducibility Notes

Operational Caveats

Code Map

Tests

License

About

Resources

Stars

Watchers

Forks

Releases

Packages

Contributors

Languages