feat: Audio file ingestion pipeline (.mp3, .wav, .ogg, .flac)

[aj1126]

Summary

Add a unified audio ingestion pipeline that handles both standalone audio files and audio tracks extracted from video files. Currently the video pipeline in worker.js runs Whisper transcription on video audio, but it's tightly coupled to the video processing path. This issue proposes a dedicated, reusable audio ingestion module that serves as the single entry point for all audio processing — whether the source is a .mp3 file or the audio track demuxed from a .mp4.

Motivation

Many UAP/UAV datasets include standalone audio recordings — cockpit comms, ATC radio intercepts, field observer narrations, and ambient sensor captures. These files currently get silently skipped by walkFiles() in file-ingestion.js, losing potentially critical data.

Additionally, the existing video pipeline already extracts audio via ffmpeg for Whisper transcription, but this logic is embedded directly in the video processing path. A unified audio module would:

• Eliminate duplication — one transcription pipeline for all audio sources
• Enable richer audio analysis — speaker diarization, ambient noise classification, signal anomaly detection
• Improve maintainability — video ingestion delegates to the audio module instead of inlining Whisper calls

Proposed Scope

Supported Standalone Formats

• .mp3, .wav, .ogg, .flac, .m4a, .aac

Video Audio Extraction Integration

• The existing video pipeline (worker.js / video_ingestion.py) should delegate its audio processing to this module instead of running Whisper inline.
• When processing a video file, ffmpeg demuxes the audio track → the unified audio module handles transcription and metadata extraction → results are merged back into the video's SIR entry.
• This creates a single code path: video → demux audio track → audio ingestion module → transcription + metadata.

Core Audio Ingestion Pipeline

1. Audio Detection — Extend the SUPPORTED_EXTENSIONS set in src/ingestion/file-ingestion.js to include audio formats.
2. Audio Extraction from Video — Refactor video_ingestion.py to expose audio extraction as a callable sub-step that feeds into the shared audio pipeline.
3. Transcription — Route all audio (standalone files + demuxed video tracks) through Whisper via a dedicated audio_ingestion.py script or an expanded video_ingestion.py.
4. Metadata Extraction — Extract audio metadata (duration, sample rate, channels, bitrate, codec) and attach to the SIR schema.
5. NLP Pass — Feed the transcribed text through the existing NLP entity extraction pipeline (extractDates, extractLocations, term frequency) so audio content participates in all four analytics tiers.

Worker Thread Integration

• Process standalone audio files through the existing worker_threads pool, following the same deferred-termination patterns established in AGENTS.md.
• Leverage the existing Python offloading pattern from scripts/video_ingestion.py for Whisper transcription.
• Video processing calls into the same audio sub-pipeline, avoiding duplicate Whisper codepaths.

Output Schema

Standalone audio files:

{
  "name": "recording_001.mp3",
  "content": "<transcribed text>",
  "metadata": {
    "type": "audio",
    "duration_seconds": 142,
    "sample_rate": 44100,
    "channels": 2,
    "codec": "mp3",
    "bitrate_kbps": 320
  },
  "dates": [...],
  "locations": [...]
}

Video files (audio portion merged into existing video SIR):

{
  "name": "DOD_111689133.mp4",
  "content": "<transcribed audio text>",
  "metadata": {
    "type": "video",
    "duration_seconds": 34,
    "audio": {
      "sample_rate": 48000,
      "channels": 2,
      "codec": "aac",
      "has_speech": true
    }
  },
  "dates": [...],
  "locations": [...]
}

Acceptance Criteria

• Standalone audio files (.mp3, .wav, .ogg, .flac, .m4a, .aac) are discovered by walkFiles() recursive scanner
• Whisper transcription produces text output for downstream NLP on standalone audio
• Video pipeline delegates audio processing to the shared audio ingestion module
• Audio metadata (duration, sample rate, codec, channels) is preserved in the SIR schema for both standalone audio and video audio tracks
• Single Whisper transcription codepath for both standalone audio and video-extracted audio
• Existing test suite passes with no regressions
• New test cases cover standalone audio ingestion path
• New test cases verify video→audio delegation path
• docs/architecture.md and docs/USER_GUIDE.md updated to list audio formats and describe the unified audio pipeline

Related Files

• src/ingestion/file-ingestion.js — file discovery and worker dispatch
• src/ingestion/worker.js — per-file processing logic (video audio currently inlined here)
• scripts/video_ingestion.py — existing Python Whisper/OCR offloading (audio extraction to be refactored)
• src/pipeline.js — analytics pipeline orchestrator
• docs/ROADMAP.md — feature tracking

Relationship to #20

This issue provides the audio foundation that #20 (Advanced Video Content Analysis) builds on. Once audio is handled by a dedicated module, #20 can focus purely on the visual analysis layer without duplicating audio concerns.