SENS-ASR

Overview

This repository contains the codebase for the training and evaluation experiments used in the paper**"SENS-ASR: Semantic Embedding injection in Neural-transducer for Streaming Automatic Speech Recognition"**.

What is in this folder

• train_with_context_emb.py — train/inference main script.
• librispeech_prepare.py, tedlium2_prepare.py — dataset preparation helpers.
• requirements.txt — Python dependencies for running the code in this folder.
• decoders/ — decoder implementations (for example transducer.py).
• utils/ — utilities for training/evaluation (e.g. logger.py, metric helpers, loss functions).

Quick start

1. Install necessary apt packages:

sudo apt-get install ffmpeg python3.10 pip

2. Create and activate a virtual environment:

python -m venv .venv
source .venv/bin/activate

3. Install dependencies from the local requirements.txt:

pip install --upgrade pip
pip install -r requirements.txt

Basic usage

• Inspect and adapt the example hyperparameters in hparams/.
• Run training (adjust arguments as needed):
  • using Dynamic Chunk Training (DCT):

    python train_with_context_emb.py hparams/context_emb_conformer.yaml

  • Training with Targeted Chunk Size (e.g. 320ms):

    python train_with_context_emb.py hparams/context_emb_conformer.yaml --cs_min=8 --cs_max=8

• Run inference (example with LibriSpeech using 1280ms chunk size):

  python train_with_context_emb.py hparams/context_emb_conformer.yaml --cs_test=32 --lc_test=2 --test_only

Notes

• requirements.txt is intended as the canonical dependency list for reproducibility -> pin versions if you need stable runs.
• Models and decoders are implemented in models/ and decoders/. Use hparams/ to configure training settings.

Contributing

• Open an issue for questions or proposed changes.
• Submit PRs with clear descriptions and reproduction steps.

License

• MIT