Catalog of the runnable scripts in this repository. All paths are relative to the repository root:
scripts/— single-run PBS job wrappers (Metacentrum),multidetector/— the per-detector experiment pipeline (recommended way to reproduce the paper),analysis/— offline CPU analyses; they operate on saved.npzprofiles and never re-run the model, so they can also be executed directly withpython3.
PBS wrappers contain site-specific storage paths; override them via the
environment variables documented in each script (PROJECT_DIR, STORAGE_BASE,
REF_DIR, ...).
generate_drift_reference.py
(wrappers: scripts/run_generate_drift.sh, multidetector/md_reference.sh)
Runs the detector over its own training split and saves the reference profiles used by the monitor:
reference_bonafide.npz— bona fide training samples,reference_spoof.npz— spoof training samples (known attacks),reference_all.npz— combined (legacy, not used by the monitor decision).
Each .npz stores embs, file_names, mean, precision, mode. The
training set (ASVspoof 2019 LA / ASVspoof 5) follows the dataset argument; the
output directory can be namespaced per detector via the DRIFT_PROFILES_DIR
environment variable.
eval.py
(wrappers: scripts/run_eval.sh — parametric, qsub -v DATASET=...;
multidetector/md_eval.sh)
Runs the detector over one evaluation dataset and saves everything needed for
offline analysis: eval_profile_*.npz (embeddings + logits + labels +
file_names), summary.json, per-attack CSV, and t-SNE/UMAP visualisations.
| Script | Purpose | PBS wrapper |
|---|---|---|
analysis/per_cluster_classification.py |
Two-reference Mahalanobis categorization: distances (d_b, d_s) to the bona fide and spoof references → 5 categories (known_bonafide, known_attack, borderline, novelty_bonafide_like, novelty_spoof_like). Options: --cov_reg (none / ledoitwolf / ridge:c / shrink:a / pca:k; the paper uses shrink:0.9), --standardize, --threshold_method (percentile / parametric tail fits), --percentile (default 95), --export_csv (per-sample distance dump for the overlay figure). |
scripts/run_per_cluster_classification.sh, multidetector/md_analysis.sh |
analysis/drift_detector_simulator.py |
Deployment simulation on top of the per-cluster results: TRUSTED vs ALERT zones and the operational outcomes (correctly trusted / alerted / wrongly trusted). | multidetector/md_analysis.sh |
analysis/detector_baseline_fnr.py |
Detector-only error rates without the monitor — the "before monitoring" side of the false-acceptance comparison. | multidetector/md_baseline_fnr.sh |
analysis/analyze_covariance_conditioning.py |
Covariance conditioning diagnostics per reference (effective rank, condition numbers) — motivates the regularization. | multidetector/md_cov_diag.sh |
analysis/cov_regularization_sweep.py |
Grid sweep of covariance regularization × threshold method → one CSV; the covariance regularization ablation. | multidetector/md_cov_sweep.sh |
analysis/plot_monitor_overlay.py |
Monitor-region figure: category zones at the real thresholds with real (d_b, d_s) samples overlaid. Input = the CSVs from per_cluster_classification.py --export_csv; also runs locally (needs matplotlib + scipy). |
multidetector/md_analysis.sh |
Shared library: cov_utils.py — RefModel (covariance estimators) and
fit_threshold (percentile and parametric self-distance tail fits). It is used
by the per-cluster categorization, the bootstrap CIs, and the sweep, so all of
them score with an identical metric.
multidetector/drive_detector.sh <name> submits the whole dependency chain on
PBS: reference build → evals on the three datasets → analysis job (per-cluster
categorization, overlay figures, per-attack EER, bootstrap CIs, deployment
simulator), all namespaced by the detector name from detectors.tsv. Metric
variants are selected with COV_REG / STANDARDIZE / THRESHOLD_METHOD
environment variables. See multidetector/README.md.
These supported the early single-detector exploration and additional
diagnostics; they are not part of the paper's final pipeline but are kept
for reference. Their PBS wrappers read the eval-profile paths from
scripts/eval_profiles.env (a small path registry — update it after re-running
evals) and accept a REF_DIR override to select the reference directory.
| Script | Purpose | PBS wrapper |
|---|---|---|
analysis/analyze_drift.py |
Re-analyze one saved eval profile against the reference modes (incl. the legacy all mode): aggregate metrics + t-SNE/UMAP overlays. |
scripts/run_analyse_drift.sh |
analysis/analyze_bonafide_drift.py |
Bona fide cluster shift across datasets (Wasserstein + Mahalanobis vs the bona fide reference). | scripts/run_bonafide_drift.sh |
analysis/threshold_analysis.py |
Single-reference alarm thresholds (95th/99th percentile of training self-distances), per-dataset TPR/FPR, ROC curves. | scripts/run_threshold_analysis.sh |
analysis/per_attack_eer_analysis.py |
Per-attack EER vs per-attack Wasserstein drift, with correlation statistics (also run as a supplementary stage by md_analysis.sh). |
scripts/run_per_attack_eer.sh |
analysis/bootstrap_ci_analysis.py |
Bootstrap 95 % confidence intervals for AUROC / Mahalanobis / Wasserstein (also run as a supplementary stage by md_analysis.sh). |
scripts/run_bootstrap_ci.sh |
analysis/multivariate_wasserstein.py |
Marginal Wasserstein vs Sliced Wasserstein Distance — methodological check that marginals are sufficient. | scripts/run_multivariate_wasserstein.sh |
extract_asvspoof5_train_bonafide.py |
GPU helper: bona fide embeddings from the ASVspoof 5 train set, input for the combined reference below. | scripts/run_extract_asvspoof5_train_bonafide.sh |
analysis/combine_references.py |
Combined 2019 + ASVspoof 5 bona fide ("domain-robust") reference experiment. | scripts/run_combine_references.sh |
analysis/compare_reference_results.py |
Side-by-side metric comparison of two reference setups from their bootstrap-CI outputs (runs locally, seconds). | — |
train_and_eval.py and finetune.py train / fine-tune the detectors
themselves and come from the original detector codebase — see
docs/ORIGINAL_README.md for usage.
| File | Role |
|---|---|
common.py |
Model/dataset registry, build_model(), get_dataloaders(), get_drift_dataloaders() |
config.py |
Dataset paths and batch sizes per environment (local / metacentrum / sge) |
drift_config.py |
Metadata file paths, columns, separators, known attack IDs |
cov_utils.py |
Covariance regularization (RefModel) + threshold fitting (fit_threshold) |
parse_arguments.py |
Shared CLI argument parser |
trainers/DriftMonitor.py |
Drift monitoring layer: per-sample metrics, per-attack analysis, visualisations |
trainers/BaseFFTrainer.py |
Eval loop with drift integration; generate_drift_reference() |
trainers/FFTrainer.py |
val_epoch() — returns embeddings + logits + file_names |
extractors/, feature_processors/, classifiers/ |
Model components (XLS-R, MHFA/AASIST, FF, ...) |
datasets/ |
Per-dataset loaders (ASVspoof 2019/2021/5, In-the-Wild, ...) |
augmentation/ |
Data augmentations (not used in the drift pipeline) |