Feature/ab#2096 integrate rf detr

[edblu1]

AB#2096 — Integrate RF-DETR backend (multi-backend training pipeline)

This PR introduces a config-driven multi-backend training pipeline that supports both Ultralytics YOLO and RF-DETR, while keeping the existing workflow stable: DVC stages, baseline comparison, unified metrics export, and side-by-side visualizations.

It also promotes a new "current best" baseline checkpoint for YOLO11m trained on the mapped classes: waste, cigarette, leaves_dense, and leaves_sparse.

Why

• We want to include RF-DETR because it outperforms YOLO on many benchmarks and is supposed to also run on edge devices.
• RF-DETR has an Apache 2.0 license.
• We want to train/evaluate RF-DETR on the same datasets and get the same outputs we rely on today (metrics, plots, comparisons, baseline promotion).

What changed

1) Config-driven backend selection (params.yaml)

• train.model selects a model key from models.<key>
• models.<key>.backend determines the backend (yolo or rfdetr)

This establishes an extensible pattern: add a new model family by adding a new backend module + adapter while keeping evaluation/reporting stable.

2) Pipeline modularization (backend separation)

• yolov8_training/train_pipeline.py is now primarily:
  • stage orchestration (prepare + train)
  • config resolution
  • shared evaluation/finalize (_evaluate_and_finalize)
  • baseline loading/fallback logic
• Backend-specific training code moved into:
  • yolov8_training/backends/yolo_backend.py (YOLO train + finetune path)
  • yolov8_training/backends/rfdetr_backend.py (RF-DETR train + dataset bridging + weights layout)

This keeps train_pipeline.py predictable and makes adding "backend 3" straightforward.

3) RF-DETR training path + YOLO-compatible artifacts

• RF-DETR runs write under runs/rfdetr/...
• Best checkpoint is copied into the same convention as YOLO: weights/best.pt (so baseline export/downstream tooling work unchanged).
• RF-DETR dataset bridging uses a lightweight YOLO-layout with symlinks + a generated data.yaml (fast; no COCO JSON export step).
• RF-DETR training is hardened for an upstream edge case where some small/broken runs never produce checkpoint_best_* (fallback to a regular checkpoint so the pipeline can continue).

4) Adapter to reuse the existing evaluation stack

• Added yolov8_training/utils/rfdetr_adapter.py (RFDETRModelAdapter) that makes an RF-DETR model look like ultralytics.YOLO for:
  • predict() (plotting + side-by-side comparisons)
  • val() (metrics contract evaluate.py expects)
• val() computes COCO metrics via pycocotools and aligns with native RF-DETR defaults (maxDets=500 + patched COCOeval summarize).
• Macro precision/recall/F1 is computed via RF-DETR’s confidence-threshold sweep (coco_extended_metrics) and exposed as metrics/f1(B) so the shared logger reports the native value.

5) Unified evaluation/reporting improvements

• yolov8_training/utils/evaluate.py now:
  • records model_backend / model_variant in metadata
  • exports per-class metrics and merged-class subset results into results_comparison/
  • accepts both YOLO metrics and RF-DETR adapter metrics
  • prefers metrics/f1(B) when present (otherwise computes F1 from precision/recall)

6) DVC + docs + tests

• Updated dvc.yaml to use --model ${train.model} and track models in params.
• Added backend files as DVC deps so backend changes trigger repro.
• Added dependencies for RF-DETR + evaluation (rfdetr, pycocotools, supervision) and updated ignores for new artifact types (*.pth, outputs).
• Added/updated tests:
  • RF-DETR adapter unit tests
  • E2E tests using stubs to keep CI fast/deterministic
  • opt-in heavy integration tests (pytest --heavy) to run real backend training
  • regression test for unsafe RF-DETR dataset export dir names
• Prepare stage now fails fast when no training frames are found (instead of producing confusing downstream errors).
• Dataset class-id parsing is normalized (handles YAML dict keys like "0" vs 0 consistently).

How to use

Switching backends is a one-line change:

# RF-DETR
dvc exp run -n "rfdetr-test" -S train.model=rfdetr-medium

# YOLO
dvc exp run -n "yolo-test" -S train.model=yolo11m