test/add-test-for-metrics

[juniorcl]

Test: Metrics (Scores & Plots)

Motivation

Ensure that all metrics available under /metrics/scores and /metrics/plots are:

• Fully compatible with the scikit-learn API
• Reusable within pipelines, cross-validation, and AutoML workflows
• Deterministic, reproducible, and easy to test
• Safe for production usage (no hidden state or side effects)

---

Scope

Implement unit tests and light integration tests for:

• Metric score functions (/metrics/scores)
• Custom sklearn scorers (*_scorer.py)
• Plotting utilities (/metrics/plots)

The initial scope focuses on binary classification, with future extensibility to multiclass classification and regression.

---

Metrics – Scores

Components Covered

• ks_score.py
• average_precision_lift_score.py
• ks_scorer.py
• average_precision_lift_scorer.py

Acceptance Criteria – Scores

API & sklearn Compatibility

• Implements a signature compatible with:
  • (y_true, y_score) or
  • (estimator, X, y) for sklearn scorers
• Works correctly with:
  • sklearn.model_selection.cross_val_score
  • GridSearchCV / RandomizedSearchCV
• Can be used via the scoring= argument in sklearn estimators

Input Types

• Accepts numpy.ndarray inputs
• Accepts pandas.Series and pandas.DataFrame
• Correctly handles 1D and 2D y_score (positive-class probabilities)

Output

• Always returns a scalar float
• Output is deterministic given a fixed random_state
• Does not modify input objects (X, y, y_score)

Robustness

• Handles:
  • Class imbalance
  • Constant prediction scores
• Raises clear and explicit errors for:
  • Incompatible inputs
  • Invalid y_score formats

---

Metrics – Plots

Components Covered

• roc_curve.py
• precision_recall_curve.py
• calibration_curve.py
• ks_curve.py

Acceptance Criteria – Plots

API & Usability

• Functions explicitly receive:
  • y_true
  • y_score or y_pred
• No reliance on global state (e.g., implicit plt.gca())
• Can be called multiple times in the same process without side effects

Input Compatibility

• Accepts:
  • numpy.ndarray
  • pandas.Series
• Validates consistent lengths between y_true and y_score

Output & Testability

• Explicitly returns:
  • matplotlib.figure.Figure or
  • (Figure, Axes)
• Does not call plt.show() internally
• Allows inspection of:
  • Number of plotted lines
  • Axis labels
  • Axis limits

Robustness

• Does not fail with:
  • Constant prediction scores
  • Small datasets
• Raises explicit errors for invalid inputs