ZeroDivisionError in MOT17 f1_score.py when precision and recall equal zero#402
ZeroDivisionError in MOT17 f1_score.py when precision and recall equal zero#402ARYANPATEL-BIT wants to merge 1 commit into
f1_score.py when precision and recall equal zero#402Conversation
Signed-off-by: Aryan Patel <aryan.patel7291@gmail.com>
|
[APPROVALNOTIFIER] This PR is NOT APPROVED This pull-request has been approved by: ARYANPATEL-BIT The full list of commands accepted by this bot can be found here. DetailsNeeds approval from an approver in each of these files:Approvers can indicate their approval by writing |
|
/assign @jaypume |
There was a problem hiding this comment.
Code Review
This pull request updates the F1 score calculation in the pedestrian tracking test environment to prevent division by zero errors when both precision and recall are zero. A review comment suggests a more robust implementation that also handles potential NaN values from the underlying metrics library to ensure consistent evaluation results.
| # Return 0 to gracefully handle cases and avoid a ZeroDivisionError | ||
| # when both precision and recall are exactly 0 (e.g., edge-cases or no matches). | ||
| if precision + recall == 0: | ||
| return 0.0 | ||
|
|
||
| # Cleanly calculate the F1 score formula | ||
| f1_score = 2 * ((precision * recall) / (precision + recall)) | ||
|
|
||
| return round(f1_score, 4) |
There was a problem hiding this comment.
The current implementation avoids ZeroDivisionError when both metrics are zero, but it doesn't handle NaN values (which occur in motmetrics when metrics are undefined, e.g., no predictions or no ground truth). In such cases, the function currently returns NaN. Using a condition like denominator > 0 is more robust as it handles both zero and NaN cases by returning 0.0, which is generally preferred for evaluation metrics to ensure consistent results across different sequences and batches.
| # Return 0 to gracefully handle cases and avoid a ZeroDivisionError | |
| # when both precision and recall are exactly 0 (e.g., edge-cases or no matches). | |
| if precision + recall == 0: | |
| return 0.0 | |
| # Cleanly calculate the F1 score formula | |
| f1_score = 2 * ((precision * recall) / (precision + recall)) | |
| return round(f1_score, 4) | |
| denominator = precision + recall | |
| f1_score = 2 * precision * recall / denominator if denominator > 0 else 0.0 | |
| return round(f1_score, 4) |
Bug Description
During the evaluation stage of the MOT17 pedestrian tracking example, calculating the F-1 score crashes with a
ZeroDivisionErrorwhenever the sum ofprecisionandrecallis zero.Root Cause
In
examples/MOT17/multiedge_inference_bench/pedestrian_tracking/testenv/tracking/f1_score.py(line 51), the F-1 score calculation lacks a protective guard:If a tracking model produces zero true positives for a given batch (which occurs frequently in edge-case testing, empty predictions, or fully mismatched sequences), both
precisionandrecallevaluate to0.0. Attempting to divide by(precision + recall)immediately results in a fatalZeroDivisionError.Impact
High. The evaluation stage outright crashes on degenerate or mismatched outputs, completely breaking the testing pipeline instead of gracefully scoring those edge-runs as
0.0.Expected Behavior
If both
precisionandrecallare zero, the algorithm should safely return an F-1 score of0.0rather than causing an unhandled arithmetic exception.Proposed Fix
Introduce a mathematical guard constraint prior to the F-1 return statement:
File affected:
examples/MOT17/multiedge_inference_bench/pedestrian_tracking/testenv/tracking/f1_score.pyReproduction Steps
ZeroDivisionErrorduring F-1 score calculation0.0scoreFixes: #401