Skip to content

ZeroDivisionError in MOT17 f1_score.py when precision and recall equal zero#402

Open
ARYANPATEL-BIT wants to merge 1 commit into
kubeedge:mainfrom
ARYANPATEL-BIT:fix/mot17-f1-score-zerodivision
Open

ZeroDivisionError in MOT17 f1_score.py when precision and recall equal zero#402
ARYANPATEL-BIT wants to merge 1 commit into
kubeedge:mainfrom
ARYANPATEL-BIT:fix/mot17-f1-score-zerodivision

Conversation

@ARYANPATEL-BIT
Copy link
Copy Markdown

Bug Description

During the evaluation stage of the MOT17 pedestrian tracking example, calculating the F-1 score crashes with a ZeroDivisionError whenever the sum of precision and recall is zero.

Root Cause

In examples/MOT17/multiedge_inference_bench/pedestrian_tracking/testenv/tracking/f1_score.py (line 51), the F-1 score calculation lacks a protective guard:

round(2*((precision*recall)/(precision+recall)), 4)

If a tracking model produces zero true positives for a given batch (which occurs frequently in edge-case testing, empty predictions, or fully mismatched sequences), both precision and recall evaluate to 0.0. Attempting to divide by (precision + recall) immediately results in a fatal ZeroDivisionError.

Impact

High. The evaluation stage outright crashes on degenerate or mismatched outputs, completely breaking the testing pipeline instead of gracefully scoring those edge-runs as 0.0.

Expected Behavior

If both precision and recall are zero, the algorithm should safely return an F-1 score of 0.0 rather than causing an unhandled arithmetic exception.

Proposed Fix

Introduce a mathematical guard constraint prior to the F-1 return statement:

if precision + recall == 0:
    return 0.0
round(2*((precision*recall)/(precision+recall)), 4)

File affected:
examples/MOT17/multiedge_inference_bench/pedestrian_tracking/testenv/tracking/f1_score.py

Reproduction Steps

  1. Run MOT17 evaluation with a model producing zero true positives (empty predictions or complete mismatches)
  2. Observe ZeroDivisionError during F-1 score calculation
  3. Evaluation pipeline crashes instead of continuing with 0.0 score

Fixes: #401

Signed-off-by: Aryan Patel <aryan.patel7291@gmail.com>
@kubeedge-bot
Copy link
Copy Markdown
Collaborator

[APPROVALNOTIFIER] This PR is NOT APPROVED

This pull-request has been approved by: ARYANPATEL-BIT
To complete the pull request process, please assign jaypume after the PR has been reviewed.
You can assign the PR to them by writing /assign @jaypume in a comment when ready.

The full list of commands accepted by this bot can be found here.

Details Needs approval from an approver in each of these files:

Approvers can indicate their approval by writing /approve in a comment
Approvers can cancel approval by writing /approve cancel in a comment

@kubeedge-bot kubeedge-bot added the size/S Denotes a PR that changes 10-29 lines, ignoring generated files. label Apr 13, 2026
@ARYANPATEL-BIT
Copy link
Copy Markdown
Author

/assign @jaypume

Copy link
Copy Markdown

@gemini-code-assist gemini-code-assist Bot left a comment

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

Code Review

This pull request updates the F1 score calculation in the pedestrian tracking test environment to prevent division by zero errors when both precision and recall are zero. A review comment suggests a more robust implementation that also handles potential NaN values from the underlying metrics library to ensure consistent evaluation results.

Comment on lines +52 to +60
# Return 0 to gracefully handle cases and avoid a ZeroDivisionError
# when both precision and recall are exactly 0 (e.g., edge-cases or no matches).
if precision + recall == 0:
return 0.0

# Cleanly calculate the F1 score formula
f1_score = 2 * ((precision * recall) / (precision + recall))

return round(f1_score, 4)
Copy link
Copy Markdown

Choose a reason for hiding this comment

The reason will be displayed to describe this comment to others. Learn more.

medium

The current implementation avoids ZeroDivisionError when both metrics are zero, but it doesn't handle NaN values (which occur in motmetrics when metrics are undefined, e.g., no predictions or no ground truth). In such cases, the function currently returns NaN. Using a condition like denominator > 0 is more robust as it handles both zero and NaN cases by returning 0.0, which is generally preferred for evaluation metrics to ensure consistent results across different sequences and batches.

Suggested change
# Return 0 to gracefully handle cases and avoid a ZeroDivisionError
# when both precision and recall are exactly 0 (e.g., edge-cases or no matches).
if precision + recall == 0:
return 0.0
# Cleanly calculate the F1 score formula
f1_score = 2 * ((precision * recall) / (precision + recall))
return round(f1_score, 4)
denominator = precision + recall
f1_score = 2 * precision * recall / denominator if denominator > 0 else 0.0
return round(f1_score, 4)

Sign up for free to join this conversation on GitHub. Already have an account? Sign in to comment

Labels

size/S Denotes a PR that changes 10-29 lines, ignoring generated files.

Projects

None yet

Development

Successfully merging this pull request may close these issues.

ZeroDivisionError in MOT17 f1_score.py when precision and recall equal zero

3 participants