Skip to content

Episode 9, Exercise 1: Concept of Validation proposal - "Segmentation Challenge" #21

@MalloryWittwer

Description

@MalloryWittwer

Brief description

This exercise takes the form of a "segmentation challenge": a few challenge images are presented to learners, who are invited to come up with their best workflow to segment these images.

The images contain multiple recognizable objects, suitable for instance segmentation. For example:

Image

Each image is also accompanied with a carefully curated, manually edited segmentation mask to use as a "ground truth" for evaluating segmentation accuracy.

The evaluation metrics are:

  • (1) object count error (relative to the true object count)
  • (2) intersection over union (computed on the binary version of the masks)

Learners are invited to load and process the images using scikit-image and the tools and techniques they've learned during the course. When they feel like they have come to an acceptable solution, they can calculate the evaluation metrics for their solution against the ground truth mask.

➡️ Setup with an interactive "leaderboard"

To make it a more fun and engaging session:

  • Learners can "submit" their proposed segmentation masks by uploading them to a shared folder.
  • A little script automatically retrieves the masks in the shared folder, computes the metrics, and updates a "leaderboard" panel displayed on the main screen.

Learning objective(s)

Related episode: Concept of Validation

Main objectives:

  • Understand how segmentation masks can be compared to a ground truth (or other segmentation masks) quantitatively.
  • Practice concepts and techniques seen during the rest of the course.

Volunteer(s)

@MalloryWittwer

Metadata

Metadata

Assignees

No one assigned

    Labels

    exercise-proposalProposed exercises for the new curriculum

    Type

    No type

    Projects

    No projects

    Milestone

    No milestone

    Relationships

    None yet

    Development

    No branches or pull requests

    Issue actions