ear-abnormalities

Public demo link: https://huggingface.co/spaces/rebeccassova/Infant_Ear_Abnormalities

Overview

This project develops a machine learning–based framework for objective assessment of infant ear deformities. Current diagnosis in pediatric plastic surgery is largely subjective. Our goal is to build a standardized, quantitative pipeline that:

• Detects ear landmarks automatically
• Computes geometric measurements (distances, ratios, curvature)
• Predicts deformity types

Instead of treating deformity detection as a black-box classification problem, we focus on measurement-driven and clinically interpretable modeling.

As shown by the above screenshots, once the user uploads an image, 23 ear landmarks are predicted and mapped. The left side of the screen shows the confidence levels the model has for each landmark. The right side shows the landmarks, calculated lines and measurements, and predicted diagnosis.

Technologies

• Python
• PyTorch
• TorchVision
• NumPy
• Gradio

Folder Overview

• frontend: This folder has the Gradio-integrated user accessible app for the model. There are also versions of the trained model inside, with the most current version being infant_ear_model_23lm_best_v4.pth.
• backend/EDA_scripts: This folder has various scripts used to understand the breakdown of infant ear data available and labeled.
• backend/classification_model: This folder has the model that takes the ear landmarks and measurements and classifies the ears into the possible ear abnormality diagnoses.
• backend/landmark_model: This folder has the model that predicts adult and infant ear landmarks.
• backend/tests: This folder tests aspects of our landmark prediction model with preprocessing, model output, accuracy, and robustness tests.
• backend/utils: This folder has some other files used across the backend.
• archive: This was a previous version of the ear landmark prediction model based on this CNN model. It was modified and finetuned, but ultimately discontinued due to the ResNet-18 heatmap model being more accurate.

How to Train the Ear Landmark Prediction model

1. Clone the main branch of this repository.
2. Download the adult ear data from iBug. (If iBug link is down, use this Dropbox link).
3. Download the infant ear data from BabyEar4k.
4. Download the infant ear landmark data from the Google Drive link.
5. In the terminal, go to the backend/landmark_model folder.
6. Update the config.yaml file, specifically considering the variables that are labeled “UPDATE”, such as file paths of the downloaded dataset.
7. Run python pipeline.py to run the full pipeline, starting to train the adult ear model first and then the infant ear model. If the adult ear model is already trained, you can skip the adult model steps by running python pipeline.py –skip adult. See further configurable options in the README.md in that folder.

Credits

The final model found in the backend folder is based on an existing transfer learning model from resnet-18. We replaced the classification head of the model and replaced it with a regression head. We then further replaced direct coordinate regression with a multi-stage heatmap-based landmark localization architecture.

To train the model on adult ears, we used data made available through iBug (also available at this Dropbox link).

To train the model on infant ears, we worked on manually labeling this dataset of infant ears. 23 points were labeled on each of the ear images, specifically #0-19, 37, 26, and a new point between 48 and 53 on the intersection of the cartilage. This dataset has not been labelled before for ear landmarks.