Oral Cancer AI

Multimodal AI-assisted oral cancer screening and patient monitoring prototype built with Flask, TensorFlow/Keras, OpenCV, and SQLite.

This project combines clinical oral image classification, patient metadata, histopathology image prediction, explainability, uncertainty estimation, and scan history tracking in a single web application. It is intended as a research and decision-support prototype, not as a standalone diagnostic medical device.

Features

• Clinical oral image prediction for:
  • Normal
  • Variations from normal
  • Oral Potentially Malignant Disorders
  • Oral Cancer
• Multimodal prediction using image input plus patient metadata:
  • Age
  • Sex
  • Smoking
  • Chewing
  • Areca nut use
  • Alcohol use
• Histopathology ensemble prediction for binary OSCC-style screening.
• Monte Carlo dropout based uncertainty estimation.
• Grad-CAM heatmap generation for image explainability.
• SHAP-style metadata explanation endpoint.
• Patient registration and scan history storage using SQLite.
• Risk trend tracking across patient visits.
• Doctor login, admin page, reminders, and habit tracker pages.

Project Structure

oral_cancer_ai/
|-- app.py                         # Main Flask application
|-- database.py                    # SQLite database setup and connection helpers
|-- index.html                     # Main screening interface
|-- patient_history.html           # Patient history interface
|-- reminders.html                 # Reminder page
|-- habit_tracker.html             # Habit tracking page
|-- admin.html                     # Admin interface
|-- patients.db                    # SQLite patient and scan database
|-- data/                          # Dataset CSVs, splits, and image folders
|-- Metadata/                      # Patient metadata files
|-- Descriptors/                   # Annotation descriptor spreadsheets
|-- models/                        # Trained model files
|-- notebooks/                     # Training, evaluation, Grad-CAM, and SHAP scripts
|-- results/                       # Training curves, confusion matrices, reports
|-- scripts/                       # Utility scripts
|-- project_report.md              # Short project report
|-- research_paper.md              # Detailed research paper
`-- research_paper.docx            # Word version of research paper

Main Models

The application expects trained models inside the models/ directory:

models/
|-- best_model.h5                  # Clinical oral image classifier
|-- multimodal_model.h5            # Image + metadata classifier
|-- gradcam_model.h5               # Grad-CAM related model artifact
|-- histo_custom_1.keras           # Histopathology ensemble model
|-- histo_custom_2.keras
|-- histo_custom_3.keras
|-- histo_custom_4.keras
|-- histopath_resnet50.h5
|-- histopath_densenet121.h5
`-- generated/                     # Generated model experiments

If one or more histopathology model files are missing, the Flask app will skip those models and report how many were loaded.

Dataset

The project uses clinical oral images, metadata, and histopathology images. The available split CSV files are:

data/train.csv
data/val.csv
data/test.csv

Current clinical split counts:

Split	Normal	Variations	OPMD	OC	Total
Train	1501	125	88	14	1728
Validation	322	27	18	3	370
Test	322	27	19	3	371

The data is highly imbalanced, especially for the Oral Cancer class. Model performance should therefore be evaluated with class-wise recall, precision, F1-score, balanced accuracy, confusion matrices, and external validation rather than accuracy alone.

Requirements

Recommended environment:

• Python 3.10 or newer
• TensorFlow / Keras
• Flask
• Flask-CORS
• NumPy
• OpenCV
• SHAP
• scikit-learn
• pandas
• matplotlib
• seaborn

The project already contains a local venv/ folder in this workspace. If you are setting up from scratch, create and activate a virtual environment first.

Setup

From the project root:

cd "E:\Intern\AI-COE\Multimodal for oral cancer\Final Project\oral_cancer_ai"

Activate the existing virtual environment:

.\venv\Scripts\Activate.ps1

If PowerShell blocks activation, run:

Set-ExecutionPolicy -Scope Process -ExecutionPolicy Bypass
.\venv\Scripts\Activate.ps1

Install dependencies if needed:

pip install flask flask-cors tensorflow opencv-python numpy pandas scikit-learn matplotlib seaborn shap

Running the Application

Start the Flask app:

python app.py

The app runs on:

http://localhost:5000

Open the main interface in a browser:

http://localhost:5000/

Other pages:

http://localhost:5000/history
http://localhost:5000/reminders
http://localhost:5000/habit-tracker

API Endpoints

Health Check

GET /health

Returns:

{
  "status": "ok"
}

Clinical and Multimodal Prediction

POST /predict

Expected form data:

• image: oral image file
• age: patient age
• sex: numeric encoded sex value
• smoking: 0 or 1
• chewing: 0 or 1
• arecanut: 0 or 1
• alcohol: 0 or 1

Returns image-only prediction, multimodal prediction, risk category, uncertainty, class probabilities, Grad-CAM image data, and number of Monte Carlo runs.

Histopathology Ensemble Prediction

POST /predict_histo_ensemble

Expected form data:

• image: histopathology image file

Returns ensemble class, confidence, OSCC probability, uncertainty, per-model votes, and model count.

Metadata Explanation

POST /shap_explain

Expected JSON body:

{
  "age": 40,
  "sex": 0.5,
  "smoking": 0,
  "chewing": 0,
  "arecanut": 0,
  "alcohol": 0
}

Returns SHAP-style metadata contribution values and predicted class probabilities.

Save Scan

POST /save_scan

Stores patient details, prediction output, risk state, and Grad-CAM data in patients.db.

Patient History

GET /patient_history/<patient_id>

Returns patient details, scan history, and risk trend information.

Patient List

GET /patients

Returns all registered patients, scan counts, latest scan date, and latest risk level.

Authentication

POST /api/login
POST /api/logout

The default fallback login in development is:

Username: admin
Password: admin123

Change this before any real deployment.

Training and Evaluation Scripts

Important scripts include:

prepare_dataset.py                       # Prepare histopathology train/val/test split
train_histopath.py                       # Train transfer-learning histopathology models
train_histopath_4models_scratch.py       # Train custom histopathology models
train_scratch.py                         # Train custom CNN variants
train_generated_models.py                # Train generated model architectures
notebooks/train_model.py                 # Clinical image model training workflow
notebooks/train_multimodal.py            # Multimodal image + metadata training workflow
notebooks/evaluate_model.py              # Clinical model evaluation
notebooks/evaluate_multimodal.py         # Multimodal model evaluation
notebooks/gradcam.py                     # Grad-CAM generation workflow
notebooks/shap_metadata.py               # Metadata explainability workflow

Some scripts contain absolute paths from the original development machine. Update those paths before retraining or evaluating on a different system.

Database

database.py initializes two SQLite tables:

• patients: stores patient ID, name, age, sex, phone, address, and creation date.
• scans: stores scan date, doctor, department, clinical notes, habits, model outputs, uncertainty labels, risk key, Grad-CAM image data, and timestamp.

The database file is:

patients.db

Results and Reports

Generated artifacts are stored in results/, including:

confusion_matrix.png
multimodal_confusion_matrix.png
training_curves.png
multimodal_training.png
gradcam_results.png
sample_images.png
project_report.pdf

Project documentation:

project_report.md
research_paper.md
research_paper.docx

Clinical Safety Notice

This project is a research prototype for AI-assisted oral cancer screening. It must not be used as a replacement for clinical examination, biopsy, histopathology, or specialist diagnosis.

High-risk predictions should be reviewed by a qualified clinician. Low-risk predictions should not override clinical concern, persistent symptoms, suspicious lesions, or the need for follow-up.

Before clinical deployment, the system requires:

• Larger and more balanced datasets
• Patient-level train/validation/test splitting
• External validation
• Calibration and threshold analysis
• Clinical usability testing
• Security hardening
• Privacy and regulatory review

Known Limitations

• The clinical dataset split is highly imbalanced.
• Oral Cancer examples are limited in the available CSV splits.
• Some training scripts use absolute local paths.
• Structured numerical evaluation reports are not currently stored for every model.
• Authentication is suitable for development only.
• Grad-CAM and SHAP outputs are explanatory aids, not proof of correctness.
• Histopathology prediction is patch/image-level support and does not replace pathologist review.

Suggested Next Steps

• Add a requirements.txt file for reproducible installation.
• Replace hard-coded local paths in training scripts with config variables.
• Add model cards for every trained model.
• Add automated evaluation reports in JSON or CSV format.
• Add patient-level split validation to reduce leakage risk.
• Improve authentication and data protection before deployment.
• Add image quality checks before prediction.

License

Add your project license here before public release.

Acknowledgement

This project was developed as part of a multimodal oral cancer AI workflow, combining clinical imaging, metadata, histopathology, explainability, and patient history tracking into a single prototype application.