SuperPoint – Synthetic Keypoint Detection (PyTorch)

This repository implements a SuperPoint-style keypoint detector trained entirely on synthetic geometric data, using homography adaptation for robustness and generalization. The project follows a research-oriented design and is implemented from scratch in PyTorch.

The pipeline covers:

• 🧩 Synthetic dataset generation with precise keypoint supervision
• 🔁 Homography-based data augmentation & adaptation
• 🧠 SuperPoint-style CNN detector (encoder + detector head)
• 🎯 Grid-based keypoint classification (65-channel formulation)
• 🏋️ Training on infinite synthetic data
• 🖼️ Evaluation on synthetic and real images
• 🔥 Rich visualizations (heatmaps, predicted keypoints, overlays)

This project is suitable for computer vision coursework, research experiments, and as a foundation for learned local features.

---

Repository Structure

.
├── generate_dataset.py        # Synthetic shape generation + homography augmentation
├── model.py                   # SuperPoint-style detector (encoder + head + decoder)
├── train.py                   # Training on synthetic data
├── evaluate.py                # Evaluation on synthetic samples
├── evaluate_real.py           # Evaluation on real images
├── evaluate_real_ha.py        # Real images + homography adaptation
│
├── dataset/                   # Generated synthetic dataset (images, masks, keypoints)
├── eval_output/               # Synthetic evaluation visualizations
├── real_output/               # Real-image evaluation results
├── homography_adaptation_outputs/
│                               # HA heatmaps & keypoints on real images
├── training_loss.png          # Training loss curve
├── loss_history.csv           # Logged training loss
└── README.md

---

Synthetic Dataset Generation

Below is a direct visualization of the synthetic dataset generated by this repository.
All images are produced automatically by generate_dataset.py.

Each _pair.png image contains:

• Left: RGB image
• Right: Keypoint visualization (green dots)

---

Example: Quadrilateral shape

From left to right:

1. Original synthetic image (RGB + keypoints)
2. Homography-transformed image (RGB + transformed keypoints)

---

Example: Chessboard shape

From left to right:

1. Original synthetic chessboard (RGB + keypoints)
2. Homography-transformed chessboard (RGB + transformed keypoints)

---

The synthetic data includes:

• Clean geometric primitives (triangle, quadrilateral, star, chessboard, cube)
• Multiple objects per image (multi-shape scenes)
• Precisely defined keypoints
• Controlled noise and color variation

Model Architecture

Encoder

• Fully convolutional ResNet-style encoder
• Input: grayscale image (B, 1, H, W)
• Output: feature map (B, 128, H/8, W/8)

Detector Head

• Predicts 65 channels per cell:

  • 64 sub-pixel locations (8×8 grid)
  • 1 dustbin channel

Decoder (Inference only)

• Softmax over channels
• Reshape + permute (no PixelShuffle)
• Produces dense probability map (B, 1, H, W)

This matches the original SuperPoint formulation, while remaining fully transparent and readable.

---

Training

Training loss curve produced during synthetic training.

---

Evaluation – Synthetic Data

The following examples show the model output on synthetic data, taken directly from the synthetic_output/ folder.

Legend:

• 🔴 Red crosses — ground-truth keypoints
• 🟢 Green dots — predicted keypoints

---

Evaluation – Real Images

The detector generalizes to real photographs, despite being trained only on synthetic data.

Each result is obtained by:

• Forward pass through the detector
• Probability-map thresholding
• Non-maximum suppression (NMS)

---

Homography Adaptation (Real Images)

Homography Adaptation (HA) significantly improves keypoint stability by aggregating predictions across many random homographies.

Below is a real example from homography_adaptation_outputs/house/:

From left to right:

1. Input image
2. Single-pass probability heatmap
3. Homography-adapted heatmap (averaged)
4. Final keypoints after NMS

---

Visualizations

The repository includes:

• Training loss curve
• Synthetic GT vs prediction overlays
• Probability heatmaps
• Real-image detections
• Homography-adapted heatmaps

All visualizations are generated by the code in this repository.

---

Requirements

• Python ≥ 3.9
• PyTorch ≥ 2.0
• NumPy
• OpenCV
• Matplotlib
• Pandas

CUDA is optional but recommended.

---

Notes

• Input dimensions must be divisible by 8
• The detector outputs logits during training and probability maps during inference
• The implementation avoids PixelShuffle for full transparency

---

Author

Developed as an academic / research project focusing on:

• Learned keypoint detection
• Synthetic supervision
• Geometric consistency
• Homography-based robustness

---

If you use this repository for coursework or research, feel free to adapt and extend it 🚀