percent-face

Pure Rust facial landmark detection using Ensemble of Regression Trees (ERT).

This crate implements the algorithm from "One Millisecond Face Alignment with an Ensemble of Regression Trees" (Kazemi & Sullivan, CVPR 2014).

Quick Start

# Download required models and build
make setup

# Run the GUI
make run-gui

Features

• Pure Rust - No C++ dependencies, native dlib model loading
• Sub-pixel accuracy - Bilinear interpolation and similarity transforms
• Multiple models - Supports 5, 68, and 81-point landmark models
• CLI tool - Analyze images with human-readable or JSON output
• GUI application - Visual face detection and landmark overlay
• Face metrics - Calculate feature areas as percentage of face/image
• Cross-platform - Linux, macOS, and Windows

CLI Tool

Analyze images from the command line:

# Human-readable output
percent-face image.jpg

# JSON output
percent-face image.jpg --json

# Save to file
percent-face image.jpg --json -o results.json

# With custom models
percent-face image.jpg --detector model.bin --landmarks model.dat

Example output:

Image: photo.jpg (1920x1080)
Faces detected: 1

--- Face 1 ---
Bounding box: 400x400 at (760, 200)
Landmarks: 81 points

Image Coverage:
  Face: 7.7% of image
  Head: 10.9% of image

Features (% of face):
  Eyes:     1.8% (L: 0.9%, R: 0.9%)
  Eyebrows: 2.1%
  Nose:     3.2%
  Mouth:    4.5% (lips: 3.8%)
  Forehead: 5.8%

Ratios:
  Face/Head:    70.5%
  Eye symmetry: 98.2%
  Eye/Mouth:    0.40x

Build and run:

make build-cli
make run-cli ARGS="image.jpg --json"

GUI Demo

The GUI application provides:

• Face detection using SeetaFace/rustface
• Landmark visualization with connections
• Image rotation controls for testing
• Face area calculations (jawline, head outline)
• Support for 68-point and 81-point models

make run-gui

Or manually:

cargo run --features gui --bin percent-face-gui

Face Metrics

The GUI calculates and displays:

• Box area - Detection bounding box as % of image
• Jawline area - Area enclosed by landmarks 0-16
• Head area - Full head outline including forehead
  • 81-point model: Uses actual forehead landmarks (68-80)
  • 68-point model: Estimates forehead from facial proportions

Supported Models

Model	Points	Description	Size
shape_predictor_5_face_landmarks.dat	5	Eyes + nose tip	~9 MB
shape_predictor_68_face_landmarks.dat	68	Full iBUG model (jaw, brows, eyes, nose, mouth)	~99 MB
shape_predictor_81_face_landmarks.dat	81	68-point + forehead/hairline	~19 MB
seeta_fd_frontal_v1.0.bin	-	Face detector (SeetaFace)	~1 MB

Download all models:

make download-models

Library Usage

use percent_face::{BoundingBox, GrayImage};

// Load a dlib model (supports .dat and .dat.bz2)
let model = percent_face::dlib::load_dlib_model("shape_predictor_68_face_landmarks.dat.bz2")?;

// Create/load a grayscale image
let image = GrayImage::new(pixels, width, height);

// Face bounding box from your face detector
let face_rect = BoundingBox::new(x, y, width, height);

// Predict landmarks
let landmarks = model.predict(&image, &face_rect);

for (i, point) in landmarks.points.iter().enumerate() {
    println!("Landmark {}: ({}, {})", i, point.x, point.y);
}

Face Metrics

Calculate facial feature areas and proportions:

use percent_face::FaceMetrics;

// After getting landmarks...
if let Some(metrics) = FaceMetrics::from_shape(&landmarks) {
    // Areas (in square pixels)
    println!("Face area: {:.0} px²", metrics.jawline_area);
    println!("Left eye: {:.0} px²", metrics.left_eye_area);
    println!("Nose: {:.0} px²", metrics.nose_area);

    // Proportions (as % of face area)
    println!("Eyes: {:.1}% of face", metrics.eyes_ratio());
    println!("Nose: {:.1}% of face", metrics.nose_ratio());
    println!("Mouth: {:.1}% of face", metrics.mouth_ratio());

    // Inter-feature ratios
    println!("Eye symmetry: {:.1}%", metrics.eye_symmetry());
    println!("Eye/Mouth ratio: {:.2}x", metrics.eye_to_mouth_ratio() / 100.0);
}

Available measurements:

• Areas: jawline (lower face), head (full face), left/right eyes, eyebrows, nose, mouth (outer/inner), forehead
• Ratios: Each feature as % of full face area, lower-face ratio, inter-feature comparisons
• Symmetry: Eye symmetry score (100% = perfect symmetry)

Note: Feature ratios are calculated against head_area (full face including forehead), not just the jawline. This gives more intuitive percentages where all features sum closer to 100%.

Custom Image Types

Implement the ImageAccess trait for your own image types:

use percent_face::ImageAccess;

impl ImageAccess for MyImage {
    fn get_pixel(&self, x: i32, y: i32) -> u8 {
        // Return grayscale intensity, 0 for out-of-bounds
    }
    fn width(&self) -> u32 { self.width }
    fn height(&self) -> u32 { self.height }
}

Algorithm Overview

The ERT algorithm achieves real-time facial landmark detection through:

1. Initial Estimate: Start with mean face shape scaled to the detected face bounding box
2. Cascade Refinement: For each of ~10 cascade levels:
   • Sample sparse pixel intensity differences as features
   • Each regression tree in the ensemble votes on landmark position adjustments
   • Sum tree predictions to get shape update
   • Apply update to refine current shape estimate
3. Output: Final landmark positions

Why This Works

• Sparse features: Only ~400 pixel comparisons per cascade level (not dense HOG/SIFT)
• Shape-indexed features: Pixel locations are relative to current landmark estimates
• Gradient boosting: Each cascade level corrects errors from previous levels
• Simple inference: Just tree traversal and additions - no convolutions or matrix ops

Project Structure

src/
├── lib.rs          # Public API and module exports
├── types.rs        # Core types: Point, Shape, BoundingBox
├── tree.rs         # RegressionTree, TreeEnsemble, SplitFeature
├── features.rs     # Pixel feature extraction, similarity transforms
├── model.rs        # ShapePredictor (main entry point)
├── dlib.rs         # dlib .dat/.dat.bz2 format loader
├── error.rs        # Error types
└── bin/
    └── gui.rs      # GUI application (optional feature)

Building

Prerequisites

• Rust 1.70+ (for workspace features)
• curl (for model downloads)
• make (optional, for convenience targets)

Commands

# Build library only
cargo build

# Build with GUI
cargo build --features gui

# Run tests
cargo test

# Run GUI
cargo run --features gui --bin percent-face-gui

Implementation Status

Phase 1: Core Implementation

• Core data structures (Point, Shape, BoundingBox)
• Regression tree structure and traversal
• Tree ensemble (gradient boosting)
• Pixel feature extraction framework
• Cascade inference loop
• Model serialization (bincode)
• dlib .dat/.dat.bz2 format loader (pure Rust)

Phase 2: Accuracy & Compatibility

• Integrate anchor_idx and deltas into split features
• Test inference against dlib reference (sub-pixel accuracy)
• Bilinear interpolation for sub-pixel sampling
• Similarity transform normalization (rotated faces)
• 81-point model support with forehead landmarks

Phase 3: GUI & Visualization

• Face detection integration (rustface)
• Landmark visualization with connections
• Image rotation controls
• Face area metrics calculation
• Auto-load models on startup

Phase 4: Performance (Planned)

• Benchmarks
• SIMD optimization for feature extraction
• Parallel tree evaluation
• Memory layout optimization

References

• One Millisecond Face Alignment with an Ensemble of Regression Trees - Original paper
• dlib shape_predictor - Reference implementation
• 81-point model - Extended forehead landmarks
• iBUG 300-W dataset - Standard benchmark

License

MIT