AI Interview Performance Analyzer

An AI-driven application that analyzes mock interview responses and provides quantitative feedback on confidence, fluency, and communication quality using speech recognition, audio signal processing, NLP, and machine learning.

This project is designed as a local ML system with a thin UI layer, focusing on correctness of methodology, interpretability, and clean software architecture.

---

Features

• Speech-to-Text using OpenAI Whisper
• Audio Signal Processing
  • Speaking pace (words per minute)
  • Pitch variation
  • Pause ratio
• NLP-Based Text Analysis
  • Filler word detection
  • Grammar & fluency proxy scoring
• Machine Learning Confidence Model
  • Supervised regression model
  • Uses acoustic, linguistic, and transformer-based emotion features
• Visual Analytics
  • Radar chart for overall performance
  • Bar chart for metric breakdown
• Automated PDF Report
  • Transcript
  • Metrics
  • Confidence score
  • Actionable feedback
• Clean, user-friendly Streamlit UI

---

Project Architecture

ai_interview_analyzer/
│
├── app.py                        # Streamlit UI
├── requirements.txt
├── README.md
│
├── audio/
│   └── sample.wav
│
├── processing/                   # Feature extraction
│   ├── speech_to_text.py
│   ├── audio_features.py
│   ├── text_analysis.py
│   └── confidence_score.py       # Rule-based baseline
│
├── models/                       # ML model & weights
│   ├── confidence_model.py
│   └── confidence_model.pkl
│
├── training/                     # Offline training & evaluation
│   ├── training_data.py
│   ├── train_confidence_model.py
│   └── evaluate_confidence_model.py
│
├── utils/
│   ├── feedback.py
│   └── report_pdf.py

---

Machine Learning Approach

Problem Type

• Regression
• Output: Confidence score ∈ [0, 100]

Input Features

1. Speaking pace (WPM)
2. Filler words per minute
3. Grammar & fluency score (NLP-based proxy)
4. Pitch variation
5. Pause ratio
6. Emotion probabilities from a pretrained Hugging Face transformer

Model Design

• RandomForestRegressor
• Trained on heuristic-labeled samples
• Designed to learn relative importance of features
• Optimized for interpretability and stability

Why Not a Fully Pretrained Model?

There is no pretrained model that directly outputs interview confidence. Instead, this project follows an industry-standard hybrid approach: pretrained models for feature extraction + a lightweight supervised regressor.

---

Model Evaluation

Metric	Value
MAE	1.16
RMSE	1.54
R² Score	0.992

Interpretation

• Low error is expected due to small dataset size and heuristic labeling.
• Results validate the ML pipeline and feature relevance, not real-world generalization.

---

User Interface

Built using Streamlit with a simple, intuitive flow:

1. Upload interview audio
2. View transcript
3. Inspect performance metrics
4. Understand confidence score
5. Receive actionable feedback
6. Download a PDF performance report

---

How to Run

Environment Setup

python3.10 -m venv venv
source venv/bin/activate
pip install -r requirements.txt
python -m spacy download en_core_web_sm

Train Confidence Model (one-time)

python -m training.train_confidence_model

Evaluate Model

python -m training.evaluate_confidence_model

Run Application

streamlit run app.py

---

PDF Report

The application generates a downloadable PDF containing:

• Interview transcript
• Extracted metrics
• Confidence score
• Personalized feedback

---

Limitations & Ethics

• Confidence is subjective.
• Labels are heuristic-based, not psychological ground truth.
• Predictions represent perceived confidence from observable signals.

---

Future Improvements

• Larger labeled dataset
• Cross-validation & robustness testing
• Video-based cues (eye contact, posture)
• Backend API and cloud deployment

---