📈 Linear Regression from Scratch

A two-part project: a fully explained Jupyter notebook implementing linear regression using only NumPy — and a live, interactive browser visualizer that lets you watch gradient descent work in real time.

---

---

🗂️ What's in This Repo

This project has two distinct but connected parts:

Part	File	Description
Jupyter Notebook	linear-regression-from-scratch.ipynb	Full educational walkthrough — concept, math, implementation, evaluation, experiments
Web Visualizer	index.html + script.js + style.css	Interactive browser app — place data, train in real time, watch the fit evolve

---

📓 Part 1 — The Notebook

The notebook is a complete, well-explained implementation of linear regression built from scratch using only NumPy — no sklearn, no shortcuts.

What it covers

1. The concept — what linear regression is and what the model equation means
2. The math — MSE loss function and how gradient descent minimizes it
3. Data generation — synthetic salary vs. experience dataset with a known true line
4. The LinearRegressionScratch class — fit(), predict(), score(), mse() built from scratch
5. Feature scaling — standardization to stabilize convergence (and why it matters)
6. Training + visualization — loss curve, fitted line, and residual plot
7. Evaluation — RMSE and R² on train and test sets
8. Learning rate experiment — comparing α = 0.001, 0.01, 0.1, 0.5 side by side
9. Sanity check — comparing results directly against sklearn.LinearRegression
10. Prediction — using the trained model to predict unseen values

Quick start — notebook

git clone https://github.com/yourusername/linear-regression-from-scratch.git
cd linear-regression-from-scratch

pip install numpy matplotlib scikit-learn jupyter
jupyter notebook "linear-regression-from-scratch.ipynb"

---

🌐 Part 2 — The Web Visualizer

An interactive single-page app where you can generate synthetic datasets, configure a gradient descent linear regression model, and watch it train — live.

Features

• Configurable dataset — adjust point count (20–300), noise σ (0–1), and train/test split (50%–90%) with live regeneration
• Hyperparameter controls — set learning rate (0.001–0.5), epochs (10–1000), and weight initialization (zeros or random)
• Animated training — gradient descent runs frame by frame via requestAnimationFrame, updating the scatter plot and loss curve as it trains
• Live regression line — the fitted line updates every animation frame; the true (ground truth) line is shown dashed for comparison
• Live metrics — slope (w), intercept (b), train MSE, test MSE, R² score, and progress all update in real time
• Loss summary — initial loss, final loss, and percentage reduction shown after training completes
• Stop & reset — interrupt training at any epoch or clear everything and start fresh
• Responsive canvas — both canvases use devicePixelRatio scaling for crisp rendering on retina/HiDPI screens
• Zero dependencies — pure vanilla HTML, CSS, and JavaScript; no libraries

How to use the visualizer

1. Adjust dataset — change point count, noise, and split ratio in the left panel
2. Set hyperparameters — tune learning rate and epochs; choose zero or random initialization
3. Click Train — watch the line fit the data and loss decrease in real time
4. Experiment — try high noise + low epochs, compare zero vs random init, crank up the learning rate and watch it diverge

---

🏗️ Project Structure

linear-regression-from-scratch/
├── index.html                              # Web visualizer markup
├── script.js                               # Gradient descent engine + canvas rendering
├── style.css                               # Layout, sliders, typography
├── vercel.json                             # Deployment config
└── linear-regression-from-scratch.ipynb   # Full educational notebook

---

🚀 Getting Started (Web App)

Option 1 — Just open it

git clone https://github.com/yourusername/linear-regression-from-scratch.git
open index.html

Option 2 — Local dev server

npx serve .
# or
python3 -m http.server 8080

Option 3 — Deploy to Vercel

npm i -g vercel
vercel

---

🧮 The Math (Quick Reference)

The model:   ŷ = wx + b

Mean Squared Error (loss):

$$\text{MSE} = \frac{1}{n} \sum_{i=1}^{n} (\hat{y}_i - y_i)^2$$

Gradient descent update rules:

$$w \leftarrow w - \alpha \cdot \frac{\partial L}{\partial w} \qquad b \leftarrow b - \alpha \cdot \frac{\partial L}{\partial b}$$

Gradients (derived analytically):

$$\frac{\partial L}{\partial w} = \frac{2}{n} \sum (wx_i + b - y_i) \cdot x_i \qquad \frac{\partial L}{\partial b} = \frac{2}{n} \sum (wx_i + b - y_i)$$

R² Score:

$$R^2 = 1 - \frac{SS_{res}}{SS_{tot}} = 1 - \frac{\sum (y_i - \hat{y}_i)^2}{\sum (y_i - \bar{y})^2}$$

---

🙏 Attribution & Usage

This project is licensed under CC BY-NC 4.0 (see LICENSE).

If you use, fork, adapt, or share this project:

📣 Please tag me — a mention, a link, or a shoutout is genuinely appreciated.
GitHub: @UZi-Senpai · Portfolio: umairzia.pythonanywhere.com

You're free to:

• ✅ Use it for personal learning and projects
• ✅ Fork, modify, and share with attribution
• ✅ Reference the notebook in your own ML learning

You may not:

• ❌ Sell it or use it commercially without permission
• ❌ Strip attribution and present it as your own

---

🤝 Contributing

PRs and issues welcome. Keep the web app vanilla (no bundlers/frameworks). For the notebook, keep it NumPy-only for the core implementation — sklearn usage is fine only in the sanity-check section.

---

📄 License

Creative Commons Attribution-NonCommercial 4.0 International (CC BY-NC 4.0)

© Umair Zia. All Rights Reserved. See LICENSE.

---

No sklearn. No shortcuts. Just math, NumPy, and a browser.