nn

This readme is mostly generated automatically - yes I requested humour. I started this to learn NN by doing. Similar to how rl_gridworld experiments. RL_gridworld was exploding in state-action pairs so it told me to try DQN, so I decided to first learn NNs and then try DQN on gridworlds.

I have put my learnings in learnings.md. This part is self-written and probably the most useful thing.

TL;DR (how to run)

./run.sh

That one-liner will:

1. conjure a virtual environment
2. pip-install the meaning of life (and torch)
3. let nn.py contemplate the decimal system

"In a world where calculators exist, one neural network dared to dream..."

Sometimes you have to throw 2 billion FLOPs at a problem solvable on a napkin. It builds character—and weight matrices.

"This project is a gentle reminder that sometimes the most elegant solution is to throw a neural network at a problem that could be solved with a single line of code. But where's the fun in that?"

Flexible Polynomial System

The project now includes a flexible polynomial system that can create and train on polynomial datasets of any order. Here's how it works:

Key Features

• Configurable Coefficients: Polynomial coefficients are stored in an array/list instead of individual variables
• Any Degree: Support for polynomials of any degree (linear, quadratic, cubic, quartic, etc.)
• Automatic Model Configuration: The PolynomialModel automatically adjusts to the configured polynomial degree
• Comprehensive Evaluation: Specialized evaluation functions for polynomial relationships

Usage Examples

Basic Usage

from dataset import set_polynomial_coefficients, create_dataset, create_model

# Set coefficients for y = 2 + 3x + 4x² (quadratic)
set_polynomial_coefficients([2, 3, 4])

# Create dataset and model
dl, x_train, y_train = create_dataset('polynomial')
model = create_model('polynomial')  # Automatically uses degree 2

Different Polynomial Degrees

# Linear: y = 10 + 5x
set_polynomial_coefficients([10, 5])

# Cubic: y = 1 + 2x + 3x² + 4x³
set_polynomial_coefficients([1, 2, 3, 4])

# 4th degree: y = 5 + 2x + x² + 0.5x³ + 0.1x⁴
set_polynomial_coefficients([5, 2, 1, 0.5, 0.1])

Running Examples

# Run the main training with default quadratic polynomial
python nn.py

# Run comprehensive examples with different polynomial degrees
python polynomial_examples.py

Configuration

The polynomial coefficients are configured in dataset.py:

# Coefficients for polynomial: y = a₀ + a₁x + a₂x² + a₃x³ + ... + aₙxⁿ
# The coefficients are in ascending order: [constant, linear, quadratic, cubic, ...]
POLYNOMIAL_COEFFICIENTS = [88, 5, 24]  # Default: y = 88 + 5x + 24x²

Model Architecture

The PolynomialModel explicitly includes polynomial terms in its architecture:

• Learns coefficients for each polynomial term (x⁰, x¹, x², x³, ...)
• Automatically adjusts the number of parameters based on polynomial degree
• Provides interpretable coefficients that directly correspond to the polynomial terms

Evaluation

The system includes specialized evaluation functions:

• verify_predictions_polynomial(): Polynomial-specific metrics (MAE, RMSE, R²)
• print_model_summary(): Detailed coefficient analysis and comparison
• Coefficient-wise accuracy assessment

MPS (Metal Performance Shaders) Test Script

This repository contains a test script (test_mps.py) to verify and benchmark the Metal Performance Shaders (MPS) backend for PyTorch on Apple Silicon (M1/M2/M3) Macs.

Running the Tests

python test_mps.py

Expected Results

On an Apple Silicon Mac, you should see:

• MPS being available and built into PyTorch
• Basic operations successfully running on the MPS device
• MPS (GPU) operations being significantly faster than CPU operations for large matrix multiplications