Neural Network for Differential Equations

Course work (4th semester). A feedforward neural network that approximates solutions to simple differential equations. Scientific director: Kasatkin A.A.

The project is a C++ implementation built with clang and C++17, following the Google C++ Style Guide. Network configuration and training data are read from text files in the repository root.

How It Works

Architecture

The network is a fully connected multilayer perceptron:

1. Input layer — training samples (e.g. coordinates or parameters).
2. Hidden layers — optional; count and size are defined in Nodes.txt.
3. Output layer — target values the network should predict.

Each layer is a vector of neuron activations. Connections between adjacent layers are stored as weight matrices (NetMatrix). Forward propagation:

next_layer = sigmoid(weights * current_layer)

The sigmoid activation function maps values to the range (0, 1).

Training (backpropagation)

When training is enabled via --train, the network:

1. Runs forward pass for one training example.
2. Computes output error: target - output.
3. Propagates errors backward through hidden layers.
4. Updates weights with gradient descent: weights -= learning_rate * gradient.

The learning rate is set via set_learn_coef().

Weight update derivation

For a single output neuron with two inputs, the loss is:

E = ½·(O₁ − T₁)²

where O₁ = sigmoid(I₁·W₁₁ + I₂·W₂₁). Differentiating with respect to W₁₁:

∂E/∂W₁₁ = (O₁ − T₁) · O₁·(1 − O₁) · I₁

The gradient descent update becomes:

W₁₁ ← W₁₁ − lr · (O₁ − T₁) · O₁·(1 − O₁) · I₁

In code (neural_network.cc, train()), errors = target − output = T − O, so:

gradient = (O − 1) * (T − O) * O * lr   // = O·(1−O)·(O−T)·lr
weights  -= gradient ⊗ prev_layer        // equivalent to the formula above

Using ½·(O−T)² as the loss eliminates the factor of 2 that would appear with (O−T)²; the result is numerically equivalent — the constant is absorbed into the learning rate.

Inference (query mode)

In query mode (default), the network loads saved weights, runs forward pass for each input example, prints results to the console, and writes the last result to output_layer.txt.

Project Structure

.
├── include/
│   ├── neural_net.h        # Abstract interface for neural networks
│   ├── neural_network.h    # Neural network class declarations
│   ├── layer.h             # Layer class declarations
│   └── net_matrix.h        # Matrix class declarations
├── src/
│   ├── main.cc             # Entry point: load data, train or query
│   ├── layer.cc            # Layer implementations
│   ├── net_matrix.cc       # Matrix implementations
│   └── neural_network.cc   # Neural network implementations
├── Makefile                # Build with clang++
├── Nodes.txt               # Layer sizes (input ... hidden ... output)
├── input_layer.txt         # Training/query inputs
├── targets.txt             # Expected outputs (used for training)
├── weights.txt             # Saved weight matrices
└── output_layer.txt        # Last inference result (generated)

Data File Formats

Nodes.txt

Space-separated neuron counts per layer, left to right:

3 1

Example above: 3 inputs, 1 output (no hidden layer). A deeper network might look like 10 20 10 2.

input_layer.txt

Values grouped by input layer size. For 3 inputs, every 3 lines form one example:

0
0
1

Blank lines between groups are allowed.

targets.txt

One target value per line for each training example (size = output layer size). For a single output neuron:

0
0.1
0.2

weights.txt

Weight matrices between layers, row by row. Each row contains num_cols() values; matrices are separated by blank lines. Loaded with sync_weights(); saved with save_weights() after training.

Build and Run

Requirements: clang with C++17 support (macOS Xcode CLI tools or LLVM).

From the repository root:

make
./neuralnet --inference

Or run in one step:

make run

Command-line options

./neuralnet --inference          # run inference (default)
./neuralnet --train              # train with default 1000 epochs
./neuralnet --train --epochs 10000

Option	Description
--inference	Load weights, run forward pass on all inputs, print and save output (default)
--train	Train the network and save updated weights to weights.txt
--epochs N	Number of training iterations (default: 1000; only used with --train)

Manual build:

clang++ -std=c++17 -Wall -Wextra -O2 -Iinclude -o neuralnet \
  src/main.cc src/layer.cc src/net_matrix.cc src/neural_network.cc

Clean build artifacts:

make clean

Run the binary from the repository root so relative paths to data files resolve correctly.

Example Workflow

1. Edit Nodes.txt to define network topology.
2. Prepare input_layer.txt and targets.txt with matching example counts.
3. Train: ./neuralnet --train --epochs 10000
4. Evaluate: ./neuralnet --inference

Class	Role
NeuralNet	Virtual interface: weights init, input/target loading, query
NeuralNetwork	Concrete network: sigmoid, backprop, file I/O
Layer	Neuron values with +, -, * operations for training math
NetMatrix	Dense matrix: random init, multiply, transpose, element access

Code Style

The codebase follows Google C++ conventions:

• 2-space indentation, 80-column-friendly formatting
• snake_case for functions and variables, PascalCase for types
• Trailing underscore for member variables (learn_coef_, query_mode_)
• constexpr constants instead of macros
• Include guards (#ifndef / #define) in headers
• Declarations in include/, implementations in src/
• const correctness and explicit constructors where appropriate

Class Overview