ann-backprop-cpp

A simple educational implementation of the Backpropagation Algorithm for a small Artificial Neural Network (ANN) written entirely in C++.

This project reproduces the well-known 2–2–2 neural network example used to explain backpropagation step by step. ¹

---

Overview

The program demonstrates:

• Forward propagation
• Sigmoid activation
• Mean Squared Error (MSE)
• Backpropagation using the chain rule
• Gradient calculation
• Weight updates using Gradient Descent
• Error reduction after one training iteration

Everything is implemented manually without using any machine learning libraries.

---

Neural Network Architecture

The network contains: ¹

Layer Sizes

Layer	Neurons
Input	2
Hidden	2
Output	2

Architecture:

2 → 2 → 2

---

Initial Parameters

Inputs

i1 = 0.05
i2 = 0.10

Target Outputs

t1 = 0.01
t2 = 0.99

Hidden Layer Weights

w1 = 0.15
w2 = 0.20
w3 = 0.25
w4 = 0.30

Output Layer Weights

w5 = 0.40
w6 = 0.45
w7 = 0.50
w8 = 0.55

Biases

Hidden bias = 0.35
Output bias = 0.60

Learning Rate

η = 0.5

---

Activation Function

The network uses the Sigmoid activation function.

σ(x) = 1/(1+ e^-x)

Derivative:

σ'(x) = σ(x)(1 − σ(x))

---

Loss Function

The program uses Mean Squared Error (MSE).

E = (1/2)(target − output)²

Total error:

Etotal = E1 + E2

---

Algorithm

The training process consists of:

1. Forward propagation
2. Compute network outputs
3. Calculate total error
4. Compute output layer gradients
5. Compute hidden layer gradients
6. Update all weights
7. Forward propagate again
8. Observe reduced error

---

Expected Output

Initial network output:

Output1 = 0.751365070
Output2 = 0.772928465

Total Error = 0.298371109

Updated weights:

w1 = 0.149780716
w2 = 0.199561430
w3 = 0.249751140
w4 = 0.299502290

w5 = 0.358916480
w6 = 0.408666186
w7 = 0.511301270
w8 = 0.561370121

After one training iteration:

Total Error = 0.291027924

The decrease in error demonstrates that backpropagation successfully adjusted the network weights in the correct direction.

---

Build

Compile with any C++17 compatible compiler.

g++ -std=c++17 backprop.cpp -o backprop

Run:

Linux/macOS

./backprop

Windows

backprop.exe

---

Project Structure

.
├── README.md
├── LICENSE
└── backprop.cpp

---

Educational Purpose

This project is intended for learning how backpropagation works mathematically and computationally. It is not intended to be a production machine learning framework.

Everything is implemented from scratch to make each step of the algorithm easy to follow.

---

Reference

This implementation follows the classic numerical backpropagation example popularized by Matt Mazur and reproduces its forward-pass outputs, gradients, updated weights, and error reduction.