How a Neural Network Learns

An interactive, real-time visualizer designed to demystify neural networks, backpropagation, and gradient descent.

Interactive Controls
In A In B Target Prediction Error
Live Network Architecture
INPUTS HIDDEN LAYER (SIGMOID) OUTPUT (SIGMOID)

Interactive Inspector

Hover or tap any neuron or weight in the diagram above to inspect its real-time mathematical operations, gradient directions, and role in learning.

Learning Metrics
0
Epochs Trained
0.0000
Mean Squared Error
Loss: 0.25
Loss: 0.00
Goal: Minimize the loss curve. A lower loss means the network's predictions are matching the targets.
Neural Network Masterclass

A neural network is a collection of mathematical "neurons" organized in layers. In this interactive demo, we have three layers:

  • Input Layer: Receives raw inputs (A and B). These represent features of our data.
  • Hidden Layer: Detects intermediate patterns. These neurons use a non-linear activation function (Sigmoid) to combine inputs. Without hidden layers, networks can only solve linear problems.
  • Output Layer: Produces the final prediction. In our case, a value between 0 and 1.

The arrows connecting neurons represent weights (w), which acts as multipliers. The bias (b) acts like an adjustable offset threshold for each neuron.

During a Forward Pass, data flows from left to right to create a prediction:

1. For any neuron, we sum all incoming inputs multiplied by their respective weights, and then add the neuron's bias:

Sum (z) = (Input₁ × Weight₁) + (Input₂ × Weight₂) + Bias

2. We then pass this sum z through an activation function, like the Sigmoid function:

Activation (a) = σ(z) = 1 / (1 + e⁻ᶻ)

This squashes the value beautifully between 0 and 1, representing the neuron's "firing rate" or confidence.

How do we know if our network is doing a good job? We calculate its Loss (or cost). In this demo, we use Mean Squared Error (MSE):

Error = ½ × (Target - Prediction)²

The higher the difference between what the network predicted and the actual target, the higher the error. Our global objective is to find a set of weights and biases that makes this error as close to zero as possible across all training examples.

Backpropagation is the magic behind learning. It calculates how much each individual weight and bias contributed to the final error. It does this using calculus (the chain rule), flowing the error backwards from right to left:

  • We check the output error: Prediction - Target.
  • We calculate how much nudging each weight would change that error. This derivative is called the gradient.
  • We update every weight and bias by subtracting a tiny fraction of its gradient, controlled by the Learning Rate (η):
New Weight = Weight - (Learning Rate × Gradient)

As you click "Train", watch how the weights adjust and the loss curve trends downwards! If the learning rate is too low, progress is slow. If it's too high, the network might overshoot and fail to learn.