Neural networks — the intuition

A straight line cannot describe everything. Ice cream sales may level off when it gets too hot. A neural network is a way of building bent, complicated relationships out of many simple parts.

Every neuron does three things: it multiplies each input by a weight, sums them, and sends the sum through a simple "bend" (for instance everything below zero becomes zero). Put neurons into layers and connect layer after layer — and you have a network.

A neuron with inputs (2, 3), weights (0.5, −1) and bias 1:

sum = 2·0.5 + 3·(−1) + 1 = 1 − 3 + 1 = −1 → the bend (ReLU) turns −1 into 0. Output: 0.

Why the bend? Without it every layer is just another straight line, and a line of a line is still a line. The bend is what lets the network learn curves, corners and "if-then" patterns.

Training = adjusting all the weights so that the error becomes small. Exactly as for the line — only with thousands (or billions) of weights instead of two.