The normal distribution and standardisation

The normal distribution: a symmetric bell around μ, of width σ. 68–95–99.7: 68 % of the values within ±1σ, 95 % within ±2σ, 99.7 % within ±3σ.

The z-score = (x − μ)/σ: how many standard deviations from the mean. z = 2 is unusual (the top 2.3 %), z = 0 entirely average. With z you can compare apples and pears: 180 cm tall (z ≈ 1.2) against 95 kg (z ≈ 2) — the weight is the more unusual.

The central limit theorem: the mean of many independent values becomes normally distributed whatever the original distribution. That is why the normal is everywhere — and why standardising features (z-scores) is a standard step before training.

import numpy as np
from scipy.stats import norm

mu, sigma = 170, 8
z = (186 - mu) / sigma                 # 2.0
print(norm.cdf(z))                     # 0.977 — the share shorter than 186
print(1 - norm.cdf(z))                 # 0.023 — the share taller
print(norm.ppf(0.95) * sigma + mu)     # 183.2 — the 95th percentile

# The CLT in practice: the mean of 30 dice throws
rng = np.random.default_rng(0)
m = rng.integers(1, 7, size=(10_000, 30)).mean(axis=1)
print(round(m.mean(), 2), round(m.std(), 2))   # 3.5  0.31 ≈ 1.71/√30

Standardise the features: (X - X.mean(0)) / X.std(0) — computed on the training data.