Probability distributions

A distribution says how likely the different outcomes are.

• Uniform: all the outcomes equally likely — a die, a random number from 0 to 1.
• Binomial: the number of «successes» out of n independent trials with probability p — the number of sixes in 10 throws, the number correct out of 20 guesses. The expectation is n·p.
• Normal: the bell curve — the sum of many small independent contributions (heights, measurement errors, the means of samples). It is described entirely by the mean μ and the standard deviation σ.

The ML connection: a model's initial weights are drawn from a normal or a uniform distribution; classification is the binomial in disguise; the test error over many examples is approximately normally distributed.

import numpy as np
rng = np.random.default_rng(0)

t = rng.integers(1, 7, size=10_000)              # uniform: a die
print(np.bincount(t)[1:] / 10_000)                # ≈ 0.167 each

b = rng.binomial(n=10, p=1/6, size=10_000)        # the number of sixes in 10 throws
print(b.mean(), 10 * 1/6)                          # ≈ 1.67  1.67

x = rng.normal(loc=170, scale=8, size=10_000)     # heights
print(np.mean((x > 162) & (x < 178)))              # ≈ 0.68 (±1σ)

# the binomial probability exactly:
from math import comb
print(comb(10, 2) * (1/6)**2 * (5/6)**8)          # P(exactly 2 sixes) ≈ 0.29