Integrals — the basics

The derivative answers «how fast is it changing?». The integral answers «how much did it come to in total?».

If you drive at 80 km/h for 2 hours you have driven 160 km — speed integrated over time gives distance. Graphically it is the area under the speed curve.

The basic rule: ∫xⁿ dx = xⁿ⁺¹/(n+1) + C (for n ≠ −1).

The definite integral: ∫ₐᵇ f(x) dx = F(b) − F(a), where F is an antiderivative.

An example: ∫₀² x² dx = [x³/3]₀² = 8/3 − 0 = 8/3 ≈ 2.67.

The fundamental theorem of calculus: if $F'(x) = f(x)$ then $\int_a^b f(x)\,dx = F(b) - F(a)$. Differentiation and integration are each other's opposites.

The integral is defined as the limit of a Riemann sum: split the interval into $n$ strips of width $\Delta x$, sum $f(x_i)\Delta x$, let $n\to\infty$.

Why it matters in AI:

• A probability density is integrated to give probabilities: $P(a < X < b) = \int_a^b p(x)\,dx$, and $\int_{-\infty}^{\infty} p(x)\,dx = 1$.
• The expectation: $\mathbb E[X] = \int x\,p(x)\,dx$ — the basis of nearly every loss function, which are expectations over the data distribution.
• In practice most integrals in ML are approximated with averages over samples (Monte Carlo), since they can rarely be computed exactly.

import numpy as np
from scipy import integrate

f = lambda x: x**2
print(integrate.quad(f, 0, 2)[0])          # 2.6667  (= 8/3)

# A Riemann sum by hand — the same answer as n grows
for n in (10, 100, 10_000):
    x = np.linspace(0, 2, n, endpoint=False)
    print(n, round(float((f(x) * (2 / n)).sum()), 4))
# 10     2.3080
# 100    2.6268
# 10000  2.6663

The Monte Carlo variant: draw random x, take the mean of f(x), multiply by the length of the interval. That works even in a hundred dimensions, where strips become impossible.