Functions: domain, composition, inverse

A function takes a value in and gives exactly one value out. Three things to keep track of:

• The domain: which x are allowed. For f(x) = 1/x it is every x ≠ 0. For √x it is x ≥ 0.
• The range: which values can come out. For x² it is y ≥ 0.
• Composition: (f∘g)(x) = f(g(x)) — run g first, then f. The order matters.

The inverse f⁻¹ does the opposite: if f(3) = 10 then f⁻¹(10) = 3. It exists only if the function is one-to-one — no two x give the same y. x² has no inverse on the whole of ℝ (both 2 and −2 give 4), but on x ≥ 0 the inverse is √x.

import math

f = lambda x: 2 * x + 1          # f(x) = 2x + 1
g = lambda x: x ** 2             # g(x) = x²

print(f(g(3)))                   # f(9) = 19
print(g(f(3)))                   # g(7) = 49   ← the order matters

f_inv = lambda y: (y - 1) / 2    # the inverse of f
print(f_inv(f(4)))               # 4.0

Why this is the foundation of neural networks: a network is a composition of functions:

output = f₃(f₂(f₁(x)))

Every layer is a function. Training the network requires the derivative of the whole composition with respect to each layer's parameters — and the rule for differentiating composed functions is the chain rule. That is exactly what backpropagation computes, layer by layer from the back.