The summation sign and indices

Σ (capital sigma) simply means «add up». Everything around it says what is to be added up and how many times.

$\sum_{i=1}^{4} i^2 = 1^2 + 2^2 + 3^2 + 4^2 = 30$

Σ is a for loop. That is the whole secret:

total = 0
for i in range(1, 5):     # 1 up to and including 4
    total += i ** 2

When you see a formula with Σ in a paper — read it as a loop, and it immediately becomes less frightening.

Common variants you will run into:

Rules that get used all the time:

$\sum_i (a_i + b_i) = \sum_i a_i + \sum_i b_i, \qquad \sum_i c\,a_i = c\sum_i a_i$

A constant that does not depend on the counter can be taken outside. A constant inside the sum gets multiplied by the number of terms: $\sum_{i=1}^{n} c = nc$.

Double indices. A matrix is summed over rows and columns:

$\sum_{i=1}^{m}\sum_{j=1}^{n} a_{ij}$

The inner sum runs to completion for every value of the outer one — exactly like nested loops. If the limits are independent of each other, the order may be swapped freely.

Three formulas you will recognise if you read ML:

All three are «add something up for every data point and divide by the count». Once you have seen that, the notation is no longer in the way.

# Σ and a for loop are the same thing
total = 0
for i in range(1, 5):
    total += i ** 2
print(total)                                   # 30
print(sum(i ** 2 for i in range(1, 5)))        # 30 — the same thing, shorter

# The mean: (1/n) Σ x_i
x = [3, 7, 7, 2, 11]
print(sum(x) / len(x))                         # 6.0

# The dot product: Σ a_i · b_i
a, b = [1, 2, 3], [4, 5, 6]
print(sum(ai * bi for ai, bi in zip(a, b)))    # 32

# Double indices: ΣΣ a_ij — nested loops
A = [[1, 2, 3],
     [4, 5, 6]]
print(sum(A[i][j] for i in range(2) for j in range(3)))   # 21

# The order does not matter when the limits are independent
print(sum(sum(row) for row in A))                          # 21 — row by row
print(sum(sum(A[i][j] for i in range(2)) for j in range(3)))  # 21 — column by column

# MSE — the formula you will see most often of all
y     = [3.0, 5.0, 2.0, 8.0]
y_hat = [2.5, 5.5, 2.0, 7.0]
mse = sum((yi - hi) ** 2 for yi, hi in zip(y, y_hat)) / len(y)
print(round(mse, 4))                            # 0.375