Dataloaders, batches and epochs

An epoch = one pass through all the training data. A batch = the examples processed together before the weights are updated. An iteration = one batch.

1 000 examples with a batch size of 32 → 32 iterations per epoch (the last batch has 8).

The effect of the batch size:

Small batch (8–32)	Large batch (256+)
noisy gradients (can help generalisation)	stable gradients
less memory	more memory — often the hard limit
more updates per epoch	faster per epoch on a GPU

Shuffling every epoch matters: if the data is sorted by class the model first sees nothing but class 0, then nothing but class 1, and the gradients pull in different directions in each half.

import torch
from torch.utils.data import Dataset, DataLoader

class TableData(Dataset):
    def __init__(self, X, y):
        self.X = torch.tensor(X, dtype=torch.float32)
        self.y = torch.tensor(y, dtype=torch.long)
    def __len__(self):
        return len(self.y)
    def __getitem__(self, i):
        return self.X[i], self.y[i]

train = DataLoader(TableData(X_tr, y_tr), batch_size=32, shuffle=True, num_workers=2, drop_last=False)
val   = DataLoader(TableData(X_va, y_va), batch_size=256, shuffle=False)

for epoch in range(3):
    model.train()
    for xb, yb in train:                  # xb: (32, features)
        loss = criterion(model(xb), yb)
        loss.backward(); opt.step(); opt.zero_grad()
    model.eval()
    with torch.no_grad():
        acc = sum((model(xb).argmax(1) == yb).sum().item() for xb, yb in val) / len(val.dataset)
    print(epoch, round(acc, 3))

Three common mistakes: shuffle=True on the validation data (pointless, and it makes comparison harder), forgetting model.eval() and torch.no_grad() at evaluation (dropout still active and memory wasted), and too large a batch giving «CUDA out of memory» — solve that with gradient accumulation rather than by lowering the lr.