Diffusion models

The forward process destroys an image: add a little Gaussian noise, over and over, in T steps until only noise remains. It is fixed and requires no learning.

The reverse process is what the model learns: given a noisy image at step t, predict the noise that was added. If you can do that you can take a step back towards a cleaner image. Repeat from pure noise and an image emerges.

The training is surprisingly simple: draw an image, draw a random t, add the corresponding noise, let the network guess the noise, and measure with MSE. No discriminator, no instability — which is why diffusion beat GANs.

Forward: $q(x_t\mid x_{t-1}) = \mathcal N(\sqrt{1-\beta_t}\,x_{t-1},\ \beta_t I)$. With $\alpha_t = 1-\beta_t$ and $\bar\alpha_t = \prod_{s\le t}\alpha_s$ you can jump straight to any step: $x_t = \sqrt{\bar\alpha_t}\,x_0 + \sqrt{1-\bar\alpha_t}\,\epsilon,\qquad \epsilon\sim\mathcal N(0,I)$

The training objective (Ho et al. 2020) — surprisingly, just an MSE: $\mathcal L = \mathbb E_{x_0,\epsilon,t}\big[\|\epsilon - \epsilon_\theta(x_t, t)\|^2\big]$

Sampling: start at $x_T\sim\mathcal N(0,I)$ and iterate backwards. DDPM uses all T steps (often 1 000); DDIM makes the process deterministic and manages 20–50 steps at almost the same quality.

Classifier-free guidance: train the model both with and without the text condition (the condition is dropped at random in ~10 % of cases) and extrapolate at sampling time: $\hat\epsilon = \epsilon_\theta(x_t,\varnothing) + w\big(\epsilon_\theta(x_t,c) - \epsilon_\theta(x_t,\varnothing)\big)$ A higher $w$ gives stronger adherence to the prompt but less variation — the parameter users know as the «guidance scale».

Latent diffusion (Stable Diffusion) runs the whole process in a compressed latent space from an autoencoder, which cuts the computation ~50× and is why image generation runs on consumer hardware.

import torch, torch.nn as nn

T = 1000
betas = torch.linspace(1e-4, 0.02, T)
alphas = 1.0 - betas
abar = torch.cumprod(alphas, dim=0)

def add_noise(x0, t, eps=None):
    """Jump straight to step t."""
    eps = torch.randn_like(x0) if eps is None else eps
    a = abar[t].view(-1, 1, 1, 1)
    return a.sqrt() * x0 + (1 - a).sqrt() * eps, eps

def train_step(model, x0, opt):
    t = torch.randint(0, T, (x0.size(0),))
    xt, eps = add_noise(x0, t)
    loss = nn.functional.mse_loss(model(xt, t), eps)      # predict the noise
    loss.backward(); opt.step(); opt.zero_grad()
    return loss.item()

@torch.no_grad()
def sample(model, shape, steps=50):
    x = torch.randn(shape)
    for t in reversed(range(0, T, T // steps)):           # DDIM-like jumps
        tt = torch.full((shape[0],), t, dtype=torch.long)
        eps = model(x, tt)
        a = abar[t]
        x0_hat = (x - (1 - a).sqrt() * eps) / a.sqrt()
        a_prev = abar[max(t - T // steps, 0)]
        x = a_prev.sqrt() * x0_hat + (1 - a_prev).sqrt() * eps
    return x