What does the network see? Images through the layers

An image goes through the network and is transformed step by step. What each step produces can be looked at.

Layer	What happens	Size (MNIST)
Input	the raw pixels	1 × 28 × 28
Conv 1 + ReLU	edges and brightness transitions	16 × 28 × 28
Pooling	halves the resolution	16 × 14 × 14
Conv 2 + ReLU	corners, arcs, pieces of pattern	32 × 14 × 14
Pooling		32 × 7 × 7
Flatten	turns it into a vector	1 568
Fully connected	combines it into class evidence	10

The hierarchy is the central point: the first layers see small local patterns, the later ones combine them into larger ones. Edges → corners → shapes → parts → objects. Nobody programmed that order; it arises from each layer seeing only the output of the previous one.

Two things get smaller, one gets larger. The height and the width shrink, while the number of channels grows. The network is trading where something is for what it is.

import torch, torch.nn as nn

net = nn.Sequential(
    nn.Conv2d(1, 16, 3, padding=1), nn.ReLU(), nn.MaxPool2d(2),   # 28 → 14
    nn.Conv2d(16, 32, 3, padding=1), nn.ReLU(), nn.MaxPool2d(2),  # 14 → 7
    nn.Flatten(), nn.Linear(32 * 7 * 7, 10),
)

image = torch.randn(1, 1, 28, 28)
x = image
for i, layer in enumerate(net):
    x = layer(x)
    share_active = float((x > 0).float().mean()) if x.dim() == 4 else float("nan")
    print(f"{i} {layer.__class__.__name__:<10} {tuple(x.shape)}  active {share_active:.2f}")
# 0 Conv2d     (1, 16, 28, 28)  active 0.50
# 1 ReLU       (1, 16, 28, 28)  active 0.50   ← half are zeroed
# 2 MaxPool2d  (1, 16, 14, 14)  active 0.76
# 3 Conv2d     (1, 32, 14, 14)  active 0.50
# ...
# 7 Linear     (1, 10)

# Look at individual channels after the first layer
captured = {}
net[0].register_forward_hook(lambda m, i, o: captured.update(conv1=o.detach()))
net(image)
a = captured["conv1"][0]                    # (16, 28, 28)
for k in range(3):
    print(f"channel {k}: mean {a[k].mean():+.3f}  max {a[k].max():.3f}")

# Which channel responds most strongly to this particular image?
print("strongest channel:", int(a.flatten(1).max(dim=1).values.argmax()))

register_forward_hook is the tool for looking inside a network without changing it. It is called every time the layer runs and gets both the input and the output — exactly what you need in order to visualise the activations.

The share of active units after ReLU is worth keeping an eye on. If it is near 0 in a layer, those neurons have «died» and contribute nothing — a sign of too high a learning rate. Around 0.3–0.6 is normal.

Look at a real network without writing any code. Open CNN Explainer and do the following, in order:

1. Click on an image. Follow it through the layers and look at how many channels there are in each.
2. Click on a square in the first conv layer. You can see which 3×3 kernel was used and how it slides across the image.
3. Compare two channels in the first layer. They respond to different things — often edges in different directions.
4. Go to the last conv layer. The images are now small and abstract; you can no longer see what they depict.
5. Change the input image and see which channels change the most.

Three questions to answer while you look:

• Why do the images get smaller but more numerous with each layer?
• Why are the last activations impossible to interpret as images?
• If the network classifies something wrongly — which layer would you start looking in?

The answer to the last is usually «the last»: the error is easiest to see in the fully connected layer, where you can see which two classes were competing. But the cause is nearly always earlier.