Skip to content
AI-grafen
Sign in
☰ Menu

The goal F AI engineering

Interpreting a language model

Attention patterns, the logit lens and explanations for end users — how to see what a model is doing, and where the limits of what an explanation proves lie.

Knowledge nodes
66
From zero
about 48 h
Labs
6
See what you already know — no account

The diagnostic removes what you already know, so your path is usually much shorter.

What you can do afterwards

Labs along the way

You write the code. Tests you cannot see decide whether it holds up.

Lab: a neural network in pure NumPy — forward, backprop, gradient checkDa sandbox · about 75 minLab: dot product, norm and cosine similarityDin the browser · about 40 minLab: gradient descent from scratchDin the browser · about 50 minLab: matrix multiplication and one layer of a neural networkDin the browser · about 45 minLab: scaled dot-product attention with a causal maskDa sandbox · about 60 minLab: semantic search with embeddingsDa sandbox · about 45 min

The whole path

Everything the goal builds on, grouped by level and in the order it builds on itself. Show on the map

AExplorer4 knowledge nodes
  1. Patterns and categories
  2. Sequences and precise instructions
  3. Integer arithmetic and order of operations
  4. Comparing and sorting values
BInvestigator9 knowledge nodes
  1. Data in everyday life
  2. Algorithmic thinking
  3. Rule-based systems and machine learning
  4. Source criticism and responsibility in AI use
  5. Binary numbers and bits
  6. Fractions, decimals, and percentages
  7. Coordinates: positions on a grid
  8. Training data, features and labels
  9. Classification: how a model sorts information
CBuilder18 knowledge nodes
  1. Functions and coordinate systems
  2. Programming logic — variables, conditions, loops
  3. Python — the basics
  4. Python — lists, loops and dictionaries
  5. Statistics — mean, median and spread
  6. Probability — the basics
  7. Digital representation: text, images and audio
  8. Proportionality and scale
  9. Linear relationships in tables
  10. Linear regression: fitting a line to data
  11. Linear regression: fitting a straight line to data
  12. Neural networks — the intuition
  13. Prompting — steering a language model
  14. Responsible use of AI
  15. Images as matrices
  16. Turn the knobs: weights
  17. Why multiple layers are needed
  18. Training a neural network in the browser
DAI developer19 knowledge nodes
  1. Derivatives and optimisation
  2. Loss functions
  3. Tokenisation
  4. Gradient descent
  5. Vectors
  6. Matrices and matrix multiplication
  7. Linear regression with several features
  8. Logistic regression and decision trees
  9. Neural networks — the forward pass with matrices
  10. Backpropagation
  11. What does the network see? Images through the layers
  12. Embeddings — words as vectors
  13. Attention
  14. Overfitting and generalisation
  15. PyTorch — tensors and autograd
  16. Training, validation and test
  17. Train a neural network in PyTorch
  18. Transformers — the architecture
  19. Looking inside the network
EUniversity12 knowledge nodes
  1. Explainability for end users
  2. Linear maps
  3. Eigenvalues and eigenvectors
  4. Matrix factorisation and low-rank approximation
  5. Visualising attention patterns
  6. Singular value decomposition (SVD)
  7. PCA — principal component analysis
  8. Autoencoders
  9. Multi-head attention in detail
  10. Language models — training and generation
  11. Model evaluation
  12. Scientific method in AI
FAI engineering1 knowledge nodes
  1. Activations and linear probes
GFrontier Lab3 knowledge nodes
  1. Mechanistic interpretability — the basics
  2. The logit lens and the residual stream
  3. Sparse autoencoders for features