Linear regression: fitting a straight line to data
Be able to fit a straight line to data points, explain what 'error' (loss) means, and understand that training means minimising the error.
Intuition
You have measured how many cups of coffee the café sells at different temperatures: (15 °C, 20 cups), (20 °C, 35 cups), (25 °C, 50 cups). It looks like a line. If you find the line, you can predict sales at 30 °C.
A model here is simply sales = k · temperature + m. "Training the model" means:
find k and m so the line fits the points as well as possible.
Formal
How well does a line fit? Calculate the error for each point: actual value − line's prediction. Square the errors (so positive and negative values do not cancel out, and large errors are penalised heavily) and take the average. This is called MSE, mean squared error, or loss.
With k = 3, m = −25 on the coffee points:
| temp | actual | prediction | error | error² |
|---|---|---|---|---|
| 15 | 20 | 20 | 0 | 0 |
| 20 | 35 | 35 | 0 | 0 |
| 25 | 50 | 50 | 0 | 0 |
Loss = 0 — perfect. With real data, it is never zero; you seek the minimum. How to find the best k and m automatically is the next node: gradient descent.
Mastery means
- Calculates the prediction and error for a given line and data point
- Explains why errors are squared
Prerequisites
Sign in to do the exercises and build your mastery up.
Sources
Part of the goals (33)
- Training neural networks for real
- Language models in practice
- Train an agent with reward
- Classical machine learning in practice
- Classical ML for real
- Understand how generative AI works
- Image classification with convolutional networks
- Deep reinforcement learning
- Run models more cheaply: quantisation
- Fine-tune a model with LoRA
- Train your first neural network
- AI safety in practice
- Fine-tune and run your own models
- Interpreting a language model
- Generative models in depth
- Build a voice interface
- Responsible AI in practice
- Build a transformer from scratch
- Seeing and hearing with AI
- Frontier Lab — an independent research project
- Multimodal systems
- Data: collect, clean, document
- Build a RAG pipeline with traceable sources
- Reproduce a paper
- Statistics for experiments
- Build an NLP system end to end
- Evals in practice
- AI in production
- Build a memory system for an agent
- Build an AI service that survives production
- Build an agent you can trust
- An AI service in operation
- AI, ethics and society