Classification: how a model sorts information
Be able to explain what a classifier does, read how well it performs, and reason about why it makes mistakes on certain examples.
Everyday explanation
A classifier is a model that sorts information into categories. For example: spam or important email, approved application or rejected, or whether a transaction is legitimate or fraud. The model has been trained on previous examples and now guesses on new cases it has never seen.
How good is it? Test it on examples where you already know the answer, but which the model has not been trained on. If it gets 90 out of 100 cases right, the accuracy is 90 %.
Intuition
There are two types of errors, and they have different consequences:
- The model marks an important email as spam. You miss it. This is a false positive error.
- The model lets spam through as a normal email. Your inbox becomes full of junk. This is a false negative error.
Which error is worse depends on the context. A model that detects diseases would rather alarm unnecessarily (false positive) than miss a sick patient (false negative). Accuracy alone does not tell the whole story about the risk.
Mastery means
- Calculates accuracy from the number of correct predictions and the total number
- Explains the difference between two types of errors
Prerequisites
Sign in to do the exercises and build your mastery up.
Sources
Leads to
Part of the goals (34)
- AI for beginners
- Classical machine learning in practice
- AI, ethics and society
- Data: collect, clean, document
- Classical ML for real
- Build a RAG pipeline with traceable sources
- Train your first neural network
- Responsible AI in practice
- Interpreting a language model
- AI safety in practice
- Build a memory system for an agent
- Build an agent you can trust
- Build an AI service that survives production
- An AI service in operation
- Language models in practice
- AI in production
- Multimodal systems
- Training neural networks for real
- Train an agent with reward
- Build an NLP system end to end
- Fine-tune and run your own models
- Frontier Lab — an independent research project
- Evals in practice
- Understand how generative AI works
- Image classification with convolutional networks
- Deep reinforcement learning
- Run models more cheaply: quantisation
- Fine-tune a model with LoRA
- Reproduce a paper
- Generative models in depth
- Build a voice interface
- Build a transformer from scratch
- Seeing and hearing with AI
- Statistics for experiments