When the model is wrong — and who it hits
Be able to give examples of mistakes that matter (health care, school) and mistakes that do not.
Prerequisites
- AAccuracy: how good is the model?required
Everyday explanation
An AI that suggests the wrong song for your playlist — no harm done. An AI that says a mole is harmless when it is not — serious harm.
The same kind of mistake, a very different consequence. So you have to ask: who is hit if the model is wrong, and how badly?
That is why AI in health care, schools and policing must not decide on its own. A human has to be able to look, understand and say no.
Intuition
Two kinds of mistake:
- The model says «ill» when the person is healthy → unnecessary worry and tests.
- The model says «healthy» when the person is ill → missed treatment. Worse.
When the model is unsure (say 55 % / 45 %) it ought to say so — «I don't know, ask a doctor» — instead of guessing. A good AI service is built so that uncertain cases go to a human.
Think about it: in which situations do you not want an AI deciding for itself?
Mastery means
- Gives examples of mistakes that matter a great deal and mistakes that do not
- Suggests what should happen when the model is unsure
Sign in to do the exercises and build your mastery up.
Sources
- Skolverket — About AI in school (in Swedish) — Skolverket's open terms
- Wikipedia — Algoritmisk partiskhet (CC BY-SA 4.0) — CC BY-SA 4.0