When the model is wrong — and who it hits
Be able to give examples of mistakes that matter (health care, school) and mistakes that do not.
Everyday explanation
An AI that suggests the wrong song for your playlist — no harm done. An AI that says a mole is harmless when it is not — serious harm.
The same kind of mistake, a very different consequence. So you have to ask: who is hit if the model is wrong, and how badly?
That is why AI in health care, schools and policing must not decide on its own. A human has to be able to look, understand and say no.
Intuition
Two kinds of mistake:
- The model says «ill» when the person is healthy → unnecessary worry and tests.
- The model says «healthy» when the person is ill → missed treatment. Worse.
When the model is unsure (say 55 % / 45 %) it ought to say so — «I don't know, ask a doctor» — instead of guessing. A good AI service is built so that uncertain cases go to a human.
Think about it: in which situations do you not want an AI deciding for itself?
Mastery means
- Gives examples of mistakes that matter a great deal and mistakes that do not
- Suggests what should happen when the model is unsure
Prerequisites
- AAccuracy: how good is the model?required
Sign in to do the exercises and build your mastery up.
Sources
- Skolverket — About AI in school (in Swedish) — Skolverket's open terms
- Wikipedia — Algoritmisk partiskhet (CC BY-SA 4.0) — CC BY-SA 4.0