Designing for the AI being wrong

An AI feature is sometimes wrong. That is not a bug to be fixed away — it is a property the interface has to be designed for.

Four questions every AI feature should answer in the design:

1. How does the user notice that it went wrong?
2. How easy is it to correct?
3. What happens when the feature does not work at all?
4. Is the user encouraged to check, or to trust?

A common anti-pattern: a finished, confident answer with no source, no way to edit it, and no way forward if it is wrong. Then the design has made the error the user's problem without giving them any tools.

The opposite: a suggestion that can be edited, with the source beside it, and a button for «do it manually instead».

Patterns that work, and what each one solves:

Pattern	Solves
A suggestion, not a decision	the user keeps control and responsibility
An editable result	correcting costs a second instead of starting over
The source beside the answer	checking is one click away
Uncertainty in words	«I am unsure here» says more than «0.62»
Undo, always	it lowers the threshold for trying
A route past the AI	the feature never becomes a dead end
Clear AI labelling	a requirement in the EU AI Act for synthetic content

Anti-patterns:

Anti-pattern	Why it does harm
A confidence figure without an explanation	«87 % sure» of what?
An explanation that is a post hoc construction	it increases trust without increasing reliability
Hidden AI use	the user cannot calibrate their trust
Harder to correct than to accept	the design chooses for the user

A sketch in three states. Every AI feature should be drawn in three states before it is built:

┌─ IT WORKS ─────────────┐  ┌─ UNCERTAIN ────────────┐  ┌─ BROKEN ───────────────┐
│ A suggestion:          │  │ A suggestion (unsure): │  │ Could not create a     │
│ ┌────────────────────┐ │  │ ┌────────────────────┐ │  │ suggestion just now.   │
│ │ editable text      │ │  │ │ editable text      │ │  │                        │
│ └────────────────────┘ │  │ └────────────────────┘ │  │ [Write it yourself]    │
│ The source: node X  ⓘ  │  │ ⚠ Check it against the │  │ [Try again]            │
│ [Use] [Change]         │  │   source before you    │  │ [Contact the teacher]  │
│ 👍 👎                  │  │   use this.            │  │                        │
└────────────────────────┘  └────────────────────────┘  └────────────────────────┘

The third state is the one most often forgotten — and the one that decides whether the user can get anything done on a bad day.

Particularly for children and young people: it should be clear that it is a machine, no dark patterns that prolong the use, and a visible route to an adult. Those are both design principles and requirements in the AI Act and data protection law.

Sketch a real feature. Take AI-grafen's «suggest the next exercise» and draw it in the three states above — on paper, that is faster.

Then answer the seven questions:

#	The question	Your answer
1	Is it clear that the suggestion comes from AI?
2	What does the pupil see when the system is unsure?
3	How does the pupil change the suggestion?
4	What happens if the service is down?
5	Can the pupil see why the suggestion was made?
6	Is there a route to a person?
7	Is the pupil encouraged to think for themselves?

Then test the sketch on somebody else with three tasks:

1. «Use the suggestion.»
2. «The suggestion is wrong — do something else instead.»
3. «Find out why the system suggested this.»

Tasks 2 and 3 are the ones that reveal the problems. If the test person hesitates for more than a few seconds the design is unclear — and that is much cheaper to discover on paper than after the feature is built.

A final check: go back to the sketch and count the number of clicks to accept against the number to correct. If there are more to correct, the design has taken a position for the user.