Children as users of AI

Products for children have stricter rules and higher requirements — and they apply as soon as children actually use the service, whatever the terms say.

Consent and age in Sweden: GDPR article 8 applies to information society services offered directly to children. Sweden has set the age limit at 13 — below that, a guardian's consent is required. Above it the child can consent themselves, but the information has to be comprehensible to a child.

Four requirements that are stricter for children:

Requirement	Means
Clear that it is a machine	not hidden in a footnote
No dark patterns	nothing that artificially prolongs the use
Moderation in both directions	both what the child writes and what the model answers
A route to an adult	visible, always

The third takes the most work and is the one most often done half-heartedly.

Moderation in both directions is the core of a child-safe AI service:

Direction	What gets caught	What happens
Incoming	personal data the child writes about themselves or others	stop it, explain why, do not store it
Incoming	signs of vulnerability, self-harm, abuse	escalate to a human following a documented procedure
Outgoing	inappropriate content, violence, sexual content, dangerous instructions	block it before it is shown
Outgoing	advice that needs an adult (medicine, law, relationships)	refer them to an adult

The hardest trade-off is between stopping and pushing away. A child who tells you about something difficult should not be met with «I can't talk about that» — it may be their only attempt to tell somebody. The procedure has to be: acknowledge, do not break the conversation off abruptly, and guide them on to somebody who can help.

That is a design and a policy question together, and it has to be thought through before launch, not when the first case turns up.

Dark patterns that are particularly problematic for children:

Pattern	Why
Streaks and daily rewards	create obligation, not learning
Infinite scroll or autoplay	remove the natural stopping points
Social comparison	applies pressure, especially to those falling behind
Artificial urgency	«the offer expires in 2 hours»
Being hard to quit	it should be as easy as starting

Age adaptation in practice is about more than a content filter: the language level, the sentence length, how much is explained, and which examples are used. A ten-year-old and an upper-secondary pupil need different answers to the same question.

AI-grafen's level system A–G does this structurally: the level governs both which nodes are offered and how the tutor phrases things.

Documentation that has to exist before launch:

Document	Contents
A DPIA with a child perspective	the risks specific to children
A moderation policy	what is stopped, how, and what is logged
An escalation procedure	who is contacted, within what time, when there is concern
Information for guardians	in comprehensible Swedish
Information for the child	in even simpler language

The Convention on the Rights of the Child has been Swedish law since 2020. The best interests of the child are to be taken into account in every decision concerning children — which includes product decisions. That is not a recommendation.

Review a service with ten questions. Use them on something a child you know actually uses.

#	Question
1	Is it clearly stated that it is a machine?
2	What happens if the child types in their name and address?
3	What happens if the child tells it about something difficult at home?
4	Are the model's answers moderated, or only the child's input?
5	Is there a visible route to an adult?
6	Are there streaks, rewards or anything else pulling the child back?
7	How easy is it to quit and delete everything?
8	Is the language adapted to the age?
9	What is stored, and for how long?
10	Who has tested the service with children in the target group?

Questions 3 and 4 are the ones that separate well-considered services from the rest.

For question 3 there are three possible behaviours, and only one is right:

Behaviour	The verdict
«I can't talk about that» and changes the subject	bad — the child is pushed away
Gives advice about a serious situation	bad — the model is not qualified
Acknowledges it, says that sounds hard, and guides them to an adult or a helpline	right

Question 10 is the most underrated. A service for children that has never been tested with children will have problems no adult foresaw — children use things in ways nobody designed for.