The EU AI Act

The EU AI Act (2024/1689) is risk-based: the requirements follow from what the system is used for, not from the technology.

Level	Examples	Requirements
Prohibited	social scoring, emotion recognition in the workplace and in education, untargeted face scraping	forbidden
High risk	education (admission, assessment), recruitment, credit, justice, critical infrastructure	risk management, data quality, technical documentation, logging, human oversight, accuracy and robustness
Limited risk	chatbots, deepfakes, emotion recognition (where it is allowed)	transparency requirements — the user has to know
Minimal risk	spam filters, game AI	no particular requirements

The most important row for a learning platform is the second. Systems used to assess pupils or steer their education are high risk, with everything that entails.

And the first row contains something easy to miss: emotion recognition in education is prohibited, not regulated.

The requirements on high-risk systems (chapter III) — nine areas:

Requirement	Means
A risk management system	ongoing, documented, across the whole lifecycle
Data and data governance	relevant, representative, reviewed for bias
Technical documentation	sufficient for an authority to assess compliance
Logging	automatic recording throughout operation
Transparency towards the user	enough information to interpret the output
Human oversight	a human must be able to understand, intervene and stop it
Accuracy, robustness, cybersecurity	measured and reported
A quality management system	for the provider
A conformity assessment	before it is placed on the market

Generative models (GPAI) have their own rules in chapter V:

All GPAI models	Models with systemic risk
technical documentation	model evaluation and adversarial testing
information to downstream providers	incident reporting
a copyright policy	cybersecurity protection
a summary of the training data	risk assessment and risk mitigation

Article 50 — transparency applies to everybody and is the one that shows up most in practice:

• The user has to be informed that they are interacting with an AI system.
• Synthetic content has to be machine-readably marked.
• Deepfakes have to be clearly labelled.
• Emotion recognition requires the person concerned to be informed.

What you do concretely, in order:

1. Classify every use separately — the same technology can be high risk in one context and minimal in another.
2. Check that nothing falls into the prohibited category.
3. If it is high risk: start with the documentation, the logging and the human oversight — the three that take longest to build afterwards.
4. Label synthetic content from the start. Adding labelling later is hard.

The AI Act does not replace the GDPR. They apply in parallel: the GDPR regulates the personal data, the AI Act the system. A system can be entirely compliant with one and breach the other.

Classify eight uses. Write prohibited, high, limited or minimal — and a justification.

#	Use	Level?
1	A spam filter in the school's email
2	A system that scores pupils for admission
3	A chatbot answering questions about the school's rules
4	A system that analyses pupils' facial expressions to measure engagement
5	Generated images for teaching materials
6	A system that suggests the next exercise without affecting grades
7	Face recognition for attendance checking
8	A tool that marks multiple-choice questions automatically

The answers:

1. Minimal — no particular requirements.
2. High risk — education and admission are explicitly in annex III.
3. Limited — a transparency requirement: the pupil has to know it is an AI.
4. Prohibited — emotion recognition in an educational setting is forbidden.
5. Limited — synthetic content has to be labelled.
6. Probably limited, but the borderline case is interesting: if the suggestion in practice steers the pupil's educational path it approaches high risk. Which is why it has to be freely overridable.
7. High risk and questionable — biometric identification, and problematic under the GDPR too; the Swedish data protection authority has issued sanctions over exactly this in a Swedish school.
8. Minimal to limited — marking multiple-choice questions is deterministic; it only becomes high risk if the result is used for formal assessment.

Questions 6 and 8 show the principle: it is not the technology that decides, but the consequence for the individual. The same model can be minimal risk in a practice mode and high risk if the result sets a grade.

That is also why AI-grafen is built the way it is: a competence certificate is documented evidence, not a grade, and the model never issues a judgement with formal consequences.