How we know what you know

Having read something is not knowing it. Here is exactly how AI-grafen decides where you start, when you know something and what the certificate means.

1. The knowledge graph

All content is nodes in a graph: a node is one well-defined piece of knowledge, e.g. «cosine similarity» or «reranking». Each node has prerequisites, learning objectives, exercises and sources.

A goal is a set of end nodes. The path there is everything they build on, in the order it builds on itself.

2. The diagnostic

The diagnostic asks questions on nodes along the prerequisite chain and updates a probability for every node in the goal after each answer. A correct answer makes it likely that you also know what the node builds on; a wrong one makes it likely that what builds on the node is missing too.

The next question is chosen where uncertainty is greatest. If you answer wrong, the level below is probed. The number of questions depends on the size of the goal — between 8 and 20 — and the diagnostic stops earlier when it is confident.

The diagnostic never counts as proof of mastery. It only decides where you start.

3. Mastery

A node counts as mastered only when the probability is at least 85 % and it rests on at least two different kinds of evidence — for example a multiple-choice question and an explanation in your own words.

From level D a practical piece of evidence is also required: code, debugging, an evaluation or a transfer task in a new context.

Knowledge fades. Each node has a half-life that grows as you review; when the probability drops, the path adds a review.

4. Labs with hidden tests

In the labs you write the code yourself — in the browser or in an isolated sandbox on the server (gVisor, no network, with time and memory limits).

Grading is done by tests you cannot see. Code that looks right or prints the right thing is not enough; it has to handle cases you didn't know about.

5. The tutor

The tutor is an AI model given the node's content and sources. It gives hints and counter-questions, not finished answers.

It can be wrong. That is why exercises and tests — never the tutor — decide what you have mastered.

6. The certificate

The certificate lists the nodes you have mastered, the kinds of evidence behind each node, and which labs you passed with how many tests.

It is signed and has a verification link, so whoever receives it can check that it has not been changed.

It is documented evidence of what you have done — not a formal qualification, and not a replacement for a degree.

7. We measure whether the engine works

Every answer is logged with what the engine predicted beforehand. Three measures are tracked every week:

• Calibration: when the engine says a 70 % chance of a correct answer — are 70 % correct?
• Diagnostic accuracy: do those judged to know a node pass it on the first try more often than those judged to lack it?
• Value of prerequisites: do people do better on a node when the prerequisite has been shown? Edges that don't help are reviewed.

During the beta the data is small. We will publish the measures when they say something.

Why not just ask ChatGPT?

A chatbot answers what you ask well. It doesn't know what you already know, what you are missing or whether you can actually do what it explained. AI-grafen keeps track of that, picks the next step from it and lets tests decide.