Skip to content
AI-grafen
BInvestigatorTransformer architecture· about 20 min· fundamentals that rarely change· verified 2026-09-20· EN

Language models and probabilities

Understand how a language model generates text by calculating the probability of the next word.

Everyday explanation

Consider the sentence «I am going to send an email to …». Most people think of a colleague, a client, or a supplier. Few think of a robot or an asteroid.

You guess based on what you have experienced before. A language model does exactly that — except it has processed vast amounts of text and calculates: “colleague” 40%, “client” 20%, “boss” 10% … Then it selects a word, adds it, and calculates the probability of the next word.

An entire text is just a series of these probability calculations.

Interactive

Try it yourself: Write the beginning of a sentence, e.g. «The meeting started at …». What is the most likely answer? Perhaps “9:00” or “10:00”. That is the word’s “probability” in that context.

«I paid the invoice with …» → cash, card, check. «The weather in Stockholm is …» → rainy, sunny, windy.

Try a sentence where it is hard to guess: «My favourite colour is …» — here the answers are scattered, and a language model also becomes uncertain.

Mastery means

  • Can explain why certain words are more likely than others in a given context.
  • Can link the concept of probability to how a language model constructs sentences.

Prerequisites

Sign in to do the exercises and build your mastery up.

Sources

All the sources and licences