Encoder–decoder transformers

Three families of transformers, with different jobs:

Type	Attention	Trained with	Examples	Good at
Encoder-only	full, bidirectional	masked tokens	BERT	understanding: classifying, extracting
Decoder-only	causal	the next token	GPT, Llama	generating
Encoder–decoder	the encoder full, the decoder causal plus cross	denoising / seq2seq	T5, BART, Whisper	transforming A → B

Cross-attention is the key in the third: the decoder's queries fetch from the encoder's keys and values. The decoder therefore generates its answer while being able to «look at» the whole input the whole time — bidirectionally, in its entirety.

In an encoder–decoder block the decoder has three substeps:

1. Causal self-attention over what has been generated so far: $Q,K,V$ from the decoder's own states, with a triangular mask.
2. Cross-attention: $Q$ from the decoder, $K$ and $V$ from the encoder's output $H_{enc}$ — no mask, the whole input is visible.
3. A feedforward layer.

That means the input is processed bidirectionally (the encoder sees the whole source text in both directions) while the output is generated autoregressively.

Why decoder-only dominates anyway: a sufficiently large decoder-only model can solve transformation tasks by having both the input and the output in the same sequence («translate into English: … →»). That gives a single simple architecture, simpler scaling and simpler training on unstructured text. Encoder–decoder keeps its advantage when the input and the output have clearly different roles and the input is long — translation, summarisation, speech-to-text (Whisper).