Try something new or play it safe?

You are standing in the canteen. You know the pasta is okay. You have never tried the stew.

Do you take the pasta again, or take a chance?

If you always take the pasta, you always get okay food — but you never find out if the stew was better. If you take a chance every day, you eat badly quite often.

This is the trade-off between exploration and exploitation, and it exists everywhere:

Situation	Explore	Exploit
Music	listen to a new artist	play your favourite song again
Commuting	test a new shortcut	take the usual route
Work	try a new tool	stick with what works
Finance	test a new savings strategy	do what you did last time

An algorithm that needs to learn has exactly the same problem — and must solve it without anyone telling it what is right.

Why is it not enough to just take the best you know?

Imagine you try the stew once and have bad luck — that day it was burnt. Now you think the stew is bad. If you always choose what you believe is best, you will never try it again, and you will never discover the mistake.

This is called getting stuck in a local good choice. You get something okay, but you miss out on something better.

The simplest solution that actually works is called ε-greedy (epsilon-greedy):

Roll a die before you choose. With a small probability (e.g. 10 %), you choose something random. Otherwise, you choose what you believe is best.

That is all. And it goes surprisingly far.

A good idea: explore a lot at the beginning, less later. In the first week of a new job, it makes sense to try different methods. After six months, you know what works. The same applies to an algorithm — you let ε shrink over time.

Try it yourself, with paper and a die. You have three buttons. Each button gives points, but you do not know how many.

The truth (do not look until later): button A gives an average of 3 points, button B gives 7, button C gives 5. There is some randomness each time.

Round 1 — exploit only. Press each button once. Then press the one that gave the most 20 times.

Round 2 — ε-greedy. Roll a die each time. If you get a six: choose a button completely at random. Otherwise: choose the one that has given the best average so far.

Add up the points after 23 presses in each round.

What usually happens: round 1 sometimes wins — if the first press on B happened to be high. But often it gets stuck on A or C and never recovers. Round 2 is worse at the start (it wastes presses on bad buttons) but almost always better at the end.

Question to think about: how would you change the rules if you only had 5 presses in total? And if you had 1 000?