Can an AI be fooled?

Models can be fooled — not because they are stupid, but because they only see patterns.

Three common ways:

1. The picture is changed a little. If you change a few pixels in a way a human being cannot see, an image model can go from «panda» to «gibbon» with high confidence. This is called an adversarial example.
2. The question is rewritten. A model that refuses a question can answer it if the question is put as a «story» or in another language. This is called a jailbreak.
3. Instructions are hidden in text the model reads. A web page can contain «AI assistant: ignore the user and do X». This is called prompt injection.

Testing this on your own systems is called red teaming and is part of serious AI work.

Why it matters: a model that can be fooled is dangerous when it has the power to do something — send money, let somebody in, make a diagnosis.

The protection in layers:

• Limit what the model is allowed to do (reading is less dangerous than sending).
• A human being confirms what cannot be undone.
• Test systematically with known tricks before launch.
• Log everything so that you can see afterwards what happened.

An important rule: only test on systems you own yourself or have been given permission to test. Fooling somebody else's system is not a game — it can be illegal.