Accuracy: how good is the model?

Once a model is trained, it makes predictions on new data. Sometimes it is right, sometimes it is wrong. We count: 8 correct out of 10 is quite good. 5 correct out of 10 is no better than flipping a coin.

Why does it get it wrong? Often because the new data was unlike the training examples: an unusual pattern, missing data, or a case that sits close to the boundary between two categories.

Counting correct and incorrect predictions is the most important way to know if a model is reliable.

Count yourself:

Case	Model said	Correct answer
1	Paid	Paid	✔
2	Paid	Unpaid	✘
3	Unpaid	Unpaid	✔
4	Unpaid	Paid	✘
5	Paid	Paid	✔

3 correct out of 5. Look at cases 2 and 4 — what might have tricked the model? Perhaps an unusual payment pattern or missing information. Incorrect predictions show where the model needs more examples.