Classification: how a model sorts information

A classifier is a model that sorts information into categories. For example: spam or important email, approved application or rejected, or whether a transaction is legitimate or fraud. The model has been trained on previous examples and now guesses on new cases it has never seen.

How good is it? Test it on examples where you already know the answer, but which the model has not been trained on. If it gets 90 out of 100 cases right, the accuracy is 90 %.

There are two types of errors, and they have different consequences:

• The model marks an important email as spam. You miss it. This is a false positive error.
• The model lets spam through as a normal email. Your inbox becomes full of junk. This is a false negative error.

Which error is worse depends on the context. A model that detects diseases would rather alarm unnecessarily (false positive) than miss a sick patient (false negative). Accuracy alone does not tell the whole story about the risk.