Grouping without an answer key: clustering by hand

Sometimes you have data without an answer key. Nobody has said what the things are called — you have to find the groups yourself.

  y
  |   ● ●●
  |  ●●● ●            ◆◆
  |    ●             ◆◆◆
  |                   ◆◆
  |        ▲▲▲
  |       ▲▲▲▲
  |________________________ x

You see three clumps without anyone having told you. That is clustering: finding groups in data where there are no labels.

How does the computer do it? Much as you do:

1. Guess where the centre of each group should be.
2. Let every point belong to the nearest centre.
3. Move each centre to the middle of its points.
4. Repeat until nothing changes.

That is called k-means, and k is how many groups you have decided to look for.

The catch: you have to say how many groups you want. The computer does not know whether it is two, three or seven.

Different people group differently — and both can be right.

Think of a pile of clothes. You can group them by:

By	Groups
Colour	red, blue, black
Kind	tops, trousers, socks
Season	summer, winter
Whose	mine, my sibling's

All four are reasonable. Which one is «right» depends on what you are going to do with the groups.

The same holds for the computer. It groups by what you fed in. Give it colour and it groups by colour — and it does not know that you were really thinking about kind.

Two things govern the result:

1. Which properties you use. Height and weight give different groups from age and favourite food.
2. How many groups you ask for. With k = 2 two clumps are merged. With k = 10 one clump is split into little pieces.

How do you know the right number? You do not know for certain. One trick is to try several and see when it stops getting better — but often the best method is to look at the groups and ask: do these mean anything?

And the most important thing: the computer having found groups does not mean the groups are real. Random points can also be divided into three «groups» if you ask for three.

Group by hand. You need squared paper.

Task 1 — draw and group.

Draw these twelve points in a coordinate system:

(1,1) (2,1) (1,2) (2,2)
(8,1) (9,1) (8,2) (9,2)
(5,8) (6,8) (5,9) (6,9)

Draw rings around the groups you see. How many were there? Where did you draw the boundaries?

Task 2 — do what the computer does.

1. Put three crosses somewhere on the paper — your first guesses at the group centres.
2. Draw a line from each point to the nearest cross.
3. Move each cross to the middle of its points.
4. Do steps 2 and 3 again.

After two or three rounds the crosses stop moving. Did they end up in the middle of your rings?

Task 3 — try the wrong number.

Do it again with two crosses instead of three. What happens? (Two groups are merged — but which ones?)

Do it again with six crosses. What happens? (The groups are split, and the boundaries become arbitrary.)

Task 4 — the important one.

Draw twelve points completely at random, without any clumps. Run the same method with three crosses.

You get three groups. But are they real? No — the method always gives the number of groups you ask for, whether or not they exist in the data.

That is the most important lesson in this whole node.