Vectors

A vector is a list of numbers: (3, 4). Draw it as an arrow from the origin. Two numbers give an arrow in the plane, three in space, 768 in… well, a space we cannot draw but can calculate in.

Almost everything in AI is vectors: an image is a long vector of pixel values, a word becomes a vector of hundreds of numbers, a neuron's input is a vector.

For u = (u₁, u₂, …) and v = (v₁, v₂, …):

• Addition: u + v = (u₁+v₁, u₂+v₂, …)
• Scalar times vector: 2u = (2u₁, 2u₂, …)
• Dot product: u · v = u₁v₁ + u₂v₂ + …
• Length (norm): ‖u‖ = √(u₁² + u₂² + …)
• Cosine similarity: cos θ = (u · v) / (‖u‖ ‖v‖)

The dot product is large when the arrows point the same way, zero when they are perpendicular, negative when they point in opposite directions. That is why it is used to measure similarity: two words with similar meanings have vectors that point in similar directions. (3, 4) · (4, 3) = 24; ‖(3, 4)‖ = 5.