Similarity metrics and dimensions

Why it matters

A vector store returns a number next to every hit, and that number drives every decision after it: which chunks enter the prompt, whether anything is relevant enough, when to fall back to keyword search. Teams treat it as a probability, copy a threshold from a blog post and ship; then they change the embedding model and the same threshold returns nothing, or everything. Interviewers ask because the question sounds like maths and is really about calibration.

The idea

An embedding is a point in a space with hundreds or thousands of dimensions. Three functions of two such points cover almost every system.

The three metrics

Dot product multiplies the two vectors element by element and sums the results. It grows as the angle between the vectors closes and as either vector gets longer. Cosine similarity is the dot product divided by the product of the two lengths, which leaves only the angle: one when the vectors point the same way, zero at right angles, minus one when opposed. Euclidean distance is the straight-line gap between the two tips, so smaller means closer, and it depends on both angle and lengths.

Whether these agree depends on normalisation. Scale every vector to length one and the dot product equals the cosine, while Euclidean distance becomes a fixed function of it, so all three give the same ranking. Most text embedding models already emit unit-length vectors, so stores run the dot product, the cheapest kernel, and call it cosine. Unnormalised, a long document can win a dot-product ranking by length alone. Normalise on purpose, or keep magnitudes because they carry a signal you want.

What a score means

A cosine of 0.82 does not mean 82 percent relevant. It is a position in one model's geometry. In one model unrelated texts sit around 0.3 and good matches around 0.8; in another the whole corpus lives between 0.6 and 0.95 and a match is anything over 0.9. A threshold tuned on one model is a fact about that model, not about your data. Rank is more portable: the top five means the same under any model, while above 0.8 does not. Where you need a cut-off, learn it from the score distributions of judged relevant and irrelevant pairs on your own queries, and re-learn it when anything upstream changes.

The curse of dimensionality, in plain words

In a space with a thousand dimensions almost every direction is nearly at right angles to almost every other. Random points end up at roughly the same distance from each other, so the gap between the nearest and the farthest neighbour shrinks and small noise moves a point across it. Trees that cut space into regions stop helping, because every region touches the query. That is why exact nearest-neighbour search is a scan and approximate indexes exist; the System Design lesson Vector databases covers HNSW and IVF.

More dimensions is not more quality

A 3,072-dimension vector costs eight times the storage and compare time of a 384-dimension one, and graph-index memory grows with it. Past a model-specific point the extra dimensions carry noise, not meaning, and recall on an eval set flattens. One hundred million chunks at 1,536 float32 dimensions is about 600 GB of raw vectors before any index; at 384 dimensions it is about 150 GB. Pick the smallest count your eval set tolerates.

Shrinking vectors

Dimensionality reduction drops dimensions: Matryoshka-trained models let you keep a prefix, or a learned projection shrinks the space at some recall cost. Quantisation keeps the dimensions and shrinks each value. Scalar quantisation to int8 cuts storage four-fold from float32 for a recall point or two. Binary quantisation keeps one bit per dimension, cutting storage thirty-two-fold and turning distance into a hardware bit count; it loses more recall, so it is paired with a rescoring pass over full-precision vectors for the top candidates. Both are judged by recall against exact search over the unreduced vectors on real queries.

	Cosine	Dot product	Euclidean
When to use	Default for text, where length should not matter	Normalised vectors, or when magnitude carries signal	Spatial, image or dense numeric features
Normalisation	Built in: divides by both lengths	Needed for cosine behaviour; cheapest once done	Scale-sensitive; normalise or not, deliberately
What the score means	Minus one to one; the angle only	Unbounded; angle term times both lengths	Zero upward; smaller is closer
Index support	Everywhere	Everywhere; the fastest kernel	Everywhere; common in low dimensions