Vector databases

Why it matters

Semantic search, recommendations, retrieval for language models, duplicate detection and image similarity all reduce to one operation: given a point in a high-dimensional space, find the points nearest to it. At a million points this is a brute-force scan; at a billion it is impossible without an approximate index. Vector databases exist to make that operation fast, and because every product now wants "search that understands meaning", the design question appears in interviews constantly.

The idea

An embedding model turns a piece of content (text, image, user behaviour) into a fixed-length vector of floats, typically 384 to 3,072 dimensions, such that semantically similar content lands close together. A vector database stores those vectors with metadata and answers k-nearest-neighbour queries: the k vectors closest to a query vector by cosine similarity, dot product or Euclidean distance.

Why exact search does not scale

Exact nearest neighbour means computing the distance from the query to every stored vector: a million 1,536-dimension vectors is 1.5 billion multiplications per query. Tree structures that work in low dimensions collapse in high ones (the curse of dimensionality). So production systems use approximate nearest neighbour (ANN) indexes that return very nearly the right answers in a tiny fraction of the time, measured by recall: the fraction of the true top-k the index actually returned.

The two index families

	HNSW (graph-based)	IVF (clustering-based)	Flat (exact)
How	A multi-layer navigable graph; search greedily descends from coarse to fine layers	Vectors clustered around centroids; search probes the nearest few clusters	Compare against everything
Query speed	Fastest at high recall	Fast; tuned by number of clusters probed	Slow, linear
Memory	High: graph links plus vectors, usually all in RAM	Lower; combines with product quantisation to compress vectors	Vectors only
Build and updates	Incremental inserts fine; deletes leave holes until rebuild	Needs training on a sample; re-clustering as data drifts	Trivial
Fits	Up to tens of millions of vectors needing low latency	Hundreds of millions to billions with compression	Under a hundred thousand vectors, or as the ground truth for measuring recall

Filtering and hybrid search

Real queries are "similar to this, but only in-stock items in the user's region". Pre-filtering restricts the candidate set first and can starve the ANN index of neighbours; post-filtering fetches extra neighbours and then filters, which can return fewer than k. Good engines integrate filters into the traversal. Hybrid search combines vector similarity with keyword (BM25) scores, usually by reciprocal rank fusion, because embeddings miss exact identifiers and rare terms that keyword search nails. Most production retrieval is hybrid.

Operating a vector store

Embeddings are tied to a model: change the model and every vector must be regenerated, so store the source content and version the embedding space. Vectors are large (1,536 floats is 6 KB), so a hundred million of them is 600 GB before any index; quantisation (int8, product quantisation, binary) cuts that several-fold at a small recall cost. Measure recall against a flat index on a sample, and watch latency at the 99th percentile as the index grows. Keep the vector store a derived view: the documents live elsewhere, and the store holds vectors, ids and the metadata needed for filtering.

Dedicated store or an extension

Postgres with pgvector (HNSW and IVF indexes) handles millions of vectors alongside the relational data with full SQL filtering and transactions, and removes a system. Dedicated engines (Pinecone, Milvus, Weaviate, Qdrant, Vespa) win at hundreds of millions of vectors, very low latency, advanced quantisation and filtered search at scale. Start with the extension; move when measurements say so.