passdrill
AI & LLM Engineering · RAG & Embeddings · Card 026/032 hard

A team's HNSW-based vector index gives fast, high-recall search, but as their corpus grows into the hundreds of millions of vectors, the index no longer fits in memory. They switch to an IVF+PQ index instead, which partitions vectors into clusters and additionally compresses each vector by splitting it into subvectors and replacing each subvector with the ID of its nearest centroid from a small codebook. Compared to storing full-precision vectors, what does this quantization step trade away, and why would a team accept that?

  1. Nothing is traded away; IVF+PQ produces exactly the same similarity ranking as an exhaustive full-precision search, just organized differently in memory
  2. Recall is reduced, because replacing each subvector with the ID of its nearest codebook centroid is a lossy approximation of the original values, so similarity computed from the compressed representation only approximates the true distance; a team accepts this because the memory footprint can shrink dramatically, often by roughly an order of magnitude or more, letting a large index fit in memory at all, while search remains far faster than an exhaustive scan
  3. Only the ability to add new vectors to the index after it is built is lost; existing search accuracy is completely unaffected by the quantization step
  4. Only the ability to filter search results by metadata is lost; the vector similarity ranking itself remains exactly as accurate as full-precision search
Next card → Shuffle