A pipeline splits long documents into chunks by comparing the embedding similarity between each pair of adjacent sentences and starting a new chunk whenever that similarity drops sharply, rather than cutting every fixed number of characters regardless of content. What is this chunking approach called, and why can it retrieve better than fixed-size chunking on a document that mixes several unrelated topics?
- This is called cross-encoder reranking, and it retrieves better because a joint query-passage pass scores each chunk more accurately than comparing two independently computed embeddings
- This is called query expansion, and it retrieves better because appending related terms to the chunk's text before embedding gives the embedding model more signal to work with
- This is called semantic chunking, and it retrieves better because placing chunk boundaries where the topic actually shifts keeps each chunk focused on a single topic, so a chunk's embedding is not an average of unrelated content and a query about one of the topics is less likely to be diluted by the others sharing its chunk
- This is called vector quantization, and it retrieves better because compressing each chunk's embedding to lower numeric precision makes nearest-neighbor comparisons faster and therefore more accurate
Why C? And why not the others?
Correct answer: C. This is called semantic chunking, and it retrieves better because placing chunk boundaries where the topic actually shifts keeps each chunk focused on a single topic, so a chunk's embedding is not an average of unrelated content and a query about one of the topics is less likely to be diluted by the others sharing its chunk
Semantic chunking places a chunk boundary wherever the similarity between consecutive sentences drops, which is exactly the point where the text moves from one topic to another; because the boundary tracks the content rather than an arbitrary character count, each resulting chunk stays about one topic, so its embedding represents that topic rather than an average of several unrelated ones, and a query about just one of those topics is more likely to land close to the chunk in vector space instead of being diluted by unrelated material stuffed in alongside it. Cross-encoder reranking is a separate technique that re-scores already-retrieved candidates by feeding the query and passage through a transformer together; it has nothing to do with how chunk boundaries are chosen before indexing. Query expansion changes the text of the query or chunk by adding related terms, which is a lexical-augmentation idea unrelated to deciding where to cut a long document into chunks. Vector quantization compresses embeddings to save memory and speed up comparisons, and it is a lossy compression step that if anything can slightly reduce retrieval accuracy in exchange for speed, not a technique for choosing chunk boundaries at all.
Source: Pinecone, 'Chunking Strategies for LLM Applications,' https://www.pinecone.io/learn/chunking-strategies/; LangChain, semantic chunking documentation (percentile/standard-deviation breakpoint thresholds), https://python.langchain.com/docs/how_to/semantic-chunker/