A support-ticket search system built purely on dense embedding similarity performs poorly when a user searches for an exact error code like 'ERR-4471', because the embedding model treats the code as similar to other short alphanumeric strings rather than as a specific identifier that must match exactly. Which change would most directly address this particular weakness?
- Retrain the embedding model on a larger general-purpose text corpus so that it becomes more accurate at every kind of query
- Add a sparse, keyword-based retrieval method such as BM25 alongside the dense embedding search, and combine both rankings into a single hybrid result set, since exact-term matching methods are specifically strong at the rare-token, exact-identifier queries where dense embeddings tend to struggle
- Increase the number of dimensions in the embedding vectors so the model can represent more information about each ticket
- Reduce the size of the text chunks so that each error code ends up stored as its own, isolated chunk
Why B? And why not the others?
Correct answer: B. Add a sparse, keyword-based retrieval method such as BM25 alongside the dense embedding search, and combine both rankings into a single hybrid result set, since exact-term matching methods are specifically strong at the rare-token, exact-identifier queries where dense embeddings tend to struggle
Dense embedding models are trained to capture semantic similarity and tend to smooth over exact tokens such as rare identifiers, treating them as similar to other short alphanumeric strings rather than requiring an exact character match; a sparse, term-based method like BM25 scores documents on the exact terms they contain, so it is specifically good at surfacing a document because it contains the literal string 'ERR-4471,' and combining that ranking with the dense ranking in a hybrid setup covers the weakness that either method has on its own. The option about retraining on a larger general corpus is wrong because a bigger corpus does not change the fundamental tendency of a dense embedding to generalize over exact tokens; it would not reliably fix retrieval of a specific rare identifier. The option about increasing embedding dimensionality is wrong because more dimensions increase representational capacity in general but do not add a mechanism for exact lexical matching. The option about shrinking chunk size is wrong because isolating the code into its own chunk does not change how the embedding model represents it, so a dense search would still return approximate neighbors rather than an exact match.
Source: Robertson & Zaragoza, 'The Probabilistic Relevance Framework: BM25 and Beyond' (2009); Karpukhin et al., 'Dense Passage Retrieval for Open-Domain Question Answering' (2020), arXiv:2004.04906