A team building a RAG system needs to choose an embedding model for two different scenarios: (1) matching a short user question against long, several-paragraph reference documents, and (2) matching one previously-asked question against a database of other previously-asked questions to detect duplicates. According to guidance from Sentence-Transformers (SBERT) on choosing a semantic search model, why should these two scenarios not necessarily use the same kind of pre-trained embedding model?
- Scenario 1 is asymmetric semantic search, since the query and the matching text are of very different length and character (reversing them would not make sense), while scenario 2 is symmetric semantic search, since query and matching text are of comparable length and structure (reversing them would still make sense); using a model trained for the wrong one of these two cases can produce misaligned embeddings and degrade search quality
- Scenario 1 requires a model trained only on English text, while scenario 2 requires a model trained only on question-formatted text, and no model can ever be trained on both kinds of text at once
- Scenario 2 cannot be done with embeddings at all and must instead use a plain keyword search, while scenario 1 is the only one of the two that embeddings can ever be used for
- There is no real difference between the two scenarios once cosine similarity is used for comparison, since cosine similarity is defined identically regardless of what the two compared texts represent
Why A? And why not the others?
Correct answer: A. Scenario 1 is asymmetric semantic search, since the query and the matching text are of very different length and character (reversing them would not make sense), while scenario 2 is symmetric semantic search, since query and matching text are of comparable length and structure (reversing them would still make sense); using a model trained for the wrong one of these two cases can produce misaligned embeddings and degrade search quality
SBERT's guidance distinguishes asymmetric semantic search, where the query and the matching text differ substantially in length and character such that swapping them would not make sense (a short question against a long reference passage), from symmetric semantic search, where query and matching text are comparable in length and structure such that swapping them would still make sense (one question against a database of other questions); because a model can be pre-trained with either case's structure in mind, using a model built for one case on data shaped like the other can misalign the resulting embeddings and quietly degrade search quality even though the pipeline still runs without errors. This is a different failure mode from mixing embeddings produced by two different model versions or checkpoints within the same index (a versioning-compatibility problem); this is instead a design decision about choosing the right category of pre-trained model for the shape of the task before any index is ever built. The option describing a language restriction is wrong because the symmetric/asymmetric distinction is about query-versus-document length and structure, not language. The option claiming scenario 2 cannot use embeddings at all is wrong because duplicate-question detection is a standard symmetric semantic search use case. The option claiming cosine similarity erases any difference between the scenarios is wrong because the metric used for comparison does not fix a mismatch between what the underlying embedding model was actually trained to represent well.
Source: Sentence-Transformers (SBERT), 'Semantic Search' documentation, https://sbert.net/examples/sentence_transformer/applications/semantic-search/README.html