Embeddings are remarkable right up until the exact moment a user searches for a serial number.
If someone queries your RAG application for TX-9942-B connection timeout, semantic search fails. The vector space maps TX-9942-B to a neighborhood of abstract alphanumeric patterns. It loses the precise sequence of characters that makes that identifier distinct.
Semantic search understands intent across different phrasing. Lexical search understands exactness. You need both. This intersection is Hybrid Search.
Here is how it actually works, why scaling scores is a trap, and how to build it from first principles.
Dense vs Sparse: A Primer
We retrieve context via two completely different mathematical mechanisms. You need to understand both to merge them effectively.
Dense Search (Embeddings)
Text is projected into a high-dimensional vector space. We find matches by calculating the Cosine Similarity between the query vector and document vectors. It captures concepts. It doesn't care if the words match exactly.
In frameworks like LangChain, this is what you are doing when you initialize a standard vector store:
vector_retriever = chroma_db.as_retriever(search_kwargs={"k": 4})Sparse Search (BM25)
Sparse search is the direct descendant of TF-IDF. It physically counts word overlaps between the query and the document. However, unlike a naive word counter, BM25 natively handles two critical realities:
- Document Frequency Penalty: If a word (like "the" or "error") appears in 90% of your documents, it carries zero information. BM25 discounts it heavily.
- Term Frequency Saturation: If a document repeats "router" 15 times, it isn't 15 times more relevant than a document that says it twice. BM25 caps the reward for repetition.
Setting this up requires an explicit lexical retriever:
from langchain_community.retrievers import BM25Retriever
bm25_retriever = BM25Retriever.from_texts(documents)The Trap: Why You Can't Just Add Them
The intuitive approach to hybrid search is to run both retrievers, extract the relevance scores, and mash them together with a weight: score = (0.5 * dense) + (0.5 * sparse).
Do not do this.
Dense cosine similarity scores are bounded—usually between 0.0 and 1.0. BM25 scores are mathematically unbounded. Depending on keyword rarity and query length, a BM25 score can be 1.4, 15.6, or 86.0.
If you add them directly together, the unbounded BM25 score will completely overrun your dense search signal. Normalizing BM25 limits (via MinMax scaling) is notoriously fragile because a single outlier document with a massive score will crush the rest of your distribution into a tiny decimal range.
We need a system that entirely ignores the absolute scores. Make way for Reciprocal Rank Fusion.
The Solution: Reciprocal Rank Fusion (RRF)
Reciprocal Rank Fusion (RRF) is the current industry standard for merging disparate retrieval pipelines.
Instead of looking at the magnitudes of the scores, we look exclusively at the ranking permutations. We take the rank of a document in the dense results, the rank of the document in the sparse results, and run them through a simple inversion formula.
TIP
The RRF Formula:score = 1 / (k + rank)
kis a smoothing constant. Standard practice definesk = 60.
If Document A is Rank 1 in your Dense search, it scores 1 / (60 + 1) = 0.0163.
If it's also Rank 4 in your Sparse search, it gets an additional 1 / (60 + 4) = 0.0156.
Total RRF Score: 0.0319.
RRF works because it massively rewards documents that perform well across both ranking mechanisms, while smoothly decaying the score of documents that only appear in one.
Merging the Streams: Step-By-Step
Let's walk through how to actually orchestrate this. We will extract the exact logic, and then show how abstraction handles it for you.
Phase 1: Retrieve Independent Rankings
First, you query both algorithms independently to get two separate sorted lists of results. Under the hood, this looks like extracting indexes based on highest similarity mathematically:
# np.argsort sorts low-to-high, so [::-1] flips it to highest-score-first.
sparse_ranks = { idx: rank + 1 for rank, idx in enumerate(np.argsort(sparse_scores)[::-1]) }
dense_ranks = { idx: rank + 1 for rank, idx in enumerate(np.argsort(dense_scores)[::-1]) }Phase 2: Compute the RRF
Once you have the isolated ranks, you calculate the RRF fusion by iterating over every document that surfaced. You completely throw away the underlying cosine or TF-IDF metric and calculate purely based on position.
k = 60
rrf_scores = {}
for i in range(len(documents)):
rrf_scores[i] = (1 / (k + sparse_ranks[i])) + (1 / (k + dense_ranks[i]))
# The final hybrid ranking
final_ranking = sorted(rrf_scores.keys(), key=lambda x: rrf_scores[x], reverse=True)Phase 3: The Framework Level
If you are building production systems, you don't want to maintain that sorting and looping logic manually for every query. This is exactly where abstractions like LangChain become valuable.
LangChain abstracts the execution of both algorithms and handles the fusion scoring (often defaulting to an implemented RRF or weighted interpolation) using EnsembleRetriever.
from langchain.retrievers import EnsembleRetriever
# Combine our previously initialized retrievers
ensemble_retriever = EnsembleRetriever(
retrievers=[bm25_retriever, vector_retriever],
weights=[0.5, 0.5] # Weights dynamically bias the fusion mapping
)
# A single call fires both algorithms, calculates ranking logic, and returns top documents
docs = ensemble_retriever.invoke("How do I reset my TX-9942-B router?")Notice how clean the architecture is.
By separating the retrieval completely from the fusion logic, you decouple your architecture. The fusion logic doesn't care if you're using Pinecone, Chroma, or Postgres. It simply asks for a ranked list, processes the math, and returns the optimal intersection of lexical exactness and semantic intent.
Stop throwing away exact keyword matches just because embeddings are trendy. Build Hybrid Search.