Hybrid search

What is hybrid search?

Hybrid search is a retrieval design that runs one query against two indexes, a sparse keyword index and a dense vector index, and merges the two ranked lists into a single set of passages. The merge, called fusion, is the part that makes it hybrid, and it is where most of the tuning lives.

A support query such as "refund policy for order 88214-B" carries both kinds of signal at once: an exact token no embedding model preserves reliably, and a concept that lives in an article titled "returns and cancellations". Each retriever catches one half of that question.

How hybrid search works

A hybrid pipeline runs in five stages. The query fans out to both indexes simultaneously. The sparse side, usually BM25, scores documents on how often the query's terms appear in them and how rare those terms are across the corpus. The dense side converts the query into a vector embedding and scores stored document vectors by cosine similarity, which is the mechanism behind semantic search and the reason a paraphrase still matches.

Stage four is fusion. Two ranked lists arrive carrying scores on incompatible scales, and the system has to produce one ordering from them. Reciprocal rank fusion solves this by ignoring the scores and using positions: each document scores 1/(k + rank) in each list, and the sums are added. With the common constant k of 60, a passage ranked third by both retrievers sums to roughly 0.032, ahead of a passage ranked first by one retriever and missed entirely by the other at roughly 0.016. Agreement between the two indexes wins.

Stage five is reranking, where a cross-encoder scores the top fused candidates against the full query text and reorders them, so a retrieval-augmented generation prompt carries the five best passages and not the fifty plausible ones.

Types of hybrid search fusion

• Reciprocal rank fusion: Merges on rank position alone, needing no score normalization, which is why it survives contact with BM25 and cosine scores at once.

• Weighted score fusion: Normalizes both score sets onto a shared range and blends them with a tunable alpha, giving finer control at the cost of recalibration whenever either index changes.

• Cascade retrieval: Uses one retriever to produce a wide candidate pool and the other to filter or rescore it, which is cheaper but inherits the first stage's blind spots.

• Learned fusion: Trains a small model on labeled relevance judgements to combine the two signals, appropriate only when a team already maintains a labeled query set.

Hybrid search vs lexical search vs dense vector search vs reranking

These four get discussed as though they were competing products, and the confusion costs teams months of tuning aimed at the wrong layer. Lexical search ranks documents by term overlap, so it is exact and literal. Dense vector search ranks documents by embedding distance, so it is tolerant of rephrasing. Reranking rescores a shortlist with a slower, more accurate model, so it improves an ordering it did not produce. Hybrid search is the layer that runs the first two together and reconciles what they return.

If your queries carry account numbers, error codes, or SKUs alongside plain description, you need the hybrid path. If the corpus is small and the vocabulary is controlled, a single retriever with a reranker will usually get you there for far less operational cost.

Why hybrid search matters for customer experience

When retrieval misses, an AI agent does not go quiet. It answers from the closest thing it found, which is how a retrieval gap reaches a customer as a confident statement of the wrong policy. A dense-only stack handles "my package never arrived" and stumbles on "error 0x8007". A keyword-only stack does the reverse, returning nothing for a customer who described the symptom in their own words.

Both failures land in the same place: an unnecessary escalation, or a wrong answer that a correct passage sitting in the knowledge base would have prevented.

The tradeoff is real. Hybrid retrieval doubles the index footprint, adds a fusion hop to every query, and introduces weights someone has to own. On a corpus with tightly controlled vocabulary the recall you gain is small and the latency you add is measurable.

How is hybrid search measured?

Hybrid search is measured at the retrieval step, before the model writes anything, because a fluent wrong answer usually begins as a bad candidate list. Two numbers carry the weight: recall at the fusion cutoff, meaning whether the passage containing the answer made the merged list at all, and precision at the top of that list, meaning how much of what survived is genuinely about the question.

Both require a labeled set: a few hundred real support questions, each tagged with the passage that answers it. Run that set three times, against the sparse retriever alone, the dense retriever alone, and the fused output, so every failure can be attributed to a side. Dense-side failures often trace back to AI embeddings trained on language unlike your product's. For an outside reference point, the Stanford HAI AI Index reports year-over-year movement on public retrieval and reasoning benchmarks across its 2023 to 2025 editions.

How AI agents change hybrid search

A person types one query. An agent issues several: it rewrites the customer's phrasing, splits a compound question into parts, and often runs a follow-up search after reading the first result set. Fusion stops being a single merge and becomes a repeated one, and agentic RAG systems now reconcile results across queries as well as across indexes.

The second change is budget. Every fused candidate that reaches the prompt consumes tokens and dilutes the model's attention, so reranking moves from optimization to load-bearing infrastructure. Teams building platforms that ground answers in documentation typically discover this in the first week of production traffic, when latency and cost climb faster than ticket volume does.

Agents also demand filtering at query time. A retrieval call scoped to one customer's locale, plan tier, and entitlements returns a different fused list than the same words asked globally.

What to look for in a hybrid search implementation

Coverage comes first: both indexes must be built over the same chunks of the same corpus, or the fused ranks are comparing different objects. Integration surface comes next, since the retriever has to accept metadata filters for locale, product line, and plan tier, and enforce user permissions during the query itself.

Governance decides the rest. Fusion weights, the k constant, and candidate depth change retrieval behaviour for every customer at once, so they need a named owner and a change log, with edits reviewed like code. Regulated buyers will ask how permission filtering is evidenced, usually against SOC 2 Type II or ISO 27001, and the answer has to be a repeatable test.

The constraint that bites hardest is reindexing: changing the embedding model means re-embedding every document, while the sparse index sits untouched, so model upgrades are a scheduling problem as much as a quality one.

Hybrid search and context engineering

Retrieval hands its output to the next stage, and that stage is context engineering: deciding what actually enters the model's context window, in what order, and with what labels attached. Fusion determines the candidate pool; context engineering determines which of those candidates survive and how they are framed for the model.

The stage after that is verification, which is the subject of grounding and RAG. Hybrid search improves grounding in one narrow and useful way: the passage carrying the exact policy number is far more likely to be inside the set the model is permitted to cite from.

What does hybrid search mean in plain terms?

Think of hybrid search as sending the same question to two researchers with opposite habits. One trusts only the exact words you used and finds the file labelled with your order number. The other ignores your wording entirely and chases the idea behind it, so it surfaces the returns article even though you wrote "send it back". A third step lays their two shortlists side by side and promotes whatever both of them ranked highly.

Drop the second researcher and a customer who describes a problem in their own language gets nothing back, because they never typed the words printed in the article. Drop the first and a customer who quotes a policy number gets a warm, plausible article about a different policy.

The price is upkeep. Two systems have to stay in step, and one dial in the middle quietly decides which researcher wins the ties.

Common hybrid search mistakes

Fusing across mismatched indexes is the first pattern. Teams build the sparse index over whole articles and the dense index over paragraph chunks, then merge two rank orders describing different units of content. The fused list looks reasonable and its ordering means nothing.

Tuning weights on anecdotes is the second. Someone complains about one query, the alpha moves, and last week's good results quietly regress. Without a labeled question set, every adjustment is a trade nobody measured.

Skipping the rerank is the third. Fifty fused candidates entering a prompt push the correct passage into the middle of a long context, where the model gives it the same weight as the forty-nine near-misses around it.

The fourth is reading retrieval failures as model failures. Swapping models rarely repairs a corpus with duplicate, contradictory, or unchunkable articles, which is a knowledge architecture problem that retrieval only exposes.