Hybrid search and RRF

Vector search and keyword search fail on different questions. That is precisely why they can be combined.

The question	BM25	Vector
«error code E404»	finds it	blurs the exact string away
«how do I fix the page not existing»	misses	finds it
«Södertälje municipality reference number 2026-114»	finds it	misses
«what does it cost to live here»	misses the paraphrases	finds it

A hybrid that takes both lists and merges them finds everything each of them finds — plus it ranks up what both agree on.

The problem with merging the scores: BM25 gives numbers between 0 and 30, cosine similarity between −1 and 1. They are not comparable, and normalisation is arbitrary since the scales vary with the query.

RRF solves that by throwing the scores away and using only the order.

$\mathrm{RRF}(d) = \sum_{r \in R} \frac{1}{k + \mathrm{rank}_r(d)}, \qquad k \approx 60$

Three properties that make it hard to beat:

1. No calibration is needed. Only the ranking is used, so different score scales make no difference.
2. Consensus is rewarded. A document that is second in both lists beats one that is first in one and fifth in the other.
3. Robust against outliers. A single list cannot dominate, since the contribution is limited to $1/(k+1)$.

What k does: a large $k$ flattens the difference between the ranks out and makes the fusion more democratic; a small $k$ gives the top placings more weight. $k = 60$ comes from the original paper and works surprisingly well without adjustment.

The alternatives and when they are better:

The method	When
RRF	the default choice — no scores needed
A weighted score sum	when you have calibrated scores and time to set the weights
Reranking with a cross-encoder	when quality comes before latency; it often beats both
Learnt fusion (LTR)	when you have click data in quantity

The most effective setup in practice is two-stage: retrieve broadly with the hybrid (RRF over BM25 and vector search), and then rerank the top 50 with a cross-encoder. The first part is cheap and has high recall; the second is expensive but is only run on 50 documents.

Always evaluate the hybrid against each part on its own. Sometimes it is worse — particularly if one source is much worse than the other, since RRF then lets noise in. Measure:

The measure	The question
recall@k per source	what does each of them find?
recall@k for the hybrid	does it find more?
nDCG@10	is the right thing high up?
The share of queries where the hybrid is worse	is there a regression?

The last row is what reveals when the hybrid does harm: an average can improve at the same time as a quarter of the queries get worse.

def rrf(rankings: list[list[str]], k: int = 60, top: int = 10) -> list[str]:
    scores: dict[str, float] = {}
    for lst in rankings:
        for pos, doc in enumerate(lst):
            scores[doc] = scores.get(doc, 0.0) + 1.0 / (k + pos + 1)
    return sorted(scores, key=lambda d: (-scores[d], d))[:top]

# Consensus beats a lone first place
text  = ["a", "b", "c", "d"]
image = ["d", "b", "e", "f"]
print(rrf([text, image], top=3))        # ['b', 'd', 'a']
#  b is second in both → 1/62 + 1/62 = 0.03226
#  d is first and fourth → 1/61 + 1/64 = 0.03202

def hybrid_search(query, bm25_search, vector_search, retrieve=50, top=10, k=60):
    b = bm25_search(query, retrieve)
    v = vector_search(query, retrieve)
    return rrf([b, v], k=k, top=top)

# Evaluate: the hybrid AGAINST each part on its own, and the share of regressions
def evaluate(key, bm25_search, vector_search, k=10):
    results = {"bm25": 0, "vector": 0, "hybrid": 0}
    worse = 0
    for case in key:
        b = bm25_search(case["query"], k)
        v = vector_search(case["query"], k)
        h = rrf([b, v], top=k)
        hit = {"bm25": case["right"] in b, "vector": case["right"] in v,
               "hybrid": case["right"] in h}
        for name, t in hit.items():
            results[name] += int(t)
        if (hit["bm25"] or hit["vector"]) and not hit["hybrid"]:
            worse += 1
    n = len(key)
    return {name: round(v / n, 3) for name, v in results.items()} | {
        "regressions": round(worse / n, 3)}

# Two-stage: broad retrieval with the hybrid, then an expensive rerank of the top 50
def two_stage(query, bm25_search, vector_search, cross_encoder, top=10):
    candidates = hybrid_search(query, bm25_search, vector_search, retrieve=50, top=50)
    scores = cross_encoder.predict([(query, d) for d in candidates])
    return [d for _, d in sorted(zip(scores, candidates), reverse=True)][:top]

regressions in the evaluation is the number most often missing: the share of queries where one of the parts found the right thing but the hybrid did not. If the average rises while that figure is 0.15 you have made a sixth of the queries worse.