BM25 Retriever¶
📖 Overview¶
BM25 is a classical lexical information-retrieval algorithm that ranks documents based on how well their terms match the user's query.
Unlike dense vector retrieval, which represents text as embeddings, BM25 works directly with the words and terms present in the query and documents.
The fundamental flow is:
At query time:
BM25 remains highly relevant to modern enterprise RAG because exact lexical matching is extremely valuable for:
Product IDs
Error Codes
API Names
Class Names
Database Tables
Policy Numbers
Ticket IDs
Version Numbers
Technical Terms
Acronyms
The key architectural idea is:
Dense retrieval is strong at semantic similarity; BM25 is strong at lexical precision.
This makes BM25 an important building block for hybrid retrieval.
🎯 Learning Objectives¶
After completing this chapter, you will be able to:
- Understand lexical retrieval
- Understand BM25
- Understand TF-IDF and how BM25 improves upon it
- Understand term frequency
- Understand inverse document frequency
- Understand document-length normalization
- Understand BM25 parameters
- Understand inverted indexes
- Build a basic BM25 retriever
- Integrate BM25 with RAG
- Understand BM25 retrieval in LlamaIndex
- Compare BM25 with vector retrieval
- Understand hybrid search
- Understand BM25 limitations
- Tune BM25 for enterprise workloads
- Evaluate BM25 retrieval quality
- Design production lexical retrieval pipelines
1. What Is Lexical Retrieval?¶
Lexical retrieval searches for terms that occur in the query and documents.
Consider:
A lexical retriever looks for documents containing terms such as:
The more important query terms a document contains, the higher it may rank.
2. Lexical vs Semantic Retrieval¶
Consider:
Query:
Lexical retrieval can work well because:
appear directly or in closely related forms.
Now consider:
There may be little exact word overlap.
A dense vector retriever may perform better because it understands semantic similarity.
3. Core Difference¶
This distinction is fundamental.
4. BM25 Architecture¶
flowchart TD
A["Documents"] --> B["Tokenization"]
B --> C["Inverted Index"]
C --> D["BM25 Index"]
E["User Query"] --> F["Query Tokenization"]
F --> G["Term Lookup"]
D --> G
G --> H["BM25 Scoring"]
H --> I["Ranking"]
I --> J["Top-K Documents"]BM25 does not require an embedding model.
5. Why BM25 Still Matters¶
Modern RAG systems often focus heavily on embeddings.
However, enterprise queries frequently contain exact identifiers.
Examples:
A semantic embedding may understand these concepts imperfectly.
Lexical retrieval can directly match them.
6. Example: Error Codes¶
Suppose a document contains:
Query:
BM25 can strongly benefit from the exact match:
This is a classic lexical retrieval problem.
7. Example: Product Identifiers¶
Document:
Query:
Exact lexical matching can be extremely effective.
8. BM25 Mental Model¶
A simplified ranking idea is:
The actual BM25 formula is more sophisticated.
9. TF-IDF Foundation¶
BM25 evolved from classical information retrieval techniques such as TF-IDF.
TF:
measures how frequently a term occurs in a document.
IDF:
measures how distinctive the term is across the corpus.
10. Term Frequency¶
Suppose:
The term:
appears four times.
A basic TF concept is:
More occurrences generally increase relevance.
But simply rewarding frequency has problems.
11. Why Raw Frequency Is Not Enough¶
Consider:
and:
Document A contains more occurrences but may not actually provide more useful information.
BM25 therefore uses saturation.
Repeated occurrences provide diminishing returns.
12. Term Frequency Saturation¶
Conceptually:
1 occurrence
↓
Large relevance increase
2 occurrences
↓
Additional increase
10 occurrences
↓
Much smaller additional increase
This prevents documents from dominating merely because they repeat a term many times.
13. Inverse Document Frequency¶
Some words occur in many documents:
They are less distinctive.
Other terms occur rarely:
These are more informative.
IDF captures this distinction.
14. IDF Intuition¶
Therefore:
15. Document Frequency¶
Suppose the corpus contains:
Term:
appears in:
Term:
appears in:
The second term carries much more discriminative information.
16. BM25 Formula¶
A common BM25 formulation is:
IDF(t) × f(t,D) × (k₁ + 1)
score(D,Q) = Σ ───────────────────────────────
f(t,D) + k₁ × (1 - b + b × |D|/avgdl)
Where:
t
=
Query term
D
=
Document
f(t,D)
=
Frequency of term t in document D
|D|
=
Document length
avgdl
=
Average document length
k₁
=
Term-frequency saturation parameter
b
=
Document-length normalization parameter
17. BM25 Formula Visualization¶
The formula captures three important ideas:
18. BM25 Parameter k₁¶
k₁ controls how quickly term-frequency contribution saturates.
Conceptually:
A commonly used starting value is around:
but this should be treated as a tuning starting point rather than a universal optimum.
19. BM25 Parameter b¶
b controls document-length normalization.
Conceptually:
means little/no length normalization.
means strong length normalization.
A commonly used starting point is:
Again, production values should be evaluated against the actual corpus.
20. Why Document Length Matters¶
Suppose:
Both contain:
If term frequency were treated without normalization, the longer document could gain an unfair advantage simply because it contains more words.
BM25 accounts for document length.
21. Document-Length Normalization¶
Conceptually:
Short Document
↓
Term appears frequently
↓
Strong signal
Very Long Document
↓
Same term frequency
↓
Potentially weaker signal
This makes ranking more balanced.
22. BM25 Tokenization¶
Before retrieval, text is generally processed into terms.
Example:
might become:
Real tokenization depends on the implementation.
23. Stop Words¶
Some lexical retrieval systems remove common words such as:
These terms often provide little retrieval value.
However, blindly removing words can be harmful in some domains.
For example:
contains a meaningful negation.
Enterprise search pipelines should therefore treat stop-word configuration carefully.
24. Stemming and Lemmatization¶
Consider:
A lexical system may treat these as different terms unless normalization is applied.
Possible approaches include:
But aggressive normalization can also remove useful distinctions.
25. Enterprise Tokenization¶
Technical corpora contain:
A generic natural-language analyzer may not handle these perfectly.
Therefore enterprise lexical retrieval may require domain-specific analysis.
26. Inverted Index¶
BM25 commonly operates over an inverted index.
Instead of storing:
the inverted index stores:
Example:
27. Inverted Index Architecture¶
flowchart LR
A["Documents"] --> B["Analyzer"]
B --> C["Terms"]
C --> D["Inverted Index"]
D --> E["Kafka → D1, D7, D21"]
D --> F["OAuth → D3, D8"]
D --> G["Retry → D1, D9, D21"]
H["Query Terms"] --> DThis makes term lookup efficient.
28. BM25 Retrieval Flow¶
Query
↓
Analyze
↓
Extract Terms
↓
Lookup Inverted Index
↓
Find Candidate Documents
↓
Calculate BM25 Scores
↓
Rank
↓
Top-K
This differs substantially from vector retrieval.
29. BM25 vs Vector Retrieval¶
| Capability | BM25 | Vector |
|---|---|---|
| Exact term matching | Excellent | Variable |
| Semantic similarity | Limited | Excellent |
| IDs / error codes | Excellent | Variable |
| Synonyms | Limited | Strong |
| Vocabulary mismatch | Weak | Strong |
| Explainability | Strong | Lower |
| Embedding required | No | Yes |
| Inverted index | Yes | No |
| Semantic paraphrases | Weak | Strong |
This is why the two approaches complement each other.
30. Example Comparison¶
Document:
Query A:
BM25:
Query B:
Vector retrieval may have an advantage.
31. Exact Matching Strength¶
BM25 can be particularly useful for:
These are difficult cases where semantic retrieval may not always preserve exact lexical signals.
32. Semantic Matching Limitation¶
Suppose the document says:
Query:
The terms may not overlap strongly.
BM25 may perform poorly.
A dense retriever may identify the semantic relationship more effectively.
33. Hybrid Retrieval¶
The strongest enterprise architecture often combines:
Architecture:
flowchart TD
A["User Query"] --> B["BM25 Retriever"]
A --> C["Vector Retriever"]
B --> D["Lexical Candidates"]
C --> E["Semantic Candidates"]
D --> F["Fusion"]
E --> F
F --> G["Re-ranking"]
G --> H["Final Context"]34. Why Hybrid Retrieval Works¶
BM25 provides:
Vector search provides:
Together:
This is one of the most important enterprise RAG patterns.
35. Fusion Strategies¶
BM25 and vector results can be combined using:
A simple architecture:
Then:
36. Reciprocal Rank Fusion¶
A common rank-fusion approach is Reciprocal Rank Fusion (RRF).
Conceptually:
The idea is to reward documents that appear near the top of multiple ranked lists.
This avoids requiring BM25 and vector scores to be directly comparable.
37. Why Rank Fusion Helps¶
Suppose:
Vector:
Documents:
appear strongly across both retrieval strategies.
Fusion can identify this consensus.
38. Hybrid Retrieval with RRF¶
flowchart LR
A["Query"] --> B["BM25"]
A --> C["Vector Search"]
B --> D["Ranked List A"]
C --> E["Ranked List B"]
D --> F["RRF"]
E --> F
F --> G["Unified Ranking"]
G --> H["Re-ranker"]39. BM25 in LlamaIndex¶
LlamaIndex can integrate with lexical retrieval components and external retrieval backends.
A practical enterprise architecture may look like:
The exact implementation depends on the LlamaIndex version and selected integration.
40. Basic BM25 Example¶
A simple Python implementation can be built using a BM25 library.
For example:
from rank_bm25 import BM25Okapi
documents = [
"Kafka consumers retry failed events.",
"OAuth tokens authenticate payment APIs.",
"PostgreSQL stores transaction records."
]
tokenized_documents = [
document.lower().split()
for document in documents
]
bm25 = BM25Okapi(
tokenized_documents
)
query = "Kafka consumer retry"
tokenized_query = query.lower().split()
scores = bm25.get_scores(
tokenized_query
)
print(scores)
The library shown here is a standalone BM25 implementation rather than a LlamaIndex-specific API.
41. Ranking Documents¶
results = bm25.get_top_n(
tokenized_query,
documents,
n=3
)
for document in results:
print(document)
Conceptually:
42. LlamaIndex-Oriented Architecture¶
A framework-integrated design can expose:
The application should ideally not need to know:
Those should remain inside the adapter layer.
43. Capability-Based Retriever Interface¶
from abc import ABC, abstractmethod
class LexicalRetriever(ABC):
@abstractmethod
def retrieve(
self,
query: str,
top_k: int
):
pass
Implementation:
class BM25LexicalRetriever(
LexicalRetriever
):
def __init__(self, index):
self.index = index
def retrieve(self, query, top_k):
return self.index.retrieve(
query,
top_k=top_k
)
This keeps the application framework-agnostic.
44. Ports & Adapters Architecture¶
flowchart LR
A["RAG Application"] --> B["LexicalRetriever Port"]
B --> C["BM25 Adapter"]
B --> D["OpenSearch Adapter"]
B --> E["Other Lexical Adapter"]
C --> F["BM25 Engine"]
D --> G["Search Backend"]The enterprise application depends on:
rather than:
45. BM25 + Vector Adapter¶
A unified retrieval interface can expose:
Implementations:
Then:
This fits naturally into an enterprise retrieval factory.
46. Retriever Factory¶
Factory:
def create_retriever(
retriever_type,
config
):
if retriever_type == "vector":
return VectorRetriever(config)
if retriever_type == "bm25":
return BM25Retriever(config)
if retriever_type == "hybrid":
return HybridRetriever(config)
raise ValueError(
f"Unsupported retriever: {retriever_type}"
)
This is a simple conceptual implementation.
47. BM25 Metadata Filtering¶
BM25 itself primarily scores lexical relevance.
Enterprise retrieval may still need:
Therefore:
or an index/backend capable of combining filtering and lexical scoring.
48. Filter Before Retrieval¶
Conceptually:
This prevents irrelevant or unauthorized documents from entering the candidate pool.
49. BM25 + Metadata Example¶
Query:
Filters:
The retrieval operation becomes:
This is much more useful in enterprise systems than unrestricted lexical search.
50. BM25 for Technical Documentation¶
BM25 is particularly effective for:
API documentation
Error messages
Runbooks
Incident reports
Configuration guides
CLI documentation
Architecture references
because users frequently search using the exact terminology appearing in the source.
51. BM25 for Incident Management¶
Suppose an incident document contains:
Query:
A lexical retriever is extremely well suited to this problem.
52. BM25 for Code Search¶
Consider:
Query:
BM25-style lexical retrieval can be very effective because the exact identifier is the key signal.
Dense retrieval can complement this for natural-language questions such as:
53. BM25 for Policy Search¶
Policy:
Query:
BM25 can strongly match:
This is a useful lexical signal.
54. BM25 and Numeric Values¶
Numeric and version-sensitive queries require careful analyzer design.
Examples:
A generic analyzer may tokenize these differently.
Production search systems should explicitly test:
55. BM25 and Acronyms¶
Enterprise documents frequently contain:
Lexical matching can preserve these exact terms.
This is another reason BM25 remains useful in technical knowledge bases.
56. Query Expansion¶
BM25 can benefit from query expansion.
Original:
Possible related terms:
However, adding too many terms can introduce noise.
Query expansion should therefore be evaluated carefully.
57. Query Rewriting¶
A query can be rewritten for lexical retrieval.
User:
Lexical rewrite:
The rewritten query can be passed to BM25.
This is particularly useful when the user's natural-language phrasing differs from enterprise terminology.
58. BM25 + Query Rewriting¶
flowchart TD
A["Natural Language Query"] --> B["Query Rewriter"]
B --> C["Lexical Query"]
C --> D["BM25"]
D --> E["Ranked Results"]
E --> F["Re-ranking"]59. BM25 and Multi-Query¶
Multiple lexical queries can also be generated:
Each can retrieve candidates.
Then:
This extends BM25 beyond simple single-query matching.
60. BM25 and MMR¶
BM25 can produce duplicate or highly similar documents.
Example:
MMR or deduplication can improve result diversity.
Architecture:
61. BM25 and Re-ranking¶
A strong lexical pipeline:
BM25 provides:
The re-ranker provides:
62. Production Lexical Retrieval¶
A mature architecture can be:
Query
│
▼
Query Processing
│
▼
Security Filtering
│
┌───────┴────────┐
▼ ▼
BM25 Vector Search
│ │
▼ ▼
Lexical Top-K Semantic Top-K
│ │
└───────┬────────┘
▼
Fusion
│
▼
Re-ranking
│
▼
Context
This is the core of hybrid enterprise retrieval.
63. BM25 Latency¶
BM25 is generally computationally efficient because it operates over an inverted index.
Potential pipeline:
Latency depends on:
64. BM25 Scalability¶
For large corpora, use a production search engine that provides:
Examples include systems based on:
The appropriate platform depends on enterprise infrastructure and operational requirements.
65. BM25 and OpenSearch¶
A common enterprise architecture is:
OpenSearch can provide:
The LlamaIndex integration should be treated as an adapter rather than embedding the entire search platform inside application logic.
66. BM25 vs Database LIKE¶
It is tempting to implement lexical retrieval with:
This is generally not equivalent to BM25.
LIKE primarily performs pattern matching.
BM25 provides:
Therefore:
67. BM25 vs Full-Text Search¶
Database engines may provide full-text search features.
These can be useful for smaller applications.
However, production search requirements may require:
A dedicated search engine can be more appropriate at scale.
68. BM25 Limitations¶
BM25 does not inherently understand:
For example:
and:
may not match strongly unless the retrieval pipeline handles synonyms or query expansion.
Dense retrieval can provide semantic matching.
69. BM25 Failure Cases¶
Query:
Document:
Low lexical overlap may hurt BM25.
A vector retriever may perform better.
70. Dense Retrieval Failure Cases¶
Query:
Document:
Semantic retrieval may not always rank the exact identifier as strongly as lexical search.
BM25 provides an important complementary signal.
71. Best Practice¶
Do not ask:
For many enterprise applications, ask:
The answer may be:
72. Evaluation Dataset¶
Create queries covering:
Exact identifiers
Natural language
Technical terms
Synonyms
Acronyms
Numbers
Versions
Multi-term queries
Long queries
Short queries
Example:
[
{
"query": "HTTP 429",
"type": "exact"
},
{
"query": "How does API rate limiting work?",
"type": "semantic"
},
{
"query": "PAY-GW-2026",
"type": "identifier"
}
]
73. BM25 Evaluation¶
Measure:
Compare:
using the same evaluation dataset.
74. Example Evaluation Matrix¶
| Query Type | BM25 | Vector | Hybrid |
|---|---|---|---|
| Exact ID | Strong | Variable | Strong |
| Error Code | Strong | Variable | Strong |
| Natural Language | Moderate | Strong | Strong |
| Synonym | Weak | Strong | Strong |
| Technical Term | Strong | Strong | Strong |
| Paraphrase | Weak | Strong | Strong |
The ratings are conceptual and should be validated against your own corpus.
75. BM25 Tuning¶
Tune:
Do not tune parameters independently without measuring overall retrieval quality.
76. k₁ Tuning Intuition¶
If the corpus contains:
repeated terms may carry meaningful signals.
If the corpus contains:
repetition may be less informative.
Therefore k₁ should be evaluated against corpus characteristics.
77. b Tuning Intuition¶
If document lengths vary significantly:
length normalization can become important.
If documents are already highly uniform in size, aggressive normalization may provide less benefit.
78. Analyzer Tuning¶
For enterprise technical data, test:
Example:
should ideally remain searchable as a meaningful identifier.
79. BM25 Observability¶
Track:
Example:
{
"retriever": "bm25",
"index_version": "docs-v8",
"query_terms": [
"http",
"429"
],
"top_k": 10,
"result_count": 10,
"latency_ms": 18
}
Avoid logging sensitive document content.
80. BM25 Monitoring¶
Important production metrics:
P50 Latency
P95 Latency
P99 Latency
Zero-result Rate
Average Result Count
Score Distribution
Index Size
Index Freshness
Query Volume
Error Rate
For hybrid systems also monitor:
81. BM25 Index Freshness¶
Enterprise search systems must update when source documents change.
Track:
to measure freshness.
82. Incremental BM25 Indexing¶
When one document changes:
A production search engine manages these index operations.
The important architectural principle is:
unless there is a specific reason.
83. BM25 Deletion¶
If a document is deleted:
Otherwise the document can remain searchable.
84. BM25 Security¶
Lexical search can expose the same sensitive information as vector search.
Security must cover:
Do not assume:
85. Tenant Isolation¶
A shared search index might contain:
The query must include a trusted tenant filter:
The filter must be generated by the application's security context.
86. BM25 in a Multi-Stage Retriever¶
A mature pipeline can use:
This allows each component to specialize.
87. Multi-Stage Architecture¶
flowchart TD
A["Query"] --> B["BM25"]
A --> C["Vector"]
B --> D["Lexical Candidates"]
C --> E["Semantic Candidates"]
D --> F["Fusion"]
E --> F
F --> G["Top-50"]
G --> H["Re-ranking"]
H --> I["Top-10"]
I --> J["Context Compression"]
J --> K["LLM"]88. BM25 as Candidate Generator¶
BM25 does not necessarily need to produce the final five documents.
It can produce:
and allow:
to select:
This separates:
from:
89. Hybrid Retrieval Decision¶
Use BM25 when:
Use vector retrieval when:
Use hybrid retrieval when:
Example:
This contains:
Hybrid retrieval is ideal.
90. Enterprise Search Architecture¶
flowchart TD
A["Enterprise Query"] --> B["Query Planner"]
B --> C["Security Context"]
C --> D["Metadata Filters"]
D --> E["BM25 Search"]
D --> F["Vector Search"]
E --> G["Lexical Results"]
F --> H["Semantic Results"]
G --> I["Fusion"]
H --> I
I --> J["Re-ranking"]
J --> K["Context Engineering"]
K --> L["LLM"]
L --> M["Response Validation"]
M --> N["Citations"]This is the practical role of BM25 in enterprise RAG.
91. Practical Configuration¶
A conceptual configuration might be:
retrieval:
lexical:
enabled: true
engine: bm25
top_k: 50
parameters:
k1: 1.2
b: 0.75
analyzer:
lowercase: true
stemming: false
hybrid:
enabled: true
fusion: rrf
reranking:
enabled: true
top_k: 5
These values are starting points, not universal production defaults.
92. Testing BM25¶
Test at least:
Exact match
Partial match
Case differences
Pluralization
Acronyms
Numbers
Versions
Identifiers
Long queries
Short queries
No-result queries
Example:
93. BM25 Test Case¶
def test_exact_error_code_retrieval():
query = "HTTP 429"
results = retriever.retrieve(
query,
top_k=5
)
assert any(
"429" in result.text
for result in results
)
The exact assertion depends on your retrieval abstraction.
94. BM25 Regression Testing¶
Whenever changing:
run the retrieval benchmark again.
Track:
This prevents silent retrieval degradation.
95. BM25 Deployment Lifecycle¶
Source Documents
↓
Analyzer
↓
BM25 Index
↓
Validation
↓
Retrieval Benchmark
↓
Quality Gate
↓
Publish
↓
Monitor
This is similar to an ML model deployment lifecycle.
96. BM25 Index Versioning¶
Example:
Version changes may represent:
Track these explicitly.
97. Blue-Green Search Index¶
After validation:
If retrieval quality drops:
98. Common Anti-Patterns¶
Anti-Pattern 1 — Treating BM25 as Semantic Search¶
Incorrect.
BM25 is fundamentally lexical.
Anti-Pattern 2 — Removing BM25 Because Embeddings Exist¶
Dense retrieval does not eliminate the value of exact lexical matching.
99. Common Anti-Patterns — Continued¶
Anti-Pattern 3 — Using Default Tokenization Everywhere¶
Technical corpora may require custom analyzers.
Anti-Pattern 4 — Ignoring Document Length¶
Document length can influence BM25 ranking.
Anti-Pattern 5 — Treating Scores as Probabilities¶
does not mean:
Scores are ranking signals.
100. Common Anti-Patterns — Continued¶
Anti-Pattern 6 — No Hybrid Evaluation¶
If hybrid retrieval is introduced, measure:
independently.
Anti-Pattern 7 — No Security Filter¶
Never search the entire enterprise corpus and hope the LLM removes unauthorized information.
101. Production Checklist¶
☐ Define lexical retrieval use cases
☐ Identify exact-match requirements
☐ Select BM25 backend
☐ Define analyzer
☐ Define tokenizer
☐ Evaluate stemming
☐ Evaluate stop words
☐ Test identifiers
☐ Test numbers
☐ Test versions
☐ Configure k₁
☐ Configure b
☐ Configure Top-K
☐ Implement metadata filtering
☐ Implement tenant isolation
☐ Evaluate Recall@K
☐ Evaluate Precision@K
☐ Evaluate MRR / NDCG
☐ Measure P95 latency
☐ Monitor zero-result rate
☐ Version the index
☐ Version analyzer configuration
☐ Implement incremental updates
☐ Implement deletion
☐ Consider hybrid retrieval
☐ Consider re-ranking
102. Key Takeaways¶
- BM25 is a classical lexical retrieval algorithm.
- BM25 does not require embeddings.
- BM25 operates over terms and an inverted index.
- Term frequency contributes to relevance.
- BM25 applies term-frequency saturation.
- IDF gives greater importance to rare terms.
- Document-length normalization prevents long documents from receiving unfair advantages.
k₁controls term-frequency saturation.bcontrols document-length normalization.- Exact identifiers are one of BM25's strongest use cases.
- Error codes, API names, versions, and technical identifiers benefit from lexical retrieval.
- BM25 is weaker at semantic paraphrases and synonyms.
- Vector retrieval is stronger for semantic similarity.
- BM25 and vector retrieval complement each other.
- Hybrid retrieval is a strong enterprise RAG pattern.
- Reciprocal Rank Fusion can combine BM25 and vector rankings.
- BM25 can serve as a first-stage candidate generator.
- Re-ranking can improve final precision.
- MMR can reduce duplicate lexical results.
- Query rewriting can improve lexical retrieval.
- Enterprise analyzers should be tested against technical identifiers and terminology.
- Metadata filtering and tenant isolation remain important for BM25.
- BM25 scores should be treated as ranking signals rather than probabilities.
- Retrieval quality must be evaluated independently from LLM generation.
- Index freshness, latency, zero-result rate, and ranking quality should be monitored.
- BM25 indexes should support versioning, validation, rollback, and deletion.
- A capability-based
LexicalRetrieverabstraction keeps enterprise applications independent of a specific search engine. - BM25 is not a replacement for vector retrieval; it is an important complementary retrieval capability.
The central architecture is:
USER QUERY
│
▼
Query Processing
│
▼
Security Context
│
▼
Metadata Filtering
│
▼
┌─────────┐
│ BM25 │
└────┬────┘
│
▼
Lexical Candidates
│
▼
Top-K Results
│
▼
Hybrid Fusion / MMR
│
▼
Re-ranking
│
▼
Context Selection
│
▼
LLM
│
▼
Validated Response
BM25 provides the lexical precision that dense retrieval alone can miss. In enterprise RAG, the strongest architecture often combines lexical signals, semantic signals, metadata constraints, and re-ranking into a multi-stage retrieval pipeline.
🧭 Chapter Navigation¶
Part V — Advanced Retrieval-Augmented Generation¶
Previous:
03. Vector Index Retriever
Next:
05. Document Summary Retriever
Section:
03 — LlamaIndex Retrieval Engineering
LlamaIndex Retrieval Engineering Path¶
01 LlamaIndex Retrievers Overview
↓
02 LlamaIndex Indexes
↓
03 Vector Index Retriever
↓
04 BM25 Retriever
↓
05 Document Summary Retriever
↓
06 Recursive Retriever
↓
07 Query Fusion Retriever
↓
08 Auto-Merging Retriever
↓
04 Vector Search Engineering
Enterprise AI Engineering Handbook
Building Production-Grade Enterprise AI Systems — One Chapter at a Time.