LlamaIndex Retrievers Overview¶
📖 Overview¶
LlamaIndex is a framework for building applications that connect Large Language Models (LLMs) with external data.
In Retrieval-Augmented Generation (RAG), LlamaIndex provides abstractions for:
While the previous chapters focused on retrieval engineering concepts and framework-independent architectures, this section focuses specifically on how LlamaIndex implements and composes retrieval capabilities.
The goal is not simply to learn individual LlamaIndex classes.
The goal is to understand:
LlamaIndex Retrieval
↓
Indexes
↓
Retriever Abstractions
↓
Specialized Retrievers
↓
Composable Retrieval
↓
Enterprise RAG Architecture
🎯 Learning Objectives¶
After completing this chapter, you will be able to:
- Understand the LlamaIndex retrieval architecture
- Understand Documents, Nodes, Indexes, and Retrievers
- Understand how retrieval fits into the LlamaIndex query pipeline
- Understand the relationship between indexes and retrievers
- Identify major LlamaIndex retriever types
- Understand vector-based retrieval
- Understand keyword and BM25 retrieval
- Understand recursive retrieval
- Understand query fusion
- Understand document-summary retrieval
- Understand auto-merging retrieval
- Understand how LlamaIndex supports advanced RAG architectures
- Compare LlamaIndex retrieval concepts with generic RAG concepts
- Design framework-aware retrieval components
- Integrate LlamaIndex retrieval into enterprise RAG systems
1. LlamaIndex in the RAG Architecture¶
A typical RAG application looks like:
┌──────────────────┐
│ User Query │
└────────┬─────────┘
↓
┌──────────────────┐
│ Query Processing │
└────────┬─────────┘
↓
┌──────────────────┐
│ Retriever │
└────────┬─────────┘
↓
┌──────────────────┐
│ Retrieved Nodes │
└────────┬─────────┘
↓
┌──────────────────┐
│ Context Builder │
└────────┬─────────┘
↓
┌──────────────────┐
│ LLM │
└────────┬─────────┘
↓
┌──────────────────┐
│ Response │
└──────────────────┘
LlamaIndex provides abstractions for many of these stages.
2. High-Level LlamaIndex Architecture¶
flowchart TD
A["Source Data"] --> B["Documents"]
B --> C["Nodes"]
C --> D["Index"]
D --> E["Retriever"]
E --> F["Query Engine"]
F --> G["Response Synthesizer"]
G --> H["LLM Response"]The important relationship is:
3. Documents¶
A Document represents a source piece of information.
Sources can include:
Conceptually:
from llama_index.core import Document
document = Document(
text="""
Payment authentication requires
OAuth-based authorization.
"""
)
The document represents the source content before retrieval-specific processing.
4. Nodes¶
LlamaIndex works extensively with Nodes.
A document can be transformed into smaller units:
A node typically contains:
Conceptually:
from llama_index.core import Document
from llama_index.core.node_parser import SentenceSplitter
document = Document(
text="Payment authentication requires OAuth..."
)
parser = SentenceSplitter(
chunk_size=512,
chunk_overlap=50
)
nodes = parser.get_nodes_from_documents(
[document]
)
5. Nodes Are Central to Retrieval¶
Retrieval usually returns:
rather than entire documents.
For example:
Each node can carry:
This is important for:
6. Documents vs Nodes¶
| Concept | Purpose |
|---|---|
| Document | Represents source content |
| Node | Represents a retrievable unit |
| Metadata | Describes the content |
| Relationship | Connects nodes |
| Index | Organizes data for retrieval |
| Retriever | Finds relevant nodes |
Think of:
7. What Is an Index?¶
An index organizes data so that it can be efficiently searched.
Conceptually:
Different indexes can support different retrieval strategies.
Common LlamaIndex index concepts include:
The exact available classes and APIs can vary by LlamaIndex version.
8. Why Multiple Indexes?¶
Different information needs different retrieval strategies.
For example:
Semantic Search
↓
Vector Index
Keyword Search
↓
Keyword / BM25
Document Summaries
↓
Summary Index
Graph Relationships
↓
Knowledge Graph
This leads to an important principle:
The index should match the retrieval problem.
9. VectorStoreIndex¶
A vector index stores embeddings representing semantic meaning.
Architecture:
Query:
10. Vector Retrieval¶
flowchart TD
A["Documents"] --> B["Chunking"]
B --> C["Nodes"]
C --> D["Embedding Model"]
D --> E["Vector Store"]
F["User Query"] --> G["Query Embedding"]
G --> H["Similarity Search"]
E --> H
H --> I["Top-K Nodes"]This is the standard semantic retrieval pattern.
11. Basic Vector Index Example¶
A simplified LlamaIndex example:
from llama_index.core import VectorStoreIndex
from llama_index.core import Document
documents = [
Document(
text="Payment authentication uses OAuth."
),
Document(
text="Payment failures are retried."
)
]
index = VectorStoreIndex.from_documents(
documents
)
retriever = index.as_retriever(
similarity_top_k=3
)
nodes = retriever.retrieve(
"How does payment authentication work?"
)
for node in nodes:
print(node.text)
The exact API may vary with the installed LlamaIndex version.
12. Retriever Abstraction¶
The important architectural concept is the retriever.
Conceptually:
returns relevant nodes.
This provides an abstraction over:
The application can therefore work with a retrieval capability rather than directly managing every storage mechanism.
13. Retriever Architecture¶
flowchart LR
A["Query"] --> B["Retriever"]
B --> C["Index / Store"]
C --> D["Candidate Nodes"]
D --> B
B --> E["NodeWithScore"]A retrieval result can contain:
14. NodeWithScore¶
A retriever commonly returns a node together with a relevance score.
Conceptually:
The score can later be used by:
15. Retrieval Is Not Generation¶
This distinction is fundamental.
Retriever:
LLM:
Therefore:
A retriever should not be responsible for generating the final response.
16. Retriever vs Query Engine¶
A retriever answers:
A query engine typically handles a larger workflow:
Conceptually:
17. LlamaIndex Query Pipeline¶
flowchart TD
A["User Query"] --> B["Query Engine"]
B --> C["Retriever"]
C --> D["Relevant Nodes"]
D --> E["Response Synthesizer"]
E --> F["LLM"]
F --> G["Final Response"]The retriever is therefore one component inside the larger RAG execution pipeline.
18. Retriever Types in This Section¶
This section will explore several LlamaIndex retrieval capabilities:
Vector Index Retriever
↓
BM25 Retriever
↓
Document Summary Retriever
↓
Recursive Retriever
↓
Query Fusion Retriever
↓
Auto-Merging Retriever
These concepts progressively move from:
toward:
19. Vector Retrieval¶
Vector retrieval answers:
"Which nodes are semantically similar to the query?"
Example:
Query:
How does authentication work?
Retrieved:
OAuth authentication architecture
Token validation
Identity service
Even if the exact word:
is not present in every retrieved node, semantic similarity can identify related content.
20. Keyword Retrieval¶
Keyword retrieval answers:
"Which documents contain important query terms?"
Example:
Keyword retrieval can be especially useful for:
21. Why Vector Search Is Not Enough¶
Suppose the query is:
Dense retrieval may understand:
But exact matching of:
can be extremely valuable.
This is one reason enterprise systems often combine:
22. LlamaIndex and Retrieval Composition¶
One of LlamaIndex's strengths is the ability to compose retrieval components.
Instead of:
you can build:
This becomes important in enterprise RAG.
23. Composable Retrieval¶
flowchart TD
A["User Query"] --> B["Retriever A"]
A --> C["Retriever B"]
A --> D["Retriever C"]
B --> E["Candidates"]
C --> E
D --> E
E --> F["Fusion / Ranking"]
F --> G["Final Nodes"]Possible components:
24. Retrieval Abstraction¶
From an architecture perspective:
This is useful because the application does not need to know every internal retrieval implementation.
25. Generic Retrieval Interface¶
A framework-agnostic application might define:
A LlamaIndex adapter can implement that capability.
class LlamaIndexRetrieverAdapter:
def __init__(self, retriever):
self.retriever = retriever
def retrieve(self, query, top_k):
return self.retriever.retrieve(query)
This keeps the application architecture independent from the framework.
26. Framework Adapter Architecture¶
flowchart TD
A["Enterprise RAG Application"] --> B["Retriever Port"]
B --> C["LlamaIndex Adapter"]
B --> D["LangChain Adapter"]
B --> E["Custom Retriever"]
C --> F["LlamaIndex"]
D --> G["LangChain"]
E --> H["Custom Retrieval Engine"]This architecture is particularly useful when building reusable enterprise AI platforms.
27. LlamaIndex Storage Context¶
LlamaIndex applications can separate:
from:
The storage layer may include:
This separation allows persistence and integration with external storage systems.
28. Conceptual Storage Architecture¶
┌─────────────────┐
│ Index │
└────────┬────────┘
↓
┌─────────────────┼─────────────────┐
↓ ↓ ↓
Vector Store Document Store Index Store
Each storage component has a different responsibility.
29. Vector Store¶
The vector store holds:
and often associated:
Its primary responsibility is similarity retrieval.
30. Document Store¶
A document store can maintain:
This is useful when the retrieval process needs to resolve:
31. Index Store¶
The index store can persist information about:
This helps LlamaIndex applications reconstruct indexes without rebuilding everything from the source documents.
32. Separation of Concerns¶
A production architecture benefits from:
Each layer has a clear responsibility.
33. Metadata in LlamaIndex¶
Nodes can contain metadata.
Example:
from llama_index.core import Document
document = Document(
text="Payment authentication uses OAuth.",
metadata={
"department": "payments",
"document_type": "architecture",
"status": "approved"
}
)
Metadata can later participate in retrieval filtering and downstream processing.
34. Metadata-Aware Retrieval¶
flowchart TD
A["User Query"] --> B["Query Processing"]
B --> C["Metadata Filters"]
C --> D["LlamaIndex Retriever"]
D --> E["Filtered Candidate Nodes"]
E --> F["Ranking"]
F --> G["Context"]This connects the previous chapter's metadata-aware retrieval concepts with LlamaIndex.
35. Node Relationships¶
LlamaIndex nodes can represent relationships between content units.
Conceptually:
Relationships can support advanced retrieval strategies.
For example:
36. Why Relationships Matter¶
A small chunk may be highly relevant:
But answering the question may require the larger section:
Relationships help connect the retrieved fragment with surrounding context.
37. Recursive Retrieval¶
Recursive retrieval can follow relationships between retrieval objects.
Conceptually:
This is particularly useful for:
The dedicated recursive retriever chapter will explore this in detail.
38. Query Fusion¶
Multiple retrieval queries can be combined.
Example:
This improves coverage for ambiguous or complex questions.
39. Auto-Merging Retrieval¶
Auto-merging retrieval addresses hierarchical context.
Suppose:
Document
├── Section A
│ ├── Chunk 1
│ ├── Chunk 2
│ └── Chunk 3
│
└── Section B
├── Chunk 4
└── Chunk 5
Retrieval may identify:
Instead of returning only those fragments, the system can merge them into a larger parent context.
40. Hierarchical Retrieval¶
flowchart TD
A["Document"] --> B["Section A"]
A --> C["Section B"]
B --> D["Chunk 1"]
B --> E["Chunk 2"]
B --> F["Chunk 3"]
C --> G["Chunk 4"]
C --> H["Chunk 5"]
I["Query"] --> D
I --> E
D --> J["Parent Resolution"]
E --> J
J --> BThis improves contextual completeness.
41. LlamaIndex Retrieval Ecosystem¶
The retrieval ecosystem can be viewed as:
LlamaIndex Retrieval
│
┌────────────────┼─────────────────┐
↓ ↓ ↓
Indexes Retrievers Query Engines
│ │
┌────┼────┐ ┌────┼────┐
↓ ↓ ↓ ↓ ↓ ↓
Vector Summary Keyword Recursive Fusion
This architecture allows retrieval to become composable.
42. Index vs Retriever¶
This distinction is extremely important.
Index¶
Answers:
Retriever¶
Answers:
For example:
The index provides the underlying structure.
The retriever defines the query-time retrieval behavior.
43. Retriever vs Query Engine¶
Another important distinction:
while:
Architecture:
44. Retrieval vs Response Synthesis¶
A retrieval system should answer:
A response synthesizer answers:
This separation is valuable for:
45. Retrieval Pipeline¶
A mature LlamaIndex pipeline may look like:
Documents
↓
Node Parsing
↓
Indexing
↓
Retriever
↓
Candidate Nodes
↓
Postprocessing
↓
Context
↓
Response Synthesizer
↓
LLM
46. Node Postprocessors¶
Between retrieval and generation, additional processing may occur.
Conceptually:
Possible operations include:
This connects LlamaIndex retrieval with the broader retrieval engineering techniques covered earlier.
47. Retrieval Score Threshold¶
A system may retrieve:
but then remove low-confidence nodes:
Conceptually:
Thresholds should be calibrated rather than chosen arbitrarily.
48. LlamaIndex and Re-ranking¶
A LlamaIndex retrieval pipeline can be extended with re-ranking.
Architecture:
This follows the general retrieval architecture:
49. LlamaIndex and MMR¶
Maximum Marginal Relevance can be used to reduce redundancy.
Example:
MMR may produce:
This is useful when multiple chunks contain almost identical information.
50. LlamaIndex and Hybrid Retrieval¶
Enterprise systems often combine:
LlamaIndex can participate in this architecture through its retrieval abstractions and integrations.
The exact implementation depends on the configured retrievers and storage backends.
51. LlamaIndex Retrieval Composition¶
flowchart TD
A["User Query"] --> B["Query Processing"]
B --> C["Vector Retriever"]
B --> D["Keyword Retriever"]
B --> E["Metadata Filter"]
C --> F["Fusion"]
D --> F
E --> F
F --> G["Re-ranking"]
G --> H["Context"]
H --> I["LLM"]This is a practical enterprise retrieval pattern.
52. LlamaIndex and Enterprise RAG¶
A production enterprise architecture can be:
User
↓
API Gateway
↓
Query Orchestrator
↓
┌─────────┴──────────┐
↓ ↓
Security Context Query Rewriting
│ │
└─────────┬──────────┘
↓
LlamaIndex Layer
↓
┌──────────┼──────────┐
↓ ↓ ↓
Vector BM25 Graph
Retrieval Retrieval Retrieval
└──────────┼──────────┘
↓
Fusion
↓
Re-ranking
↓
MMR
↓
Context Engineering
↓
LLM / SLM
↓
Response Validation
↓
Citation
↓
Enterprise Response
LlamaIndex can therefore be one layer inside a larger enterprise architecture rather than the entire architecture.
53. Framework-Aware vs Framework-Agnostic Design¶
A useful engineering distinction:
Framework-Agnostic¶
Framework Adapter¶
This allows an enterprise platform to support multiple frameworks.
54. Why This Matters¶
If application code directly depends on:
throughout the codebase, migration becomes difficult.
Instead:
provides a cleaner architecture.
55. Ports & Adapters Pattern¶
flowchart LR
A["RAG Application"] --> B["Retriever Port"]
B --> C["LlamaIndex Adapter"]
B --> D["LangChain Adapter"]
B --> E["Custom Adapter"]
C --> F["LlamaIndex"]
D --> G["LangChain"]
E --> H["Custom Retrieval"]This is especially valuable for enterprise platforms where framework choices may evolve.
56. Capability-Based Retrieval¶
Instead of abstracting an entire framework, define capabilities:
VectorRetriever
KeywordRetriever
MetadataRetriever
HybridRetriever
Reranker
QueryRewriter
ContextCompressor
Then map framework implementations to those capabilities.
57. Capability Example¶
LlamaIndex implementation:
class LlamaIndexVectorRetriever(
VectorRetriever
):
def __init__(self, retriever):
self.retriever = retriever
def retrieve(self, query, top_k):
return self.retriever.retrieve(query)
This keeps the enterprise architecture focused on capabilities.
58. Retrieval Engineering Perspective¶
LlamaIndex should not be viewed merely as:
It can be viewed as a toolkit for:
The important engineering question is:
Which retrieval capability should be used for this information problem?
59. Choosing a Retrieval Strategy¶
| Requirement | Potential Strategy |
|---|---|
| Semantic similarity | Vector retrieval |
| Exact identifiers | BM25 / keyword |
| Hierarchical context | Auto-merging / recursive |
| Multiple query perspectives | Query fusion |
| Document-level summaries | Summary retrieval |
| Structured metadata | Metadata filtering |
| Diverse evidence | MMR |
| High precision | Re-ranking |
| Complex reasoning | Agentic retrieval |
These techniques can also be composed.
60. Example: Enterprise Payment Knowledge Base¶
Suppose the knowledge base contains:
A single vector retriever may not be optimal.
Instead:
Query
↓
Query Classification
↓
Router
├── API → API Retriever
├── Policy → Policy Retriever
├── Incident → Incident Retriever
├── Architecture → Vector Retriever
└── Database → SQL Retriever
LlamaIndex retrieval components can participate in such an architecture.
61. Retrieval Router¶
flowchart TD
A["Enterprise Query"] --> B["Query Classifier"]
B --> C{"Domain"}
C -->|API| D["API Retriever"]
C -->|Policy| E["Policy Retriever"]
C -->|Incident| F["Incident Retriever"]
C -->|Architecture| G["Vector Retriever"]
C -->|Database| H["SQL Retriever"]
D --> I["Evidence"]
E --> I
F --> I
G --> I
H --> IThis architecture will become increasingly important in later chapters.
62. LlamaIndex as a Retrieval Layer¶
A useful mental model:
Enterprise AI Application
↓
Retrieval API
↓
Retrieval Engine
↓
LlamaIndex
↓
┌──────────┼───────────┐
↓ ↓ ↓
Index Retriever Postprocessor
LlamaIndex provides implementation capabilities while the enterprise application owns business rules.
63. What Should Stay Outside the Framework?¶
Enterprise concerns such as:
Authentication
Authorization
Tenant Isolation
Business Policy
Cost Budgets
Audit Requirements
Compliance
Service-Level Objectives
should generally remain controlled by the application/platform architecture.
The framework should not become the security boundary.
64. Enterprise Security Boundary¶
flowchart TD
A["User"] --> B["Enterprise API"]
B --> C["Authentication"]
C --> D["Authorization"]
D --> E["Tenant Context"]
E --> F["Query Orchestrator"]
F --> G["LlamaIndex Retrieval"]
G --> H["Retrieved Nodes"]
H --> I["Response Pipeline"]Security should be enforced before untrusted retrieval results can influence the response.
65. Observability¶
Production retrieval should capture:
Query
Retriever
Index
Top-K
Scores
Metadata Filters
Latency
Candidate Count
Postprocessor
Final Nodes
LLM
Example trace:
66. Retrieval Evaluation¶
LlamaIndex retrieval should be evaluated independently from generation.
Measure:
Then evaluate generation:
Separating these metrics makes debugging much easier.
67. Retrieval Evaluation Flow¶
Question
↓
Retriever
↓
Retrieved Nodes
↓
Retrieval Evaluation
↓
Context
↓
LLM
↓
Generation Evaluation
This distinction is fundamental to production RAG engineering.
68. LlamaIndex Retrieval Learning Path¶
This section progresses through:
01 LlamaIndex Retrievers Overview
↓
02 LlamaIndex Indexes
↓
03 Vector Index Retriever
↓
04 BM25 Retriever
↓
05 Document Summary Retriever
↓
06 Recursive Retriever
↓
07 Query Fusion Retriever
↓
08 Auto-Merging Retriever
Each chapter builds on the concepts introduced here.
69. Recommended Mental Model¶
Think of LlamaIndex retrieval as:
Source Data
↓
Documents
↓
Nodes
↓
Indexes
↓
Retrievers
↓
Postprocessors
↓
Context
↓
Response Synthesis
And think of enterprise retrieval as:
LlamaIndex provides tools for several of these layers, but the enterprise architecture should remain responsible for coordinating them.
70. Key Takeaways¶
- LlamaIndex provides abstractions for connecting LLMs with external data.
- Documents represent source information.
- Nodes are fundamental retrieval units.
- Metadata provides contextual information about nodes.
- Indexes organize information for efficient retrieval.
- Retrievers determine how relevant nodes are selected.
- Query engines combine retrieval with response synthesis.
- VectorStoreIndex supports semantic vector retrieval.
- Keyword retrieval is valuable for exact identifiers and terminology.
- Recursive retrieval can follow relationships between nodes.
- Query fusion can combine multiple retrieval perspectives.
- Auto-merging retrieval can reconstruct larger contextual structures.
- LlamaIndex supports composable retrieval architectures.
- Retrieval and generation should remain separate concerns.
- Retrieval results should be evaluated independently from generated answers.
- Metadata filtering can be combined with LlamaIndex retrieval.
- Re-ranking and MMR can be added after initial retrieval.
- Hybrid retrieval can combine dense and sparse search.
- LlamaIndex can serve as a retrieval implementation inside a larger enterprise architecture.
- Enterprise security, authorization, tenant isolation, and compliance should not depend solely on the framework.
- A framework-agnostic retrieval interface can prevent application-level coupling to LlamaIndex.
- Ports & Adapters architecture can allow LlamaIndex, LangChain, or custom retrieval implementations to coexist.
- Capability-based abstractions are often more useful than abstracting an entire framework.
- The objective is not to learn framework APIs in isolation.
- The objective is to understand how LlamaIndex retrieval capabilities map to production RAG requirements.
The core architecture is:
USER QUERY
│
▼
┌─────────────────┐
│ Query Processing│
└────────┬────────┘
│
▼
┌─────────────────┐
│ Retriever │
└────────┬────────┘
│
┌───────────┼───────────┐
▼ ▼ ▼
Vector Keyword Graph /
Search Search Structured
│ │ │
└───────────┼───────────┘
▼
Candidates
│
▼
Re-ranking
│
▼
MMR
│
▼
Context Engineering
│
▼
LLM
│
▼
Validated Response
The framework provides retrieval capabilities; the architecture determines how those capabilities become a production-grade RAG system.
🧭 Chapter Navigation¶
Part V — Advanced Retrieval-Augmented Generation¶
Previous:
13. Advanced Query Rewriting
Next:
02. LlamaIndex Indexes
Section:
03 — LlamaIndex Retrieval Engineering
LlamaIndex Retrieval Engineering Path¶
01 LlamaIndex Retrievers Overview
↓
02 LlamaIndex Indexes
↓
03 Vector Index Retriever
↓
04 BM25 Retriever
↓
05 Document Summary Retriever
↓
06 Recursive Retriever
↓
07 Query Fusion Retriever
↓
08 Auto-Merging Retriever
↓
04 Vector Search Engineering
Enterprise AI Engineering Handbook
Building Production-Grade Enterprise AI Systems — One Chapter at a Time.