Multi-Vector Retriever¶
📖 Overview¶
A Multi-Vector Retriever represents a single logical document using multiple vectors instead of relying on one embedding vector.
Traditional vector retrieval usually follows:
A Multi-Vector Retriever expands this approach:
Document
│
┌─────────────┼─────────────┐
↓ ↓ ↓
Summary Content Questions
↓ ↓ ↓
Vector A Vector B Vector C
└─────────────┼─────────────┘
↓
Vector Store
↓
Query
↓
Multiple Matches
↓
Parent Document
↓
LLM
The key idea is:
One document can have multiple representations optimized for different retrieval signals.
This is particularly useful when a single embedding does not adequately represent all the ways users may search for a document.
🎯 Learning Objectives¶
After completing this chapter, you will be able to:
- Understand Multi-Vector Retrieval
- Understand why one vector may not be sufficient for complex documents
- Differentiate single-vector and multi-vector retrieval
- Understand parent-document and child-vector relationships
- Represent documents using summaries
- Represent documents using hypothetical questions
- Understand multiple embeddings per document
- Implement a Multi-Vector Retriever
- Understand document ID mapping
- Combine multiple retrieval representations
- Understand score and ranking considerations
- Combine Multi-Vector Retrieval with reranking
- Combine Multi-Vector Retrieval with contextual compression
- Design production-ready Multi-Vector Retrieval architectures
- Evaluate Multi-Vector Retrieval against a single-vector baseline
1. The Limitation of Single-Vector Retrieval¶
A traditional RAG ingestion pipeline often looks like:
For example:
Document Chunk
"OAuth 2.0 authorization allows applications
to obtain access tokens from an authorization
server before accessing protected resources."
The embedding represents the semantic meaning of the chunk.
This works well for many queries.
However, users may ask questions in many different ways.
For example:
A single vector may not represent all possible retrieval perspectives equally well.
2. Multi-Vector Retrieval Concept¶
Instead of creating one vector representation, we can create multiple representations.
For example:
Document
│
├── Original Content
│
├── Summary
│
├── Hypothetical Questions
│
├── Keywords
│
└── Other Representations
Each representation can have its own embedding:
The vector store therefore contains multiple vectors that point back to the same logical document.
3. Core Architecture¶
flowchart TD
A["Parent Document"] --> B["Representation Generator"]
B --> C["Content Representation"]
B --> D["Summary Representation"]
B --> E["Question Representation"]
B --> F["Keyword Representation"]
C --> G["Embedding"]
D --> H["Embedding"]
E --> I["Embedding"]
F --> J["Embedding"]
G --> K["Vector Store"]
H --> K
I --> K
J --> K
K --> L["Query"]
L --> M["Retrieved Representations"]
M --> N["Parent Document Lookup"]
N --> O["Original Document"]
O --> P["LLM"]The vector store primarily handles retrieval representations.
The original document is maintained separately or in a parent-document store.
4. Single Vector vs Multi-Vector¶
Single-Vector Retrieval¶
Example:
Multi-Vector Retrieval¶
Document A
├── Vector A1
├── Vector A2
├── Vector A3
Document B
├── Vector B1
├── Vector B2
├── Vector B3
The important distinction is:
Single Vector
→ One representation per retrieval unit
Multi-Vector
→ Multiple representations per logical document
5. Why Multiple Representations Help¶
Consider a long technical document:
Enterprise Authentication Architecture
Sections:
1. OAuth 2.0
2. OpenID Connect
3. Access Tokens
4. Refresh Tokens
5. Authorization Code Flow
6. Client Credentials Flow
7. Security Considerations
8. Token Validation
A summary may capture:
Enterprise authentication architecture
covering OAuth 2.0, OpenID Connect,
token management, authorization flows,
and security considerations.
A generated question may be:
Another question:
These different representations expose different semantic paths to the same parent document.
6. Representation Types¶
Multi-Vector Retrieval can use several representation types.
Common examples include:
Original Content
Summaries
Hypothetical Questions
Keywords
Entities
Metadata Descriptions
Generated Captions
Synthetic Queries
Conceptually:
Parent Document
│
┌───────────────┼───────────────┐
↓ ↓ ↓
Summary Questions Content
↓ ↓ ↓
Embedding Embeddings Embedding
│ │ │
└───────────────┼───────────────┘
↓
Vector Store
7. Parent Document and Vector Representations¶
A common architecture maintains:
for retrieval representations and:
for the original document.
For example:
Vector Store
vector_001 → doc_100
vector_002 → doc_100
vector_003 → doc_100
vector_004 → doc_200
vector_005 → doc_200
The document store contains:
The relationship is:
8. Why Store the Parent ID?¶
Suppose the vector database returns:
The application needs to know which logical document produced that vector.
Metadata can therefore contain:
This allows the application to perform:
9. Basic Data Model¶
A simple representation could look like:
class VectorRepresentation:
def __init__(
self,
vector_id,
parent_id,
representation_type,
content
):
self.vector_id = vector_id
self.parent_id = parent_id
self.representation_type = representation_type
self.content = content
Example:
VectorRepresentation(
vector_id="vec-001",
parent_id="doc-100",
representation_type="summary",
content="OAuth 2.0 authentication architecture"
)
Another:
VectorRepresentation(
vector_id="vec-002",
parent_id="doc-100",
representation_type="question",
content="How does OAuth 2.0 authorization work?"
)
Both representations point to:
10. Summary-Based Multi-Vector Retrieval¶
One common strategy is to generate a summary for each document.
The vector represents the summary rather than the complete document.
When a query matches the summary:
This can be useful for large documents where a concise summary provides a stronger semantic representation.
11. Summary Example¶
Original document:
Enterprise API Security Guide
This document explains authentication,
authorization, OAuth 2.0, OpenID Connect,
access tokens, refresh tokens, API gateways,
rate limiting, logging, monitoring, and
security best practices for production APIs.
Generated summary:
Production API security covering OAuth 2.0,
OpenID Connect, tokens, authorization,
rate limiting, monitoring, and security
best practices.
The summary becomes a retrieval representation.
12. Hypothetical Question Representations¶
Another powerful approach is to generate hypothetical questions that a document could answer.
For example:
Document
↓
Question Generator
↓
Question 1
Question 2
Question 3
Question 4
↓
Embeddings
↓
Vector Store
For a document about OAuth:
Question 1:
"What is OAuth 2.0?"
Question 2:
"How does OAuth authorization work?"
Question 3:
"How are access tokens obtained?"
Question 4:
"When should OAuth be used?"
Each question becomes a vector representation of the parent document.
13. Why Hypothetical Questions Can Help¶
Users naturally ask questions.
Documents are not necessarily written in question form.
For example:
User:
A generated hypothetical question:
can create a closer semantic representation.
The retrieval path becomes:
14. Question Generation Example¶
A simple prompt could be:
question_prompt = """
Generate five questions that this document
could answer.
Document:
{document}
Requirements:
- Questions should represent different user intents.
- Questions should be specific.
- Questions should not introduce information
that is not present in the document.
"""
Example output:
1. What is OAuth 2.0?
2. How does the authorization code flow work?
3. How are access tokens issued?
4. When should client credentials flow be used?
5. How should access tokens be validated?
Each question can then be embedded independently.
15. Multi-Vector Representation Pipeline¶
flowchart TD
A["Original Document"] --> B["Representation Generator"]
B --> C["Summary"]
B --> D["Hypothetical Questions"]
B --> E["Original Content"]
C --> F["Embedding Model"]
D --> F
E --> F
F --> G["Multiple Vectors"]
G --> H["Vector Store"]
H --> I["User Query"]
I --> J["Vector Search"]
J --> K["Matched Representations"]
K --> L["Parent IDs"]
L --> M["Parent Document Store"]
M --> N["Original Document"]16. Multiple Vectors per Document¶
Suppose:
produces:
The vector store may contain:
Vector 1 → Document A → Summary
Vector 2 → Document A → Question A1
Vector 3 → Document A → Question A2
Vector 4 → Document A → Question A3
Vector 5 → Document A → Content
Therefore:
This is the fundamental Multi-Vector Retrieval pattern.
17. Query-Time Retrieval¶
At query time:
Example:
Vector search might return:
The application then resolves:
After deduplication:
The parent documents are returned to the generation layer.
18. Parent Deduplication¶
Multiple representations may point to the same parent.
For example:
Vector Search Results
Question A1 → Document A
Question A2 → Document A
Summary A → Document A
Question B1 → Document B
Without deduplication:
With parent-level deduplication:
This prevents one document from dominating the final context simply because it has more representations.
19. Parent-Level Ranking¶
A production Multi-Vector Retriever should consider how multiple representation matches affect parent ranking.
Suppose:
Document B:
The system can aggregate these signals.
For example:
Or use another aggregation strategy.
The important point is:
Representation-level retrieval eventually needs to become document-level ranking.
20. Representation-Level vs Parent-Level Ranking¶
The retrieval process has two levels.
Level 1 — Representation Retrieval¶
Level 2 — Parent Resolution¶
Architecture:
flowchart LR
A["Query"] --> B["Vector Search"]
B --> C["Representation Matches"]
C --> D["Parent ID Resolution"]
D --> E["Parent Aggregation"]
E --> F["Final Document Ranking"]This distinction becomes important when multiple vectors belong to the same document.
21. Aggregation Strategies¶
Possible parent-level aggregation strategies include:
Maximum Score
Average Score
Weighted Average
Top-N Representation Score
Reciprocal Rank Fusion
Weighted Rank Fusion
For example:
or:
The appropriate strategy depends on the representation types and evaluation results.
22. Representation-Type Weighting¶
Different representations may have different importance.
For example:
The system may therefore assign higher importance to generated question representations if they consistently improve query matching.
Conceptually:
Query
↓
┌─────────────┬─────────────┬─────────────┐
↓ ↓ ↓
Content Summary Questions
0.3 0.3 0.4
└─────────────┴─────────────┴─────────────┘
↓
Parent Ranking
These values should be treated as configuration parameters to evaluate, not fixed defaults.
23. LangChain Multi-Vector Retriever¶
LangChain provides a MultiVectorRetriever abstraction.
A simplified example:
from langchain.retrievers.multi_vector import MultiVectorRetriever
from langchain.storage import InMemoryByteStore
retriever = MultiVectorRetriever(
vectorstore=vector_store,
byte_store=InMemoryByteStore(),
id_key="doc_id"
)
The important relationship is:
The id_key connects the vector representation to the original document.
24. Adding Parent Documents¶
A simplified pattern:
from uuid import uuid4
doc_id = str(uuid4())
parent_document = {
"id": doc_id,
"content": original_content
}
Representations can then contain the same ID:
This creates:
25. Adding Multiple Representations¶
Example:
representations = [
summary_document,
question_1,
question_2,
question_3
]
for representation in representations:
representation.metadata["doc_id"] = doc_id
vector_store.add_documents(
representations
)
The parent document is stored separately.
At query time:
26. End-to-End Example¶
Consider a document:
Representations:
Summary:
"Guide covering OAuth, tokens,
authorization, API security."
Question 1:
"How does OAuth authentication work?"
Question 2:
"How are access tokens validated?"
Question 3:
"What security controls are required
for production APIs?"
The vector store contains:
All point to:
Query:
Possible retrieval:
27. Multi-Vector Retrieval with Chunked Content¶
Multi-Vector Retrieval can also work with chunks.
Instead of:
we can use:
For example:
Document
├── Chunk 1
│ ├── Content Vector
│ └── Summary Vector
│
├── Chunk 2
│ ├── Content Vector
│ └── Question Vector
│
└── Chunk 3
├── Content Vector
└── Question Vector
This creates a more granular representation structure.
28. Multi-Vector vs Parent-Document Retrieval¶
These approaches are related but not identical.
Parent-Document Retrieval¶
Usually focuses on:
Multi-Vector Retrieval¶
Focuses on:
They can be combined.
For example:
This creates:
29. Multi-Vector + Parent Document Architecture¶
flowchart TD
A["Parent Document"] --> B["Chunking"]
B --> C["Chunk 1"]
B --> D["Chunk 2"]
B --> E["Chunk 3"]
C --> F["Representations"]
D --> G["Representations"]
E --> H["Representations"]
F --> I["Vector Store"]
G --> I
H --> I
I --> J["User Query"]
J --> K["Representation Retrieval"]
K --> L["Matched Chunks"]
L --> M["Parent Resolution"]
M --> N["Parent Document"]
N --> O["LLM"]30. Multi-Vector + Reranking¶
A reranker can operate after representation retrieval.
Query
↓
Multi-Vector Retrieval
↓
Candidate Representations
↓
Parent Resolution
↓
Candidate Documents
↓
Reranker
↓
Top Documents
Architecture:
flowchart LR
A["Query"] --> B["Multi-Vector Retriever"]
B --> C["Representation Matches"]
C --> D["Parent Resolution"]
D --> E["Candidate Documents"]
E --> F["Reranker"]
F --> G["Top Documents"]This can help when multiple representation matches produce a broad candidate set.
31. Multi-Vector + Contextual Compression¶
Multi-Vector Retrieval can also be combined with contextual compression.
Query
↓
Multi-Vector Retrieval
↓
Parent Documents
↓
Contextual Compression
↓
Relevant Sections
↓
LLM
Architecture:
flowchart TD
A["Query"] --> B["Multi-Vector Retriever"]
B --> C["Representation Matches"]
C --> D["Parent Resolution"]
D --> E["Parent Documents"]
E --> F["Contextual Compression"]
F --> G["Relevant Context"]
G --> H["LLM"]This is useful when parent documents are significantly larger than the information actually required by the query.
32. Multi-Vector + Ensemble Retrieval¶
Multiple retrieval representations can themselves be treated as an ensemble.
For example:
Architecture:
flowchart TD
A["User Query"] --> B["Content Retriever"]
A --> C["Summary Retriever"]
A --> D["Question Retriever"]
B --> E["Content Results"]
C --> F["Summary Results"]
D --> G["Question Results"]
E --> H["Fusion"]
F --> H
G --> H
H --> I["Parent Ranking"]
I --> J["Final Documents"]This approach makes the relationship between Multi-Vector Retrieval and Ensemble Retrieval explicit.
33. Multi-Vector + Hybrid Search¶
A production system can also combine:
For example:
Query
│
┌───────────┴───────────┐
↓ ↓
Multi-Vector Search BM25
│ │
↓ ↓
Representation Results Keyword Results
│ │
└───────────┬───────────┘
↓
Fusion
↓
Parent Ranking
This can provide:
34. Representation Generation Cost¶
Multi-Vector Retrieval introduces additional ingestion work.
For example:
Multi-Vector:
The ingestion pipeline may therefore become more expensive.
Example:
This creates additional:
- LLM generation cost
- Embedding cost
- Storage requirements
- Indexing time
- Metadata management
Therefore, the retrieval improvement must justify the additional ingestion complexity.
35. Storage Considerations¶
If a corpus contains:
and each document produces:
then approximately:
may need to be stored.
This affects:
Multi-Vector Retrieval should therefore be designed with scale in mind.
36. Representation Explosion¶
A common mistake is generating too many representations.
For example:
can rapidly increase vector count.
More vectors do not automatically mean better retrieval.
The objective should be:
rather than:
37. Representation Quality¶
Generated representations are only useful if they are high quality.
A poor generated question can introduce incorrect assumptions.
Example:
Bad generated question:
The document may not support that claim.
The representation generator should therefore remain grounded in the source document.
38. Guardrails for Representation Generation¶
A safe generation prompt could be:
Generate retrieval representations for the document.
Rules:
1. Use only information present in the document.
2. Do not introduce unsupported facts.
3. Do not make assumptions.
4. Preserve important terminology.
5. Generate questions that the document can actually answer.
6. Avoid duplicate questions.
7. Keep representations concise and specific.
This reduces the risk of creating misleading retrieval vectors.
39. Metadata Design¶
A production vector representation should carry enough metadata to support traceability.
Example:
{
"doc_id": "doc-100",
"representation_id": "rep-004",
"representation_type": "question",
"source": "security-guide.pdf",
"page": 14,
"section": "OAuth",
"version": "v3"
}
Useful fields include:
This supports:
- Citation
- Debugging
- Evaluation
- Versioning
- Observability
40. Representation Lifecycle¶
Representations should have a lifecycle.
Source Document
↓
Representation Generation
↓
Validation
↓
Embedding
↓
Indexing
↓
Retrieval
↓
Evaluation
When the source document changes:
This is important for enterprise knowledge systems.
41. Versioning¶
Consider:
producing:
After an update:
the representations should be regenerated.
A metadata model can contain:
{
"doc_id": "security-001",
"document_version": "2",
"representation_version": "2",
"representation_type": "summary"
}
This prevents stale representations from being returned.
42. Multi-Vector Retrieval Evaluation¶
The correct baseline is:
The experiment is:
Compare:
Architecture:
flowchart LR
A["Evaluation Dataset"] --> B["Single-Vector Retriever"]
A --> C["Multi-Vector Retriever"]
B --> D["Baseline Metrics"]
C --> E["Multi-Vector Metrics"]
D --> F["Comparison"]
E --> F
F --> G["Production Decision"]43. Retrieval Metrics¶
Important retrieval metrics include:
Recall@K¶
Does the correct document appear in the retrieved candidates?
is particularly important when Multi-Vector Retrieval is being used for candidate generation.
MRR¶
Measures how highly the first relevant result appears.
can show whether Multi-Vector Retrieval moves relevant documents closer to the top.
NDCG¶
Measures ranking quality while considering relevance and position.
44. End-to-End RAG Evaluation¶
Retrieval quality is not the only consideration.
The final system should also evaluate:
Metrics may include:
A retrieval improvement is valuable only if it translates into meaningful downstream improvement.
45. Common Failure Modes¶
45.1 Too Many Representations¶
45.2 Poor Generated Questions¶
Incorrect or unsupported questions can create misleading retrieval paths.
45.3 Duplicate Parent Documents¶
Multiple vectors can return the same parent.
Parent-level deduplication is required.
45.4 Parent Ranking Problems¶
A document with many representations may dominate simply because it has more vectors.
The system should carefully design parent-level aggregation.
45.5 Stale Representations¶
When source documents change, old summaries or questions can remain indexed.
This can produce stale retrieval results.
45.6 Increased Storage¶
Multiple vectors per document can significantly increase vector database size.
45.7 Increased Ingestion Cost¶
LLM-generated summaries and questions increase preprocessing cost.
45.8 Retrieval Latency¶
A larger vector index and more complex resolution logic can affect latency.
46. Production Architecture¶
A mature Multi-Vector Retrieval architecture can look like:
flowchart TD
A["Source Documents"] --> B["Document Processing"]
B --> C["Parent Document Store"]
B --> D["Representation Generator"]
D --> E["Content Representation"]
D --> F["Summary Representation"]
D --> G["Question Representations"]
E --> H["Embedding Model"]
F --> H
G --> H
H --> I["Vector Store"]
J["User Query"] --> K["Query Processing"]
K --> I
I --> L["Representation Matches"]
L --> M["Parent ID Resolution"]
M --> N["Parent Deduplication"]
N --> O["Parent Ranking"]
O --> P["Reranker"]
P --> Q["Contextual Compression"]
Q --> R["Context Selection"]
R --> S["Prompt Assembly"]
S --> T["LLM"]This architecture separates:
Representation Generation
↓
Vector Retrieval
↓
Parent Resolution
↓
Ranking
↓
Context Optimization
↓
Generation
47. Enterprise Design Principle¶
The most important architectural principle is:
A representation optimized for retrieval does not necessarily need to be sent to the LLM.
For example:
may be excellent for retrieval.
But the LLM should receive:
rather than the generated question.
Therefore:
This separation is fundamental to Multi-Vector Retrieval.
48. Decision Flow¶
flowchart TD
A["Single Vector Retrieval"] --> B{"Retrieval Recall Sufficient?"}
B -->|Yes| C["Keep Single Representation"]
B -->|No| D{"Are Multiple Retrieval Perspectives Useful?"}
D -->|No| E["Improve Embedding / Chunking"]
D -->|Yes| F["Add Multiple Representations"]
F --> G{"Choose Representation Types"}
G --> H["Summary"]
G --> I["Hypothetical Questions"]
G --> J["Content"]
G --> K["Other Domain-Specific Representations"]
H --> L["Embed"]
I --> L
J --> L
K --> L
L --> M["Vector Store"]
M --> N["Evaluate"]
N --> O{"Quality Improvement Justifies Cost?"}
O -->|Yes| P["Production Multi-Vector Retrieval"]
O -->|No| Q["Reconsider Representation Strategy"]49. When to Use Multi-Vector Retrieval¶
Multi-Vector Retrieval is especially useful when:
- Documents have multiple semantic aspects
- Queries can be phrased in many different ways
- Long documents are difficult to represent with one vector
- Synthetic questions improve retrieval recall
- Summaries provide useful high-level representations
- Different representations capture different retrieval intents
- Parent documents should be returned after representation-level retrieval
- Enterprise knowledge contains complex heterogeneous documents
Typical applications include:
Enterprise Knowledge Assistants
Technical Documentation
Research Systems
Legal Document Search
Financial Knowledge Systems
Healthcare Knowledge Bases
Product Documentation
Enterprise Policy Search
50. When It May Not Be Necessary¶
Multi-Vector Retrieval may not be appropriate when:
or:
or:
or:
or:
A simpler architecture is often preferable when it already satisfies production requirements.
51. Recommended Starting Architecture¶
A practical starting point is:
Parent Document
↓
┌────┴──────────┐
↓ ↓
Summary Questions
↓ ↓
Embedding Embeddings
└───────┬───────┘
↓
Vector Store
↓
Query
↓
Representation Retrieval
↓
Parent Resolution
↓
Deduplication
↓
Reranking
↓
Contextual Compression
↓
LLM
This provides:
Multiple Retrieval Perspectives
+
Parent-Level Context
+
Precision Optimization
+
Context Optimization
52. Production Checklist¶
Before deploying a Multi-Vector Retriever:
☐ Single-vector baseline has been evaluated
☐ Representation types are clearly defined
☐ Generated representations are grounded in source documents
☐ Duplicate representations are controlled
☐ Parent IDs are stable
☐ Parent-level deduplication is implemented
☐ Parent ranking strategy is defined
☐ Representation metadata is preserved
☐ Source metadata is preserved
☐ Document versioning is supported
☐ Stale representations are removed
☐ Vector storage growth is measured
☐ Embedding costs are measured
☐ LLM representation-generation costs are measured
☐ Retrieval latency is measured
☐ End-to-end RAG quality is evaluated
☐ Citation traceability is preserved
☐ Regression tests are implemented
53. Key Takeaways¶
- Multi-Vector Retrieval represents a logical document using multiple vectors.
- A single document can have content, summary, question, and other retrieval representations.
- Multiple representations provide multiple semantic paths to the same document.
- Vector retrieval operates against representations rather than necessarily the final generation context.
- Parent IDs connect representations to original documents.
- Parent-level deduplication is essential.
- Parent-level ranking must account for multiple representation matches.
- Summaries can provide useful high-level retrieval representations.
- Hypothetical questions can align document representations with natural user queries.
- Multiple representations increase ingestion and storage costs.
- More vectors do not automatically mean better retrieval.
- Representation generation must remain grounded in source content.
- Multi-Vector Retrieval can be combined with Parent-Document Retrieval.
- It can also be combined with ensemble retrieval, reranking, hybrid search, and contextual compression.
- Generated retrieval representations should generally not replace source evidence during generation.
- Source metadata and document versioning are critical for enterprise systems.
- Multi-Vector Retrieval should be evaluated against a single-vector baseline.
- The objective is not to maximize the number of vectors.
- The objective is to create useful retrieval representations that improve recall and relevance without introducing unnecessary operational complexity.
The central pattern is:
One Logical Document
↓
Multiple Retrieval Representations
↓
Multiple Vector Signals
↓
Representation Retrieval
↓
Parent Resolution
↓
Parent Ranking
↓
Context Optimization
↓
LLM
Or simply:
🧭 Chapter Navigation¶
Part V — Advanced Retrieval-Augmented Generation¶
Previous:
02. Ensemble Retriever
Next:
04. Time-Weighted Retriever
Section:
02 — Enterprise Retrieval Engineering
Enterprise Retrieval Engineering Path¶
01 Contextual Compression Retriever
↓
02 Ensemble Retriever
↓
03 Multi-Vector Retriever
↓
04 Time-Weighted Retriever
↓
05 Hybrid Search Retriever
↓
06 HyDE Retriever
↓
07 Router Retriever
↓
08 Multi-Stage Retrieval
↓
09 Agentic Retrieval
↓
10 Re-ranking Techniques
↓
11 MMR & Diversity-Aware Retrieval
↓
12 Metadata-Aware Retrieval
↓
13 Advanced Query Rewriting
Enterprise AI Engineering Handbook
Building Production-Grade Enterprise AI Systems — One Chapter at a Time.