Re-ranking Techniques¶
📖 Overview¶
Re-ranking is a second-stage retrieval technique used to improve the relevance of documents returned by an initial retriever.
A typical retrieval system works in two stages:
The initial retriever is optimized for high recall and fast candidate generation.
The re-ranker is optimized for high precision and better query-document relevance ordering.
This separation is fundamental to production RAG:
🎯 Learning Objectives¶
After completing this chapter, you will be able to:
- Understand why re-ranking is required in RAG
- Understand first-stage retrieval vs second-stage ranking
- Understand bi-encoder and cross-encoder architectures
- Implement cross-encoder re-ranking
- Understand score-based re-ranking
- Combine dense and sparse retrieval with re-ranking
- Apply metadata-aware re-ranking
- Understand reciprocal rank fusion
- Understand weighted score fusion
- Implement multi-stage re-ranking
- Understand reranking thresholds
- Tune candidate count and final result count
- Evaluate re-ranking quality
- Understand latency and cost trade-offs
- Design production-grade re-ranking pipelines
- Implement fallback strategies
- Understand when re-ranking should and should not be used
1. Why Re-ranking?¶
A vector retriever usually calculates similarity between:
This is highly efficient.
However, the similarity score may not perfectly represent the relevance of a document to the complete query.
Consider:
Initial retrieval might return:
1. OAuth overview
2. Payment authentication
3. OAuth token expiration
4. Payment API reference
5. Authentication troubleshooting
The document that directly explains token expiration may not necessarily have the highest embedding similarity.
A re-ranker can analyze the query and candidate document together and produce a more precise ranking.
2. First-Stage Retrieval¶
The first-stage retriever should generally optimize for:
For example:
The retriever does not need to perfectly rank those 100 documents.
It needs to ensure that the relevant documents have a good chance of entering the candidate set.
3. Second-Stage Ranking¶
The second stage focuses on:
Example:
The re-ranker performs more detailed relevance analysis on a much smaller candidate set.
4. Retrieval vs Re-ranking¶
flowchart LR
A["User Query"] --> B["First-Stage Retriever"]
B --> C["100 Candidates"]
C --> D["Re-ranker"]
D --> E["Top 10 Documents"]
E --> F["Context Selection"]
F --> G["LLM"]The architecture separates:
from:
5. Why Not Re-rank the Entire Corpus?¶
Suppose:
A sophisticated re-ranker may need to evaluate:
for every document.
That would be expensive.
Instead:
This is the fundamental efficiency advantage of two-stage retrieval.
6. Two-Stage Retrieval Architecture¶
┌─────────────────────┐
│ Large Corpus │
└──────────┬──────────┘
↓
┌─────────────────────┐
│ Candidate Retriever │
│ Fast / High Recall │
└──────────┬──────────┘
↓
100 Candidates
↓
┌─────────────────────┐
│ Re-ranker │
│ Precise / Expensive │
└──────────┬──────────┘
↓
Top 10 Results
↓
┌─────────────────────┐
│ Context Selection │
└──────────┬──────────┘
↓
LLM
7. Bi-Encoder Retrieval¶
Most dense retrieval systems use a bi-encoder architecture.
The query and document are encoded independently.
Then:
is calculated.
8. Bi-Encoder Architecture¶
flowchart TD
A["Query"] --> B["Query Encoder"]
B --> C["Query Embedding"]
D["Document"] --> E["Document Encoder"]
E --> F["Document Embedding"]
C --> G["Similarity"]
F --> G
G --> H["Similarity Score"]The advantage is that document embeddings can be precomputed.
This makes large-scale vector search practical.
9. Limitation of Bi-Encoder Retrieval¶
The query and document are encoded independently:
The model does not deeply process their interaction at retrieval time.
For many queries this is sufficient.
For nuanced questions, however, a more precise relevance model can improve ranking.
10. Cross-Encoder Re-ranking¶
A cross-encoder processes the query and document together.
The model can directly analyze relationships between:
11. Cross-Encoder Architecture¶
flowchart TD
A["Query"] --> C["Query + Document Pair"]
B["Candidate Document"] --> C
C --> D["Cross-Encoder"]
D --> E["Relevance Score"]For multiple candidates:
The candidates are then sorted by score.
12. Bi-Encoder vs Cross-Encoder¶
| Characteristic | Bi-Encoder | Cross-Encoder |
|---|---|---|
| Query Encoding | Independent | Joint |
| Document Encoding | Independent | Joint with query |
| Speed | Very Fast | Slower |
| Corpus Scale | Excellent | Poor for full corpus |
| Recall | High | Depends on candidates |
| Precision | Good | Usually better ranking |
| Embeddings | Precomputed | Query-dependent |
| Typical Usage | First Stage | Second Stage |
The ideal architecture often combines both.
13. The Two-Stage Pattern¶
This pattern combines:
14. Candidate Pool Size¶
The candidate pool is one of the most important tuning parameters.
Example:
may miss relevant documents.
Increasing to:
may improve recall.
But:
can make re-ranking expensive.
Therefore:
must be evaluated experimentally.
15. Candidate K vs Final K¶
These values should be separate.
Example:
Architecture:
The values are application-specific.
16. Basic Cross-Encoder Example¶
A common implementation uses a cross-encoder model.
from sentence_transformers import CrossEncoder
model = CrossEncoder(
"cross-encoder/ms-marco-MiniLM-L-6-v2"
)
query = "How does OAuth token expiration work?"
documents = [
"OAuth is an authorization framework...",
"Tokens expire after a configured period...",
"Payment APIs provide authentication..."
]
pairs = [
[query, document]
for document in documents
]
scores = model.predict(pairs)
ranked = sorted(
zip(documents, scores),
key=lambda item: item[1],
reverse=True
)
for document, score in ranked:
print(score, document)
The exact model should be selected and evaluated for the target domain.
17. Re-ranking Function¶
A reusable implementation:
def rerank(
query,
documents,
reranker,
top_k=10
):
pairs = [
[query, doc.page_content]
for doc in documents
]
scores = reranker.predict(pairs)
ranked = sorted(
zip(documents, scores),
key=lambda x: x[1],
reverse=True
)
return [
document
for document, score in ranked[:top_k]
]
18. Preserving Scores¶
In production systems, do not discard the score.
def rerank(
query,
documents,
reranker,
top_k=10
):
pairs = [
[query, doc.page_content]
for doc in documents
]
scores = reranker.predict(pairs)
ranked = sorted(
zip(documents, scores),
key=lambda x: x[1],
reverse=True
)
return [
{
"document": document,
"score": float(score)
}
for document, score in ranked[:top_k]
]
Scores are useful for:
19. Score Thresholding¶
Instead of always returning:
the system can require:
Example:
Be careful:
Re-ranking scores are often model-specific and should not be treated as universally calibrated probabilities.
Thresholds should be determined using an evaluation dataset.
20. Ranking vs Thresholding¶
These are different operations.
Ranking¶
Thresholding¶
Pipeline:
21. Dense Retrieval + Re-ranking¶
A standard architecture:
candidates = vector_store.similarity_search(
query,
k=100
)
ranked = rerank(
query,
candidates,
reranker,
top_k=10
)
Pipeline:
This is often an excellent baseline for RAG systems.
22. Hybrid Search + Re-ranking¶
Hybrid search can improve candidate recall.
flowchart TD
A["Query"] --> B["Dense Search"]
A --> C["BM25"]
B --> D["Dense Candidates"]
C --> E["Sparse Candidates"]
D --> F["Fusion"]
E --> F
F --> G["Candidate Pool"]
G --> H["Re-ranker"]
H --> I["Final Results"]This architecture combines:
23. Why Re-rank Hybrid Results?¶
Dense and sparse retrieval may produce different rankings.
Example:
Fusion produces:
The re-ranker can then determine which documents are truly most relevant to the query.
24. Reciprocal Rank Fusion¶
Reciprocal Rank Fusion (RRF) combines ranked lists.
A common formulation is:
where:
RRF is useful when combining:
before the re-ranking stage.
25. Weighted Score Fusion¶
Another approach combines normalized scores.
Conceptually:
where:
The exact weighting should be tuned experimentally.
26. Score Normalization¶
Different retrieval systems may produce incompatible score ranges.
Example:
These scores cannot simply be added.
They may require normalization before weighted fusion.
Possible approaches include:
RRF often avoids the need to directly compare raw score magnitudes.
27. Re-ranking with Metadata¶
Relevance is not always the only ranking factor.
Enterprise systems may consider:
A conceptual score might be:
Final Score =
0.70 × Semantic Relevance
+
0.15 × Authority
+
0.10 × Recency
+
0.05 × Business Priority
The weights are illustrative.
28. Metadata-Aware Ranking¶
Example:
def business_score(
semantic_score,
authority_score,
recency_score,
priority_score
):
return (
0.70 * semantic_score
+ 0.15 * authority_score
+ 0.10 * recency_score
+ 0.05 * priority_score
)
This should be applied carefully.
Hard authorization rules should remain separate from ranking.
29. Hard Filters vs Soft Ranking¶
Hard Filter¶
Result:
Soft Ranking¶
Result:
Never turn an authorization rule into a ranking signal.
30. Re-ranking and Time-Weighted Retrieval¶
Time-sensitive enterprise knowledge may require:
Example:
An older but semantically similar document should not automatically outrank the latest approved policy.
Pipeline:
Candidate Retrieval
↓
Security Filter
↓
Relevance Ranking
↓
Recency / Authority Adjustment
↓
Final Ranking
31. Re-ranking and MMR¶
Relevance alone can produce redundant results.
Example:
All may be highly relevant but describe the same paragraph.
MMR can introduce diversity:
The pipeline becomes:
32. Re-ranking vs MMR¶
| Technique | Primary Goal |
|---|---|
| Re-ranking | Relevance |
| MMR | Relevance + Diversity |
| Metadata Ranking | Business / Contextual Priority |
| RRF | Rank Fusion |
They can be combined.
33. Multi-Stage Re-ranking¶
A production pipeline may have multiple ranking stages:
1,000,000 Documents
↓
Vector Search
↓
500 Candidates
↓
Hybrid Fusion
↓
200 Candidates
↓
Cross-Encoder
↓
30 Documents
↓
MMR
↓
10 Documents
Each stage has a specific purpose.
34. Re-ranking After Parent Resolution¶
With Parent-Document Retrieval:
This prevents the final ranking from being based only on isolated child chunks.
35. Re-ranking with Multi-Vector Retrieval¶
Multi-vector retrieval can produce:
representations.
After candidate generation:
This is useful for complex documents.
36. Re-ranking After Multi-Query Retrieval¶
Multi-query retrieval produces multiple search results:
The re-ranker can provide a common relevance model across the merged candidate set.
37. Re-ranking After HyDE¶
HyDE may produce:
The re-ranker then evaluates the actual:
rather than relying only on the hypothetical representation.
38. Re-ranking and Agentic Retrieval¶
Agentic Retrieval can dynamically decide whether re-ranking is necessary.
flowchart TD
A["Query"] --> B["Agent"]
B --> C["Candidate Retrieval"]
C --> D{"Need Precise Ranking?"}
D -->|Yes| E["Re-ranker"]
D -->|No| F["Candidate Results"]
E --> G["Evidence Evaluation"]
F --> G
G --> H{"Sufficient?"}
H -->|No| B
H -->|Yes| I["Context"]This allows expensive re-ranking to be used selectively.
39. Re-ranking in Multi-Stage Retrieval¶
The previous chapter introduced:
Re-ranking is the precision stage in this architecture.
Its purpose is not to retrieve missing documents.
Its purpose is:
Improve the ordering and selection of documents already present in the candidate set.
40. Re-ranking Cannot Recover Missing Documents¶
This is a critical limitation.
Suppose:
is not included in:
Then:
cannot rank it.
Therefore:
still matters.
The complete system requires:
41. Candidate Recall vs Re-ranking Precision¶
Think of the pipeline as:
If recall is poor:
If re-ranking is poor:
Both stages matter.
42. Re-ranking Model Selection¶
When selecting a re-ranker consider:
A model that performs well on a public benchmark may not perform best on your enterprise corpus.
43. Domain-Specific Re-rankers¶
General-purpose models may struggle with:
Legal terminology
Medical terminology
Financial terminology
Internal product names
Engineering identifiers
Evaluate models on your own data.
A domain-specific model may provide better ranking quality.
44. Multilingual Re-ranking¶
For multilingual applications, verify:
A multilingual embedding model does not automatically guarantee that the chosen re-ranker performs equally well across all languages.
45. Long Documents¶
Cross-encoders can become expensive for long documents.
Instead of:
consider:
or:
This also helps control latency.
46. Chunk Size and Re-ranking¶
Chunking affects ranking quality.
If chunks are:
they may lack context.
If chunks are:
the re-ranker may see too much irrelevant content.
Therefore:
should be evaluated together.
47. Batch Re-ranking¶
Candidate documents can often be scored in batches.
pairs = [
[query, document.page_content]
for document in documents
]
scores = reranker.predict(
pairs,
batch_size=32
)
Batching can improve throughput depending on the model and hardware.
48. GPU Re-ranking¶
For high-throughput systems:
A dedicated inference service can isolate:
from the application service.
49. Re-ranking Service Architecture¶
flowchart LR
A["RAG Application"] --> B["Retrieval Service"]
B --> C["Candidate Pool"]
C --> D["Re-ranking Service"]
D --> E["GPU / CPU Model"]
E --> F["Ranked Results"]
F --> G["RAG Application"]This is useful when multiple applications share the same re-ranking infrastructure.
50. Re-ranking API¶
A service might expose:
Request:
{
"query": "How does OAuth token expiration work?",
"documents": [
{
"id": "doc-1",
"text": "..."
},
{
"id": "doc-2",
"text": "..."
}
],
"top_k": 5
}
Response:
The exact API contract should be designed around your platform requirements.
51. Flask Example¶
Since this handbook also covers Flask deployment, a simple re-ranking service can be exposed through Flask.
from flask import Flask, request, jsonify
from sentence_transformers import CrossEncoder
app = Flask(__name__)
reranker = CrossEncoder(
"cross-encoder/ms-marco-MiniLM-L-6-v2"
)
@app.post("/rerank")
def rerank_documents():
payload = request.get_json()
query = payload["query"]
documents = payload["documents"]
top_k = payload.get("top_k", 10)
pairs = [
[query, document["text"]]
for document in documents
]
scores = reranker.predict(pairs)
results = sorted(
zip(documents, scores),
key=lambda item: item[1],
reverse=True
)
return jsonify({
"results": [
{
"id": document["id"],
"score": float(score)
}
for document, score in results[:top_k]
]
})
For production, add:
Authentication
Authorization
Input Validation
Timeouts
Batching
Metrics
Tracing
Model Warmup
Health Checks
Rate Limiting
52. Re-ranking Service Health Check¶
Example:
A production service should distinguish:
from:
when appropriate.
53. Re-ranking Latency¶
Suppose:
Total:
If re-ranking increases:
the user experience may degrade substantially.
Therefore re-ranking must be evaluated as part of the complete request path.
54. Re-ranking Cost¶
Cost depends on:
Example:
is much cheaper to re-rank than:
The objective is:
55. Adaptive Candidate K¶
A sophisticated system can adjust candidate count.
Simple query:
Complex query:
Low confidence:
This can be combined with Agentic Retrieval.
56. Confidence-Based Re-ranking¶
Example:
The ranking is ambiguous.
The system may benefit from stronger re-ranking.
If:
additional ranking may provide little benefit.
This is an optimization strategy, not a universal rule.
57. Score Gap¶
A simple signal is:
Large gap:
Small gap:
However, raw scores and gaps are model-specific and should be calibrated empirically.
58. Re-ranking Evaluation¶
Do not evaluate re-ranking only by:
Evaluate the ranking stage directly.
Useful metrics include:
Then evaluate downstream:
59. NDCG¶
Normalized Discounted Cumulative Gain rewards relevant documents appearing higher in the ranking.
Conceptually:
It is especially useful when documents have graded relevance.
60. MRR¶
Mean Reciprocal Rank focuses on the position of the first relevant result.
For a query:
the reciprocal rank is:
At rank 5:
MRR is useful when finding at least one highly relevant document is particularly important.
61. Re-ranking Evaluation Dataset¶
Create examples:
{
"query": "How does OAuth token expiration work?",
"documents": [
{
"id": "a",
"relevance": 1
},
{
"id": "b",
"relevance": 3
},
{
"id": "c",
"relevance": 0
}
]
}
Use graded relevance when appropriate:
This enables NDCG-style evaluation.
62. Before vs After Re-ranking¶
Example:
After re-ranking:
If the ground-truth relevant document is C, the ranking has improved.
63. A/B Testing¶
Compare:
Measure:
Do not assume re-ranking improves every workload.
64. Re-ranking Failure Modes¶
64.1 Missing Candidate¶
The relevant document never entered the candidate pool.
Solution:
64.2 Poor Re-ranker¶
The model ranks relevant documents incorrectly.
Solution:
64.3 Candidate Pool Too Small¶
The re-ranker has too little evidence to work with.
Solution:
64.4 Candidate Pool Too Large¶
Latency and cost increase.
Solution:
64.5 Context Redundancy¶
Top-ranked documents repeat the same information.
Solution:
65. Re-ranking and Hallucination¶
Re-ranking can reduce hallucination risk indirectly.
Better ranking:
However:
Re-ranking does not guarantee factual correctness or eliminate hallucinations.
The final system still requires:
66. Re-ranking and Citation¶
If final context comes from:
their source metadata should be preserved.
Example:
Do not discard provenance during re-ranking.
67. Re-ranking and Source Authority¶
Consider:
For a policy question, Document B may deserve higher final priority.
This demonstrates why production ranking can involve:
rather than semantic score alone.
68. Enterprise Re-ranking¶
Enterprise systems may use:
Architecture:
flowchart TD
A["Candidate Documents"] --> B["Semantic Re-ranker"]
B --> C["Relevance Scores"]
C --> D["Business Ranking"]
D --> E["Authority / Recency"]
E --> F["Diversity Selection"]
F --> G["Final Context"]Security authorization should remain a separate hard constraint.
69. Re-ranking Pipeline Contract¶
A clean service interface might be:
This allows different implementations:
70. Capability-Based Architecture¶
The application should depend on:
rather than:
Example:
Implementations can include:
This fits a provider-based enterprise architecture.
71. Reranking with LLMs¶
An LLM can also evaluate:
and produce a relevance score.
Conceptually:
This can be powerful but usually has higher:
than specialized cross-encoder models.
LLM-based reranking should therefore be evaluated against specialized rerankers.
72. Structured LLM Re-ranking¶
A structured approach:
prompt = """
Evaluate the relevance of the document
to the query.
Return JSON:
{
"relevance": 0-3,
"reason": "..."
}
"""
Example output:
The response should be schema-validated before use.
73. Cross-Encoder vs LLM Re-ranking¶
| Characteristic | Cross-Encoder | LLM |
|---|---|---|
| Latency | Lower | Higher |
| Cost | Lower | Higher |
| Ranking Scale | Better | Limited |
| Flexibility | Moderate | High |
| Structured Reasoning | Limited | Strong |
| Production Throughput | Excellent | More challenging |
For high-volume retrieval, specialized re-rankers are often a strong starting point.
74. When LLM Re-ranking Makes Sense¶
LLM re-ranking may be useful when:
For example:
Even then, measure the additional cost and latency.
75. Cascaded Re-ranking¶
A sophisticated architecture can use:
Example:
This is useful only when the additional quality justifies the complexity.
76. Cascaded Ranking¶
flowchart LR
A["Large Candidate Pool"] --> B["Cheap Ranker"]
B --> C["500"]
C --> D["Cross-Encoder"]
D --> E["50"]
E --> F["Optional LLM Judge"]
F --> G["10 Final Documents"]The guiding principle remains:
77. Re-ranking and Context Engineering¶
The re-ranker determines which evidence deserves attention.
The next stage decides how that evidence is assembled:
Therefore ranking and context engineering should be designed together.
78. Position Bias¶
LLMs may not treat every context position equally.
Therefore:
should generally be placed intentionally in the final context.
However, context ordering should be validated against the target model and application.
79. Context Ordering¶
Possible strategy:
Another strategy may use:
The best strategy should be evaluated experimentally.
80. Re-ranking and Context Compression¶
A useful production pipeline:
This prevents expensive compression from operating on an unnecessarily large candidate set.
81. Re-ranking and Response Validation¶
The complete pipeline can be:
flowchart TD
A["User Query"] --> B["Candidate Retrieval"]
B --> C["Re-ranking"]
C --> D["Context Selection"]
D --> E["Prompt Assembly"]
E --> F["LLM Generation"]
F --> G["Response Validation"]
G --> H["Citation"]
H --> I["Enterprise Response"]Re-ranking is therefore one component of a larger production RAG architecture.
82. Production Observability¶
Track:
Candidate Count
Reranked Count
Final Count
Reranker Model
Reranking Latency
Average Score
Top Score
Score Distribution
Threshold Rejections
Fallback Count
Token Usage
Cost
Example:
{
"stage": "reranking",
"candidate_count": 100,
"output_count": 10,
"latency_ms": 118,
"top_score": 0.94,
"model": "cross-encoder"
}
83. Score Distribution Monitoring¶
Track score distributions over time.
Example:
Suddenly:
Potential causes:
Embedding model change
Reranker model change
Corpus change
Chunking change
Query distribution change
Index degradation
This is useful for production monitoring.
84. Model Versioning¶
Track:
Reranker Model
Model Version
Tokenizer Version
Candidate Retriever Version
Embedding Model
Index Version
Example:
{
"retriever_version": "v4",
"embedding_version": "v3",
"reranker_version": "v2",
"pipeline_version": "v7"
}
This is essential for reproducible evaluation.
85. Fallback Strategy¶
If the re-ranker fails:
Do not necessarily fail the entire RAG request.
Example:
In production, the exception should also be:
and the fallback should have appropriate safeguards.
86. Timeout Strategy¶
Set a timeout:
If the timeout is exceeded:
This prevents a ranking service from dominating the total request latency.
87. Circuit Breaker¶
For a remote re-ranking service:
repeated failures should trigger:
Then:
This protects the overall RAG service.
88. Production Re-ranking Architecture¶
flowchart TD
A["User Query"] --> B["Retrieval Layer"]
B --> C["Dense Search"]
B --> D["Sparse Search"]
C --> E["Fusion"]
D --> E
E --> F["Security / Metadata Filtering"]
F --> G["Candidate Pool"]
G --> H["Re-ranking Service"]
H --> I{"Available?"}
I -->|Yes| J["Cross-Encoder Ranking"]
I -->|No| K["Fallback Ranking"]
J --> L["MMR / Diversity"]
K --> L
L --> M["Context Selection"]
M --> N["Prompt Assembly"]
N --> O["LLM"]
O --> P["Validation"]
P --> Q["Citation"]89. Performance Optimization Checklist¶
☐ Tune candidate K
☐ Tune final K
☐ Batch candidates
☐ Use GPU when justified
☐ Cache repeated queries
☐ Keep documents reasonably sized
☐ Apply security filters early
☐ Avoid unnecessary re-ranking
☐ Use timeout limits
☐ Implement fallback ranking
☐ Monitor P95 latency
☐ Monitor throughput
☐ Monitor model cost
90. Re-ranking Evaluation Checklist¶
☐ Build relevance-labeled dataset
☐ Measure Recall@K
☐ Measure MRR
☐ Measure NDCG@K
☐ Compare baseline retrieval
☐ Compare candidate sizes
☐ Compare reranker models
☐ Evaluate latency
☐ Evaluate cost
☐ Evaluate downstream answer quality
☐ Evaluate citation quality
☐ Test domain-specific queries
☐ Test multilingual queries if required
91. Practical Tuning Strategy¶
Start with:
Then experiment:
Measure:
Do not choose K values based solely on intuition.
92. Recommended Baseline¶
For many enterprise RAG systems, a strong baseline is:
Then evaluate whether additional techniques provide measurable improvement.
93. When to Use Re-ranking¶
Use re-ranking when:
- Initial retrieval returns noisy results
- Top results are often incorrectly ordered
- Search quality is more important than minimum latency
- Candidate pools can be kept reasonably small
- Enterprise questions require precise evidence
- Hybrid retrieval produces a broad candidate set
- You need better context selection
- Retrieval evaluation shows ranking weaknesses
94. When Re-ranking May Not Be Necessary¶
You may not need re-ranking when:
or:
or:
or:
Always compare:
against:
95. Common Anti-Patterns¶
Anti-Pattern 1 — Re-ranking Too Many Documents¶
Usually expensive.
Anti-Pattern 2 — Candidate Pool Too Small¶
The relevant document may already be missing.
Anti-Pattern 3 — Treating Score as Probability¶
This is generally unsafe unless explicitly calibrated.
Anti-Pattern 4 — Ignoring Domain Evaluation¶
A model benchmark does not guarantee enterprise performance.
Anti-Pattern 5 — Discarding Provenance¶
Never lose:
during ranking.
96. Key Takeaways¶
- Re-ranking is a second-stage retrieval technique.
- First-stage retrieval should generally optimize for recall and speed.
- Re-ranking should generally optimize for precision and ordering.
- Bi-encoders are efficient for large-scale candidate retrieval.
- Cross-encoders provide deeper query-document interaction.
- Re-ranking cannot recover documents missing from the candidate pool.
- Candidate K and final K should be tuned independently.
- Hybrid retrieval followed by re-ranking is a strong production pattern.
- RRF can combine ranked results from different retrieval systems.
- Weighted score fusion can combine normalized retrieval scores.
- Metadata, authority, and recency can influence ranking when appropriate.
- Authorization must remain a hard constraint rather than a ranking signal.
- MMR can complement re-ranking by improving diversity.
- LLM-based re-ranking provides flexibility but usually increases cost and latency.
- Cascaded ranking can progressively apply more expensive ranking models.
- Re-ranking should preserve source and document provenance.
- Re-ranking quality should be evaluated using metrics such as NDCG, MRR, Precision@K, and Recall@K.
- The best candidate size depends on the corpus, retriever, re-ranker, and latency budget.
- Production systems should support timeouts and fallback ranking.
- Score thresholds must be calibrated using real evaluation data.
- Re-ranking should be treated as one component of the complete RAG pipeline.
- The objective is not simply higher ranking quality; it is better grounded answers at an acceptable latency and cost.
The central pattern is:
Large Corpus
↓
Fast Retrieval
↓
Broad Candidate Pool
↓
Precise Re-ranking
↓
Small High-Quality Context
↓
Grounded Generation
Or:
🧭 Chapter Navigation¶
Part V — Advanced Retrieval-Augmented Generation¶
Previous:
09. Agentic Retrieval
Next:
11. MMR and Diversity-Aware Retrieval
Section:
02 — Enterprise Retrieval Engineering
Enterprise Retrieval Engineering Path¶
01 Contextual Compression Retriever
↓
02 Ensemble Retriever
↓
03 Multi-Vector Retriever
↓
04 Time-Weighted Retriever
↓
05 Hybrid Search Retriever
↓
06 HyDE Retriever
↓
07 Router Retriever
↓
08 Multi-Stage Retrieval
↓
09 Agentic Retrieval
↓
10 Re-ranking Techniques
↓
11 MMR & Diversity-Aware Retrieval
↓
12 Metadata-Aware Retrieval
↓
13 Advanced Query Rewriting
Enterprise AI Engineering Handbook
Building Production-Grade Enterprise AI Systems — One Chapter at a Time.