Time-Weighted Retriever¶
📖 Overview¶
A Time-Weighted Retriever considers both relevance and recency when selecting documents.
Traditional vector retrieval primarily asks:
"Which documents are most semantically similar to this query?"
Time-weighted retrieval adds another question:
"How recent is this information?"
This is especially useful for knowledge bases where newer information should gradually become more important.
Examples include:
News
Product Documentation
Support Tickets
Operational Knowledge
Incident Reports
Engineering Discussions
Policy Updates
Research Notes
Conversation History
The core idea is:
🎯 Learning Objectives¶
After completing this chapter, you will be able to:
- Understand time-weighted retrieval
- Understand why recency matters in RAG
- Differentiate relevance ranking from recency-aware ranking
- Understand exponential time decay
- Configure decay parameters
- Combine semantic similarity with recency
- Implement time-weighted retrieval
- Understand the role of timestamps and metadata
- Combine time weighting with vector retrieval
- Combine time weighting with other retrieval strategies
- Understand common failure modes
- Design production-ready time-aware retrieval systems
- Evaluate retrieval quality across different time windows
1. Why Recency Matters¶
Consider an enterprise API policy.
API Security Policy
2022 Version
→ OAuth 2.0
→ Token expiration: 24 hours
2024 Version
→ OAuth 2.0
→ Token expiration: 12 hours
2026 Version
→ OAuth 2.0
→ Token expiration: 1 hour
A semantic retriever may find all three documents because they discuss the same topic.
However, if the user asks:
the latest version should normally receive higher priority.
Therefore:
2. Traditional Retrieval¶
A conventional vector retriever may rank documents based primarily on semantic similarity.
Example:
A traditional retriever might return:
even though the 2026 document is the most current.
3. Time-Weighted Retrieval¶
Time-weighted retrieval introduces a temporal signal.
Conceptually:
or, depending on the implementation:
The exact scoring model depends on the implementation.
The architecture becomes:
flowchart LR
A["User Query"] --> B["Vector Retriever"]
B --> C["Candidate Documents"]
C --> D["Similarity Score"]
C --> E["Document Timestamp"]
D --> F["Time-Weighted Ranking"]
E --> F
F --> G["Final Ranking"]
G --> H["Top-K Documents"]
4. Core Concept¶
A time-weighted retriever uses:
to determine the final ranking.
For example:
A time-aware system may prefer:
if recency is sufficiently important.
5. Time Decay¶
The influence of a document can gradually decrease as it becomes older.
A common conceptual model is exponential decay:
where:
The behavior is:
The decay should be selected according to the domain.
6. Decay Visualization¶
Recency Weight
↑
1.0 ●
│\
│ \
0.8 │ ●
│ \
0.6 │ ●
│ \
0.4 │ ●
│ \
0.2 │ ●
│ \
0.0 └────────────────────────→
New Older
The exact curve depends on the decay configuration.
The important idea is:
Older documents gradually lose temporal preference.
7. Half-Life Concept¶
A useful way to reason about decay is half-life.
Suppose the recency weight has a half-life of:
Then approximately:
This does not necessarily mean the document becomes unusable.
It means its recency contribution decreases over time.
8. Why Half-Life Matters¶
Different domains require different temporal behavior.
Breaking News¶
Incident Management¶
Product Documentation¶
Enterprise Policies¶
Historical Research¶
Therefore:
Time decay must be domain-specific.
9. Time-Weighted Retrieval Architecture¶
flowchart TD
A["User Query"] --> B["Semantic Retriever"]
B --> C["Candidate Documents"]
C --> D["Similarity Signal"]
C --> E["Created / Updated Timestamp"]
E --> F["Age Calculation"]
F --> G["Recency Function"]
D --> H["Score Combination"]
G --> H
H --> I["Final Ranking"]
I --> J["Top-K Results"]
J --> K["LLM"]
The retrieval system therefore adds a temporal ranking stage after candidate generation.
10. Timestamp Metadata¶
Time-weighted retrieval depends on reliable timestamps.
A document might contain:
{
"source": "api-security-policy.pdf",
"created_at": "2024-05-12T10:30:00Z",
"updated_at": "2026-07-20T09:15:00Z"
}
Possible temporal fields include:
The correct field depends on the business meaning of "current."
11. Created Time vs Updated Time¶
These fields are not interchangeable.
Consider:
If the system uses:
the document appears old.
If the business requirement is to prioritize the latest version, then:
may be more appropriate.
Therefore:
12. Effective Dates¶
Enterprise systems often need more than timestamps.
For example:
A policy may have been created earlier but become effective later.
Therefore:
For policies, contracts, pricing, and regulations, effective dates can be more important than modification dates.
13. Validity Windows¶
Some documents have explicit validity periods.
Example:
A production retrieval system can use:
before applying semantic retrieval.
This is different from simply applying time decay.
14. Time-Weighted Retrieval vs Temporal Filtering¶
These concepts should be distinguished.
Temporal Filtering¶
Hard constraint:
Documents outside the range are excluded.
Time Weighting¶
Soft preference:
Older documents may still be returned if they are highly relevant.
Therefore:
15. Combining Filtering and Weighting¶
A production system can combine both.
For example:
then:
This can be safer for time-sensitive enterprise information.
16. LangChain Example¶
LangChain provides a time-weighted retriever abstraction.
A simplified example is:
from langchain.retrievers import TimeWeightedVectorStoreRetriever
retriever = TimeWeightedVectorStoreRetriever(
vectorstore=vector_store,
decay_rate=0.01,
k=5
)
Documents need temporal metadata so that their age can be considered.
For example:
The exact metadata requirements depend on the framework version and implementation being used.
17. Adding Documents with Timestamps¶
Example:
from datetime import datetime, timezone
document.metadata["created_at"] = (
datetime.now(timezone.utc).isoformat()
)
vector_store.add_documents(
[document]
)
The retrieval layer can then use temporal information during ranking.
For production systems, timestamps should preferably be assigned from authoritative ingestion metadata rather than generated arbitrarily during retrieval.
18. Basic Retrieval Example¶
results = retriever.invoke(
"What is the current deployment policy?"
)
for document in results:
print(document.page_content)
print(document.metadata)
The result can contain:
This allows downstream components to understand why a particular version was retrieved.
19. Relevance + Recency¶
Consider three documents:
Document A
Similarity = 0.95
Age = 3 years
Document B
Similarity = 0.90
Age = 6 months
Document C
Similarity = 0.85
Age = 1 week
A purely semantic retriever may prefer:
A time-aware retriever might prefer:
depending on the decay configuration.
This illustrates the central trade-off:
20. The Freshness-Relevance Trade-Off¶
A document can be:
or:
For example:
If the query is:
the newer document may be more useful.
But if the query is:
the older document may be exactly what the user needs.
Therefore, time weighting should never blindly replace relevance.
21. Query-Aware Temporal Behavior¶
Temporal importance can depend on the query.
Compare:
with:
The first query requires:
The second requires:
Therefore, advanced systems may adapt temporal behavior based on query intent.
22. Query Intent and Recency¶
Conceptually:
flowchart TD
A["User Query"] --> B["Query Intent Detection"]
B --> C["Current / Latest"]
B --> D["Historical"]
B --> E["General Knowledge"]
B --> F["Time-Specific"]
C --> G["High Recency Weight"]
D --> H["Low / Targeted Recency"]
E --> I["Normal Recency"]
F --> J["Temporal Filter"]
G --> K["Retrieval"]
H --> K
I --> K
J --> K
This is a more sophisticated approach than applying one decay rate to every query.
23. Explicit Temporal Queries¶
Queries often contain temporal language:
These terms can be extracted during query processing.
Example:
Possible interpretation:
The system can then apply:
rather than relying only on generic recency decay.
24. Time-Weighted Retrieval with Hybrid Search¶
Time weighting can be combined with lexical and semantic retrieval.
Architecture:
flowchart TD
A["User Query"] --> B["Vector Retriever"]
A --> C["BM25 Retriever"]
B --> D["Semantic Results"]
C --> E["Lexical Results"]
D --> F["Candidate Pool"]
E --> F
F --> G["Temporal Scoring"]
G --> H["Final Ranking"]
H --> I["Top-K"]
This can combine:
25. Time-Weighted Retrieval with Ensemble Retrieval¶
The temporal signal can also be added to an ensemble.
Alternatively, time-aware retrievers can participate as one of the ensemble components.
The architecture should be selected based on the scoring and evaluation strategy.
26. Time-Weighted Retrieval with Re-ranking¶
A reranker can be applied after temporal retrieval.
Architecture:
flowchart LR
A["Query"] --> B["Time-Weighted Retriever"]
B --> C["Candidate Documents"]
C --> D["Reranker"]
D --> E["Final Documents"]
This allows:
27. Time-Weighted Retrieval with Contextual Compression¶
A useful RAG pipeline is:
Query
↓
Time-Weighted Retrieval
↓
Candidate Documents
↓
Contextual Compression
↓
Relevant Context
↓
LLM
Architecture:
flowchart TD
A["Query"] --> B["Time-Weighted Retriever"]
B --> C["Candidate Documents"]
C --> D["Contextual Compression"]
D --> E["Relevant Context"]
E --> F["Prompt Assembly"]
F --> G["LLM"]
This is particularly useful when recent documents are large and contain substantial irrelevant information.
28. Time-Weighted Retrieval with Multi-Vector Retrieval¶
Multi-Vector Retrieval can provide multiple representations, while time weighting prioritizes newer documents.
Query
↓
Multi-Vector Search
↓
Representation Matches
↓
Parent Resolution
↓
Temporal Ranking
↓
Final Documents
Architecture:
flowchart TD
A["Query"] --> B["Multi-Vector Retriever"]
B --> C["Representation Matches"]
C --> D["Parent Resolution"]
D --> E["Temporal Ranking"]
E --> F["Final Documents"]
This can be useful when both:
matter.
29. Time-Weighted Retrieval with Parent Documents¶
For document versioning, the architecture can be:
Query
↓
Child / Representation Retrieval
↓
Parent Documents
↓
Version Resolution
↓
Time-Aware Ranking
↓
Current Context
For example:
The system can prefer:
if it is the latest valid version.
30. Version-Aware Retrieval¶
Versioning is often more reliable than generic recency.
Example:
{
"document_id": "policy-100",
"version": "3",
"effective_from": "2026-07-01",
"effective_until": null,
"status": "active"
}
The retrieval pipeline can apply:
before ranking.
This prevents an outdated but recently modified draft from outranking the active policy.
31. Draft vs Published Documents¶
Consider:
A naive time-weighted system may prefer:
because it is newer.
But the enterprise application may need:
because it is the current published policy.
Therefore:
Recency is not the same as validity.
This is a critical enterprise retrieval principle.
32. Temporal Metadata Model¶
A robust metadata model can include:
{
"document_id": "policy-100",
"version": "3",
"created_at": "2025-10-01T09:00:00Z",
"updated_at": "2026-07-15T11:30:00Z",
"effective_from": "2026-07-01T00:00:00Z",
"effective_until": null,
"status": "published"
}
This enables more precise temporal reasoning.
33. Temporal Retrieval Pipeline¶
flowchart TD
A["User Query"] --> B["Query Understanding"]
B --> C["Temporal Intent"]
B --> D["Semantic Intent"]
C --> E["Temporal Constraints"]
D --> F["Semantic Retrieval"]
E --> G["Candidate Filtering"]
F --> G
G --> H["Recency / Validity Scoring"]
H --> I["Reranking"]
I --> J["Context Selection"]
J --> K["LLM"]
This architecture separates:
from:
34. Decay Rate Selection¶
The decay rate determines how quickly old information loses temporal influence.
Conceptually:
While:
Example:
The correct value should be established through evaluation.
35. Decay Configuration¶
Example configuration:
retrieval:
time_weighting:
enabled: true
decay_rate: 0.01
timestamp_field: updated_at
minimum_recency_weight: 0.1
An enterprise implementation may expose:
as configuration rather than hard-coding them.
36. Domain-Specific Profiles¶
Different domains can use different temporal profiles.
Example:
profiles:
news:
decay_rate: 0.20
support:
decay_rate: 0.05
engineering:
decay_rate: 0.01
policies:
decay_rate: 0.005
These numbers are illustrative.
The important architectural idea is:
37. Query-Specific Time Windows¶
Instead of continuous decay, some applications use explicit windows.
For example:
or:
The pipeline becomes:
This is often preferable when the user explicitly specifies a time range.
38. Temporal Filtering Example¶
Suppose:
The system can derive:
Then:
Only after filtering should the system perform deeper ranking.
39. Historical Questions¶
Time weighting can become harmful for historical queries.
Example:
The latest document may describe:
while the 2021 document describes:
If recency dominates:
Therefore, explicit historical intent should override generic recency preference.
40. Temporal Query Classification¶
A query classifier may identify:
Example:
"What is the current pricing?"
→ Current
"What was the pricing in 2022?"
→ Historical
"What changed between 2024 and 2026?"
→ Time-Specific
"How does OAuth work?"
→ Time-Neutral
This enables more appropriate retrieval behavior.
41. Time-Weighted Retrieval and Citations¶
Temporal retrieval makes source metadata especially important.
A response should ideally identify:
For example:
This helps users understand why a newer policy was selected.
42. Temporal Citations¶
Example response context:
{
"content": "Production APIs require OAuth 2.0 access tokens.",
"source": "api-security-policy.pdf",
"version": "3",
"effective_from": "2026-07-01",
"page": 14
}
This is more useful than:
because the temporal context is preserved.
43. Observability¶
Time-aware retrieval should expose temporal signals.
Example:
Query
↓
Candidate Documents
Document A
Similarity: 0.94
Age: 720 days
Recency Weight: 0.18
Document B
Similarity: 0.89
Age: 30 days
Recency Weight: 0.74
Document C
Similarity: 0.86
Age: 7 days
Recency Weight: 0.91
This makes retrieval decisions explainable.
44. Retrieval Trace¶
A production trace could contain:
{
"query": "What is the current token policy?",
"retriever": "time_weighted_vector",
"candidates": 20,
"timestamp_field": "updated_at",
"decay_rate": 0.01,
"top_result": {
"document_id": "policy-100",
"similarity": 0.89,
"recency_weight": 0.92
}
}
This is useful for:
45. Evaluation Strategy¶
A time-weighted retriever should be evaluated against a baseline.
Compare:
The key additional metric is:
Does the system retrieve the correct version or time period?
46. Temporal Evaluation Dataset¶
Create test cases such as:
This helps evaluate whether the system understands temporal intent rather than simply preferring newer documents.
47. Example Evaluation Table¶
| Query Type | Vector | Time-Weighted | Temporal Filter |
|---|---|---|---|
| Current | 0.78 | 0.88 | 0.91 |
| Historical | 0.84 | 0.72 | 0.90 |
| Time-specific | 0.70 | 0.81 | 0.93 |
| Time-neutral | 0.87 | 0.86 | 0.84 |
The numbers are illustrative.
The important observation is that:
48. Common Failure Modes¶
48.1 Overweighting Recency¶
A recent document can still be unrelated.
48.2 Ignoring Historical Intent¶
A current document may incorrectly replace the historical document required by the user.
48.3 Wrong Timestamp¶
Using:
when the business requirement is:
can produce incorrect results.
48.4 Draft Documents¶
A recently updated draft may outrank a valid published document.
48.5 Stale Metadata¶
Incorrect timestamps lead directly to incorrect temporal ranking.
48.6 Excessive Decay¶
If decay is too aggressive:
Important long-lived knowledge may disappear.
48.7 Insufficient Decay¶
If decay is too weak:
The system may fail to prioritize current information.
49. Production Architecture¶
A mature enterprise temporal retrieval architecture can look like:
flowchart TD
A["User Query"] --> B["Query Understanding"]
B --> C["Semantic Intent"]
B --> D["Temporal Intent"]
D --> E["Temporal Constraints"]
C --> F["Candidate Retrieval"]
E --> G["Temporal Filtering"]
F --> G
G --> H["Candidate Documents"]
H --> I["Relevance Scoring"]
H --> J["Recency / Validity Scoring"]
I --> K["Temporal Ranking"]
J --> K
K --> L["Reranker"]
L --> M["Contextual Compression"]
M --> N["Prompt Assembly"]
N --> O["LLM"]
O --> P["Response Validation"]
P --> Q["Citation / Version Attribution"]
Q --> R["Enterprise Response"]
This architecture separates:
Temporal Intent
↓
Temporal Constraints
↓
Retrieval
↓
Recency Preference
↓
Precision Optimization
↓
Generation
50. Framework-Agnostic Interface¶
An enterprise AI platform can define a temporal retrieval interface:
from abc import ABC, abstractmethod
class TimeAwareRetriever(ABC):
@abstractmethod
def retrieve(
self,
query: str,
top_k: int,
*,
timestamp_field: str = "updated_at"
) -> list:
pass
Possible implementations:
class VectorTimeWeightedRetriever(TimeAwareRetriever):
...
class HybridTimeWeightedRetriever(TimeAwareRetriever):
...
class VersionAwareRetriever(TimeAwareRetriever):
...
This allows the application to remain independent of the underlying retrieval framework.
51. Configuration-Driven Architecture¶
A production configuration might look like:
retrieval:
temporal:
enabled: true
timestamp_field: updated_at
decay:
strategy: exponential
rate: 0.01
validity:
enabled: true
versioning:
enabled: true
historical_queries:
disable_recency_bias: true
This provides explicit control over temporal behavior.
52. Decision Framework¶
flowchart TD
A["User Query"] --> B{"Contains Temporal Intent?"}
B -->|Yes| C{"Historical or Current?"}
C -->|Historical| D["Apply Time Constraint"]
C -->|Current| E["Prefer Recent Valid Documents"]
B -->|No| F["Use Normal Retrieval"]
E --> G["Semantic + Temporal Ranking"]
D --> H["Temporal Filtering + Retrieval"]
F --> I["Semantic Retrieval"]
G --> J["Reranking"]
H --> J
I --> J
J --> K["Context Selection"]
This avoids blindly applying recency to every query.
53. When to Use Time-Weighted Retrieval¶
It is particularly useful when:
- Information changes frequently
- Newer documents are generally more useful
- Knowledge bases contain multiple document versions
- Users frequently ask for current information
- Support knowledge evolves over time
- Engineering documentation changes frequently
- Operational information becomes stale
- Conversation or interaction history should favor recent information
Examples:
Support Knowledge
Incident Management
Product Releases
Engineering Discussions
News
Operational Runbooks
Current Policies
54. When It May Not Be Appropriate¶
Time weighting may be less useful when:
or:
or:
or:
For example:
may require temporal filtering or explicit time targeting rather than generic recency decay.
55. Recommended Enterprise Pattern¶
A robust enterprise pattern is:
Query
↓
Query Intent Detection
↓
Temporal Intent Detection
↓
Validity Filtering
↓
Semantic / Hybrid Retrieval
↓
Recency-Aware Ranking
↓
Reranking
↓
Contextual Compression
↓
Citation + Version Attribution
↓
LLM
Architecture:
flowchart LR
A["Query"] --> B["Intent Detection"]
B --> C["Temporal Constraints"]
C --> D["Validity Filtering"]
D --> E["Retrieval"]
E --> F["Time-Aware Ranking"]
F --> G["Reranking"]
G --> H["Context Selection"]
H --> I["LLM"]
I --> J["Citation / Version"]
This is safer than simply applying a decay function to every retrieval request.
56. Production Checklist¶
Before deploying time-weighted retrieval:
☐ Temporal requirements are clearly defined
☐ Correct timestamp field is selected
☐ Effective dates are considered where applicable
☐ Document validity is represented
☐ Published vs draft status is represented
☐ Version information is preserved
☐ Decay configuration is externalized
☐ Decay parameters are evaluated
☐ Historical queries are handled separately
☐ Explicit temporal filters are supported
☐ Source metadata is preserved
☐ Temporal signals are observable
☐ Retrieval baseline is available
☐ Temporal correctness is evaluated
☐ Latency impact is measured
☐ Cost impact is measured
☐ Regression tests cover current and historical queries
☐ Citation and version attribution are preserved
57. Key Takeaways¶
- Time-Weighted Retrieval combines relevance with temporal information.
- It is useful when newer information should receive greater preference.
- Recency should generally be treated as a ranking signal rather than an absolute replacement for relevance.
- Exponential decay is one common conceptual model for reducing the influence of older documents.
- Decay parameters should be selected according to the domain.
- Created, updated, published, and effective timestamps have different meanings.
- Recency is not the same as validity.
- A recently updated draft should not automatically outrank a current published policy.
- Temporal filtering and temporal weighting solve different problems.
- Explicit historical queries should not be blindly biased toward recent documents.
- Query intent can determine how strongly recency should influence retrieval.
- Time-aware retrieval can be combined with vector, BM25, ensemble, multi-vector, reranking, and contextual compression techniques.
- Temporal metadata should remain available for citations and auditing.
- Production systems should evaluate both retrieval quality and temporal correctness.
- The objective is not simply to retrieve the newest information.
- The objective is to retrieve the most relevant information for the requested time context.
The central pattern is:
Understand Time Intent
↓
Apply Validity Constraints
↓
Retrieve Relevant Candidates
↓
Apply Temporal Preference
↓
Rank
↓
Generate from Correct Evidence
Or simply:
🧭 Chapter Navigation¶
Part V — Advanced Retrieval-Augmented Generation¶
Previous:
03. Multi-Vector Retriever
Next:
05. Hybrid Search Retriever
Section:
02 — Enterprise Retrieval Engineering
Enterprise Retrieval Engineering Path¶
01 Contextual Compression Retriever
↓
02 Ensemble Retriever
↓
03 Multi-Vector Retriever
↓
04 Time-Weighted Retriever
↓
05 Hybrid Search Retriever
↓
06 HyDE Retriever
↓
07 Router Retriever
↓
08 Multi-Stage Retrieval
↓
09 Agentic Retrieval
↓
10 Re-ranking Techniques
↓
11 MMR & Diversity-Aware Retrieval
↓
12 Metadata-Aware Retrieval
↓
13 Advanced Query Rewriting
Enterprise AI Engineering Handbook
Building Production-Grade Enterprise AI Systems — One Chapter at a Time.