13 — Vector Database Fundamentals¶

Understand how vector databases store, index, search, filter, and manage high-dimensional embeddings that power modern semantic search and Retrieval-Augmented Generation (RAG) systems.

📖 Overview¶

A vector database provides the infrastructure required to store and retrieve embeddings efficiently.

In a typical RAG system:

Documents
    ↓
Document Processing
    ↓
Chunking
    ↓
Embedding Model
    ↓
Vector Database
    ↓
Similarity Search
    ↓
Retrieved Context
    ↓
LLM

The embedding model converts text into numerical vectors.

The vector database provides the capabilities to:

Store vectors
Store associated content
Store metadata
Build vector indexes
Perform similarity search
Apply metadata filters
Update records
Delete records
Scale retrieval workloads

The key idea is:

An embedding model creates the representation; a vector database makes that representation searchable at scale.

1. Why Vector Databases Matter¶

Traditional databases are optimized for structured queries such as:

SELECT *
FROM employees
WHERE department = 'Engineering';

Semantic retrieval asks a different question:

Which stored pieces of content are
most semantically similar to this query?

For example:

Query:
"What is the company's annual leave policy?"

       ↓

Embedding Model

       ↓

Query Vector

       ↓

Vector Database

       ↓

Most Similar Chunks

This enables semantic search even when the query and document use different words.

2. Traditional Search vs Vector Search¶

Traditional keyword search may look for:

annual leave policy

Vector search attempts to understand semantic relationships such as:

vacation entitlement
paid time off
employee leave allowance
annual leave days

These phrases may have different words but similar meaning.

3. Vector Database Architecture¶

flowchart TD
    A["Documents"] --> B["Chunking"]
    B --> C["Embedding Model"]
    C --> D["Embedding Vectors"]

    D --> E["Vector Database"]

    E --> F["Vector Index"]
    E --> G["Metadata Store"]
    E --> H["Document / Chunk Store"]

    I["User Query"] --> J["Query Embedding"]
    J --> E

    E --> K["Similarity Search"]
    K --> L["Filtered Results"]

A production vector database is therefore more than a simple array of vectors.

4. What Is a Vector?¶

A vector is a numerical representation of data.

Conceptually:

Text
 ↓
Embedding Model
 ↓
[0.12, -0.42, 0.81, ...]

For example, a simplified vector might be:

[0.21, 0.73, -0.14, 0.62]

Real embedding models typically produce vectors with hundreds or thousands of dimensions.

5. High-Dimensional Vectors¶

Suppose an embedding model produces:

768 dimensions

Then each document chunk becomes:

Vector =
[
  x1,
  x2,
  x3,
  ...
  x768
]

Another model might produce:

1536 dimensions

The vector database index must be configured consistently with the selected embedding model.

6. Vector Database Record¶

A vector record commonly contains:

{
  "id": "policy-001-chunk-007",
  "vector": [0.12, -0.42, 0.81],
  "text": "Employees receive 25 days of annual leave.",
  "metadata": {
    "document_id": "policy-001",
    "section": "Annual Leave",
    "page": 42,
    "department": "HR"
  }
}

Conceptually:

Vector Record
├── ID
├── Vector
├── Content
└── Metadata

7. Vector Database vs Vector Index¶

These concepts are related but different.

Vector Database¶

Provides:

Storage
Query APIs
Metadata
Filtering
CRUD
Persistence
Scaling
Security
Operations

Vector Index¶

Provides:

Efficient similarity search

A vector database may use one or more vector indexing algorithms internally.

8. Similarity Search¶

Suppose the query becomes:

"How many annual leave days do employees get?"

The query is converted into a vector:

Query
 ↓
Embedding
 ↓
Query Vector

The vector database compares it against stored vectors.

Query Vector
     ↓
Similarity Search
     ↓
Vector 7
Vector 21
Vector 45
Vector 82

The most relevant vectors are returned.

9. Similarity Search Architecture¶

flowchart LR
    A["User Query"] --> B["Embedding Model"]
    B --> C["Query Vector"]

    C --> D["Vector Index"]

    D --> E["Candidate Vectors"]
    E --> F["Similarity Scoring"]
    F --> G["Top-K Results"]

10. Similarity Metrics¶

Common similarity measures include:

Cosine Similarity
Dot Product
Euclidean Distance

The correct metric depends on:

Embedding Model
Vector Normalization
Index Configuration
Retrieval Objective

11. Cosine Similarity¶

Cosine similarity measures the angle between two vectors.

Conceptually:

Similarity
   ↑
   │      A
   │     /
   │    /
   │   / θ
   │  /
   │ /
   └────────────→ B

The important idea is that cosine similarity focuses on vector direction rather than raw magnitude.

For normalized embeddings, cosine similarity and dot product are closely related.

12. Dot Product¶

The dot product measures alignment between vectors.

Conceptually:

a · b

Higher values indicate stronger alignment when the embedding setup is appropriate.

Many modern embedding systems use normalized vectors, making dot-product search particularly convenient.

13. Euclidean Distance¶

Euclidean distance measures geometric distance.

A ●
   \
    \
     ● B

Smaller distance means vectors are closer.

However, the appropriate distance function depends on the embedding model and indexing configuration.

14. Similarity vs Distance¶

Similarity metrics usually work as:

Higher = Better

Distance metrics usually work as:

Lower = Better

Therefore a retrieval implementation must understand how its vector index interprets the selected metric.

15. Exact Search¶

The simplest vector search compares the query against every stored vector.

Query Vector
     ↓
Compare with Vector 1
Compare with Vector 2
Compare with Vector 3
...
Compare with Vector N

Conceptually:

O(N)

comparisons for N vectors.

This can become expensive at large scale.

16. Approximate Nearest Neighbor Search¶

Production vector systems commonly use:

Approximate Nearest Neighbor (ANN) search.

Instead of comparing against every vector, the index narrows the search space.

Millions of Vectors
       ↓
ANN Index
       ↓
Candidate Region
       ↓
Relevant Vectors

This improves search performance.

17. ANN Trade-off¶

ANN search introduces a trade-off:

Higher Search Speed
        ↕
Search Accuracy / Recall

The goal is usually:

Very high retrieval quality
+
Much lower latency

rather than mathematically perfect exhaustive search.

18. Vector Index Architecture¶

flowchart TD
    A["Vectors"] --> B["Vector Index"]

    B --> C["Search Structure"]

    D["Query Vector"] --> B
    B --> E["Candidate Neighbors"]
    E --> F["Top-K Results"]

The exact search structure depends on the indexing algorithm.

19. HNSW¶

One widely used ANN technique is:

Hierarchical Navigable Small World (HNSW)

HNSW represents vectors as nodes in a graph.

Conceptually:

Layer 2:

A -------- D
           |
           E

Layer 1:

A --- B --- C --- D
      |     |
      E --- F

Higher layers provide longer-range navigation.

Lower layers provide more detailed neighborhood search.

20. HNSW Search Concept¶

flowchart TD
    A["Query"] --> B["Top HNSW Layer"]
    B --> C["Navigate Toward Nearest Region"]
    C --> D["Lower Layer"]
    D --> E["Local Neighborhood"]
    E --> F["Top-K Candidates"]

HNSW is popular because it can provide strong search quality with low latency.

21. HNSW Parameters¶

Vector databases may expose parameters such as:

M
efConstruction
efSearch

Conceptually:

M¶

Controls graph connectivity.

Higher values can improve search quality but increase:

Memory
Index Construction Cost

efConstruction¶

Controls effort during index construction.

efSearch¶

Controls search effort at query time.

Higher values can improve recall but increase latency.

22. IVF¶

Another family of techniques uses:

Inverted File (IVF) indexes.

Conceptually:

Vectors
   ↓
Clusters
   ↓
Cluster 1
Cluster 2
Cluster 3
...
Cluster N

During search, only relevant clusters need to be examined.

23. IVF Architecture¶

flowchart TD
    A["Vectors"] --> B["Clustering"]
    B --> C["Cluster 1"]
    B --> D["Cluster 2"]
    B --> E["Cluster 3"]
    B --> F["Cluster N"]

    G["Query"] --> H["Nearest Cluster Selection"]
    H --> C
    H --> D

    C --> I["Candidate Search"]
    D --> I
    I --> J["Top-K"]

24. PQ¶

Another technique is:

Product Quantization (PQ)

It compresses vector representations to reduce memory usage.

Conceptually:

Full Precision Vector
        ↓
Vector Quantization
        ↓
Compressed Representation

This can be useful for very large vector collections.

25. Index Selection¶

Different workloads may benefit from different indexes.

Consider:

Dataset Size
Query Latency
Recall Requirements
Memory
Update Frequency
Hardware

There is no universal vector index that is optimal for every workload.

26. Metadata Filtering¶

Vector search is often combined with metadata filters.

Example:

Query:
"What is the leave policy?"

Filter:
department = HR
country = IN

Conceptually:

Semantic Search
+
Metadata Filter

27. Metadata Filtering Architecture¶

flowchart TD
    A["User Query"] --> B["Query Embedding"]
    B --> C["Vector Search"]

    D["User Context"] --> E["Metadata Filters"]

    C --> F["Candidate Results"]
    E --> F

    F --> G["Authorized Relevant Results"]

Metadata filtering is especially important in enterprise systems.

28. Metadata Fields¶

Common metadata fields include:

document_id
document_type
department
country
language
author
created_at
updated_at
version
page
section
tenant
classification
source

29. Filtering Before vs After Search¶

Two conceptual approaches exist.

Pre-filtering¶

Metadata Filter
      ↓
Eligible Documents
      ↓
Vector Search

Post-filtering¶

Vector Search
      ↓
Candidate Results
      ↓
Metadata Filter

Pre-filtering can reduce the search space, while post-filtering may be easier to implement in some systems.

The correct approach depends on the vector database and workload.

30. Hybrid Search¶

Vector databases are often used alongside keyword search.

Semantic Search
      +
Keyword Search
      ↓
Combined Results

For example:

Vector Search

may understand:

vacation entitlement

while keyword search may be better for:

Policy ID HR-2026-0042

Hybrid retrieval combines both strengths.

31. Hybrid Search Architecture¶

flowchart TD
    A["User Query"] --> B["Semantic Search"]
    A --> C["Keyword Search"]

    B --> D["Semantic Results"]
    C --> E["Keyword Results"]

    D --> F["Result Fusion"]
    E --> F

    F --> G["Final Candidates"]

Hybrid search will be explored more deeply in the advanced retrieval module.

32. CRUD Operations¶

A vector database should support lifecycle operations.

Create
Read
Update
Delete

Example:

Insert Vector
Retrieve Vector
Update Metadata
Delete Vector

These operations become important when enterprise documents change.

33. Upsert¶

Many vector stores provide:

Upsert

which means:

Insert if new
Update if existing

Conceptually:

vector_store.upsert(
    records
)

This simplifies incremental indexing.

34. Delete Operations¶

If a source document is deleted:

Source Document
      ↓
Deleted
      ↓
Associated Chunks
      ↓
Associated Vectors
      ↓
Deleted

Vector lifecycle must follow source lifecycle.

35. Vector Database Lifecycle¶

flowchart LR
    A["Document"] --> B["Chunk"]
    B --> C["Embedding"]
    C --> D["Vector Record"]

    D --> E["Insert / Upsert"]
    E --> F["Search"]
    F --> G["Update"]
    G --> H["Delete"]

36. Persistence¶

A production vector database should persist vectors and metadata.

Possible storage architectures include:

Local Disk
Object Storage
Managed Database
Distributed Storage

Persistence requirements depend on:

Scale
Availability
Durability
Recovery Objectives

37. In-Memory vs Persistent Vector Stores¶

In-Memory¶

Advantages:

Very Fast
Simple
Useful for Experiments

Disadvantages:

Data Loss on Restart
Limited Scale

Persistent¶

Advantages:

Durability
Recovery
Production Support

Disadvantages:

Operational Complexity
Storage Cost

38. Vector Database vs Relational Database¶

A relational database excels at:

Structured Data
Transactions
Joins
Constraints
Aggregations

A vector database specializes in:

High-Dimensional Vector Search
Similarity Retrieval
Embedding Indexes

Modern architectures may use both.

39. PostgreSQL + pgvector¶

A relational database can also provide vector capabilities.

Conceptually:

PostgreSQL
 ├── Relational Tables
 ├── Metadata
 └── Vector Columns

This can be attractive when an application already depends heavily on PostgreSQL.

The decision between a specialized vector database and PostgreSQL with vector support depends on:

Scale
Workload
Operational Requirements
Existing Infrastructure
Query Patterns

40. Vector Database Examples¶

Common technologies include:

FAISS
Chroma
pgvector
Qdrant
Milvus
Weaviate
Pinecone
OpenSearch
Elasticsearch

They differ in:

Deployment
Indexing
Filtering
Scaling
Persistence
Cloud Integration
Operational Model

41. FAISS¶

FAISS is primarily a vector similarity search library.

Conceptually:

Application
    ↓
FAISS
    ↓
Vector Index

It is useful for:

Experiments
Local Applications
Research
Custom Retrieval Systems

It is not necessarily a complete distributed vector database by itself.

42. Chroma¶

Chroma provides a developer-friendly vector storage and retrieval experience.

Conceptually:

collection.add(
    ids=["chunk-1"],
    documents=["Annual leave is 25 days."],
    embeddings=[[0.1, 0.2, 0.3]]
)

It is useful for:

Prototyping
Local Development
RAG Applications

43. Qdrant¶

Qdrant is a vector search engine/database designed around vector similarity and metadata filtering.

Conceptually:

Collection
 ├── Point
 │    ├── Vector
 │    └── Payload
 └── Point

The payload can contain metadata used for filtering.

44. Managed Vector Services¶

Cloud-managed vector systems can reduce operational burden.

Typical capabilities include:

Managed Infrastructure
Scaling
High Availability
Backups
Monitoring
Security
API Access

The trade-off is greater dependency on the selected platform.

45. Vector Database Selection¶

Consider:

1. Dataset size

2. Query throughput

3. Latency requirements

4. Recall requirements

5. Metadata filtering

6. Hybrid search

7. Persistence

8. High availability

9. Backup and recovery

10. Multi-tenancy

11. Security

12. Cloud integration

13. Cost

14. Operational complexity

15. Vendor lock-in

46. Vector Store Abstraction¶

Enterprise applications should avoid tightly coupling business logic to one vector database.

A framework-independent interface could be:

from abc import ABC, abstractmethod


class VectorStore(ABC):

    @abstractmethod
    def upsert(self, records):
        pass

    @abstractmethod
    def search(
        self,
        vector,
        top_k: int,
        filters=None
    ):
        pass

    @abstractmethod
    def delete(self, ids):
        pass

This creates a stable application boundary.

47. Java VectorStore Interface¶

For a Java-first enterprise architecture:

public interface VectorStore {

    void upsert(
        List<VectorRecord> records
    );

    List<VectorRecord> search(
        List<Float> queryVector,
        int topK,
        SearchFilter filter
    );

    void delete(
        List<String> ids
    );
}

The application does not need to know which vector database implements the interface.

48. Adapter Architecture¶

flowchart TD
    A["RAG Application"] --> B["VectorStore"]

    B --> C["Chroma Adapter"]
    B --> D["FAISS Adapter"]
    B --> E["Qdrant Adapter"]
    B --> F["pgvector Adapter"]
    B --> G["Cloud Vector Adapter"]

This is a Ports & Adapters approach.

49. Vector Record Model¶

A Java record could be:

public record VectorRecord(
    String id,
    List<Float> vector,
    String content,
    Map<String, Object> metadata
) {
}

This provides a common domain representation independent of the underlying database.

50. Vector Store Factory¶

A factory can select the implementation:

public interface VectorStoreFactory {

    VectorStore create(
        VectorStoreType type
    );
}

Possible values:

CHROMA
FAISS
QDRANT
PGVECTOR
MILVUS

The exact set depends on the application.

51. Embedding Dimension Compatibility¶

One of the most important operational rules:

Embedding Model
        ↓
Vector Dimension
        ↓
Vector Index

These must be compatible.

Example:

Embedding Model
 → 768 dimensions

Vector Index
 → 768 dimensions

Switching to:

1536 dimensions

usually requires a new index or collection.

52. Embedding Model Migration¶

Suppose production currently uses:

Embedding Model V1
768 dimensions

and you want:

Embedding Model V2
1536 dimensions

A safe migration is:

Documents
   ↓
V2 Embeddings
   ↓
New Vector Index
   ↓
Evaluation
   ↓
Production Cutover

53. Blue-Green Vector Indexing¶

flowchart TD
    A["Documents"] --> B["Embedding Model V1"]
    A --> C["Embedding Model V2"]

    B --> D["Index V1"]
    C --> E["Index V2"]

    D --> F["Retrieval Evaluation"]
    E --> F

    F --> G["Production Cutover"]

This avoids abruptly replacing a working index.

54. Batch Ingestion¶

Large datasets should generally be ingested in batches.

Documents
 ↓
Chunks
 ↓
Batch
 ↓
Embeddings
 ↓
Vector Records
 ↓
Batch Upsert

Example:

for batch in batches:

    vectors = embedding_model.embed_batch(
        batch
    )

    vector_store.upsert(
        vectors
    )

Batching improves throughput and can reduce API overhead.

55. Query-Time Workflow¶

At query time:

User Query
    ↓
Query Embedding
    ↓
Vector Search
    ↓
Metadata Filtering
    ↓
Top-K Results
    ↓
Optional Reranking
    ↓
Context Assembly
    ↓
LLM

56. Query-Time Architecture¶

flowchart TD
    A["User Query"] --> B["Query Embedding"]
    B --> C["Vector Database"]

    D["User Authorization"] --> E["Metadata Filter"]
    E --> C

    C --> F["Top-K Candidates"]
    F --> G["Optional Reranking"]
    G --> H["Context Assembly"]
    H --> I["LLM"]

57. Top-K¶

K determines how many results are returned.

Example:

Top-K = 5

means:

Return the 5 most relevant candidates.

A larger K may improve recall but can increase:

Context Size
Latency
LLM Cost
Noise

The optimal value should be evaluated.

58. Similarity Threshold¶

Instead of returning every result:

Top-K = 10

a system can also apply:

Similarity >= threshold

For example:

Only return sufficiently relevant results.

However, similarity scores are model- and metric-dependent, so thresholds must be calibrated on real data.

59. Top-K + Threshold¶

A production retriever may use:

Candidate Search
       ↓
Top-K
       ↓
Similarity Threshold
       ↓
Metadata Filtering
       ↓
Final Context

The exact ordering depends on the vector database and retrieval design.

60. Empty Retrieval¶

A production RAG system must handle:

No relevant results

Do not automatically send unrelated documents to the LLM.

Possible behavior:

No Relevant Context
       ↓
Controlled Response

For example:

"I could not find sufficient information
in the available enterprise knowledge base."

61. Retrieval Confidence¶

Vector similarity should not automatically be interpreted as:

Answer Confidence

A high similarity score means:

The retrieved vector is similar to the query

It does not necessarily mean:

The retrieved content contains the correct answer.

Additional evaluation is required.

62. Vector Database Security¶

Production systems should consider:

Authentication
Authorization
Encryption
Network Isolation
Tenant Isolation
Audit Logging
Secret Management
Access Policies

A vector database may contain sensitive enterprise knowledge.

63. Multi-Tenancy¶

A multi-tenant vector system might organize data as:

Tenant A
 └── Collection / Namespace

Tenant B
 └── Collection / Namespace

Tenant C
 └── Collection / Namespace

Alternative designs may use metadata:

tenant_id = tenant-a

with strict filtering.

The correct isolation model depends on security and scale requirements.

64. Tenant Isolation¶

flowchart TD
    A["User Request"] --> B["Authentication"]
    B --> C["Tenant Context"]
    C --> D["Vector Search"]

    D --> E["Tenant Filter"]
    E --> F["Authorized Results"]

Tenant isolation should be enforced by infrastructure and application controls, not by prompt instructions.

65. Vector Database Observability¶

Monitor:

Query Latency
P50 Latency
P95 Latency
P99 Latency
Queries Per Second
Index Size
Memory Usage
CPU
Storage
Search Recall
Failed Queries
Upsert Throughput
Delete Throughput

66. Ingestion Metrics¶

Track:

Documents Processed
Chunks Indexed
Vectors Created
Vectors Updated
Vectors Deleted
Embedding Failures
Upsert Failures
Indexing Latency

67. Query Metrics¶

Useful query metrics include:

Query Count
Search Latency
Top-K Distribution
Similarity Score Distribution
Empty Retrieval Rate
Metadata Filter Rate
Reranking Rate
Context Size

These help identify retrieval problems.

68. Vector Database Monitoring¶

flowchart TD
    A["Vector Database"] --> B["Latency Metrics"]
    A --> C["Throughput Metrics"]
    A --> D["Storage Metrics"]
    A --> E["Index Metrics"]
    A --> F["Error Metrics"]

    B --> G["Observability Platform"]
    C --> G
    D --> G
    E --> G
    F --> G

69. Backup and Recovery¶

Production vector data should have an appropriate recovery strategy.

Consider:

Backups
Snapshots
Replication
Recovery Point Objective
Recovery Time Objective
Re-indexing Strategy

A useful question is:

If the vector database disappears, how quickly can we rebuild it from the source documents?

If the answer is:

Documents + Embedding Model + Configuration

then the vector store can often be treated as a rebuildable derived data layer.

70. Vector Store as Derived Data¶

A useful architecture principle:

Source Documents
       ↓
Processing
       ↓
Chunks
       ↓
Embeddings
       ↓
Vector Store

The vector database is generally derived from the authoritative source.

This makes:

Re-indexing
Migration
Recovery
Model Upgrades

more manageable.

71. Rebuilding a Vector Index¶

A rebuild may involve:

Read Source Documents
        ↓
Process
        ↓
Chunk
        ↓
Embed
        ↓
Build New Index
        ↓
Validate
        ↓
Cut Over

This is often safer than trying to modify a large production index in place.

72. Vector Database Performance¶

Performance depends on:

Number of Vectors
Vector Dimensions
Index Type
Index Parameters
Metadata Filters
Hardware
Concurrency
Query Distribution

Do not benchmark a vector database using only a tiny dataset.

73. Benchmarking¶

A useful benchmark should include:

Representative Dataset
Representative Queries
Expected Top-K
Latency Targets
Concurrency
Index Configuration
Metadata Filters

Measure:

P50
P95
P99
Recall
Throughput
Memory
Storage

74. Vector Database Evaluation¶

flowchart TD
    A["Representative Dataset"] --> B["Index"]
    C["Representative Queries"] --> D["Search"]

    B --> D
    D --> E["Latency"]
    D --> F["Recall"]
    D --> G["Throughput"]
    D --> H["Resource Usage"]

    E --> I["Evaluation"]
    F --> I
    G --> I
    H --> I

75. Common Vector Database Mistakes¶

75.1 Treating a Vector Database as Just a Vector Array¶

Production systems need:

Metadata
Filtering
Lifecycle
Security
Observability

75.2 Ignoring Embedding Dimension¶

The index and embedding model must be compatible.

75.3 Choosing an Index Without Benchmarking¶

ANN parameters directly affect:

Recall
Latency
Memory

75.4 No Metadata Filtering¶

Enterprise RAG often requires:

Tenant
Department
Country
Security Classification

filters.

75.5 Ignoring Deletions¶

Deleted source documents must not remain retrievable.

75.6 No Backup or Rebuild Strategy¶

A vector database should be recoverable.

75.7 Mixing Embedding Models¶

Do not blindly mix vectors generated by incompatible embedding models in one index.

75.8 No Query Monitoring¶

Without latency and retrieval metrics, production degradation can remain invisible.

75.9 Sending Too Many Results to the LLM¶

More vectors do not automatically mean better answers.

75.10 Using Similarity Score as Truth¶

Similarity indicates relevance, not correctness.

76. Best Practices¶

1. Treat vectors as derived data.

2. Preserve source document lineage.

3. Store chunk content with vector metadata where appropriate.

4. Track embedding model versions.

5. Track vector dimensions.

6. Choose similarity metrics intentionally.

7. Benchmark ANN indexes.

8. Tune ANN parameters using representative workloads.

9. Support metadata filtering.

10. Design for multi-tenancy.

11. Implement upsert and deletion workflows.

12. Batch ingestion workloads.

13. Validate vectors before indexing.

14. Monitor query latency.

15. Monitor indexing throughput.

16. Monitor retrieval quality.

17. Implement backups or a rebuild strategy.

18. Use blue-green index migration for major model changes.

19. Keep vector-store access behind an application interface.

20. Avoid leaking vendor-specific APIs into business logic.

21. Evaluate specialized vector databases against existing relational infrastructure.

22. Use hybrid search when keyword matching provides complementary value.

23. Calibrate similarity thresholds on real data.

24. Handle empty retrieval explicitly.

25. Secure vector databases like other enterprise data stores.

77. Production Vector Database Workflow¶

1. Process enterprise documents.

2. Generate retrieval chunks.

3. Generate embeddings.

4. Validate embedding dimensions.

5. Validate vector values.

6. Attach metadata.

7. Generate stable vector IDs.

8. Batch records.

9. Upsert into vector store.

10. Build or update vector indexes.

11. Validate indexing.

12. Record embedding model version.

13. Record document version.

14. Expose retrieval API.

15. Generate query embedding.

16. Apply tenant and security filters.

17. Execute similarity search.

18. Retrieve Top-K candidates.

19. Apply thresholds or additional retrieval logic.

20. Return relevant context.

21. Monitor latency and retrieval quality.

22. Process document updates incrementally.

23. Remove vectors for deleted documents.

24. Rebuild indexes when required.

25. Evaluate retrieval continuously.

78. Production Architecture¶

flowchart TD
    A["Enterprise Documents"] --> B["Document Processing"]
    B --> C["Chunking"]
    C --> D["Embedding Provider"]

    D --> E["VectorStore Port"]

    E --> F["Vector Database"]

    F --> G["Vector Index"]
    F --> H["Metadata Store"]

    I["User Query"] --> J["Query Embedding"]
    J --> E

    K["Identity / Tenant Context"] --> L["Security Filters"]
    L --> E

    E --> M["Top-K Results"]
    M --> N["RAG Pipeline"]
    N --> O["LLM"]

79. Enterprise Architecture Principles¶

A production vector layer should follow:

Separation of Concerns
        ↓
Document Processing
        ↓
Chunking
        ↓
Embedding
        ↓
Vector Storage
        ↓
Retrieval

Each capability should have a clear responsibility.

For example:

DocumentProcessor
DocumentChunker
EmbeddingProvider
VectorStore
Retriever

This avoids turning the vector database into the center of the entire AI architecture.

80. Framework-Agnostic Design¶

LangChain and LlamaIndex can simplify vector-store integration.

However:

Application
   ↓
Framework API
   ↓
Vendor API

can create tight coupling.

A more portable design is:

Application
   ↓
VectorStore Interface
   ↓
Adapter
   ↓
Vector Database

This makes migration easier.

81. LangChain Conceptual Example¶

A simplified LangChain-style workflow:

from langchain_chroma import Chroma

vector_store = Chroma(
    collection_name="enterprise-documents",
    embedding_function=embedding_model
)

vector_store.add_documents(
    documents
)

results = vector_store.similarity_search(
    "What is the annual leave policy?",
    k=5
)

The underlying architecture remains:

Documents
 ↓
Embeddings
 ↓
Vector Store
 ↓
Similarity Search

82. LlamaIndex Conceptual Example¶

A simplified LlamaIndex-style workflow:

from llama_index.core import VectorStoreIndex

index = VectorStoreIndex.from_documents(
    documents
)

retriever = index.as_retriever(
    similarity_top_k=5
)

results = retriever.retrieve(
    "What is the annual leave policy?"
)

The framework manages implementation details, while the underlying concept remains vector indexing and retrieval.

83. Why Framework Examples Matter¶

Framework examples help demonstrate:

How documents enter an index
How vectors are stored
How retrieval is invoked
How metadata flows through the pipeline

But the enterprise architecture should still understand:

What is happening underneath?

84. Vector Database Selection Matrix¶

Requirement	Possible Direction
Local experimentation	FAISS / Chroma
PostgreSQL-heavy application	pgvector
Dedicated vector service	Qdrant / Milvus / Weaviate / Pinecone
Existing search platform	Elasticsearch / OpenSearch
Managed cloud architecture	Cloud-managed vector service
Highly custom retrieval	FAISS / custom index
Enterprise metadata filtering	Evaluate filtering capabilities carefully

This is a starting point rather than a universal recommendation.

85. Vector Database Decision Framework¶

flowchart TD
    A["Start"] --> B{"Existing Database?"}

    B -->|PostgreSQL| C["Evaluate pgvector"]
    B -->|Search Platform| D["Evaluate Search Engine Vectors"]
    B -->|No| E{"Scale Requirements"}

    E -->|Small / Prototype| F["FAISS / Chroma"]
    E -->|Production Scale| G["Evaluate Dedicated Vector DB"]

    C --> H["Benchmark"]
    D --> H
    F --> H
    G --> H

    H --> I["Security + Cost + Performance"]
    I --> J["Production Selection"]

86. Vector Database vs Search Engine¶

Modern search platforms increasingly support vector search.

This creates architectural choices between:

Dedicated Vector Database

and:

Search Engine with Vector Capabilities

A search engine may be attractive when the application already needs:

Keyword Search
Faceting
Filtering
Full-Text Search
Vector Search

87. Vector Database vs Object Storage¶

Object storage is excellent for:

Raw Documents
PDFs
Images
Source Files
Backups

but it is not designed to perform low-latency vector similarity search.

A common architecture is:

Object Storage
      ↓
Raw Documents

Vector Database
      ↓
Embeddings + Retrieval Metadata

88. Source of Truth¶

A production RAG architecture should clearly define:

Source of Truth

For example:

SharePoint
      ↓
Authoritative Documents

Vector Database
      ↓
Derived Retrieval Representation

This distinction simplifies:

Governance
Deletion
Updates
Auditing
Recovery

89. Data Lineage¶

A retrieved vector should be traceable:

Vector
 ↓
Chunk
 ↓
Document Version
 ↓
Source

Example:

{
  "vector_id": "policy-001-v3-chunk-07",
  "document_id": "policy-001",
  "version": "v3",
  "source": "sharepoint",
  "page": 42
}

90. Citation Architecture¶

Metadata allows the RAG system to produce:

Answer

Sources:
Employee Handbook — Page 42
Annual Leave Policy

This improves:

Trust
Auditability
Debugging
Enterprise Adoption

91. Vector Database and RAG¶

The vector database is one component of the larger RAG architecture.

Document Processing
        ↓
Chunking
        ↓
Embedding
        ↓
Vector Database
        ↓
Retrieval
        ↓
Context Assembly
        ↓
LLM

Do not confuse:

Vector Database

with:

Complete RAG System

92. RAG Pipeline with Vector Database¶

flowchart LR
    A["Documents"] --> B["Processing"]
    B --> C["Chunks"]
    C --> D["Embeddings"]
    D --> E["Vector Database"]

    F["Query"] --> G["Query Embedding"]
    G --> E

    E --> H["Retrieved Chunks"]
    H --> I["Prompt Context"]
    I --> J["LLM"]
    J --> K["Answer"]

93. Key Takeaways¶

A vector database provides storage and retrieval infrastructure for embeddings.
Embedding models create vector representations.
Vector databases make those representations searchable.
Vector records commonly contain:
ID
Vector
Content
Metadata
Vector indexes accelerate similarity search.
Exact search compares against every vector.
ANN search reduces the search space for better performance.
HNSW is a popular graph-based ANN approach.
IVF groups vectors into clusters for efficient candidate search.
Product Quantization can reduce memory requirements through vector compression.
Similarity metrics include:
Cosine similarity
Dot product
Euclidean distance
The correct metric depends on the embedding model and vector configuration.
Metadata filtering is essential for enterprise retrieval.
Multi-tenant systems require strict tenant isolation.
Vector databases should support document lifecycle operations.
Upsert simplifies incremental indexing.
Deleted source documents should result in vector deletion.
Embedding model versions should be tracked.
Embedding dimension must match the vector index configuration.
Major embedding-model migrations may require rebuilding the index.
Vector databases should be benchmarked using representative workloads.
Monitor P50, P95, and P99 search latency.
Monitor indexing throughput and storage.
Retrieval quality should be evaluated alongside infrastructure performance.
Vector databases are usually derived data stores.
Source documents remain the authoritative source.
A vector store should be recoverable through backups or deterministic re-indexing.
Hybrid search can combine semantic and keyword retrieval.
Similarity scores should not automatically be treated as answer confidence.
Frameworks such as LangChain and LlamaIndex can simplify implementation.
Enterprise applications should keep vector-store access behind a capability interface.
Ports & Adapters architecture makes vector database migration easier.
There is no universally best vector database.
Vector database selection should consider:
Scale
Latency
Recall
Filtering
Security
Availability
Cost
Operations
Vendor lock-in

The central principle is:

A vector database is the retrieval infrastructure that turns embeddings into a searchable knowledge layer.

Part IV — Prompt Engineering & RAG Fundamentals¶

Previous Chapter: 12. Document Chunking Strategies

Current Chapter: 13 — Vector Database Fundamentals

Next Chapter: 14. Similarity Search Techniques

Part IV Chapters¶

References¶

FAISS documentation
Chroma documentation
Qdrant documentation
pgvector documentation
Milvus documentation
Weaviate documentation
Pinecone documentation
Elasticsearch documentation
OpenSearch documentation
LangChain documentation
LlamaIndex documentation
Hugging Face documentation
Vector database and ANN indexing documentation
Enterprise search and retrieval architecture documentation

Enterprise AI Engineering Handbook
Building Production-Grade Enterprise AI Systems — One Chapter at a Time.