Part VIII — AI Engineering Frameworks & Tooling¶
Master the modern frameworks, orchestration platforms, SDKs, and development tools used to build, evaluate, deploy, and operate production-ready Enterprise AI applications.
📖 Overview¶
Modern AI applications extend far beyond calling Large Language Models through APIs. Building production-ready Enterprise AI systems requires a rich ecosystem of frameworks, orchestration platforms, SDKs, evaluation tools, and development utilities that simplify the creation, deployment, and operation of intelligent applications.
This module provides a production-focused introduction to the AI engineering ecosystem, covering the industry's leading frameworks and tooling for developing LLM-powered applications, Retrieval-Augmented Generation (RAG) pipelines, AI Agents, Agentic AI systems, and enterprise AI platforms.
Designed for software engineers, backend developers, cloud engineers, solution architects, and AI engineers, this module equips you with the practical knowledge required to select, integrate, and leverage the right frameworks for building scalable, maintainable, and production-ready AI systems.
🎯 Learning Outcomes¶
After completing this module, you will be able to:
- Understand the modern AI Engineering ecosystem
- Compare popular AI orchestration frameworks
- Build LLM applications using industry-standard SDKs
- Develop RAG applications using specialized frameworks
- Build AI Agents and Agentic AI systems using modern orchestration platforms
- Understand workflow orchestration and execution models
- Compare framework capabilities and trade-offs
- Select the appropriate framework for different enterprise use cases
- Design modular, maintainable, and scalable AI applications
- Apply production best practices when building Enterprise AI systems
🧭 Framework Ecosystem¶
The modern AI Engineering ecosystem can be viewed as multiple layers:
flowchart TD
A[Enterprise AI Applications]
A --> B[AI Application Frameworks]
A --> C[Agent & Workflow Frameworks]
A --> D[RAG & Data Frameworks]
A --> E[AI Provider SDKs]
A --> F[AI Programming & Optimization]
B --> B1[LangChain]
B --> B2[Semantic Kernel]
C --> C1[LangGraph]
C --> C2[CrewAI]
C --> C3[AutoGen]
D --> D1[LlamaIndex]
D --> D2[Haystack]
E --> E1[OpenAI SDK]
E --> E2[Anthropic SDK]
E --> E3[Google GenAI SDK]
F --> F1[DSPy]
A --> G[Framework Comparisons]The objective is not simply to learn APIs.
The objective is to understand:
Section 1 — LangChain¶
Build LLM-powered applications using LangChain's abstractions for models, prompts, tools, retrieval, agents, and application workflows.
LangChain Coverage¶
The LangChain section covers:
- LangChain architecture
- Core abstractions
- Model integration
- Prompt abstractions
- Message handling
- Tool integration
- Function calling
- Retrieval
- RAG pipelines
- Agent integration
- Memory and state
- Production patterns
- Framework limitations and trade-offs
LangChain Architecture¶
flowchart LR
A[Application] --> B[LangChain]
B --> C[Models]
B --> D[Prompts]
B --> E[Tools]
B --> F[Retrieval]
B --> G[Agents]
B --> H[Memory / State]
C --> I[LLM Providers]
D --> I
E --> J[Enterprise APIs]
F --> K[Vector Stores]
G --> C
G --> E
H --> L[State Store]Section 2 — LlamaIndex¶
Build data-centric AI applications using LlamaIndex for document ingestion, indexing, retrieval, RAG, agents, and workflows.
LlamaIndex Coverage¶
The LlamaIndex section covers:
- LlamaIndex architecture
- Data ingestion
- Document processing
- Indexing
- Retrieval
- RAG pipelines
- Agents
- Tools
- Workflows
- Production architecture
- Framework limitations and trade-offs
LlamaIndex Architecture¶
flowchart TD
A[Enterprise Data] --> B[Data Connectors]
B --> C[Document Processing]
C --> D[Nodes / Chunks]
D --> E[Indexing]
E --> F[Retrieval]
F --> G[Context]
G --> H[LLM]
H --> I[Response]
D --> J[Metadata]
J --> F
F --> K[RAG Pipeline]
K --> HSection 3 — LangGraph¶
Build stateful, controllable, and production-oriented AI Agent workflows using graph-based orchestration.
LangGraph Coverage¶
The LangGraph section covers:
- Graph-based orchestration
- Agent state
- Nodes and edges
- Routing
- Conditional execution
- Checkpointing
- Persistence
- Human-in-the-loop workflows
- Tool execution
- Stateful Agent workflows
- Production deployment patterns
- Framework limitations and trade-offs
LangGraph Architecture¶
flowchart TD
A[User Request] --> B[Agent State]
B --> C[Planner / Agent Node]
C --> D{Decision}
D -->|Tool Required| E[Tool Node]
D -->|Need More Information| F[Retrieval Node]
D -->|Complete| G[Final Response]
E --> B
F --> B
B --> H[Checkpoint]
H --> C
G --> I[User]Section 4 — Enterprise AI Frameworks¶
Explore additional frameworks used for enterprise AI applications, Agent orchestration, multi-agent systems, and AI application development.
These frameworks are planned and will be developed later.
| Framework | Planned Coverage | Status |
|---|---|---|
| Semantic Kernel | Enterprise AI orchestration, plugins, memory, Agents, and workflows | 🚧 |
| CrewAI | Role-based Agents and multi-agent workflows | 🚧 |
| AutoGen | Conversational Agents and multi-agent orchestration | 🚧 |
| Haystack | Production RAG, pipelines, Agents, and retrieval systems | 🚧 |
Planned Framework Relationship¶
flowchart LR
A[Enterprise AI Frameworks]
A --> B[Semantic Kernel]
A --> C[CrewAI]
A --> D[AutoGen]
A --> E[Haystack]
B --> F[Enterprise Orchestration]
C --> G[Role-Based Agents]
D --> H[Multi-Agent Conversations]
E --> I[RAG & Pipelines]Section 5 — AI Programming & Optimization¶
Explore frameworks that approach AI application development from a programming, optimization, and declarative perspective.
| Framework | Planned Coverage | Status |
|---|---|---|
| DSPy | Declarative AI programming, prompt optimization, and LM pipelines | 🚧 |
Planned DSPy Coverage¶
Topics will include:
- Declarative AI programming
- Signatures
- Modules
- Optimization
- Prompt optimization
- Demonstration optimization
- Evaluation-driven development
- Production considerations
Section 6 — AI Provider SDKs¶
Learn how to build AI applications directly using provider-native SDKs and APIs.
| SDK | Planned Coverage | Status |
|---|---|---|
| OpenAI SDK | Models, Responses, tools, structured outputs, Agents and APIs | 🚧 |
| Anthropic SDK | Claude APIs, tool use, messages, and application integration | 🚧 |
| Google GenAI SDK | Gemini models, multimodal AI, tools, and application integration | 🚧 |
Provider SDK Architecture¶
flowchart TD
A[Enterprise AI Application] --> B[Provider Abstraction]
B --> C[OpenAI SDK]
B --> D[Anthropic SDK]
B --> E[Google GenAI SDK]
C --> F[OpenAI Models]
D --> G[Claude Models]
E --> H[Gemini Models]
B --> I[Common Application Contract]The SDK chapters will emphasize:
- Direct provider integration
- Authentication
- Model invocation
- Structured outputs
- Tool calling
- Streaming
- Error handling
- Retries
- Rate limits
- Observability
- Production integration
- Provider-specific capabilities
Section 7 — AI Framework Comparisons¶
Compare frameworks and SDKs from an enterprise architecture and production engineering perspective.
| Chapter | Status |
|---|---|
| Framework Architecture Comparison | 🚧 |
| LangChain vs LlamaIndex vs LangGraph | 🚧 |
| AI Agent Framework Comparison | 🚧 |
| RAG Framework Comparison | 🚧 |
| Framework vs Direct SDK | 🚧 |
| AI Framework Selection Guide | 🚧 |
| Production AI Framework Architecture | 🚧 |
Framework Comparison Model¶
flowchart TD
A[Business Requirement] --> B[Architecture Requirements]
B --> C[RAG]
B --> D[Agents]
B --> E[Workflows]
B --> F[Data]
B --> G[Provider Integration]
C --> H[Framework Selection]
D --> H
E --> H
F --> H
G --> H
H --> I[LangChain]
H --> J[LlamaIndex]
H --> K[LangGraph]
H --> L[Semantic Kernel]
H --> M[CrewAI]
H --> N[AutoGen]
H --> O[Haystack]
H --> P[Direct SDK]The comparison section will focus on:
- Architecture
- Abstractions
- Extensibility
- Integration
- RAG capabilities
- Agent capabilities
- Workflow orchestration
- State management
- Observability
- Evaluation
- Production readiness
- Performance
- Operational complexity
- Vendor / framework lock-in
- Enterprise adoption considerations
🏢 Enterprise Architecture Perspective¶
The purpose of Part VIII is not simply to teach framework APIs.
The goal is to understand how frameworks fit into a production Enterprise AI architecture.
flowchart TD
A[Enterprise AI Application]
A --> B[Application Services]
B --> C[Model Layer]
B --> D[RAG Layer]
B --> E[Agent Layer]
B --> F[Tool Layer]
B --> G[Workflow Layer]
C --> H[Provider SDKs]
D --> I[LlamaIndex]
D --> J[Haystack]
E --> K[LangGraph]
E --> L[CrewAI]
E --> M[AutoGen]
F --> N[Enterprise APIs]
F --> O[Databases]
G --> P[LangChain]
G --> Q[Semantic Kernel]
B --> R[Observability]
B --> S[Security]
B --> T[Governance]Framework selection should therefore be driven by:
Business Requirements
↓
Architecture Requirements
↓
AI Capabilities
↓
Operational Requirements
↓
Framework / SDK Selection
🔄 Frameworks vs Direct SDKs¶
One of the important architectural questions covered later in this module will be:
Frameworks can provide:
- Higher-level abstractions
- Reusable components
- Workflow orchestration
- Tool integration
- Retrieval integration
- Agent abstractions
- Ecosystem integrations
Direct SDKs can provide:
- Lower abstraction overhead
- Direct access to provider capabilities
- Greater control
- Simpler dependency graphs
- Easier access to provider-specific features
The correct choice depends on the application's requirements.
🧩 Framework-Agnostic Architecture¶
Enterprise applications should avoid allowing framework-specific abstractions to leak unnecessarily throughout the entire business domain.
A conceptual architecture can be:
flowchart TD
A[Business Application] --> B[AI Application Service]
B --> C[Capability Interfaces]
C --> D[ModelProvider]
C --> E[RetrievalProvider]
C --> F[AgentProvider]
C --> G[ToolProvider]
C --> H[WorkflowProvider]
D --> I[Provider SDK Adapter]
E --> J[RAG Framework Adapter]
F --> K[Agent Framework Adapter]
G --> L[Tool Framework Adapter]
H --> M[Workflow Framework Adapter]
I --> N[External AI Providers]
J --> O[Vector Stores / Data]
K --> P[Agent Runtime]
L --> Q[Enterprise APIs]
M --> R[Workflow Runtime]This approach can help maintain:
- Portability
- Testability
- Maintainability
- Replaceability
- Clear architectural boundaries
🏗️ Framework Abstraction Strategy¶
A production-oriented AI platform can separate:
For example:
Or:
This keeps framework-specific implementation details closer to the infrastructure boundary.
📊 Framework Selection Criteria¶
There is no universally "best" AI framework.
Selection should depend on:
| Consideration | Questions |
|---|---|
| Use Case | What are we building? |
| RAG | How complex is retrieval? |
| Agents | How complex is Agent execution? |
| Workflow | Do we need deterministic orchestration? |
| State | How much state must be persisted? |
| Integration | What enterprise systems are involved? |
| Scale | What production scale is required? |
| Observability | How deeply must execution be traced? |
| Governance | What security and compliance controls are required? |
| Portability | How important is framework independence? |
| Team Skills | What does the engineering team already know? |
| Operations | How complex is the resulting platform? |
🔬 Production Engineering Focus¶
Every framework chapter in this module should go beyond basic API usage.
Each framework note should cover:
Architecture¶
Core Abstractions¶
Sample Code¶
Each note should contain practical implementation examples.
Example:
from framework import Model
model = Model(
model="example-model"
)
response = model.invoke(
"Explain enterprise AI architecture."
)
print(response)
The exact API will depend on the framework being discussed.
Mermaid Architecture¶
Framework notes should include architecture diagrams where they improve understanding.
Example:
flowchart LR
A[Application] --> B[Framework]
B --> C[Model]
B --> D[Tools]
B --> E[Retrieval]
B --> F[State]Sample / Worked Example¶
Each major concept should include a realistic example such as:
Customer Support Agent
↓
Retrieve Customer Context
↓
Call Order Service
↓
Generate Response
↓
Audit Interaction
Graph / Visual Representation¶
Where useful, framework notes should include graphs or visual representations for:
- Execution flow
- State transitions
- Retrieval pipelines
- Agent workflows
- Tool invocation
- Framework architecture
- Performance characteristics
- Component relationships
Text Flow¶
Simple concepts can use text diagrams when they communicate the architecture more clearly:
🧪 Testing & Evaluation¶
Framework chapters should also cover how the framework integrates with:
Where applicable, examples should demonstrate:
- Mock models
- Mock tools
- Deterministic tests
- Integration tests
- Evaluation datasets
- Output validation
- Failure testing
⚡ Performance & Optimization¶
Framework notes should address production performance where relevant.
Areas include:
- Latency
- Token usage
- Concurrency
- Streaming
- Batching
- Caching
- Connection reuse
- Async execution
- Retrieval performance
- Tool execution overhead
- Framework overhead
Example production flow:
flowchart LR
A[Request] --> B[Framework Runtime]
B --> C[Optimization Layer]
C --> D[Model]
C --> E[Cache]
C --> F[Tools]
D --> G[Response]
E --> G
F --> G⚠️ Common Pitfalls¶
Framework notes should identify common problems such as:
- Excessive abstraction
- Framework lock-in
- Hidden network calls
- Uncontrolled retries
- Poor error handling
- State management issues
- Difficult debugging
- Dependency complexity
- Version compatibility problems
- Production observability gaps
- Excessive framework coupling
🧭 Relationship With Previous Parts¶
Part VIII builds on the engineering concepts covered earlier in the handbook.
flowchart LR
A[Part III<br/>Foundation Models & LLMs]
--> B[Part IV<br/>Generative AI]
B --> C[Part V<br/>Production RAG Engineering]
C --> D[Part VI<br/>AI Agents]
D --> E[Part VII<br/>Agentic AI & Multi-Agent Systems]
E --> F[Part VIII<br/>AI Engineering Frameworks & Tooling]The distinction is important.
Earlier parts explain primarily:
Part VIII focuses more heavily on:
using modern AI engineering frameworks and SDKs.
📚 Part VIII Technology Map¶
flowchart TD
A[Part VIII<br/>AI Engineering Frameworks & Tooling]
A --> B[Application Frameworks]
A --> C[Agent Frameworks]
A --> D[RAG / Data Frameworks]
A --> E[AI Programming]
A --> F[Provider SDKs]
A --> G[Framework Comparisons]
B --> B1[LangChain]
B --> B2[Semantic Kernel]
C --> C1[LangGraph]
C --> C2[CrewAI]
C --> C3[AutoGen]
D --> D1[LlamaIndex]
D --> D2[Haystack]
E --> E1[DSPy]
F --> F1[OpenAI SDK]
F --> F2[Anthropic SDK]
F --> F3[Google GenAI SDK]
G --> G1[Framework Selection]
G --> G2[Framework vs SDK]
G --> G3[Production Architecture]🚦 Current Development Status¶
✅ Completed¶
LangChain¶
- LangChain Fundamentals
- LangChain Models & Prompts
- LangChain Tools & Function Calling
- LangChain Retrieval & RAG
- LangChain Memory & State
- LangChain Agents
- LangChain Production Patterns
- LangChain Limitations & Trade-offs
LlamaIndex¶
- LlamaIndex Fundamentals
- LlamaIndex Data & Document Ingestion
- LlamaIndex Indexes & Retrieval
- LlamaIndex RAG Pipelines
- LlamaIndex Agents & Tools
- LlamaIndex Workflows
- LlamaIndex Production Patterns
- LlamaIndex Limitations & Trade-offs
LangGraph¶
- LangGraph Fundamentals
- Graph-Based Agent Architecture
- LangGraph State & Checkpointing
- LangGraph Nodes, Edges & Routing
- LangGraph Human-in-the-Loop
- LangGraph Tool Execution
- LangGraph Agent Workflows
- LangGraph Memory & Persistence
- LangGraph Production Patterns
- LangGraph Limitations & Trade-offs
🚧 Planned¶
The following areas remain under development:
Enterprise AI Frameworks¶
- Semantic Kernel
- CrewAI
- AutoGen
- Haystack
AI Programming & Optimization¶
- DSPy
Provider SDKs¶
- OpenAI SDK
- Anthropic SDK
- Google GenAI SDK
Framework Comparisons¶
- Framework Architecture Comparison
- LangChain vs LlamaIndex vs LangGraph
- AI Agent Framework Comparison
- RAG Framework Comparison
- Framework vs Direct SDK
- AI Framework Selection Guide
- Production AI Framework Architecture
🏢 Enterprise AI Engineering Principle¶
Frameworks should be treated as engineering tools, not architectural dependencies that automatically define the entire system.
A strong enterprise architecture should follow:
flowchart TD
A[Business Requirements]
--> B[Architecture]
B --> C[AI Capabilities]
C --> D[Framework / SDK Selection]
D --> E[Implementation]
E --> F[Testing]
F --> G[Evaluation]
G --> H[Production Deployment]
H --> I[Observability]
I --> J[Continuous Improvement]The framework should serve the architecture.
The architecture should not be forced to serve the framework.
🚀 Start Learning¶
The framework journey begins with the completed LangChain section:
The completed learning path is:
The future learning path will continue with:
Semantic Kernel
↓
CrewAI
↓
AutoGen
↓
DSPy
↓
Haystack
↓
OpenAI SDK
↓
Anthropic SDK
↓
Google GenAI SDK
↓
Framework Comparisons
🚧 Module Status¶
Status: 🚧 Under Active Development
The roadmap for this module has been finalized, with the LangChain, LlamaIndex, and LangGraph sections completed.
The remaining frameworks, SDKs, and comparison chapters will be developed progressively.
Each chapter will include¶
- 📖 Production-focused explanations
- 🏗️ Enterprise architecture diagrams
- 🔷 Mermaid architecture and workflow diagrams
- 💻 Sample implementation code
- 🧪 Testing and evaluation examples
- 📊 Graphs / visual representations where useful
- 🔄 Text-based execution flows where appropriate
- ⚡ Best practices & optimization techniques
- ⚠️ Common pitfalls & troubleshooting guidance
- 🏢 Enterprise architecture considerations
- ❓ Interview questions
- 📝 Quick revision notes
- 📚 References & further reading
Enterprise AI Engineering Handbook
Building Production-Grade Enterprise AI Systems — One Chapter at a Time.