05 — LlamaIndex Agents and Tools¶
A RAG system finds information. An agent can decide which capability to use next.
Overview¶
A conventional RAG application follows a known path: retrieve relevant content and generate an answer. An agent adds a decision loop. It can choose among tools, inspect their results and decide whether another action is needed.
flowchart TD
A[User Request] --> B[Agent / LLM]
B --> C{Choose capability}
C --> D[Knowledge Query]
C --> E[Enterprise API]
C --> F[Python Function]
D --> G[Tool Result]
E --> G
F --> G
G --> B
B --> H[Final Response] 1. What is a tool?¶
A tool is a capability exposed through a defined interface. It typically has a name, description, input schema, execution function and result.
Examples: search_policy(question), get_customer(customer_id), calculate_invoice_total(items) and create_support_ticket(summary).
The model proposes a structured tool call. The application validates and executes it. The model does not receive unrestricted access to application code or databases.
2. Function tools¶
A Python function can represent a narrow application capability. Keep its inputs explicit and validate them before sensitive operations.
def calculate_tax(amount: float, rate: float) -> float:
if amount < 0 or not 0 <= rate <= 1:
raise ValueError("Invalid amount or tax rate")
return round(amount * rate, 2)
LlamaIndex provides wrappers such as FunctionTool to expose functions to supported agent interfaces. A typical wrapper looks like this:
from llama_index.core.tools import FunctionTool
def lookup_policy(policy_id: str) -> str:
# The service performs its own authorization check.
return policy_service.lookup(policy_id)
policy_tool = FunctionTool.from_defaults(
fn=lookup_policy,
name="lookup_policy",
description="Find a policy using its exact policy identifier.",
)
This is the tool definition, not the authorization boundary. The service still validates the caller and decides whether the requested record can be returned. Exact agent and tool APIs vary by release and integration; pin the version used by the application and follow its matching API reference.
A good tool description should answer: what does it do, when should the agent use it, and what does it return?
3. Query engines as tools¶
A retrieval capability can be exposed to an agent so it can consult enterprise knowledge when needed.
Agent
├── Policy search tool → RAG query engine
├── Customer lookup tool → Customer API
└── Calculator tool → Deterministic function
Use a query-engine tool when the agent must decide whether a knowledge question requires retrieval. If every request is a policy question, a direct RAG pipeline may be simpler and more predictable.
4. The agent loop¶
- Receive the request and available tool descriptions.
- Select a tool or decide to answer directly.
- Produce structured arguments.
- Let the application validate and execute the call.
- Return the tool result to the agent.
- Continue until completion or an execution limit is reached.
sequenceDiagram
participant U as User
participant A as Agent
participant X as Application
participant T as Tool / Service
U->>A: Request
A->>X: Proposed tool call
X->>X: Validate identity, scope and arguments
X->>T: Execute allowed operation
T-->>X: Result
X-->>A: Tool result
A-->>U: Final response The application—not the model—owns execution authority.
5. Read tools versus write tools¶
| Tool type | Example | Control |
|---|---|---|
| Read-only | Search a policy, fetch account summary | Enforce user and tenant scope |
| Reversible write | Create a draft ticket | Validate and record operation |
| High-impact write | Transfer funds, approve a claim, change access | Strong authorization and human approval |
A model choosing a tool is not proof that the user is authorized. Check permissions at execution time, not only when the agent starts.
6. Design bounded tools¶
Prefer narrow tools over broad tools such as run_sql(query) or call_any_api(url, body). Narrow contracts reduce ambiguity and make authorization easier.
def get_invoice_status(invoice_id: str, tenant_id: str):
# Verify invoice ownership in the application service.
return invoice_service.get_status(
invoice_id=invoice_id,
tenant_id=tenant_id,
)
In real applications, derive tenant_id from trusted authentication context rather than trusting a value generated by the model.
7. Control the operating envelope¶
Set explicit limits for maximum tool calls and execution time, allowed tools per user/task, input validation and output-size limits, rate limits and cost budgets, human approval for sensitive actions, and audit records for tool name, caller and outcome.
Treat tool output as untrusted data. A retrieved document or API response may contain malicious instructions; it must not override system policy or expand the agent's permissions.
8. When an agent is the wrong abstraction¶
Use a direct function or workflow when the sequence is known in advance.
| Requirement | Better starting point |
|---|---|
| Fixed sequence of steps | Workflow (Chapter 06) |
| Search and answer from documents | RAG pipeline (Chapter 04) |
| Choose among tools dynamically | Agent |
| Deterministic calculation or update | Application service / function |
Key takeaways¶
- Tools expose capabilities; agents decide when to use them.
- Tool calls are proposals. The application validates and executes them.
- Keep tools narrow, typed, authorized and observable.
- Use agents only when dynamic selection adds value over a fixed workflow.
Further reading¶
Chapter navigation¶
⬅️ Previous: 04—RAG Pipelines
📚 Part: AI Engineering Frameworks & Tooling
➡️ Next: 06—Workflows¶
Enterprise AI Engineering Handbook
Building Production-Grade Enterprise AI Systems — One Chapter at a Time.