Build Agentic RAG in Pure Python

Learn to build an agentic RAG system from scratch using three core tools (list files, grep search, read files) in pure Python. Agentic RAG outperforms semantic RAG through iterative feedback loops where an LLM can self-correct and search multiple times. The video walks through basic tool definitions, debugging with streaming steps, structured output, and production best practices including Ripgrep integration and error handling.

Agentic RAG vs Semantic RAG

Why Agentic RAG Outperforms Semantic RAG

Semantic RAG makes a single LLM API call after gathering all information in a linear process, while agentic RAG uses a feedback loop where the LLM can iteratively call search and read tools, self-correct, and find the right information even if initial searches miss the target.

Trade-offs: When to Use Each Approach

Semantic RAG is better for low-latency, cost-optimized scenarios where speed and budget are critical. Agentic RAG is preferable when you have time and budget, as it generally outperforms semantic RAG through its iterative intelligence loop.

Core Tools: The Three Primitives

Tool 1: List Files

Uses Python's pathlib and glob to find all markdown files in a directory and return relative paths. This allows the agent to discover available knowledge sources without exposing full system paths, saving tokens.

Tool 2: Grep Search

Uses regex patterns to search for specific terms across all files. The agent defines search parameters (e.g., 'connection pool'), and grep returns matching lines with file names and line numbers, enabling precise information retrieval.

Tool 3: Read Files

Safely reads the content of a specific file after verifying it is within the notes directory (security check). Returns the file text, enabling the agent to extract detailed information once it has identified relevant documents.

Building the Agent Loop

Basic Agent Setup with Pydantic AI

Define an agent with access to the three tools (list files, grep, read files) and a language model (e.g., GPT-3.5). When given a question, the agent autonomously decides which tools to call and in what order to find the answer.

Observing Agent Decisions with Streaming

Use the agent.iter method to intercept and display each tool call the LLM makes, including the parameters it chooses. This reveals what search terms the model is using and what results it receives, critical for debugging and optimization.

Tool Definitions Drive Agent Behavior

The docstrings and parameter descriptions in tool definitions are passed to the LLM, guiding what the model searches for and how it uses each tool. Clear, domain-specific documentation helps the agent make smarter decisions.

Structured Output and Robustness

Structured Output with Citations

Define a Pydantic model specifying the output format: a plain-English answer plus citations (file name, quote, line number). This ensures the agent returns machine-readable, verifiable results that downstream systems can process reliably.

Production Best Practices

Replace Grep with Ripgrep

Ripgrep is a Rust-based search tool used by modern coding agents (Cursor, Cloud Code). It is faster than pure Python grep, ignores hidden files and gitignore entries automatically, and is the industry standard for production agentic systems.

Safety Limits and Error Handling

Implement agent request limits, max line read limits, and path containment checks to prevent the agent from getting stuck in loops or reading enormous files that blow up the context window. Return human-readable errors instead of raising exceptions so the model can self-correct.

Logging for Observability

Add logging throughout the agent execution to track what tools are called, what parameters are used, and what results are returned. This enables debugging and monitoring in production environments.

Deployment Flexibility

The same agentic RAG principles work on VPS, container apps, and serverless functions. Adjust the underlying data source (markdown files, PostgreSQL, API) and file access method to match your deployment environment without changing the core agent logic.

Implementation Workflow

Step-by-Step Build Process

Start with simple tool definitions to understand the primitives, then add a basic agent loop, then add streaming for visibility, then add structured output, and finally apply production best practices. Each step builds on the previous one.

Example Query Performance

A typical query like 'Why does our nightly deploy job run at this time?' takes 10-15 seconds with GPT-3.5, makes 5 tool calls, and returns an answer with citations showing exactly which files and lines support the response.

Notable quotes

Agentic RAG will generally outperform semantic RAG just because of the feedback loop that the agent can use. — Dave Ebbelaar
The docstring in here, this is all information that the large language model will use. — Dave Ebbelaar
With the return error, it will simply return a human readable error message that the model can then intercept and course correct on. — Dave Ebbelaar

Action items

  • Clone the GitHub repository from the description and navigate to the knowledge/agentic_rag folder to access all code examples.
  • Start with build_tools.py to understand the three core primitives (list files, grep, read files) before moving to agent implementations.
  • Use streaming_steps.py with debug=True to observe what search parameters the LLM chooses and what results it receives.
  • Install Ripgrep on your system (brew install ripgrep on macOS) before deploying to production.
  • Implement safety limits (agent request limit, max line read) and path containment checks in your production setup.
  • Replace the markdown file system with your own data source (database, API, etc.) by adjusting the tool functions while keeping the agent loop logic the same.
  • Add logging to track tool calls and parameters for debugging and monitoring in production.
Glimpse.
Dave Ebbelaar
27 min video
3 min read
Build Agentic RAG in Pure Python
You just saved 24 min.
The big takeaway
Learn to build an agentic RAG system from scratch using three core tools (list files, grep search, read files) in pure Python. Agentic RAG outperforms semantic RAG through iterative feedback loops where an LLM can self-correct and search multiple times. The video walks through basic tool definitions, debugging with streaming steps, structured output, and production best practices including Ripgrep integration and error handling.
Agentic RAG vs Semantic RAG
Why Agentic RAG Outperforms Semantic RAG
Semantic RAG makes a single LLM API call after gathering all information in a linear process, while agentic RAG uses a feedback loop where the LLM can iteratively call search and read tools, self-correct, and find the right information even if initial searches miss the target.
Semantic RAG
Single LLM call, linear process
Agentic RAG
Multiple LLM calls in feedback loop
Agentic RAG enables iterative refinement vs one-shot semantic search
Trade-offs: When to Use Each Approach
Semantic RAG is better for low-latency, cost-optimized scenarios where speed and budget are critical. Agentic RAG is preferable when you have time and budget, as it generally outperforms semantic RAG through its iterative intelligence loop.
Core Tools: The Three Primitives
Tool 1: List Files
Uses Python's pathlib and glob to find all markdown files in a directory and return relative paths. This allows the agent to discover available knowledge sources without exposing full system paths, saving tokens.
1
Use pathlib to set notes directory
2
Apply glob pattern to find .md files
3
Convert to relative paths with .relative_to()
4
Return short file names to agent
List files workflow reduces token overhead
Tool 2: Grep Search
Uses regex patterns to search for specific terms across all files. The agent defines search parameters (e.g., 'connection pool'), and grep returns matching lines with file names and line numbers, enabling precise information retrieval.
1
Compile regex pattern with case-insensitive matching
2
Loop through all files in directory
3
Split each file into lines with enumerate
4
Store matches with filename, line number, and content
Grep search finds patterns across knowledge base
Tool 3: Read Files
Safely reads the content of a specific file after verifying it is within the notes directory (security check). Returns the file text, enabling the agent to extract detailed information once it has identified relevant documents.
1
Convert file path to Path object
2
Check if target is relative to notes directory
3
Raise error if file is outside allowed folder
4
Read and return file content
Read files with safety containment checks
Building the Agent Loop
Basic Agent Setup with Pydantic AI
Define an agent with access to the three tools (list files, grep, read files) and a language model (e.g., GPT-3.5). When given a question, the agent autonomously decides which tools to call and in what order to find the answer.
Observing Agent Decisions with Streaming
Use the agent.iter method to intercept and display each tool call the LLM makes, including the parameters it chooses. This reveals what search terms the model is using and what results it receives, critical for debugging and optimization.
5
Tool calls in example query
Agent made 5 tool calls to answer one question
Tool Definitions Drive Agent Behavior
The docstrings and parameter descriptions in tool definitions are passed to the LLM, guiding what the model searches for and how it uses each tool. Clear, domain-specific documentation helps the agent make smarter decisions.
Structured Output and Robustness
Structured Output with Citations
Define a Pydantic model specifying the output format: a plain-English answer plus citations (file name, quote, line number). This ensures the agent returns machine-readable, verifiable results that downstream systems can process reliably.
1
Define SearchAnswer model with answer field
2
Add Citation model with file, quote, line_number
3
Specify output_type in agent configuration
4
Agent returns structured JSON with citations
Structured output enables downstream integration
Production Best Practices
Replace Grep with Ripgrep
Ripgrep is a Rust-based search tool used by modern coding agents (Cursor, Cloud Code). It is faster than pure Python grep, ignores hidden files and gitignore entries automatically, and is the industry standard for production agentic systems.
Python Grep
1 relative speed
Ripgrep
10 relative speed
Ripgrep is significantly faster and production-ready
Safety Limits and Error Handling
Implement agent request limits, max line read limits, and path containment checks to prevent the agent from getting stuck in loops or reading enormous files that blow up the context window. Return human-readable errors instead of raising exceptions so the model can self-correct.
1
Agent request limit
Prevent infinite loops
2
Max lines per file
Avoid context overflow
3
Path containment check
Security boundary
4
Return errors, don't raise
Allow model to recover
Production safety mechanisms
Logging for Observability
Add logging throughout the agent execution to track what tools are called, what parameters are used, and what results are returned. This enables debugging and monitoring in production environments.
Deployment Flexibility
The same agentic RAG principles work on VPS, container apps, and serverless functions. Adjust the underlying data source (markdown files, PostgreSQL, API) and file access method to match your deployment environment without changing the core agent logic.
Implementation Workflow
Step-by-Step Build Process
Start with simple tool definitions to understand the primitives, then add a basic agent loop, then add streaming for visibility, then add structured output, and finally apply production best practices. Each step builds on the previous one.
Step 1
Build tools (list, grep, read)
Step 2
Basic agent setup
Step 3
Streaming steps for debugging
Step 4
Structured output with citations
Step 5
Production best practices
Build agentic RAG incrementally
Example Query Performance
A typical query like 'Why does our nightly deploy job run at this time?' takes 10-15 seconds with GPT-3.5, makes 5 tool calls, and returns an answer with citations showing exactly which files and lines support the response.
10-15s
Query latency with GPT-3.5
Typical agentic RAG query time
Worth quoting
"Agentic RAG will generally outperform semantic RAG just because of the feedback loop that the agent can use."
— Dave Ebbelaar, at [1:01]
"The docstring in here, this is all information that the large language model will use."
— Dave Ebbelaar, at [16:56]
"With the return error, it will simply return a human readable error message that the model can then intercept and course correct on."
— Dave Ebbelaar, at [24:37]
Try this
Clone the GitHub repository from the description and navigate to the knowledge/agentic_rag folder to access all code examples.
Start with build_tools.py to understand the three core primitives (list files, grep, read files) before moving to agent implementations.
Use streaming_steps.py with debug=True to observe what search parameters the LLM chooses and what results it receives.
Install Ripgrep on your system (brew install ripgrep on macOS) before deploying to production.
Implement safety limits (agent request limit, max line read) and path containment checks in your production setup.
Replace the markdown file system with your own data source (database, API, etc.) by adjusting the tool functions while keeping the agent loop logic the same.
Add logging to track tool calls and parameters for debugging and monitoring in production.
Made with Glimpse by Wozart
glimpse.wozart.com/v/81c7ivj1
Share this infographic

More like this