Agentic AI Masterclass: LangChain, LangGraph, RAG & LLM Gateways

Complete Course Roadmap

The course covers six major topics: generative AI and agentic AI with LangChain, LangGraph crash course for building agentic applications, traditional RAG implementation, agentic RAG, vectorless RAG with comparisons, deep research agents with practical implementation, AI security (guardrails and LLM evaluation), and LLM gateways with 30-40 minutes of implementation.

Course structure spanning 10.5 hours of content

Environment Setup with UV Package Manager

UV is a fast Python package manager written in Rust. Initialize a project with 'uv init', create a virtual environment with 'uv venv', activate it, then install dependencies from requirements.txt using 'uv add -r requirements.txt'. This replaces traditional pip workflows with faster, more efficient dependency management.

UV package manager setup workflow

Key Libraries & API Keys Required

Install langchain, langchain-community, langchain-openai, langchain-groq, python-dotenv, and langchain-google-genai. Create .env file with three API keys: OpenAI API key, Google API key, and Groq API key. These enable integration with multiple LLM providers for comparison and fallback capabilities.

Core libraries for agentic AI development

What is an Agent

An agent is an autonomous system that makes decisions about which tools to use based on input. Unlike simple LLMs that only generate text, agents can reason about whether they need external information, call appropriate tools to fetch context, and generate informed responses. This enables handling of real-time data and tasks beyond the LLM's training cutoff.

Agent decision-making workflow

Creating Agents with LangChain v1

Use 'from langchain.agents import create_agent' to instantiate agents. Provide the model name (e.g., 'gpt-4-mini'), a list of tools, and optional system prompts. Tools are functions decorated with @tool that the agent can invoke. The agent automatically determines when and which tools to call based on the input and tool descriptions.

Tool Definition & Integration

Tools are created by decorating functions with @tool from langchain.tools. Each tool requires a docstring describing its functionality—this docstring is critical as the LLM uses it to understand when to invoke the tool. Tools can be simple functions returning strings or complex functions making API calls. Bind tools to models using model.bind_tools() or pass them to create_agent().

Tool creation and integration process

Multi-Provider Model Integration

LangChain supports seamless integration with OpenAI, Google Gemini, Groq, and other providers. Use init_chat_model() for a unified interface, or provider-specific classes like ChatOpenAI, ChatGoogleGenerativeAI, ChatGroq. Simply change the model name string to switch providers without code changes. All models support the same invoke() interface.

Streaming vs Batch Processing

Streaming outputs tokens as they are generated using model.stream(), improving user experience for long responses by showing progressive output. Batch processing sends multiple independent requests in parallel using model.batch(), improving throughput and reducing cost. Use max_concurrency parameter to control parallel request limits.

Streaming improves perceived latency

Cost Tracking & Token Usage

Models return usage metadata including input_tokens, output_tokens, and total_tokens. This allows tracking API costs per request. For batch operations, sum token counts across all requests. Understanding token usage is critical for budgeting and optimizing prompt design.

Four Message Types in LangChain

SystemMessage provides instructions on how the LLM should behave. HumanMessage represents user input. AIMessage is the LLM's response. ToolMessage contains output from tool execution. These can be combined in a list to maintain conversation history and context, enabling multi-turn interactions.

Message types in conversation flow

Detailed System Prompts Improve Output Quality

Generic system messages like 'you are a helpful assistant' produce generic responses. Detailed system prompts specifying domain expertise, output format, constraints, and reasoning style significantly improve response quality. For example, 'You are a senior Python developer with expertise in web frameworks. Always provide code examples and explain your reasoning' produces more targeted, practical responses.

Structured Output with Pydantic Models

Use Pydantic BaseModel to define output schemas with field validation. Decorate fields with Field() to add descriptions and constraints. Bind schema to model using model.with_structured_output(). The LLM then generates responses matching the schema exactly, enabling reliable downstream processing. Pydantic validates types automatically.

Structured output workflow with Pydantic

Field Validation & Type Safety

Pydantic enforces type validation—if a field expects int but receives string, validation fails. Field descriptions guide the LLM on what to generate. This ensures output is always in the expected format, eliminating parsing errors and enabling reliable automation.

RAG Architecture & Components

RAG combines retrieval and generation: documents are loaded, split into chunks, embedded into vectors, stored in a vector database, and retrieved based on query similarity. Retrieved documents are passed as context to the LLM, which generates answers grounded in the documents. This solves the problem of LLMs having outdated training data.

Complete RAG pipeline

Traditional RAG vs Agentic RAG vs Vectorless RAG

Traditional RAG retrieves documents once and passes them to the LLM. Agentic RAG uses agents to iteratively refine queries and retrieve multiple times. Vectorless RAG uses keyword matching or other non-embedding methods. Each has tradeoffs in accuracy, cost, and latency.

Creating Test Datasets for RAG Evaluation

Create question-answer pairs representing expected behavior. Store in LangSmith as datasets. These serve as ground truth for evaluating RAG performance. Each example should have a question and reference answer.

LLM as Judge for Evaluation

Use a powerful LLM to evaluate outputs from another LLM by comparing against reference answers or criteria. Define evaluation metrics as functions that take input, output, and reference output, then return a score. This is more scalable than manual evaluation and captures nuanced quality aspects.

LLM-as-judge evaluation workflow

Four Key RAG Evaluation Metrics

Correctness: does answer match ground truth? Relevance: does answer address the question? Groundedness: is answer supported by retrieved documents? Retrieval Relevance: are retrieved documents relevant to query? These four metrics comprehensively assess RAG quality.

Four dimensions of RAG quality

Running Evaluations in LangSmith

Use client.evaluate() to run metrics against a dataset. Specify target function, dataset, and evaluators. LangSmith runs all evaluations and displays results with per-example scores and aggregated metrics. This enables comparing different models or configurations.

What is an LLM Gateway

An LLM gateway is a smart middleware between applications and LLM providers. It provides unified API for multiple providers, automatic fallbacks if one provider fails, intelligent routing based on task type, caching for repeated queries, cost tracking, rate limiting, and security guardrails. This eliminates direct API integration complexity.

LLM gateway request flow

Core LLM Gateway Capabilities

Unified API: one function call across 100+ providers. Automatic fallbacks: switch providers if one fails. Smart routing: send different tasks to optimal providers. Load balancing: distribute requests across multiple API keys. Caching: reuse responses for identical queries. Observability: log all calls with cost and latency. Guardrails: block sensitive data before reaching LLM. Evals: integrate evaluation frameworks.

Caching Dramatically Reduces Costs

When identical queries are cached, subsequent requests return cached responses instantly without calling the LLM. This reduces latency from ~1.5 seconds to ~0.002 seconds (700x faster) and eliminates API costs entirely for cached queries. Caching can reduce overall costs by 40-60% for typical applications.

Caching impact on latency and cost

Smart Routing by Task Type

Define routing rules mapping task types to optimal models. For coding tasks, route to GPT-4. For summarization, route to GPT-4-mini (cheaper). For speed, route to Groq Llama. The gateway classifies incoming queries and automatically routes to the best model, optimizing cost and performance.

Task-based routing strategy

Guardrails: Blocking Sensitive Data

Define regex patterns for sensitive data (emails, phone numbers, SSNs, credit cards, PII). Before sending prompts to LLM, scan for patterns and redact matching data. This prevents sensitive information from reaching external LLM providers, maintaining privacy and compliance.

PII guardrail workflow

Automatic Fallbacks Prevent Outages

Configure primary and backup LLM providers. If primary fails, gateway automatically tries backup. This prevents application outages when a provider has issues. For example, if OpenAI is down, automatically use Groq or Claude. No code changes needed.

Fallback provider hierarchy

Real-World Outage: November 2023 OpenAI Incident

OpenAI experienced a 4-hour outage on November 8, 2023. Applications hardcoded to use OpenAI (like Cursor, Notion AI) went completely offline. With an LLM gateway and fallbacks, these applications would have automatically switched to alternative providers and remained operational.

4 hours

OpenAI outage duration (Nov 8, 2023)

Real production incident demonstrating need for fallbacks

LightLLM: Open-Source LLM Gateway

LightLLM is an open-source Python library providing unified API for 100+ LLM providers. Use 'from litellm import completion' to call any model by changing the model name string. Supports automatic fallbacks, caching, cost tracking, and guardrails with minimal code.

Unified Completion API Across Providers

Single completion() function works with all providers. Change model name from 'gpt-4-mini' to 'groq/llama-3.3-70b-versatile' to 'claude-opus' without changing code. Each provider uses its own API key from environment variables.

Cost Tracking with LightLLM

Use completion_cost() to get cost of each API call. Returns input tokens, output tokens, and total cost in dollars. Aggregate across requests to track total spending by model, user, or project.

Router for Task-Based Model Selection

Define routing rules mapping task types to model lists with fallbacks. Router automatically classifies queries and selects appropriate model. Supports strategies like 'simple_shuffle' (round-robin), 'least_busy' (load balancing), and 'latency_based' (fastest provider).