The AI Agent Ladder: From LLM to Production Agent
Summary of the video “Agentic AI Explained: The Complete 2026 Guide” by Aishwarya Srinivasan.
AI agents are built on a clear ladder: a language model brain that predicts tokens, context engineering that feeds it information, tools that give it hands to act, and evaluations that measure if it works. Understanding this stack explains every AI system from ChatGPT to production agents.
The Core: How Language Models Actually Work
Language Models Are Prediction Machines
A large language model's entire job is to predict the next most likely piece of text, token by token. It has read the entire internet, books, and code, learning patterns of how humans put words together. When you type 'The sky is', it knows 'blue' is far more likely than 'green'. Scale this up to billions of parameters and you get a system that can code, explain physics, and chat naturally.
Tokens, Not Words
The model doesn't process whole words—it works with tokens, which might be a word, part of a word, or punctuation. It reads your input as tokens and generates answers as tokens, one at a time. This is why ChatGPT appears to type in real time: it's literally predicting the next token, then the next, then the next.
Non-Determinism: The Weighted Dice
When predicting the next token, the model doesn't pick one answer—it produces a probability distribution (e.g., 'blue' 80%, 'clear' 10%, 'gray' 5%) and rolls a weighted dice to sample from it. This is why the same question can yield different responses. A setting called temperature controls how wild this gets: lower temperature means more predictable output, higher temperature means more creative responses.
The Transformer: The Invention That Changed Everything
Attention Is All You Need (2017)
Before the Transformer, language models read text sequentially through a keyhole, one word at a time, and forgot the beginning by the end. The Transformer introduced 'attention'—a mechanism that lets the model look at all words simultaneously and decide which ones matter to each other. This two-way relationship between all words unlocked everything modern AI can do.
Encoder vs. Decoder: Reader vs. Writer
The original Transformer had two halves. The encoder is the reader—it takes input and deeply understands it, turning words into rich internal representations. The decoder is the writer—it takes understanding and produces output one token at a time. The original design used both (like a human translator listening then speaking), but modern models like GPT use decoder-only architecture because generation and understanding happen in the same machinery.
The Shift from BERT to GPT
In 2018, BERT was the hottest model—an encoder-only system brilliant at understanding and classifying text. It learned by fill-in-the-blank games but couldn't write fluent essays. Then decoder-only generative models (GPT family) arrived, playing 'keep the story going' by predicting the next word forever. This shift from readers to writers, from understanding to generation, unlocked the chat assistants we use today.
Multimodal Models: Adding Senses
What Defines Modality
A model's modality is defined by what input it takes and what output it produces. A plain language model takes text and outputs text. Add image input and you get a Vision Language Model (VLM)—same brain, new senses. Add image output and you get image generation. Add video output and you get video generation. The fundamental prediction machine stays the same; only what flows in and out changes.
Multimodal Models: One Brain, Many Senses
A multimodal model can juggle multiple input and output types simultaneously—text and images in, text and images out. Think of it like a person who can read, see, listen, and speak from the same brain instead of needing a separate specialist for each sense. The underlying prediction mechanism is unchanged; it's just richer data flowing through.
The Invisible Plumbing: From Prompt to Context Engineering
The Restaurant Analogy: Your Message Doesn't Go Straight to the Model
When you chat with ChatGPT, your message doesn't go directly to the language model. There's a whole layer of plumbing in between. Your prompt gets transformed, wrapped, and enriched before reaching the brain—just like a restaurant order doesn't go straight to the chef; a waiter writes it down, the kitchen applies standing rules, ingredients are pulled from the pantry, and the finished plate comes back. The language model is just the chef; it's not the whole restaurant.
System Prompt vs. User Prompt
Your message is assembled with two kinds of instructions. The user prompt is what you typed—your specific request. The system prompt is a persistent set of instructions that stay with the model across the whole conversation. Think of the system prompt as a job description ('You are a helpful, caring assistant who explains things simply') and the user prompt as today's specific task ('Explain quantum mechanics'). The job description shapes how every task is handled.
Context Engineering: The New Skill
Prompt engineering has evolved into context engineering. The model only knows what's in front of it—inside its context window, which is its short-term working memory. The model has no idea who you are, what happened yesterday, or what's in your documents unless you put that information into its context. The real skill is no longer wording a clever prompt; it's engineering everything that goes into the context window to give the model exactly what it needs and nothing that distracts it.
RAG: Making Models Know Your World
RAG: Retrieval Augmented Generation
RAG stands for Retrieval Augmented Generation. A language model on its own is like a brilliant student taking a closed-book exam—limited to what it memorized during training with a knowledge cutoff date. RAG turns it into an open-book exam: before answering, the system retrieves the exact right information from your documents. Suddenly the model isn't guessing from memory; it's answering from a source in front of it.
The Three Parts of RAG
Retrieval is finding the most relevant information, usually from a vector database that searches by meaning, not just keywords. Augmentation is post-processing those retrieved documents—reranking them, trimming irrelevant parts, cleaning them up—before handing them to the model. Generation is the final step: the model takes your question plus those clean retrieved documents and generates an answer grounded in real, current, specific information.
Vector Databases and Embeddings
A vector database stores documents in a way that lets the system search by meaning, not just keywords. This works by converting text, audio, images, and PDFs into numbers called vector embeddings. Once converted, you combine the original text, vector embeddings, and metadata and store them in a vector database. This lets the system pull back chunks of text that are actually most relevant to what you meant, not just keyword matches.
Reasoning: Thinking Before Speaking
Reasoning Models Think Step-by-Step
A regular language model answers fast, blurting out the most likely next token almost like answering on instinct. A reasoning model does something different: it's trained to think before it speaks. It generates a private chain of thought, working through the problem step by step on scratch paper, before giving you the final clean answer. For hard tasks like math, logic, and complex coding, this scratch paper thinking makes a massive difference in accuracy. Models like OpenAI's O series and DeepSeek made this approach famous.
The Model-Agent Distinction: Thinking vs. Acting
Model = Brain, Agent = Brain + Body
A language model is the brain—it thinks, reasons, and processes. But a brain floating in a jar can't do anything. It can think brilliant thoughts about booking your flight but can't actually book it. An agent is a model that has been given a body: hands, legs, and tools that can go and actually do something in the world. A model thinks; an agent thinks and acts. This is the most important distinction in all of AI right now.
Tools: Giving the Brain Hands
The way you give a model hands and legs is through tools. A tool might be the ability to search the web, run code, query a database, send an email, or call another API. When the brain can decide to reach for a tool, use it, see what happened, and decide what to do next, you no longer have a model—you have an AI agent.
Agentic Design Patterns: How Agents Actually Work
ReAct: Reason + Act Loop
The most fundamental agentic pattern is ReAct (Reason + Act). The agent reasons about what to do, takes an action by calling a tool, observes the result, and then reasons again about what to do next given what it just learned. Think, act, observe, think, act, observe—exactly like a detective working a case: form a theory, check a clue, see what it reveals, update the theory, check the next clue. The agent loops through this cycle until the job is done. This single loop is the heartbeat of almost every agent being built today.
Four Major Agentic Design Patterns
Beyond ReAct, there are four major agentic design patterns. Reflection: the agent critiques and improves its own work, like a writer editing a draft. Tool use: giving the brain hands (covered above). Planning: the agent breaks a big goal into sequences of steps before executing. Multi-agent collaboration: several specialized agents work together like a team—a researcher, writer, and reviewer passing work to each other. Each pattern makes the basic brain dramatically more capable.
AI Evaluations: Measuring What Actually Works
The Uncomfortable Truth: Non-Determinism in Production
AI agents are non-deterministic. The same agent can behave differently on the same task because of the weighted dice sampling. This creates a critical problem: how do you know if your agent is actually good? How do you know if a change you made improved it or quietly broke it? This is where AI evaluations come in.
Why AI Evaluation Is Harder Than Software Testing
With regular code, the answer is either right or wrong: 2 + 2 is four, period. But agentic systems live in a world of qualitative judgment. Was that answer helpful? Was it grounded in a source or did it make something up? Was the tone appropriate? Was it safe? These are fuzzy human questions. Every agentic system is incredibly specific and niche—a medical intake agent, customer support agent, and coding agent all need different standards of what 'good' means.
Turning Fuzzy Judgment into Quantitative Metrics
The real work of AI evaluation is taking all the fuzzy qualitative things you care about and turning them into concrete quantitative metrics you can actually track. You define what good looks like, turn it into numbers, and then measure relentlessly. You can't improve something you can't measure. This is what separates a cool demo from something you can actually trust in production.
The Complete AI Ladder: From API Call to Production Agent
The Unified Ladder: Everything Stacks the Same Way
Everything in AI from a single chatbot to the most sophisticated agent on the planet is built on the same ladder. A model is the brain. Context is what you feed in. Tools are what turns thinking into doing. Evals are how you know if it's actually working. Once you see that ladder, you can never unsee it. You will understand any new AI system or product launched in about 30 seconds.
Notable quotes
The gap between making an API call to a language model and building an actual agent that can go and do work for you is enormous. — Aishwarya Srinivasan
A model thinks and an agent thinks and acts. That is the whole thing. — Aishwarya Srinivasan
You can't really improve something that you can't measure. — Aishwarya Srinivasan