Software 3.0: Programming in English
Summary of the video “Andrej Karpathy: Software Is Changing (Again)” by Y Combinator.
Andrej Karpathy explains how LLMs represent a fundamental shift in software—the third major paradigm in 70 years. Software 1.0 is explicit code; Software 2.0 is neural network weights; Software 3.0 is English prompts. LLMs function like 1960s-era operating systems available via time-sharing, enabling partial autonomy apps where humans verify AI-generated work. The key opportunities lie in building human-AI collaboration tools with fast verification loops, adapting infrastructure for agent access, and recognizing that everyone can now program because everyone speaks English.
The Three Eras of Software
Software 1.0: Explicit Code
Software 1.0 is traditional computer code written directly by programmers—instructions that explicitly tell the computer what to do. This has been the dominant paradigm for approximately 70 years.
Software 2.0: Neural Network Weights
Software 2.0 replaces explicit code with neural network weights. Instead of writing instructions, developers tune datasets and run optimizers to generate the parameters of a neural network. The weights themselves become the 'program.' Hugging Face is the GitHub equivalent for Software 2.0.
Software 3.0: English Prompts
Software 3.0 uses large language models programmed through natural language prompts. Unlike fixed-function neural networks, LLMs are programmable—your English-language instructions become the program. This is unprecedented because the programming language is human language, not code.
Fundamental Shift in 70 Years
Software has remained fundamentally unchanged for roughly 70 years, but has changed twice rapidly in the last few years. This represents an extraordinary moment to enter the industry, as massive amounts of existing code must be rewritten across all three paradigms.
LLMs as Operating Systems
LLMs Have Utility Properties
LLMs function like utilities: companies invest capex to train them, then distribute intelligence via APIs with metered access (pay per token). Users demand low latency, high uptime, and consistent quality—all hallmarks of utility services like electricity.
LLMs Have Fab-Like Properties
LLM development requires massive capital investment and proprietary research, similar to semiconductor fabrication plants (fabs). However, because LLMs are software rather than physical infrastructure, they are less defensible and more malleable than traditional fabs.
LLMs Are Operating Systems
LLMs most closely resemble 1960s-era operating systems: the LLM is the CPU, context windows are memory, and the system orchestrates compute and memory for problem-solving. The ecosystem mirrors early computing with closed-source providers (OpenAI, Anthropic, Google) and open-source alternatives (Llama ecosystem, analogous to Linux).
1960s Computing Parallels
LLM compute is expensive, forcing centralization in the cloud where users act as time-sharing clients. This mirrors 1960s mainframe computing. Personal computing hasn't happened yet because it's not economical, though Mac minis show early promise for local inference.
Inverted Technology Diffusion
Unlike electricity, cryptography, or flight—where governments and corporations adopted first—LLMs diffused to consumers first. Early uses are consumer-focused (how to boil an egg) rather than military or governmental. This is unprecedented and suggests different application priorities.
LLM Psychology and Limitations
LLMs as Stochastic Simulations of People
LLMs are auto-regressive transformers trained on human text, creating emergent psychology that mimics human behavior. They are 'people spirits'—simulations of human cognition, not deterministic algorithms.
Superhuman Memory but Jagged Intelligence
LLMs have encyclopedic knowledge exceeding any individual human (like the autistic savant in Rain Man) but display jagged intelligence: superhuman in some domains, making obvious errors in others (e.g., claiming 9.11 > 9.9 or two Rs in 'strawberry').
Anterograde Amnesia and Fixed Weights
LLMs suffer from anterograde amnesia: they don't learn or consolidate knowledge over time like humans do. Context windows are working memory; weights are fixed. They resemble the protagonists in Memento and 50 First Dates—unable to build expertise through experience.
Security and Gullibility Risks
LLMs are susceptible to prompt injection, data leakage, and manipulation. They are gullible and lack robust internal models of what they know and don't know, creating security vulnerabilities.
Partial Autonomy Apps: The Winning Pattern
Cursor: Partial Autonomy in Code
Cursor exemplifies the ideal LLM app: it preserves traditional manual interfaces while adding LLM integration for larger chunks of work. Users can adjust autonomy via tap completion, command-K (edit chunk), command-L (edit file), or command-I (full repo autonomy).
Perplexity: Partial Autonomy in Search
Perplexity packages information, orchestrates multiple LLMs, provides GUI for source inspection, and offers an autonomy slider: quick search, research, or deep research (10+ minutes). Users control the depth of autonomous work.
Key Properties of Successful LLM Apps
Effective LLM apps share: (1) context management by the LLM, (2) orchestration of multiple LLM calls, (3) application-specific GUI for human auditing, and (4) an autonomy slider allowing users to control how much work the AI does.
Generation-Verification Loop
Humans and AIs cooperate: AI generates, humans verify. The goal is to make this loop as fast as possible. Speed comes from (1) fast verification via GUI and visual representations, and (2) keeping the AI on the leash with concrete, specific prompts to reduce verification failures.
GUI Accelerates Verification
Visual interfaces (diffs in red/green, command-Y/N acceptance) are far faster than text-based verification. GUIs leverage human visual processing and reduce cognitive load, enabling faster human auditing of AI-generated work.
Keep the AI on the Leash
Overreactive agents producing massive diffs (10,000+ lines) overwhelm human verification capacity. Effective workflows use small, incremental chunks, concrete prompts, and careful verification to prevent bugs and security issues.
Tesla Autopilot: Partial Autonomy at Scale
Tesla's autopilot exemplifies partial autonomy: the GUI shows what the neural network sees, users have an autonomy slider (tap completion to full autonomy), and humans remain in the loop. Over five years, capabilities migrated from Software 1.0 (C++ code) to Software 2.0 (neural networks).
The Decade of Agents, Not the Year
Self-driving cars have been 'imminent' for 12 years (since 2013). Autonomy is hard; it requires careful human-in-the-loop design. Claims like '2025 is the year of agents' are premature. Expect 'the decade of agents' instead.
Iron Man Suit vs. Iron Man Robot
Build augmentations (Iron Man suits) rather than fully autonomous agents (Iron Man robots). Augmentations amplify human capability with an autonomy slider; robots are flashy but premature. The goal is to slide the autonomy slider rightward over time as systems improve.
Everyone Is Now a Programmer
Natural Language Removes the Barrier
Programming in English removes the 5–10 year learning curve required for traditional software development. Everyone speaks natural language, so everyone can now program. This is unprecedented and democratizes software creation.
Vibe Coding: Programming by Feeling
Vibe coding is building custom software by describing what you want in natural language without formal training. It's a gateway drug to software development, enabling kids and non-programmers to create working apps in hours.
Karpathy's Vibe Coding Experiments
Karpathy built an iOS app in Swift (which he doesn't know) in one day and a menu-image-generation app (menu.app) in hours. The coding was easy; the hard part was DevOps (authentication, payments, deployment)—clicking through browser UIs following instructions.
DevOps Is the New Bottleneck
For vibe-coded apps, writing the code is fast (hours), but integrating services (Google login, payments, deployment) is slow (days). This is because DevOps interfaces are designed for humans clicking through UIs, not for LLM agents.
Building Infrastructure for Agents
Agents as a New Consumer Category
Agents are a new category of digital consumer: humanlike but not human. They need to interact with software infrastructure. Building for agents requires rethinking how we expose information and capabilities.
LLM.txt: Robots.txt for Language Models
Just as robots.txt advises web crawlers, lm.txt can advise LLMs about a domain in markdown format. This is far more efficient than forcing LLMs to parse HTML, reducing errors and improving agent usability.
Documentation for Machines, Not Just Humans
Traditional docs use lists, bold, and pictures—designed for human reading. LLM-friendly docs use markdown, which LLMs parse easily. Vercel and Stripe are early movers, offering docs in markdown format.
Replace 'Click' with Curl Commands
Docs that say 'click this button' are useless to LLM agents. Vercel replaces every 'click' instruction with an equivalent curl command that agents can execute. This makes docs actionable for agents.
Model Context Protocol (MCP)
Anthropic's Model Context Protocol is a standard way for agents to interact with software systems. It's a protocol for speaking directly to agents as a new consumer of digital information.
URL-Based Tools for LLM Ingestion
Tools like GitIngest (change GitHub URL to get-ingest) concatenate repo files into LLM-friendly text. DeepWiki goes further, analyzing repos and generating documentation specifically for LLM consumption. These tools make existing infrastructure accessible to agents.
Meeting Agents Halfway
Even though LLMs will eventually click and interact with GUIs, it's worth making infrastructure agent-friendly now. This is cheaper and more reliable than forcing agents to parse human-designed interfaces. However, a long tail of legacy software won't adapt.
The Road Ahead
Massive Code Rewriting Opportunity
Software has fundamentally changed twice in recent years. Enormous amounts of code must be rewritten across all three paradigms (1.0, 2.0, 3.0). This creates a huge opportunity for new entrants to the industry.
Fluency Across All Three Paradigms
New software engineers must be fluent in Software 1.0 (explicit code), 2.0 (neural networks), and 3.0 (LLM prompts). Each has pros and cons; the choice depends on the problem. Developers will fluidly transition between paradigms.
The Autonomy Slider Over a Decade
Over the next decade, the autonomy slider will move from left (human-controlled) to right (autonomous). The Iron Man suit will gradually become more autonomous, but this will happen carefully with humans in the loop.
Notable quotes
Software is changing again. Fundamentally, software has not changed much on such a fundamental level for 70 years. — Andrej Karpathy
Remarkably we're now programming computers in English. — Andrej Karpathy
It's less Iron Man robots and more Iron Man suits that you want to build. — Andrej Karpathy
Action items
- Learn to work fluently across Software 1.0 (explicit code), 2.0 (neural networks), and 3.0 (LLM prompts); understand the trade-offs of each paradigm.
- Build LLM apps with partial autonomy: preserve manual interfaces, add LLM integration, and include an autonomy slider so users control how much work the AI does.
- Optimize the generation-verification loop: use GUIs and visual representations to speed up human auditing, and use concrete, specific prompts to reduce AI errors.
- Adapt your infrastructure for agents: provide documentation in markdown, replace 'click' instructions with API commands, and consider adding lm.txt or Model Context Protocol support.
- Experiment with vibe coding to understand the new programming paradigm, but plan for DevOps and infrastructure work to be the bottleneck, not code generation.
- Think about how to slide the autonomy slider in your product over time; design for eventual automation even if you start with augmentation.