Jensen Huang: NVIDIA, AI Factories & the Future of Computing

Why Extreme Co-Design is Necessary

Traditional scaling hits limits when problems exceed a single computer's capacity. To achieve 1 million times speedup across 10,000 computers requires co-designing the entire stack—GPU, CPU, memory, networking, storage, power, cooling, software, and data center architecture—because bottlenecks emerge everywhere when workloads are distributed.

Amdahl's Law: The Bottleneck Problem

If computation represents 50% of a workload and you speed it up infinitely, you only speed the total by 2x. This forces engineers to optimize across all components simultaneously, not just one. Every subsystem—CPU, GPU, networking, switching—becomes a constraint that must be addressed together.

50%

If computation is 50% of workload, infinite speedup yields only 2x total gain

Amdahl's Law demonstrates why single-component optimization fails at scale

NVIDIA's Organizational Structure for Co-Design

Jensen maintains 60+ direct reports—nearly all with engineering expertise in memory, CPUs, optics, GPUs, architecture, algorithms, and design. No one-on-ones; instead, problems are presented to the entire group so every specialist can contribute cross-disciplinary insights. This mirrors the company's mission: extreme co-design requires extreme organizational co-design.

60+

Direct reports on Jensen's staff, nearly all engineers with specialized expertise

Organizational structure reflects the co-design philosophy

The Strategic Tension: Specialization vs. Generalization

NVIDIA began as a narrow accelerator company, which limited market reach and R&D capacity. To have broader impact, the company needed to become a general computing platform—but that risked losing specialization. Jensen found a narrow path: programmable pixel shaders, then FP32 floating-point, then CUDA, each step expanding aperture without abandoning core GPU acceleration.

The CUDA Decision: Risking Everything

Putting CUDA on consumer GeForce GPUs cost NVIDIA 50% of gross margins (from 35% to near zero) and crashed the stock from ~$8B to ~$1.5B market cap. The bet: build massive developer install base on consumer PCs, universities, and clusters so researchers would discover CUDA and eventually use it everywhere. It took a decade to recover, but proved foundational for the AI revolution.

NVIDIA's market cap collapsed but the CUDA install base became the company's greatest moat

Install Base > Technology

The x86 architecture is inelegant but dominates because of install base; beautiful RISC architectures failed. CUDA succeeded not because of superior technology alone, but because NVIDIA put it everywhere, cultivated millions of developers, and committed to maintaining it for decades. Developers choose CUDA first because they trust NVIDIA will keep improving it and because it reaches hundreds of millions of devices.

Pre-Training Scaling: Data as the Limit

Larger models trained on more high-quality data produce smarter AI. Industry feared data scarcity would halt progress, but synthetic data—human-created, augmented, and regenerated—is now the primary source. Most data humans consume is already synthetic; AI can now generate it at scale, so pre-training is no longer data-limited but compute-limited.

Post-Training Scaling: Refinement Through Feedback

After pre-training, models are fine-tuned and refined using human feedback and curated data. This phase continues to scale and is increasingly driven by synthetic data generated during training and inference, creating a feedback loop that amplifies model capability.

Test-Time Scaling: Inference as Thinking

Inference is not easy; it is thinking, reasoning, planning, and search. Industry assumed inference would be compute-light, but Jensen argues thinking is harder than reading (pre-training). Test-time scaling—using more compute during inference to reason through problems—is intensely compute-intensive and will be a massive market.

Agentic Scaling: Multiplying Intelligence

Agents spawn sub-agents to parallelize work, creating teams of AI systems. This is like multiplying AI: spin off agents as fast as needed. Agents generate new data and experiences, which feed back into pre-training, creating a virtuous cycle. This is the fourth and newest scaling law.

Agentic scaling creates a feedback loop: agents generate data that improves future models

AI Model Architectures Evolve Every 6 Months, Hardware Every 3 Years

Model innovations (like mixture-of-experts) arrive faster than hardware can adapt. NVIDIA must anticipate what algorithms will dominate 2–3 years ahead. The company does this through internal research, listening to every AI lab globally, and building flexible architectures (like CUDA) that can adapt without total redesign.

The innovation cadence gap forces NVIDIA to predict the future

From Grace Blackwell to Vera Rubin: Anticipating Agents

Grace Blackwell was designed for MoE LLM inference; one year later, Vera Rubin added storage accelerators, new CPUs, and new racks because agents need to access tools, databases, and file systems. The design was completed before Claude Code and OpenClaw existed, yet perfectly anticipated their needs through first-principles reasoning about what digital workers must do.

Manifesting the Future Through Conviction

Jensen makes bold bets by reasoning through why a future outcome is inevitable, then manifesting it through belief and action. He doesn't announce major pivots suddenly; instead, he spends months or years laying down foundational reasoning in meetings, keynotes, and conversations, so when he declares a new direction, everyone says 'what took you so long?' rather than 'where did this come from?'

Continuous Knowledge Transfer Over Succession Planning

Jensen rejects traditional succession planning. Instead, he continuously passes knowledge, insights, and skills to his team in real-time—every meeting is a reasoning session where he teaches. Nothing learned sits on his desk; he immediately shares it. This empowers the organization and ensures continuity without a single point of failure.

Speed of Light Thinking: First Principles Over Incremental Improvement

Rather than optimize incrementally (74 days to 72 days), Jensen asks: what are the physical limits? If you built from scratch, how long would it take? (Maybe 6 days.) Now the conversation is grounded in physics, not habit. This forces teams to question every assumption and often reveals 10x improvements hidden in legacy processes.

Computing Shifted from Retrieval to Generation

Old computers were file-retrieval systems: pre-record, store, retrieve. AI computers are generative and contextually aware: they process and generate tokens in real-time. This fundamental shift means we need vastly more computation and less storage, changing the economics of computing forever.

The fundamental purpose of computing has shifted

Computers Became Factories, Not Warehouses

Warehouses store; factories produce revenue. AI data centers are now factories that generate tokens—a commodity with real economic value. Tokens are segmented like iPhones: free tokens, premium tokens, mid-tier tokens. Some tokens command $1,000 per million, turning compute into a direct revenue engine.

NVIDIA's Unit of Compute: From Chip to Pod to Planetary Scale

Jensen's mental model of 'one computer' has evolved: once it was a chip, then a GPU, then a system, then a cluster, now an entire AI factory (gigawatt-scale pod with power, cooling, networking, thousands of engineers). His next mental click: planetary-scale infrastructure. Each level requires rethinking architecture, supply chain, and operations.

Jensen's evolving mental model of what constitutes 'one computer'

Supply Chain Complexity: 200 Suppliers, 1.3M Components per Rack

Each Vera Rubin rack contains 1.3 million components from ~200 suppliers. NVIDIA must manufacture ~200 pods per week. Jensen spends enormous time with upstream (TSMC, ASML, SK Hynix) and downstream (cloud providers, OEMs) partners, explaining future demand and helping them invest billions in capacity. This is not a concern that keeps him up at night because he actively manages it.

200

Suppliers contributing to each Vera Rubin rack

Supply chain complexity requires constant relationship management

Power Efficiency: Tokens Per Second Per Watt

Power is the primary blocker. NVIDIA's solution: improve efficiency (tokens per second per watt) by orders of magnitude yearly through extreme co-design. In 10 years, Moore's Law would have scaled computing 100x; NVIDIA scaled it 1 million times. Token costs drop an order of magnitude per year despite rising chip costs.

NVIDIA's extreme co-design vastly outpaces Moore's Law

Grid Power: Using Idle Capacity, Not Building New

Power grids are designed for worst-case (extreme weather, peak demand). 99% of the time, grids run at ~60% capacity. Jensen proposes data centers use this idle power via dynamic load-shifting: if the grid needs power, data centers degrade performance or shift workloads. This requires contracts that allow graceful degradation, not six-nines uptime. Utilities could offer tiered power products.

CUDA Install Base: The #1 Competitive Advantage

NVIDIA's single greatest moat is not the technology but the CUDA install base: millions of developers, decades of commitment, continuous improvement, and ubiquity across every cloud, every industry, every country. Developers choose CUDA first because they trust NVIDIA will maintain it forever and because it reaches the most users.

Velocity + Install Base = Unstoppable

CUDA improves 10x every 6 months on average. Developers see this velocity and commit deeper. Combined with install base, no company in history has built systems this complex this fast. This velocity-install-base combination is nearly impossible to compete against.

Horizontal Integration Across Every Industry

NVIDIA vertically integrates the entire stack but horizontally integrates into every company's products: Google Cloud, AWS, Azure, CoreWeave, Lilly, enterprise, edge, radio base stations, cars, robots, satellites, space. One architecture in all these systems covers every industry globally.

50% of World's AI Researchers Are Chinese

China has incredible talent in AI research, strong math and science education, and a tech industry that emerged during the software era (comfortable with modern software). This concentration of talent is a structural advantage.

50%

World's AI researchers are Chinese (mostly in China)

Concentration of talent is a structural advantage

Internal Competition Drives Innovation

China is not one country but many provinces and cities with competing mayors. This creates insane internal competition: many EV companies, many AI companies, many of everything. The best survive. This competitive pressure accelerates innovation.

Open Source Culture: Family, Friends, Schoolmates

Chinese culture prioritizes family first, friends second, company third. Engineers' brothers and schoolmates work at competing companies. There's no sense keeping technology hidden; they share via open source. This rapid knowledge diffusion accelerates the entire ecosystem.

Technology Is Not TSMC's Only Moat

TSMC's deepest advantage is not just transistors or metallization but the ability to orchestrate hundreds of companies' dynamic demands—wafer starts, stops, emergency requests, shifting volumes—while maintaining high throughput, yields, costs, and customer service. This manufacturing orchestration is miraculous.

Balancing Technology Excellence + Customer Service

Many companies excel at one: either bleeding-edge technology or customer service. TSMC is world-class at both simultaneously. They're at the frontier of technology while treating promises seriously; when wafers are promised, they arrive on time.

Trust Without Contracts

NVIDIA and TSMC have done hundreds of billions of dollars of business over three decades without a formal contract. This trust is built on consistent performance, respect, and shared vision. It's the intangible that matters most.

NVIDIA Could Reach $3T+ in Revenue

If the world's GDP accelerates due to AI productivity, and computation becomes 100x more of GDP (because it's now a product factory, not storage), and NVIDIA captures a significant share, $3T revenue is not limited by physics—only by energy and supply chain, which Jensen is actively solving. Previous 'limits' ($1B, $25B, $100B) all proved illogical.

Agents as the iPhone of Tokens

OpenClaw represents the fastest-growing application in history. Agents are the iPhone moment for AI: they access files, use tools, spawn sub-agents, and generate value. This is the killer app that will drive demand for trillions of tokens.

Jobs Transform, Don't Disappear

Radiologists were predicted to disappear when computer vision became superhuman. Instead, radiologists grew in number because the purpose of radiology (diagnose disease, help patients) didn't change—only the tools. Similarly, programmers, accountants, carpenters, plumbers will grow in number as AI elevates their capabilities and reach.

AI elevated the profession, not eliminated it

Coding Expands from 30M to 1B People

If coding is defined as specification (telling a computer what to build), not syntax, then every carpenter, plumber, accountant, farmer can code. They become architects of their own tools. The number of 'coders' expands 30x, and their value increases.

30M → 1B

Potential coders if definition shifts to specification

Redefining coding as specification democratizes the skill

Handling Pressure: Decompose, Delegate, Forget

Jensen manages enormous pressure by decomposing problems into manageable pieces, delegating to capable people, and then forgetting—not carrying the burden. He tells someone who can act on it, then moves on. This prevents panic and burnout.

Resilience Through Childlike Optimism

Jensen approaches new challenges with 'How hard can it be?' rather than simulating all setbacks in advance. This childlike optimism, combined with endurance and grit when setbacks actually occur, allows him to attempt things others deem impossible. He forgets past embarrassments and focuses on the next opportunity.

Intelligence vs. Humanity: Two Different Words

Intelligence is functional and commoditizable; humanity is not. Compassion, generosity, character, tolerance for pain—these are superhuman powers distinct from intelligence. Jensen's success comes not from being the smartest but from orchestrating superhumans and embodying human values.

Hope in Reaching the Unreachable

Jensen is hopeful because so many problems are now within reach: ending disease, reducing pollution, traveling at light speed, understanding consciousness and biology. These are reasonable expectations within his lifetime. He's romantic about the future because the future is genuinely romantic.