Why AI Finds Bugs Humans Miss

AI excels at finding software bugs through three core strengths: static analysis (reading entire codebases instantly vs. humans' ~10,000 line/day limit), vulnerability research (pattern recognition across specialties without human knowledge silos), and tireless automation (running bug-finding loops indefinitely). However, AI hallucination creates massive false-positive rates, requiring human expertise and validation tools like ASan to filter real bugs from noise.

What AI Actually Is

AI is a text predictor, not magic

AI is fundamentally a large collection of parameters (vectors) that takes tokenized input and outputs predicted text. It has no consciousness or mythical properties—it's a sophisticated mathematical system optimized for specific tasks like code generation.

Code generation excellence drives bug-finding ability

Around GPT-4 era, there was exponential improvement in code generation quality. An accidental side effect of this capability is the ability to read and analyze code deeply, which enables vulnerability detection.

Three Reasons AI Dominates Bug Finding

Static analysis: understanding codebases at scale

Static analysis means reading code and internalizing what it does, what it should do, and where it fails. Humans are bottlenecked by reading speed (~10,000 lines/day max); AI has no such limit and can absorb a 400,000-line codebase in minutes using available GPU memory.

Vulnerability research: pattern matching across domains

Vulnerability researchers must specialize (IoT, browsers, hypervisors) because mastering multiple domains takes years. AI rapidly learns patterns across all specialties simultaneously—buffer overflows, use-after-free, out-of-bounds reads—and can source information on unfamiliar chipsets or frameworks instantly.

Automation: infinite loops without fatigue

AI can run bug-finding loops continuously without breaks, sleep, or cognitive fatigue. It can be tasked to write fuzzers, generate ASan harnesses, and iterate endlessly on a codebase, limited only by context window size.

How Vulnerability Patterns Work

Primitives: bad code behaviors that become exploits

Edge cases in code create primitives—patterns like out-of-bounds reads/writes, use-after-free, or buffer overflows. These are recognizable patterns (e.g., memcpy with user-controlled length in a TLV structure without bounds validation) that AI can detect across large codebases via source-to-sink analysis.

The Hallucination Problem

AI generates massive false positives

AI hallucination is a critical flaw: it invents bugs that don't exist and reports them with high confidence. In Shawn Heeland's Linux kernel SMB handler test, AI found 2 real bugs but reported ~100–200 total, a signal-to-noise ratio of 1:100, forcing humans to manually triage false alarms.

Validation tools reduce false positives

ASan (AddressSanitizer) and QEMU-based harnesses wrap code and detect real out-of-bounds violations. By tasking AI to write fuzzers or ASan harnesses that prove bugs exist, teams filter hallucinations and confirm only genuine vulnerabilities.

Industry Implications and Future

AI amplifies expert researchers rather than replacing them

Instead of eliminating vulnerability researchers, smart companies will use AI to scale their best people: expert researchers task AI agents with their intuition and domain knowledge, then automate pattern-finding across massive codebases, multiplying their output.

Skilled humans remain essential to drive the tools

AI requires human expertise to set up harnesses, interpret results, and guide investigation. The combination of skilled researcher + AI tool + validation harness is what enables rapid bug discovery; AI alone produces noise.

Practical entry point: analyze old firmware or known-buggy code

Anyone can start experimenting now: pull firmware from old IoT devices (routers, cameras) or outdated open-source versions (old Nginx), use Ghidra MCP or Claude/Opus, and discuss the codebase with AI to see its code comprehension and pattern-finding in action.

Notable quotes

AI is literally just a big bucket of vectors, a very good text predictor. — Low Level
The limiting factor is how fast I can look at text on a screen and process it as a human. — Low Level
It still takes a skilled human to drive these things, and that will never change. — Low Level

Action items

  • Pull firmware from an old IoT device (router, camera) and use Ghidra MCP or Claude/Opus to analyze it for vulnerabilities.
  • Experiment with an outdated open-source codebase (e.g., old Nginx version with known bugs) and discuss it with AI to observe its code comprehension.
  • Set up an ASan harness or QEMU-based validation framework to filter AI-generated bug reports and confirm real vulnerabilities.
  • Task an AI model to write a fuzzer or ASan harness for a target codebase to reduce false-positive bug reports.
Glimpse.
Low Level
11 min video
3 min read
Why AI Finds Bugs Humans Miss
You just saved 8 min.
The big takeaway
AI excels at finding software bugs through three core strengths: static analysis (reading entire codebases instantly vs. humans' ~10,000 line/day limit), vulnerability research (pattern recognition across specialties without human knowledge silos), and tireless automation (running bug-finding loops indefinitely). However, AI hallucination creates massive false-positive rates, requiring human expertise and validation tools like ASan to filter real bugs from noise.
What AI Actually Is
AI is a text predictor, not magic
AI is fundamentally a large collection of parameters (vectors) that takes tokenized input and outputs predicted text. It has no consciousness or mythical properties—it's a sophisticated mathematical system optimized for specific tasks like code generation.
Code generation excellence drives bug-finding ability
Around GPT-4 era, there was exponential improvement in code generation quality. An accidental side effect of this capability is the ability to read and analyze code deeply, which enables vulnerability detection.
Three Reasons AI Dominates Bug Finding
Static analysis: understanding codebases at scale
Static analysis means reading code and internalizing what it does, what it should do, and where it fails. Humans are bottlenecked by reading speed (~10,000 lines/day max); AI has no such limit and can absorb a 400,000-line codebase in minutes using available GPU memory.
Human code reading capacity
10000 lines/day
AI codebase ingestion
400000 lines/minutes
Processing speed and scale comparison
Vulnerability research: pattern matching across domains
Vulnerability researchers must specialize (IoT, browsers, hypervisors) because mastering multiple domains takes years. AI rapidly learns patterns across all specialties simultaneously—buffer overflows, use-after-free, out-of-bounds reads—and can source information on unfamiliar chipsets or frameworks instantly.
Automation: infinite loops without fatigue
AI can run bug-finding loops continuously without breaks, sleep, or cognitive fatigue. It can be tasked to write fuzzers, generate ASan harnesses, and iterate endlessly on a codebase, limited only by context window size.
How Vulnerability Patterns Work
Primitives: bad code behaviors that become exploits
Edge cases in code create primitives—patterns like out-of-bounds reads/writes, use-after-free, or buffer overflows. These are recognizable patterns (e.g., memcpy with user-controlled length in a TLV structure without bounds validation) that AI can detect across large codebases via source-to-sink analysis.
1
Out-of-bounds read
2
Out-of-bounds write
3
Use-after-free
4
Buffer overflow
Common vulnerability primitives AI detects
The Hallucination Problem
AI generates massive false positives
AI hallucination is a critical flaw: it invents bugs that don't exist and reports them with high confidence. In Shawn Heeland's Linux kernel SMB handler test, AI found 2 real bugs but reported ~100–200 total, a signal-to-noise ratio of 1:100, forcing humans to manually triage false alarms.
1:100
Signal-to-noise ratio (real bugs : false reports)
From Shawn Heeland's Linux kernel analysis
Validation tools reduce false positives
ASan (AddressSanitizer) and QEMU-based harnesses wrap code and detect real out-of-bounds violations. By tasking AI to write fuzzers or ASan harnesses that prove bugs exist, teams filter hallucinations and confirm only genuine vulnerabilities.
Industry Implications and Future
AI amplifies expert researchers rather than replacing them
Instead of eliminating vulnerability researchers, smart companies will use AI to scale their best people: expert researchers task AI agents with their intuition and domain knowledge, then automate pattern-finding across massive codebases, multiplying their output.
Skilled humans remain essential to drive the tools
AI requires human expertise to set up harnesses, interpret results, and guide investigation. The combination of skilled researcher + AI tool + validation harness is what enables rapid bug discovery; AI alone produces noise.
Practical entry point: analyze old firmware or known-buggy code
Anyone can start experimenting now: pull firmware from old IoT devices (routers, cameras) or outdated open-source versions (old Nginx), use Ghidra MCP or Claude/Opus, and discuss the codebase with AI to see its code comprehension and pattern-finding in action.
Worth quoting
"AI is literally just a big bucket of vectors, a very good text predictor."
— Low Level, at [0:30]
"The limiting factor is how fast I can look at text on a screen and process it as a human."
— Low Level, at [2:01]
"It still takes a skilled human to drive these things, and that will never change."
— Low Level, at [9:09]
Try this
Pull firmware from an old IoT device (router, camera) and use Ghidra MCP or Claude/Opus to analyze it for vulnerabilities.
Experiment with an outdated open-source codebase (e.g., old Nginx version with known bugs) and discuss it with AI to observe its code comprehension.
Set up an ASan harness or QEMU-based validation framework to filter AI-generated bug reports and confirm real vulnerabilities.
Task an AI model to write a fuzzer or ASan harness for a target codebase to reduce false-positive bug reports.
Made with Glimpse by Wozart
glimpse.wozart.com/v/kidqbygx
Share this infographic

More like this