Given enough eyeballs, all bugs are shallow: why crowdsourced testing still makes sense
Linus's Law says enough eyeballs make every bug shallow. Heartbleed showed its limits. Why crowd testing works when the eyes are diverse, guided and checked.
In this article
Originally published 2012. Rewritten October 2026.
"Given enough eyeballs, all bugs are shallow." Eric S. Raymond named this Linus's Law in his 1997 essay The Cathedral and the Bazaar, after Linux creator Linus Torvalds, though the line is Raymond's own.
The idea is that with enough people looking at software, any bug will be obvious to someone. It's a strong argument for crowd testing, as long as you take its conditions seriously.
Why the idea works
A developer tests the way they built the app: on their own phone, on office Wi-Fi, following the happy path. A bug that's invisible from there can be obvious to someone with a different phone, a weaker signal, a different language setting or a different idea of what a button means.
That's the real force of the law. It's not the number of people that matters but how differently they look. A hundred testers on the same flagship phone mostly find the same bugs. Ten testers on ten different phones, networks and habits find far more.
Where it breaks: Heartbleed
In 2014, a serious flaw was found in OpenSSL, the open-source code that secured a large share of the web's encrypted connections. The bug, later named Heartbleed, had been in released versions for two years, since March 2012, and Netcraft estimated that it affected about 17% of secure web servers with trusted certificates. Plenty of people could have read that code. Very few actually did.
The lesson isn't that the law is wrong. It's that eyeballs that could look aren't the same as eyeballs that do look.
Three conditions that make the eyeballs work
- They have to actually look. "Anyone can test it" means nobody is responsible. Crowd testing works when specific testers are assigned specific flows for a fixed, paid block of time.
- They have to be different. Choose testers for their devices, Android skins, networks and languages, not just their availability. Diversity finds bugs; duplication finds duplicates.
- Someone has to check what they find. Many eyes produce many reports, including duplicates and mistakes. A reviewer who reproduces each finding, merges duplicates and grades severity turns noise into a list you can act on.
Why this matters more in 2026
AI coding agents now write much of the code and many of its tests. In effect they add a lot of eyeballs, but they all look the same way: through the same simulated browser, the same assumptions and the same prompt. That raises the value of human eyes that look differently, on real devices in real conditions.
When more eyeballs don't help
Some bugs need a specialist rather than a crowd:
- Logic errors deep in the code that need source access and domain knowledge
- Security vulnerabilities, which need trained penetration testers
- Performance under heavy load, which needs load-testing tools
For everything users see and touch on their own devices, the crowd's breadth is hard to beat.
How 99tests applies it
99tests puts all three conditions into every job. Testers are assigned timed sessions, matched to the devices you choose from 20,000 registered testers across India, and every finding is verified by a separate reviewer before you see it. See how crowd testing works and the packages.
