How crowd testing works in 2026, step by step

Originally published 2016. Rewritten October 2026.

Crowd testing means real people run your app on the phones, laptops and networks your users actually have, and report what breaks. The idea is simple. The details decide whether you get a useful report or a pile of duplicates, so here is exactly how it works at 99tests, from the moment you share a build to the report you act on.

1. You share a build and what to test

For a mobile app, share an APK link, a Play internal-testing link or a TestFlight build. For a web app, share a staging or production URL, plus test accounts if there's a login.

Then say what to test. Test cases can come in any format: a spreadsheet, a Jira or TestRail export, or cases an AI tool wrote for you. If you don't have cases, write a one-line charter instead, such as "try to break checkout" or "use the app as a first-time user on a slow network".

2. You pick devices and a package

Choose the devices that match your users: budget and mid-range Androids such as Redmi, Realme, Galaxy A and Vivo, iPhones, or laptops running Chrome, Edge or Safari. The device list shows what testers own today.

The package sets how many sessions you get, on how many devices, and how fast the report arrives. Prices are fixed and shown up front on the pricing page.

3. Testers are matched to your devices

99tests has 20,000 registered testers across India. Each one has passed a qualification test on their own device. Your job goes to testers who own the devices you picked, so a Redmi 12 session really runs on a Redmi 12.

4. Testers run timed sessions on real devices

Each session is a fixed block of time, 45 or 90 minutes. The tester runs your cases on the phone they use every day, on their own Jio, Airtel or Vi connection, with their own storage, battery and settings. That's where emulators and clean lab devices fall short: real phones are full, slow, mid-update and on patchy networks.

Testers record the screen for every serious finding and log the device, OS build and network. They're paid for the session, not per bug, so a clean "all passed" session pays the same as one with ten findings. That removes any reason to pad reports.

5. Every finding is verified

A separate reviewer, not the tester, checks every report before you see it:

• Findings that can't be reproduced are rejected.
• Duplicates from different testers are merged into one.
• Each finding gets a severity grade against a published rubric.

The report includes the verification record: how many findings were accepted, merged and rejected.

6. You get a report you can act on

The report comes as a web page and a PDF. Each finding has steps to reproduce, expected and actual results, the device, OS and network, and a screen recording for serious bugs. You also get a coverage table, a device matrix and a session log.

When the fixes are in, re-run the same plan on your next build in one click. See a sample report to judge the format.

What crowd testing is good at, and what it isn't

Crowd testing is strongest where an in-house team is thinnest:

• Device and OS spread, especially budget Androids with aggressive battery savers
• Real networks, including payments that drop mid-transaction
• First-time user flows, seen by people who've never used your app
• Exploratory sessions before a launch
• Running AI-generated test cases on real devices, which headless self-tests can't do

It's the wrong tool for test automation, load or performance testing, security or penetration testing, and certification. Those need specialists.

Common questions

Do testers use emulators? No. Every session runs on a real device the tester owns.

What if no tester owns my device? Unfilled sessions are credited back, and you can apply the credit to another device.

Do I need written test cases? No. Exploratory sessions work from a short charter, and you can mix charters and cases in one job.