Skip to content
99tests Blog
Testing guides

Why OTP Autofill Works in Testing but Fails for Real Users (Android and iOS)

Why SMS OTP auto-read works on debug builds but fails on the Play Store build, and how to test OTP login: SMS Retriever, consent prompt, iOS code autofill, dual-SIM, late SMS, Jio, Airtel and Vi.

Hand-drawn sketch of two phones, an OTP code and weak signal bars with a cross
In this article
  1. The number one reason auto-read fails for users
  2. Set up before you test
  3. 1. Happy path
  4. 2. Validation and negative cases
  5. 3. Edge cases
  6. 4. Interruptions and network
  7. 5. Device and OS: dual-SIM and the maker's quirks
  8. 6. Security
  9. 7. Accessibility and language
  10. A short checklist before release
  11. Get the full test case set

OTP login is the front door of most Indian apps. When it fails, the user doesn't file a bug report. They just leave. The frustrating part is that OTP login usually works perfectly for the team that built it.

The reasons it fails in the field are ones a lab rarely shows: the SMS arrives late or out of order on busy carrier routes, auto-read only works on the debug build, the phone has two SIMs, or the app gets killed while the user switches to the messages app to read the code. A passing test on one developer phone says little about those situations.

This guide is based on the free OTP login test case set in the 99tests library: 38 cases with steps, expected results, priority and real-device notes. Use the scenarios below to test the complete journey, from requesting a code to recovering when the ideal flow breaks.

The number one reason auto-read fails for users

On Android, apps usually read the OTP in one of two ways:

  • SMS Retriever can deliver a matching SMS to the app without a user prompt. The SMS must include an 11-character app hash associated with that app's package name and signing certificate.
  • SMS User Consent shows a system prompt asking the user to share a matching message with the app. It doesn't need an app hash, but the user must approve the prompt before the app receives the SMS.

The most common field failure is with SMS Retriever. The SMS template carries a hash generated for the debug or upload key, while users install a build signed with the Google Play app-signing key. The hashes don't match, so auto-read silently stops working, but only for real users. Google's app-hash instructions specifically call for the Play app-signing certificate when Google Play signs the distributed app.

The fix for testing: install the build from a Play Store testing track, request a fresh OTP, and confirm that the code fills without a prompt. Compare that result with the debug build. A pass on debug and a failure on the store build is a useful clue to check the production SMS hash. That's case OTP-02 in the set.

Other causes need different checks:

  • The SMS template changed. Inspect the message received by the tester, not just the template in a dashboard. Confirm the OTP and app hash are still present in the delivered text.
  • A sender filter is too narrow. Repeat the request on Jio, Airtel and Vi. If one route uses a different sender header, auto-read should not depend on a header the app doesn't actually control.
  • The message is localized. Send the Hindi template to a real phone and check whether the complete text arrives, whether it is split, and whether the app extracts the OTP rather than another number in the message.
  • The consent listener starts too late. Request an OTP with the app in the background, then return to the OTP screen. If the app starts listening only after the message has arrived, SMS User Consent won't forward that earlier message. Manual entry still needs to work.

Set up before you test

A useful test run needs more than three phones with different carrier logos. Record the build, device, SIM, route and OTP configuration for each result so you can reproduce a failure.

  • Registered test numbers on Jio, Airtel and Vi SIMs that can receive SMS. Keep at least one number available for repeated resend and throttling checks.
  • A dual-SIM Android with both SIMs active, plus an iPhone that can receive an actual OTP SMS.
  • Android phones from at least two makers, including one with 3-4 GB RAM. Use the lower-memory device for app-switching and process-death checks.
  • The store-signed build from a Play Store testing track, as well as the debug build. Note the version and installation source with each result.
  • The configured OTP validity, resend wait and attempt limit for the build under test. Without these values, it is easy to call correct behavior a bug, or miss an expiry bug entirely.

Before starting, confirm which flow the app implements: SMS Retriever, SMS User Consent, manual entry, or a combination. Capture timestamps for the request, SMS arrival, resend and verification when investigating late messages. Use test accounts for security and throttling scenarios so failed attempts do not lock out a real customer.

1. Happy path

Start with the normal login journey, but run it on each supported route and input method. A successful API response alone is not a pass: the tester should reach the intended signed-in screen without duplicate requests or confusing intermediate states.

  • Type the OTP from the SMS on Jio, Airtel and Vi. Confirm the message arrives, identifies the request clearly and contains a code that the app accepts. This needs real SIMs; a mocked SMS won't tell you whether the carrier route delivers.
  • Auto-read on the store build. Leave the OTP screen open after requesting a code. The matching code should fill and verify without typing or a permission prompt if the app uses SMS Retriever. Repeat after a fresh installation from the testing track, not just after updating a local build.
  • The consent prompt on each carrier. When the app uses SMS User Consent, check that the prompt displays the expected message, Allow fills the right code, and the user returns to the login flow. Don't mark a flow that requires consent as a failed Retriever flow; test it against the design actually shipped.
  • iOS code suggestion filling every digit box. On a real iPhone, request the SMS and tap the code suggestion above the keyboard. The app should place the digits into the full OTP field, even if its UI displays separate boxes. Apple's one-time-code text content type supports this suggestion; the iOS Simulator can't receive an SMS for the same end-to-end test.
  • Resend unlocking only when the countdown ends, then restarting it after a resend. For example, if Resend becomes available after the configured wait, tap it once and check both the new countdown and the new message.
  • A second channel, such as a voice call, if the app offers one. Request a call after an SMS is delayed. Check that the screen makes it clear which code to use and that a late SMS does not replace digits the user has already entered.

2. Validation and negative cases

These checks tell you whether the app stops a mistake at the right point and gives the user a way forward. Verify both the message on screen and what the backend accepts.

  • Invalid phone numbers: try 9 digits, 11 digits and letters pasted into the number field. The app should stop an invalid request before sending it, rather than advancing to an OTP screen for a number it cannot use.
  • A wrong OTP: enter a valid-length code that was not sent. Expect a clear error, the ability to correct the code and, if the app displays it, an accurate attempts-left count. Confirm that the field does not leave the wrong code looking as though it was accepted.
  • The configured wrong-attempt limit: submit wrong codes until the limit is reached, then try the correct code. The block should still apply; reinstalling or reopening the app should not reset a server-side restriction. P1.
  • An expired OTP: wait beyond the configured validity, then enter the code. It should be rejected, and the next step should be obvious, such as requesting a fresh code. Test this separately from the resend countdown: the two timers may have different durations.
  • Denied SMS consent: tap Deny on the system prompt. The app should remain usable, leave the OTP field available and let the user type the SMS code manually. Requesting a code should not depend on consent being granted.

3. Edge cases

These are the timing and input collisions that often pass isolated unit tests but fail when a person uses the screen quickly.

  • Codes arriving out of order after two resends: request a code, resend twice, then compare each SMS with its request time. The latest code should work. An older one should be rejected with a message pointing the user to the newest SMS, and auto-read must not overwrite a newer code with an older one. Record the arrival order as well as the send order.
  • Auto-read and Verify at the same moment: begin typing the correct code and tap Verify as the SMS arrives. Confirm that the app sends only one verification request and does not show two success screens or conflicting errors.
  • Other numbers in the SMS: use text that includes a validity period or reference number alongside the OTP. For example, a message might contain a six-digit code and a two-digit validity time. Only the intended code should populate the field.
  • Changing the number after the OTP was sent: request a code for one number, return to edit the number, then submit the old code for the new number. It must be rejected. Check that the OTP screen displays the number currently being verified. P1.
  • Number normalization: paste a number with +91, spaces or a leading 0. The field should resolve it to the intended 10-digit mobile number without doubling the country code or silently changing the destination.
  • Pasting a copied code: copy the SMS code and paste it into the first digit box. All digits should fill in order, with no truncation or requirement to paste into each box separately.
  • Background countdown: start the resend timer, leave the app for part of the wait, then return. The countdown should reflect elapsed time rather than restart because the screen was recreated.

4. Interruptions and network

The OTP screen has to survive interruptions that are normal on a phone: poor reception, incoming calls, and switching apps to read a message. In each case, check whether the user can finish the existing login attempt instead of starting over.

  • A late OTP: let the resend countdown end without requesting another code, then enter an SMS that arrives before the OTP itself expires. The code should still work. A resend timer reaching zero should not by itself invalidate the current OTP.
  • Mobile data dropping before submission: enter the right code, turn off data and tap Verify. Show a network-related failure and let the user retry when connected; the failed network request should not be presented as a wrong-code attempt.
  • An incoming call: request an OTP, accept or dismiss a call, and return to the app. The same number, current code and resend state should still be understandable.
  • The app killed while the user reads the code: on a 3-4 GB RAM Android, request an OTP, switch to the messages app and leave the app long enough for its process to be killed. Reopen it. The user should return to the OTP screen for the same number, and the app should not silently send a new OTP. P1.
  • A slow 2G connection: tap Get OTP twice while the first request is pending. The app should show one loading state, avoid opening the OTP screen twice and make it clear whether a request succeeded before offering another attempt.

5. Device and OS: dual-SIM and the maker's quirks

A dual-SIM device can receive an SMS on one line while using the other for mobile data. Test the actual number selected for delivery, not merely the phone's default data settings.

  • The OTP sent to the non-data SIM: request the code for that SIM's number. It should be readable through the supported auto-read flow or available for manual entry, with no instruction to switch the data SIM just to log in.
  • Number suggestions on a dual-SIM phone: tap each suggested number in turn. Confirm that the field shows the correct 10-digit mobile number and that the following SMS goes to the number the user selected.
  • Keyboard suggestion and app auto-read together: keep the keyboard open while the SMS arrives. If the keyboard offers a code and the app also reads it, tapping the suggestion must not produce doubled digits or trigger duplicate verification.
  • SMS filed as spam: on Xiaomi, Vivo or Samsung messages apps, check the spam folder when an expected OTP does not appear in the main inbox. Record whether the SMS arrived, whether the app could use it and whether manual recovery is possible; these are separate outcomes.
  • A SIM with the carrier's full do-not-disturb preference on: send a real OTP and verify delivery rather than assuming a lab-generated message covers this route. If delivery fails, keep the carrier, sender and timestamp in the report. P1; this needs a real SIM.
  • No Google Play services, or an outdated version: open the OTP flow on a suitable device and request a code. The app should not crash or trap the user waiting for auto-read; manual typing must remain available.

6. Security

Security tests should probe the server's rules, not only what the UI prevents. Where possible, check the response to an old or repeated request after navigating back, restarting or reinstalling the app.

  • A used code cannot be replayed: sign in with a valid OTP, sign out or begin another login attempt, then submit that same code again. It must not authorize another session. P1.
  • Repeated OTP requests are throttled: request codes until the configured limit is reached. Confirm the limit is enforced even after going back or reinstalling, and that the user sees a useful wait or retry message. P1.
  • Registered and unregistered numbers: walk through the OTP step with both. The screen and wording should not reveal which number already has an account, while any intended account-creation flow still works.
  • Phone-clock changes: move the device clock forward during the resend wait. Resend should not unlock early just because local time changed; compare the UI with the actual request behavior.
  • An unrelated code-like SMS: while waiting for an OTP, send another message containing a similar-looking number. The unrelated message must not fill or verify the login field.

7. Accessibility and language

An OTP is often entered under time pressure. Accessibility and localization problems can turn a simple six-digit task into a complete login block, even when delivery and verification work.

  • TalkBack: navigate to the code input and confirm the digit boxes behave as one understandable code field. Trigger a wrong-code error and check that it is announced. Let the countdown run and confirm it does not interrupt the user every second.
  • Largest font size: enlarge system text, open the keyboard and check that the code input, error message, Resend and Verify remain visible or reachable without losing focus.
  • A Hindi OTP SMS: send the localized production template to a real phone. Check that the entire message arrives and that auto-read fills the intended digits. If it fails, preserve the received message text for debugging, including any split parts.
  • Devanagari digits: type a code using a Hindi keyboard. The app should either accept and convert the digits to 0-9 for verification or present a usable numeric keyboard. An unexplained invalid-code error is not an acceptable substitute.

A short checklist before release

Use this as a final pass after the deeper scenarios above. Record the device, carrier, app version and result for every failure so the team can reproduce it.

  1. Install from the Play Store testing track and confirm auto-read works with a fresh SMS, not just on debug.
  2. Send OTPs to Jio, Airtel and Vi numbers, including one with do-not-disturb on; note delivery and arrival times.
  3. Use a dual-SIM phone with the OTP sent to the non-data SIM, and confirm manual entry is still possible.
  4. On a budget Android, switch to the messages app, return to the OTP screen and check whether the attempt survived.
  5. Resend twice in a weak-signal spot, then use the latest code and verify that older arrivals do not replace it.
  6. Try an iPhone with code autofill and confirm one tap fills every box and completes verification.

Get the full test case set

All 38 OTP login test cases are free, with steps, expected results and the reason each real-device case needs a real phone. Use the cases as a repeatable run sheet, then attach the message, timestamps, build and device details to any failures you find.

Carrier routes, dual SIMs and budget-phone memory managers only show up on real phones. 99tests runs exploratory testing sessions on real Android phones and iPhones across Indian cities and networks, from ₹9,999 + GST with a report in 3 business days.