How Peer Testing Verifies a Tester Really Used Your App

A question I get from almost every developer who tries PeerPlay: how do you know a tester really used my app, and didn't just install it to look busy?

It is a fair question. Google's closed testing requirement asks for 12 testers opted in for 14 days in a row, and a tester who installs and forgets is worth very little to you.

So here is exactly what PeerPlay checks today, based on how the app's code works, and, just as important, what it cannot check.

A test starts in PeerPlay and opens your app

A tester can't count a day by opening your app from the home screen. They tap Start Test on your app's card in PeerPlay, and PeerPlay launches your app.

From that moment PeerPlay counts the time the tester spends away from PeerPlay, which in practice is time in your app. The target is 30 seconds a day. When it is reached, the day counts.

If the tester comes back too early, the test isn't thrown away. They see a message like "Counted 12s of 30s. Tap Continue and keep playing to finish today's test", and the counted seconds wait for them.

A live timer notification on the phone shows how far along the test is.

Two bugs that taught me how real testing looks

The first version reset the count every time the tester left your app for a moment. An ad, a permission dialog or a quick glance back at PeerPlay sent the timer back to zero. For ad-heavy games that made the test practically impossible to finish, even for a tester doing everything right. Now every stretch in your app adds up toward the 30 seconds.

The second one came from phones, not testers. Some phones, often in a game mode, kill background apps to free memory, and PeerPlay was one of them while the tester was playing. The time spent was lost. Now PeerPlay recovers that session on its next start and still counts the time the tester was away.

Signals against gaming the system

Hopping through apps as fast as possible doesn't work well. After a test, PeerPlay puts a short cooldown on the Start Test button, so a tester can't start one app after another in rapid fire. Continuing an unfinished test skips the cooldown, because that is real testing.

Every counted test records:

• the device model
• the Android version
• whether Android reports the device as a physical phone

Emulators, virtual devices and bots are not allowed on PeerPlay.

Testers who join a campaign and walk away are recorded too. Each tester's profile shows a quit counter, the number of campaigns they joined and abandoned, so you can see who has a habit of leaving.

What no testing app can prove

Here is the honest limit. PeerPlay doesn't see the tester's screen and doesn't read what they do inside your app. It knows your app was launched from PeerPlay and that the tester stayed away from PeerPlay for at least 30 seconds.

That is a strong signal of a real session, but it is not proof that every one of those seconds was spent looking at your app. Any service that claims to prove more than that is either installing something much more invasive on the tester's phone, or overselling.

The daily open, the device record and the quit counter are there to make lazy testing hard and visible, not impossible.

What to check on your side

Use your own analytics. If you have Firebase Analytics or a similar tool in your app, your daily active users during the test should roughly match the number of testers opening it. If PeerPlay shows 15 testers a day and your analytics shows 3 sessions, something is off, and you should tell me.

Read the feedback. A tester who writes "I had to tap several times before anything happened" was in your app. A tester who has written nothing after a week might still be opening it, but you know less about them.

And watch your opted-in count in Play Console every day. PeerPlay can make sure testers open your app, but the number Google counts toward the 14 days lives in your console.