Reading Google Play and App Store reviews straight from their JSON, no browser

Apple (App Store)

Apple publishes reviews as an RSS feed in JSON. One endpoint:

https://itunes.apple.com/{country}/rss/customerreviews/page={1-10}/id={appId}/sortby={mostrecent|mosthelpful}/json

• country - a storefront code (us, gb, de, ...). Every storefront keeps its own reviews.
• appId - the numeric id from the store URL: apps.apple.com/us/app/whatsapp-messenger/id310633997
• page - 1 to 10, and 10 is the wall. Fifty reviews a page, so ~500 per storefront.

import { gotScraping } from 'got-scraping';

async function fetchAppleReviews(appId, { country = 'us', maxReviews = 200 } = {}) {
  const out = [];
  const pages = Math.min(10, Math.ceil(maxReviews / 50));
  for (let page = 1; page <= pages; page++) {
    const url = `https://itunes.apple.com/${country}/rss/customerreviews/page=${page}/id=${appId}/sortby=mostrecent/json`;
    const res = await gotScraping({ url, responseType: 'json' });
    const entries = res.body?.feed?.entry ?? [];
    // The first entry is sometimes app metadata, not a review - guard on im:rating.
    for (const e of entries) {
      if (!e['im:rating']) continue;
      out.push({
        id: e.id.label,
        author: e.author.name.label,
        rating: Number(e['im:rating'].label),
        title: e.title.label,
        text: e.content.label,
        version: e['im:version'].label,
        date: e.updated.label,
      });
    }
    if (!entries.length) break;
  }
  return out.slice(0, maxReviews);
}

That 500-review cap is hard. No continuation token gets you past it - I looked. What does work: reviews are scoped per storefront, so pull us, then gb, au, ca, de, and the rest of the English-language storefronts, deduping on review id. There are about ten of them; at ~500 each that's roughly 5,000 recent reviews, which is usually plenty.

App metadata - title, average rating, total count, current version - lives at a second, simpler endpoint:

https://itunes.apple.com/lookup?id={appId}&country=us

Google Play, and the batchexecute rabbit hole

Play has no clean REST endpoint. The store front-end talks to an internal RPC called batchexecute, and the payload is ugly and documented nowhere. The payoff for climbing through it: Play paginates with no ceiling. Apple caps you at 500 a storefront; Play just keeps going.

The endpoint: POST https://play.google.com/_/PlayStoreUi/data/batchexecute?hl=en&gl=us

Content-Type: application/x-www-form-urlencoded;charset=UTF-8

The body is a URL-encoded f.req parameter wrapping the RPC id UsvDTd and its arguments:

function buildBody(pkg, { count = 100, token = null, sort = 2 } = {}) {
  const tok = token ? `\\"${token}\\"` : 'null';
  const inner = `[null,null,[2,${sort},[${count},null,${tok}],null,[]],[\\"${pkg}\\",7]]`;
  const freq = `[[["UsvDTd","${inner}",null,"generic"]]]`;
  return 'f.req=' + encodeURIComponent(freq);
}

sort is 2 for newest, 1 for relevance, 3 for rating. pkg is the package name (com.whatsapp). token is the continuation cursor the previous response handed back.

The response is where it gets weird. It opens with an anti-JSON-hijacking guard, the literal )]}', and then a nested envelope where the real data sits as a JSON string inside the outer JSON. You parse twice:

function parse(raw) {
  const envelope = JSON.parse(raw.slice(raw.indexOf('['))); // strip )]}'
  const inner = envelope?.[0]?.[2]; // a JSON *string*
  if (!inner) return { reviews: [], nextToken: null };
  const data = JSON.parse(inner);
  const reviews = (data[0] ?? []).map((r) => ({
    id: r[0],
    author: r[1][0],
    rating: r[2],
    text: r[4],
    date: new Date(r[5][0] * 1000).toISOString(),
    thumbsUp: r[6],
    reply: r[7]?.[1] ?? null, // developer reply text
    appVersion: r[10],
  }));
  const nextToken = data[1]?.[1] ?? null;
  return { reviews, nextToken };
}

Then loop, feeding nextToken back until you've got enough or it comes back null:

async function fetchPlayReviews(pkg, { maxReviews = 200 } = {}) {
  const out = [];
  let token = null;
  while (out.length < maxReviews) {
    const res = await gotScraping({
      url: 'https://play.google.com/_/PlayStoreUi/data/batchexecute?hl=en&gl=us',
      method: 'POST',
      body: buildBody(pkg, { count: 150, token }),
      headers: { 'content-type': 'application/x-www-form-urlencoded;charset=UTF-8' },
    });
    const { reviews, nextToken } = parse(res.body);
    if (!reviews.length) break;
    out.push(...reviews);
    if (!nextToken) break;
    token = nextToken;
  }
  return out.slice(0, maxReviews);
}

A few things I only learned by running this against real apps

• Play reviews carry no title. Just a rating and a body. Apple gives you both. If you're merging the two stores into one schema, title has to be nullable or you'll drop half your Play data on a strict validator.
• Developer replies hide in slot [7] on Play - text at [7][1], timestamp at [7][2][0]. Apple's public feed doesn't surface replies at all.
• The loop fires requests back to back with no delay. Proxyless, from my laptop and from a datacenter, I haven't been rate-limited doing this - but it's the assumption most likely to break at tens of thousands of reviews, and I'd put a throttle in front of it before trusting it at that scale.

And here's where it got me. I first pulled appVersion from slot [8], eyeballed one response, saw a version string, shipped it. Some apps came back with a country code there instead. The version is [10]. The index parsing is brittle by design - Google can reshuffle slots whenever they like, and nothing tells you. So I pinned a test against a known app with a review I can eyeball and assert the fields on it. When Google moves something, that test screams before my users notice.

Is this actually worth skipping the browser?

For me, yes - and the number that convinced me: 250 Play reviews land in about 0.6 seconds this way. You're reading the exact API the store's own frontend reads, so a store redesign doesn't touch you. My Playwright version was 10 to 20 times slower, wanted a proxy budget the moment I scaled it, and died on the next UI refresh. I don't miss it.

If you'd rather not babysit the slot indices I bundled both stores into one actor on Apify - one schema, handles the pagination and the batchexecute envelope, runs proxyless: App Reviews Scraper. It's mostly there so I stop re-fixing the [10]-versus-[8] kind of thing every quarter. But the code above is the whole trick, and rolling your own is very doable. The batchexecute envelope eats afternoons if you go in blind - if you get stuck on it, leave a comment and I'll dig in.