DEV Community

Our second market study had the opposite bug: a browser made rental sites look bookable

In September we measured how many small rental businesses in Spain's tourist hubs can be booked online, and our first number was wrong by a factor of four. I wrote about that here: a plain curl of each homepage cannot see a booking widget that JavaScript loads, so the crawl said 8% were bookable when the browser-verified answer was 31%.

The fix seemed obvious: verify a sample in a real browser. We did that for two more rounds, in Campania (Italy) and in Florida. Each round found a new way for the instrument to be wrong, and the second one ran in the opposite direction. This is the short version of what broke and how we caught it.

The setup

For each region we pull every rental business we can find on Google Maps (cars, scooters, boats, jet skis), drop the chains, and download each homepage once. That cheap pass sorts sites into strata: a booking-engine fingerprint, a date picker plus a cart, one of the two, neither, WhatsApp only, no response, no website.

Then we draw a stratified sample, weighted toward the strata where the cheap pass is least reliable, and verify it in three passes:

  • Browser render. Accept the cookie banner, follow the most booking-shaped link, and record engine hosts from network requests.
  • The words on the reservation step. A checkout says "confirm and pay", "payment method", "card number". An enquiry says "request a quote", "send request", "we'll get back to you".
  • The form itself. Field names, placeholders and button labels are dumped as text and judged by hand against a written rule.

The rule: a business is bookable online if the booking step asks a structured availability question (pick-up and return, a date range, licence or age) or reaches a cart or payment page. A name-phone-message form doesn't count, and neither does a button that opens WhatsApp. Per-stratum rates are carried back to the population with the standard stratified estimator.

Bug 1: the crawl undercounts (Spain)

On 119 verified sites, the cheap pass had 34 false negatives and zero false positives. Every error went the same way: online booking it could not see. Published crawl-only, the study would have said 92% can't book online. The real figure is 69%.

Bug 2: one browser pass overcounts (Campania)

In Campania the cheap pass was wrong on 28 of 121 sampled sites. It missed 23 online bookings and invented 5. That was the same failure as Spain.

The new finding was in the browser passes. Of the 28 sites that pass 2 (the vocabulary pass) could settle, it overturned pass 1 on 17, and 13 of those went from "bookable" to "enquiry". Pass 1 had found a booking-shaped button and stopped there. Pass 2 read the screen behind the button, and it was a quote request.

A study that stopped after one browser pass would have overstated online booking in Campania by about as much as a curl study understates it in Spain. Final figure: 83.4% of 891 businesses can't be booked online (95% CI 77.4-89.4%).

Lesson: a number produced by one method is wrong in the direction that method is blind to, and the only way to learn which direction is a second method.

Bug 3: a disguised browser looks like a dead site (Florida)

Our verifier ran headless Chromium but announced itself as desktop Chrome 131. Some web application firewalls compare the user agent with the TLS fingerprint and drop the connection on a mismatch (ERR_HTTP2_PROTOCOL_ERROR, ERR_EMPTY_RESPONSE). We recorded those sites as "no page loaded", which our rules count as dead.

In Florida, 10 of 26 "dead" sites were actually live, and five of them ran FareHarbor or Peek, which are real booking engines. The fix was a --default-ua flag: re-run every "no page loaded" row with the browser's honest user agent before trusting the dead count. With the fix, the headline moved from 57.9% to 55.4%.

We re-ran Spain and Campania the same day. Spain had one enquiry form hiding among 23 "dead" sites. Campania's 24 were all genuinely dead. Both published figures stand.

Bug 4: a review badge is not a booking channel (Florida)

We added TripAdvisor to the list of marketplace hosts, and almost every tour operator embeds a TripAdvisor review badge. 14 of 18 "bookable on a marketplace" verdicts turned out to be review badges. We removed TripAdvisor from the list and re-ran all three passes. Viator, GetYourGuide, FishingBooker, Turo and Getaround stayed, because each of them actually sells the booking.

Testing our own judgement

Hand verdicts are where the author's interest leaks in, and we sell rental software, so the bias that suits us is a bigger "can't book online" number. In each round we flipped every ambiguous hand verdict to the other side and recomputed:

  • Campania: 5 verdicts flipped, the headline moves from 83.4% to 81.3%.
  • Florida: 4 verdicts flipped, it moves from 55.2% to 57.1%.

Both shifts sit well inside confidence intervals that are 11-12 points wide.

The three results

Round Businesses Can't be booked online Data
Spain, five tourist hubs (Sep 2026) 1,759 68.9% doi:10.5281/zenodo.23789643
Campania, Italy (Oct 2026) 891 83.4% doi:10.5281/zenodo.24430539
Florida, 14 markets (Nov 2026) 1,358 55.2% doi:10.5281/zenodo.24422633

Florida does better mostly because of its captained tours, which sell seats through ticketing tools. Its independent car rentals look a lot like Spain's.

The write-ups with the full method, the fleet and town cuts and the CSVs: Spain Campania Florida

If you measure "digital adoption" in small businesses

  • Never publish a crawl-only number. It is wrong in a known direction.
  • Don't stop at the first browser pass either. Read the last screen, not the button.
  • Treat "dead" as a claim to re-test with an honest client before you count it.
  • Keep a written rule, judge the leftovers by hand, then flip the hand verdicts and see how far the headline moves.
  • Publish the strata, so anyone can redo the weighting.

I run Renviq, rental software for small car, scooter and boat rental businesses, which is why we care about this number and why we publish the whole method rather than just the headline.

Read on DEV Community ↗ ← Back to News

Comments

No comments yet. Start the discussion.