Scaling a Static Site to 4,400 Pages Without Breaking Google

Googlebot's 2 MB HTML Limit

With 3,400 hotels on one listing page, the naive approach (render all cards in HTML) produced a 9 MB page. Googlebot truncates at 2 MB and ignores the rest.

The fix: cap the initial HTML at 400 cards. The remaining 2,500+ cards are generated as a separate HTML fragment file at a predictable URL (/data/cards/{slug}-remaining/). A "Load More" button injects 48 cards at a time from the fragment. The first search or filter interaction loads the entire fragment so all cards are available for client-side filtering. This keeps every page under 2 MB for crawlers while giving users access to everything.

Content-Aware Lastmod with Cascading

A site with 4,400 pages can't update every lastmod on every build. Search engines treat that as spam, and IndexNow has rate implications.

Instead, the build hashes each hotel's SEO-relevant fields and compares against a persisted store. Only pages with actual content changes get their lastmod bumped. The interesting part is cascading: when a hotel in Paris changes, the Paris city page, France country page, and Europe region page all get their dates updated too, since their content changed (they list that hotel). Changed URLs feed into IndexNow so only genuinely modified pages get pushed to search engines.

DOM Filtering Breaks on Mobile at Scale

The site started with pure DOM filtering: every card has data-* attributes for region, country, brand, and perks. JavaScript reads attributes and toggles visibility. Zero network requests, instant results. Great on desktop.

On a mid-range phone with 2,500+ cards in the DOM, filtering took 2-3 seconds per interaction. textContent traversal across 20-40 nodes per card means ~60,000 DOM visits per keystroke. Layout thrashing with 10,000+ nodes made every show/hide cycle expensive.

The fixes that actually moved the needle:

• Pre-indexed search: Build a data-search attribute at build time with all searchable text concatenated. One attribute read per card instead of walking child nodes.
• Batch DOM updates in requestAnimationFrame: Prevents intermediate layout recalculations.
• content-visibility: auto on cards: Browser skips layout for off-screen elements entirely.
• Offload to a search API for complex queries: When the user is actively searching (not just filtering), an Edge Runtime proxy handles it server-side and returns only matching results. The DOM path remains the instant fallback for simple filters.

The result: filtering feels instant on mobile again, and the API path handles queries the DOM can't do efficiently.

Incremental Image Pipeline for Expiring URLs

Notion stores images in signed S3 URLs that expire after about an hour. You can't reference these in your HTML.

The prebuild step downloads each hotel's image and uses sharp to generate optimized WebP variants at multiple sizes. Optimized images are stored in Cloudflare R2, so they persist across builds without re-downloading or re-processing. A manifest tracks each hotel's source URL (with S3 signature params stripped) as the cache key. When Notion's cover image changes, the source URL changes and the pipeline re-processes that hotel and uploads to R2. When it doesn't change, the build skips it entirely, even though the signed URL is different on every API call.

This means a typical build processes zero images unless content actually changed. Vercel's build cache has a 1.5 GB limit that gets fully invalidated when exceeded. Moving images to R2 decoupled the image pipeline from deploy infrastructure entirely.

Stack

Astro 7 (static output), Tailwind CSS 3, TypeScript, Notion API, sharp, Cloudflare R2, Vercel (static + Edge Functions). 91-95 PageSpeed mobile. Content refreshed from Notion every 6 hours via scheduled deploy.