JavaScript SEO: What Google and AI Crawlers Actually See on Your Site
DEV Community

JavaScript SEO: What Google and AI Crawlers Actually See on Your Site

Originally published on growwithram.in. Quick answer: JavaScript SEO is making sure content built with JavaScript can still be found and read. Google runs JavaScript, usually in a second pass. The major AI crawlers do not run it at all, so a page whose text only appears after JavaScript runs is close to empty for them. The fix is to send the text in the HTML, through server-side rendering or pre-rendering. Here is the uncomfortable version of JavaScript SEO: an early version of my own website was close to invisible to the AI crawlers I was telling clients to care about. The page looked finished in a browser. What it actually sent was an almost empty file and a script that built the page afterwards. Google would eventually run that script. ChatGPT's crawler would not. JavaScript SEO is the work of making sure content built with JavaScript can still be found, read and indexed. This guide explains what Google does with JavaScript, what AI crawlers do instead, what I measured on 50 real sites, and how to check and fix your own in an afternoon. What JavaScript SEO actually means Every web page reaches a visitor in one of two ways, and the difference decides most of JavaScript SEO. - Server-side rendering (SSR) or static HTML. The server sends a finished page. The words, headings and links are already in the HTML file. Anything that reads the file, a person or a bot, gets the content immediately. - Client-side rendering (CSR). The server sends a nearly empty shell, often a single , plus a JavaScript bundle. The visitor's browser runs the bundle, and only then does the page appear. In a browser the two look identical, which is why the problem hides so well. The difference only shows up for a visitor that does not run JavaScript. That visitor sees the shell, and a shell has nothing to rank, quote or cite. React, Vue and Svelte apps built as single-page applications render on the client by default. So do many sites made with AI website builders. Frameworks such as Next.js, Nuxt and Astro can send finished HTML instead, but only if the site is set up that way. The framework name alone tells you nothing: you have to look at what the server actually sends. How Google handles JavaScript: crawl, render, index Google does run JavaScript. Google's own documentation is clear that it processes JavaScript web apps in three main phases: crawling, rendering and indexing. The important detail is the gap between the first two. Googlebot fetches the HTML, then places the page in a queue to be rendered by a headless Chromium browser. In Google's words: The page may stay on this queue for a few seconds, but it can take longer than that. (Google Search Central, JavaScript SEO basics) So a client-rendered page does get indexed by Google, but in two passes, and anything that goes wrong in the second pass (a script error, a blocked file, a slow API call) can leave Google with the empty shell. Google also warns that it won't render JavaScript from blocked files, so a robots.txt rule that blocks your JavaScript or CSS folders quietly blanks your pages for it. Google's own advice, in the same guide, is not to rely on that second pass: not all bots can run JavaScript. (Google Search Central, JavaScript SEO basics) That sentence is the whole argument of this post. Google can cope with JavaScript. Most of the other systems that read your site cannot. Do AI crawlers render JavaScript? No. This is the part most JavaScript SEO advice was written too early to cover. Vercel and MERJ analysed crawler traffic across Vercel's network and published the results in December 2024. Their finding on rendering was unambiguous: none of the major AI crawlers currently render JavaScript. (Vercel and MERJ, The rise of the AI crawler) The list they tested includes OpenAI's GPTBot, OAI-SearchBot and ChatGPT-User, Anthropic's ClaudeBot, and PerplexityBot. They found the crawlers do fetch JavaScript files but do not execute them. Two exceptions are worth knowing: Google's Gemini uses Googlebot's infrastructure, so it renders like Google, and Applebot renders pages through a browser-based crawler. For a client-rendered site, that means the answer engines people increasingly ask instead of searching read the empty shell. If your text is not in the HTML, ChatGPT, Claude and Perplexity have nothing of yours to quote. Two honest caveats. That study is from late 2024, and crawlers change; it is the best published measurement I know of, not a permanent law. And some platforms now pre-render pages only for crawlers they recognise. Lovable's documentation, for example, describes pre-rendering for verified search and AI crawlers on older apps. That helps the crawlers on the list and nobody else: link previews, smaller AI tools and anything new still get the shell. What I measured on 50 AI-built sites To see how common this is, on 20 September 2026 I measured 50 sites taken, in order, from the public showcases of three AI website builders: 20 from Lovable, 17 from Bolt and 13 from Replit. Each homepage was fetched twice: once as a plain request with no JavaScript, the way an AI crawler reads it, and once in a headless browser with JavaScript, the way a person sees it. Then I counted the visible words in each. - 20 of 32 sites depended on JavaScript for most or all of their text - 7 words the median a crawler received from the sites that were invisible to it - 631 words the median a person saw on those same sites, with JavaScript Sites with too little content to judge, or that could not be reached, are left out. 32 of the 50 could be judged. Eighteen of those twenty sent a crawler between 1 and 11 words. The other two showed part of their text. The sites were not broken. In a browser they were full pages, with a median of 631 words. They were simply built to assemble themselves in the browser. The head tags look fine, which is the trap Here is what makes this problem survive so many audits. The parts of the page that SEO tools check first were almost always present, even on the sites that sent no content at all. | In the HTML a crawler receives | Sites (of 38 fetched) | |---|---| | A title tag | 37 | | A meta description | 36 | | A canonical tag | 22 | | An H1 heading | 15 | | An empty root div waiting for JavaScript | 20 | A checklist that looks at titles and meta descriptions passes these sites. The page body is what is missing. Among the 20 JavaScript-dependent sites, only 2 had an H1 in the HTML at all, and 17 shipped the empty root div. The full method, sample rule and raw results are in the 50-site study. What this does not show: 50 homepages from three showcases is a sample that says the problem is common, not how common across the whole web, and inner pages may differ from homepages. My own site had the same problem I found this on my own site before I found it on anyone else's. The first version of one of my pages, a page called /future, was a client-rendered shell. The HTML it sent was 1,984 bytes, and my own name appeared in it exactly twice, both times inside meta tags. Everything a reader saw was built by JavaScript afterwards. For a site whose whole pitch is being readable by search engines and AI, that was a live counter-argument to its own copy. So I had the site rebuilt to publish every page as finished HTML: a build step now turns the site's content into complete pages before anything goes live, and nothing on a page needs JavaScript to exist. The animations and the chat assistant still use JavaScript. The words do not. You can check that claim yourself: open any page on growwithram.in, press Ctrl+U (Cmd+Option+U on a Mac) and search the source for a sentence you can see on the page. It will be there. How to check your own site in five minutes You do not need an SEO tool for the first check. You need the difference between what your server sends and what your browser shows. - View the page source. Open the page, press Ctrl+U, and use Ctrl+F to search for a sentence from the middle of the page. If it is not in the source, a crawler that does not run JavaScript cannot see it. - Look for the shell. If the source is mostly script tags and an empty or , the page is client-rendered. - Check what Google rendered. In Search Console, use URL Inspection, then View crawled page, and read the HTML. That is Google's second pass: it tells you whether Google's rendering worked, not what AI crawlers saw. - Check your robots.txt. Make sure it does not block the folders your JavaScript and CSS load from. Google will not render with files it is not allowed to fetch. - Repeat on one page of every template: homepage, a product or service page, a blog post, a category page. Rendering problems belong to templates, so one page per template covers the site. If you are comfortable in a terminal, one command gives you the crawler's-eye word count: curl -s https://yoursite.com/ | sed -e 's/ ]*>/ /g' | tr -s ' \n' ' ' | wc -w In the study, I counted a page with under 50 words by that measure as invisible to a crawler. How to fix it: server rendering, pre-rendering, or a workaround There are three real options, and one of them is a stopgap. | Approach | What a crawler receives | Use it when | |---|---|---| | Server-side rendering (SSR) | The full page, built on each request | Content changes often or is personalised | | Static generation / pre-rendering | The full page, built ahead of time | Content changes when you publish: most marketing sites and blogs | | Dynamic rendering | A pre-rendered copy, served only to recognised bots | Only as a temporary bridge while you migrate | For most business websites, pre-rendering is the simplest answer: the pages rarely change between publishes, so build them once and serve finished HTML to everyone. Next.js, Nuxt, Astro and similar frameworks all support it. On an AI builder, check what the platform offers now: Lovable's documentation, for one, says new apps have used server-side rendering since 13 May 2026. Google is direct about the third option: D

Read on DEV Community ↗ ← Back to News

Comments

No comments yet. Start the discussion.