JavaScript Rendering and AI Crawlers: What You Need to Know

Key Takeaways

Most AI crawlers do not execute JavaScript — OAI-SearchBot, PerplexityBot, ChatGPT-User, and ClaudeBot see only raw HTML, so JS-dependent content is invisible to them
Googlebot is the exception — it renders JavaScript using headless Chromium, but with a delay that can slow indexing and AI visibility via Gemini
If your page shows <div id="root"></div> in view-source, your content is completely invisible to most AI crawlers
SSR (Server-Side Rendering) and ISR (Incremental Static Regeneration) are the safest approaches for AI visibility — they deliver complete HTML to every crawler
A simple view-source test (Ctrl+U / Cmd+U) tells you immediately whether AI crawlers can see your content

Is your JavaScript hiding content from AI? Run a free AI visibility scan — we check what AI crawlers actually see when they visit your site.

The JavaScript Problem for AI Crawlers
Which AI Crawlers Execute JavaScript?
CSR vs SSR vs ISR: A Comparison
Why Single-Page Applications Are Problematic
How to Test What AI Crawlers See
Making Content Accessible Without JavaScript
Framework-Specific Solutions
FAQ

The JavaScript Problem for AI Crawlers

Modern websites rely heavily on JavaScript. Frameworks like React, Vue, and Angular build entire user interfaces in the browser — the server sends a minimal HTML shell, and JavaScript constructs the visible content after the page loads. For human visitors with a browser, this works perfectly. For AI crawlers, it is often a disaster.

When an AI crawler requests your page, it receives the raw HTML that your server sends. If that HTML contains your actual content — headings, paragraphs, product descriptions, FAQ answers — the crawler can read and process everything. But if the HTML contains only a JavaScript bundle and an empty <div>, the crawler sees nothing. No content. No structure. No information to index or cite.

This is not a theoretical problem. A significant portion of modern websites — particularly those built with React SPAs, Vue SPAs, or Angular — deliver content exclusively through client-side JavaScript. These sites may look beautiful and function perfectly for visitors, but they are invisible to most AI crawlers. Understanding what AI SEO is starts with understanding this fundamental access problem.

Which AI Crawlers Execute JavaScript?

Not all crawlers handle JavaScript the same way. Here is what we know about the major crawlers as of early 2026:

| Crawler | Operator | Executes JavaScript? | Notes | |---|---|---|---| | Googlebot | Google | Yes (headless Chromium) | Renders JS but with a queue delay; pages may wait seconds to days | | Bingbot | Microsoft | Limited | Can render some JS; inconsistent execution | | OAI-SearchBot | OpenAI | No | Fetches raw HTML only | | ChatGPT-User | OpenAI | No | Fetches raw HTML when browsing | | PerplexityBot | Perplexity | No | Fetches raw HTML only | | ClaudeBot | Anthropic | No | Fetches raw HTML only | | Applebot | Apple | Limited | Renders some JS; used for Siri and Safari suggestions |

The pattern is clear: the vast majority of AI crawlers do not execute JavaScript. Googlebot is the only major crawler with full JavaScript rendering, and even it incurs a delay. If your content requires JavaScript to appear in the DOM, you are invisible to ChatGPT, Perplexity, Claude, and potentially Copilot.

This is why your robots.txt configuration alone is not enough. Even if you allow all AI crawlers in robots.txt, they will see nothing if your content is rendered client-side.

The Googlebot rendering queue

Google uses a two-phase indexing process for JavaScript-dependent pages:

Phase 1: Crawl — Googlebot fetches the raw HTML and indexes whatever it finds
Phase 2: Render — The page enters a rendering queue where headless Chromium executes the JavaScript, and Google re-indexes the fully rendered content

The time between Phase 1 and Phase 2 varies. For high-authority sites, it may be seconds. For smaller sites, it can be days. During this gap, your page is either not indexed or indexed with incomplete content — which means Google Gemini and AI Mode cannot cite it properly.

CSR vs SSR vs ISR: A Comparison

The way your application renders content determines whether AI crawlers can see it. Here are the three main rendering strategies:

Client-Side Rendering (CSR)

The server sends a minimal HTML document with a JavaScript bundle. The browser downloads and executes the JavaScript, which then builds the entire page in the browser.

What the server sends:

<!DOCTYPE html>
<html>
<head><title>My App</title></head>
<body>
  <div id="root"></div>
  <script src="/bundle.js"></script>
</body>
</html>

AI crawler visibility: None. The crawler sees an empty <div> and a script reference it cannot execute.

Server-Side Rendering (SSR)

The server executes the JavaScript on every request and sends fully rendered HTML to the client. The browser receives a complete page that works even without JavaScript.

What the server sends:

<!DOCTYPE html>
<html>
<head><title>My App</title></head>
<body>
  <div id="root">
    <h1>Product Name</h1>
    <p>Full product description with all details...</p>
    <div class="faq">...</div>
  </div>
  <script src="/bundle.js"></script>
</body>
</html>

AI crawler visibility: Full. Every crawler sees the complete content immediately.

Incremental Static Regeneration (ISR)

Pages are pre-built as static HTML at build time. When a request comes in, the server serves the cached static page. In the background, pages are regenerated at configurable intervals to stay fresh.

AI crawler visibility: Full. Static HTML is always available, and content stays reasonably current through periodic regeneration.

Comparison table

| Factor | CSR | SSR | ISR | |---|---|---|---| | AI crawler visibility | None | Full | Full | | Initial page load | Slow (JS must download + execute) | Fast (HTML ready) | Fastest (cached static HTML) | | Server load | Low (client does the work) | High (renders every request) | Low (serves cached pages) | | Content freshness | Real-time | Real-time | Near real-time (regeneration interval) | | SEO compatibility | Poor | Excellent | Excellent | | AI SEO compatibility | Poor | Excellent | Excellent | | Complexity | Simple | Moderate | Moderate |

The recommendation is clear: If you care about AI visibility, use SSR or ISR. CSR should only be used for authenticated dashboard-type applications where search indexing is irrelevant.

Why Single-Page Applications Are Problematic

Single-Page Applications (SPAs) are a specific type of CSR architecture where the entire website exists as one HTML page. Navigation between "pages" happens entirely through JavaScript — the URL changes, the content swaps, but no new HTML document is ever loaded from the server.

SPAs create several compounding problems for AI crawlers:

1. Initial content is invisible

As with any CSR application, the first request returns an empty shell. AI crawlers that do not execute JavaScript see no content on any "page" of your site.

2. Internal links may not exist in HTML

In SPAs, navigation often uses JavaScript event handlers rather than standard <a href="..."> tags. Crawlers follow <a> tags to discover pages. If your navigation is JavaScript-driven, crawlers cannot discover your pages — even if they could render them.

3. URL routing is client-side

SPAs use hash-based routing (example.com/#/about) or the History API (example.com/about) for navigation. Hash-based URLs are particularly problematic because crawlers treat everything after the # as a fragment identifier, not a separate page. Your entire site may appear as a single URL.

4. Meta tags and structured data load dynamically

If your <title>, <meta description>, and JSON-LD schema markup are injected by JavaScript, crawlers that cannot render JS will see only your default (usually generic) metadata. This severely impacts how AI models categorize and understand your content. Proper semantic HTML structure must be present in the initial HTML response.

5. Content loading depends on API calls

SPAs typically fetch content from APIs after the page loads. If the API is slow, blocked, or rate-limited, even JavaScript-executing crawlers may not see your content before their timeout expires.

How to Test What AI Crawlers See

You do not need specialized tools to check whether AI crawlers can read your content. Here are four methods, from simplest to most thorough:

Method 1: View Page Source (fastest)

Open your page in Chrome, Firefox, or any browser
Press Ctrl+U (Windows/Linux) or Cmd+U (Mac) to view the page source
Search for your main content text (Ctrl+F a distinctive phrase from your page)

If your content appears in the source, AI crawlers can see it. If the source shows only <script> tags and empty container <div> elements, your content is JavaScript-dependent and invisible to most AI crawlers.

Important: "View Page Source" shows the raw HTML the server sends. "Inspect Element" (right-click > Inspect) shows the rendered DOM after JavaScript execution. For testing AI crawler visibility, always use "View Page Source."

Method 2: Disable JavaScript in browser

Open Chrome DevTools (F12)
Press Ctrl+Shift+P (Cmd+Shift+P on Mac) to open the command palette
Type "Disable JavaScript" and select it
Reload the page

What you see is approximately what non-rendering AI crawlers see. If the page is blank or shows only a loading spinner, you have a problem.

Method 3: curl from the command line

curl -s https://example.com/your-page | head -100

This fetches the raw HTML without executing any JavaScript — exactly what AI crawlers receive. Check whether your main content appears in the output.

Method 4: Google Search Console URL Inspection

Enter your URL in the URL Inspection tool
Click "Test Live URL"
Compare "HTML" tab (what Google sees before rendering) with "Rendered HTML" tab (after JavaScript execution)

If content only appears in the Rendered HTML tab, it requires JavaScript and is invisible to non-rendering AI crawlers.

Making Content Accessible Without JavaScript

If your site currently relies on client-side rendering, here are the approaches to make your content visible to AI crawlers, ordered from most to least recommended:

Option 1: Switch to SSR or ISR (best)

If you are using a JavaScript framework, migrate to a server-side rendering approach:

React — Use Next.js with getServerSideProps (SSR) or getStaticProps (SSG/ISR)
Vue — Use Nuxt with server-side rendering enabled
Angular — Use Angular Universal for SSR
Svelte — Use SvelteKit with SSR (enabled by default)

This is the most comprehensive solution because it makes your content available to all crawlers, improves page load speed, and benefits Core Web Vitals.

Option 2: Static Site Generation (SSG)

For content that does not change frequently (blog posts, documentation, product pages), generate static HTML at build time. Tools like Next.js, Gatsby, Hugo, Astro, and Eleventy produce plain HTML files that every crawler can read.

Option 3: Dynamic rendering (workaround)

Dynamic rendering serves different content to different user agents. When a crawler is detected (via user-agent string), the server returns pre-rendered HTML. When a human browser is detected, the server returns the normal JavaScript-dependent version.

Google has described this as a "workaround, not a long-term solution." It adds complexity, creates maintenance burden, and risks serving inconsistent content. Use it only as a temporary fix while migrating to SSR.

Option 4: Prerendering services

Services like Prerender.io and Rendertron act as middleware. They maintain a cache of pre-rendered pages and serve the cached HTML to crawlers. This works but adds a dependency, introduces latency, and may not always reflect your latest content.

What you must always ensure

Regardless of which approach you choose, verify these elements are present in the initial HTML response (before any JavaScript executes):

Page title in the <title> tag
Meta description in the <meta name="description"> tag
All heading tags (h1 through h6) with actual content
Main body content within semantic HTML elements (, , <div>)
JSON-LD structured data in <script type="application/ld+json"> blocks
Internal links as standard <a href="..."> tags
Image alt text on all meaningful images

Framework-Specific Solutions

Here is a quick reference for enabling server-side rendering in the most popular JavaScript frameworks:

Next.js (React)

Next.js supports SSR, SSG, and ISR out of the box. For AI visibility:

Use getStaticProps for pages with content that changes infrequently (blog posts, documentation)
Use getServerSideProps for pages with real-time data (pricing, inventory)
Enable ISR with revalidate for the best balance of performance and freshness
The App Router (Next.js 13+) uses Server Components by default — content is server-rendered automatically

Nuxt (Vue)

Enable ssr: true in your nuxt.config.ts (this is the default)
Use useAsyncData or useFetch for data fetching — these run on the server
For static content, use nuxt generate to pre-build pages

Angular Universal

Add @nguniversal/express-engine to your project
Configure server-side rendering through angular.json
Note: Angular Universal adds significant complexity; evaluate whether a simpler framework might be better for content-heavy sites

Astro

Astro renders to static HTML by default with zero JavaScript
Use "islands architecture" to add interactivity only where needed
Excellent choice for content-heavy websites, documentation, and blogs

Gatsby (React)

Generates static HTML at build time by default
All content is available without JavaScript
Note: Gatsby's build times can be slow for large sites; consider Next.js ISR as an alternative

For a comprehensive technical setup review, see our AI SEO Checklist for 2026.

Frequently Asked Questions

Can AI crawlers execute JavaScript?

Most AI crawlers cannot execute JavaScript. OAI-SearchBot, PerplexityBot, ChatGPT-User, and ClaudeBot fetch raw HTML without running JavaScript. Googlebot is the notable exception — it uses a headless Chromium renderer. If your content depends on JavaScript to appear in the DOM, most AI crawlers will see an empty page.

What is the difference between CSR, SSR, and ISR?

CSR (Client-Side Rendering) builds the page entirely in the browser using JavaScript — crawlers that cannot execute JS see an empty shell. SSR (Server-Side Rendering) generates the full HTML on the server for every request — all crawlers see complete content. ISR (Incremental Static Regeneration) pre-builds pages as static HTML and regenerates them periodically. For AI visibility, SSR and ISR are strongly preferred over CSR.

How do I test if AI crawlers can see my content?

The simplest test: press Ctrl+U (Cmd+U on Mac) to view your page source. If your main content appears in the raw HTML, crawlers can see it. If you see only script tags and an empty <div id="root"></div>, your content requires JavaScript and is invisible to most AI crawlers. You can also disable JavaScript in Chrome DevTools and reload the page.

Does using React or Vue automatically mean AI crawlers cannot see my content?

Not necessarily. React and Vue can be configured for server-side rendering. Frameworks like Next.js (React) and Nuxt (Vue) support SSR by default. The problem only occurs with pure client-side rendering, where the framework runs entirely in the browser. If you use Next.js with SSR or SSG, your React content is fully visible to AI crawlers.

Will Google's rendering queue delay my AI visibility?

Yes. JavaScript-dependent pages enter a rendering queue that can take seconds to days. During this delay, the page may not appear fully in Google's index, which means Gemini and AI Mode cannot cite it properly. Server-side rendered pages bypass this queue entirely and are indexed immediately.

Find out what AI crawlers really see on your site

Our free scan checks JavaScript rendering, crawler access, schema markup, and technical AI readiness across all major platforms.

Scan My Website

Trusted by 2,400+ websites -- No credit card required