Do AI crawlers render JavaScript? Our site is React-based and I'm worried

Short answer: AI crawlers mostly don’t render JavaScript well. Here’s the breakdown.

Crawler JavaScript capabilities:

The practical reality:

If your content requires JavaScript to render:

• It’s likely invisible to most AI crawlers
• You won’t be cited in ChatGPT responses
• Perplexity might get some content, inconsistently
• You’re losing AI visibility

The solution hierarchy:

Best: Server-Side Rendering (SSR)

• Next.js with getServerSideProps
• Nuxt.js in SSR mode
• Content in initial HTML response

Good: Static Site Generation (SSG)

• Pre-rendered HTML for all pages
• Build-time generation
• Works for content that doesn’t change often

Acceptable: Pre-rendering services

• Prerender.io, similar services
• Detects bot, serves pre-rendered HTML
• Additional complexity and cost

Not recommended for AI visibility:

• Pure client-side rendering
• Content loading via API after page load
• Dynamic content without fallback

Your situation:

Full React SPA = likely invisible to AI. SSR migration is probably necessary for AI visibility.

We went through this migration. Here’s what we learned.

Our setup before:

• Create React App (CRA)
• All content client-rendered
• API-driven content loading

Migration to Next.js:

Timeline: 6 weeks for 50 pages

Key steps:

1. Set up Next.js project
2. Migrate components (mostly worked as-is)
3. Implement getServerSideProps for data fetching
4. Update routing to Next.js conventions
5. Test with JS disabled
6. Deploy and verify

Challenges:

• Data fetching patterns changed significantly
• Some client-only libraries needed alternatives
• Build times increased (SSR has overhead)
• Had to rethink caching strategy

Results:

AI visibility:

• Before: 5% citation rate for our topics
• After: 28% citation rate
• Perplexity started citing us consistently

SEO:

• Time to first meaningful paint improved
• Google rankings improved slightly
• Core Web Vitals better

Worth it?

Absolutely. The migration effort paid off in 3 months based on improved visibility.

How to verify what AI crawlers actually see.

Testing methods:

Method 1: Disable JavaScript

In browser DevTools:

• Settings → Preferences → Disable JavaScript
• View your page
• What you see = what most AI crawlers see

Method 2: Curl/Wget

curl https://yoursite.com/page

This fetches raw HTML. If your content isn’t there, AI crawlers won’t see it.

Method 3: Check server logs

Look for requests from:

• GPTBot
• ClaudeBot
• PerplexityBot

Check response codes. 200 with empty content body = problem.

Method 4: Google Search Console

Use “View rendered page” feature. While this is Google (which renders JS), it shows what crawlers ideally should see.

Method 5: Monitor AI visibility

Use Am I Cited to track whether you’re being cited. If you’re invisible despite good content, JS rendering is likely the issue.

The quick test:

If your main content isn’t visible in curl output, you have a problem.

Next.js implementation specifics for AI visibility.

The key patterns:

For content pages:

export async function getServerSideProps() {
  const data = await fetchContent();
  return { props: { data } };
}

Content is fetched server-side, included in initial HTML.

For static content:

export async function getStaticProps() {
  const data = await fetchContent();
  return {
    props: { data },
    revalidate: 3600 // ISR, rebuild hourly
  };
}

Even better - pre-rendered at build time.

Common mistakes:

1. Using useEffect for critical content

// BAD - content only loads client-side
useEffect(() => {
  fetch('/api/content').then(setContent);
}, []);

2. Lazy loading main content

// BAD for AI - content loads after initial render
const Content = lazy(() => import('./Content'));

3. Missing fallback in dynamic routes

// GOOD - provides fallback for not-yet-generated pages
export async function getStaticPaths() {
  return { paths: [...], fallback: 'blocking' };
}

The golden rule:

If content is important for AI visibility, it must be in the initial HTML response. No exceptions.

Nuxt.js perspective for Vue users.

Same principles apply:

SSR mode (default in Nuxt 3):

// nuxt.config.ts
export default defineNuxtConfig({
  ssr: true
})

Data fetching with useAsyncData:

const { data } = await useAsyncData('content',
  () => $fetch('/api/content')
);

Runs on server, content in initial HTML.

Static generation:

npx nuxi generate

Pre-renders all pages to static HTML.

Nuxt advantages:

• SSR by default
• Hybrid mode (some pages static, some SSR)
• Good DX for migration from Vue SPA

The verification:

Same tests apply - disable JS, check if content appears.

For Vue SPAs: Nuxt migration is your path to AI visibility.

Performance considerations for SSR.

The trade-offs:

SSR adds server load:

• Each request renders the page
• More CPU usage
• Need proper caching

Mitigation strategies:

CDN with edge caching:

Cache-Control: public, max-age=3600, stale-while-revalidate=86400

Cache rendered HTML for bots and users alike.

Incremental Static Regeneration (ISR):

Best of both worlds:

• Static pages for speed
• Background regeneration for freshness
• Works great for content sites

Edge rendering:

Vercel Edge Functions, Cloudflare Workers:

• Render at the edge
• Lower latency
• Closer to users and bots

The AI bot consideration:

AI crawlers don’t need personalized content. You can cache aggressively for them:

• Detect bot user agent
• Serve cached HTML
• Fresh enough for visibility

Performance + AI visibility is achievable:

SSR doesn’t mean slow. With proper caching, you get AI visibility AND good performance.

CMS architecture for AI visibility.

The headless challenge:

Many headless setups:

• CMS stores content
• Frontend fetches via API
• Content loads client-side

This is invisible to AI crawlers.

The solution architecture:

CMS → Build/SSR Layer → CDN → Users/Bots
         ↓
    Pre-rendered HTML

Implementation options:

Static generation at build:

• Pull from CMS at build time
• Generate static HTML
• Trigger rebuild on content change

SSR with caching:

• Fetch from CMS on request
• Render server-side
• Cache at CDN

Common CMS patterns:

Contentful/Sanity + Next.js:

export async function getStaticProps() {
  const content = await cmsClient.getContent();
  return { props: { content }, revalidate: 60 };
}

WordPress + Gatsby:

• Pull at build time
• Static site generation
• Webhook rebuilds on publish

The key:

Content must get from CMS to HTML somehow before the page reaches AI crawlers.

This thread answered all my questions.

What I learned:

1. AI crawlers don’t render JS - Our SPA is invisible to them
2. SSR is the solution - Next.js migration is the path forward
3. Testing is easy - Disable JS, curl the page, check logs
4. Migration is feasible - 6-week timeline seems realistic
5. Performance is manageable - Caching and ISR address concerns

Our plan:

1. Test current state - Confirm AI visibility issue with curl
2. Proposal to team - Present Next.js migration case
3. Start with critical pages - Blog, product pages first
4. Verify AI visibility - Monitor with Am I Cited post-migration
5. Complete migration - Roll out to full site

The business case:

We’re invisible to 70%+ of Americans using AI search. That’s worth a 6-week migration effort.

Thanks for the technical depth!