VibeSecurity

Free tool, no sign-in

Can Google see my site?

Apps built with Lovable, Bolt, v0 or a plain React setup often send an almost empty page and fill it in with JavaScript. Paste a URL to see what a crawler gets in the first response next to what the page looks like after JavaScript runs, plus noindex, robots.txt and canonical problems.

We fetch the page with our own scanner user agent, not as Googlebot, so a site that treats Googlebot differently may show Google something else. For the authoritative answer, use URL Inspection in Google Search Console. Only check sites you are allowed to test.

Paste a URL above to compare what a crawler gets in the first response with what the page looks like after JavaScript runs.

Raw HTML versus rendered HTML

When a crawler requests your page, the server answers with HTML. That first response is the raw HTML. If your app is a single-page app, the raw HTML is often a near-empty shell with a script tag, and the real text only appears once the browser runs your JavaScript. That second version is the rendered HTML.

Google can run JavaScript. Its own documentation describes crawling, rendering and indexing as separate phases: pages are queued for rendering, and the rendered HTML is processed after that. Content that needs JavaScript can therefore take longer to be picked up than content that is in the raw HTML from the start.

Many other crawlers generally read only the raw HTML. Vercel's December 2024 analysis of traffic on its network found that none of the major AI crawlers it measured, including GPTBot, ClaudeBot and PerplexityBot, executed JavaScript. The bots that build link previews in chat and social apps generally read only the raw HTML too. If your content is not in the raw HTML, they may see an almost empty page.

What we check

CheckWhy it matters
HTTP statusPages that answer with an error or redirect status are not indexed as normal content.
noindex in a robots meta tag or X-Robots-Tag headerTells search engines to keep the page out of results. Google may skip rendering when it sees noindex in the raw HTML.
robots.txt rules for GooglebotWe read the site's robots.txt and apply the group for Googlebot, or the * group if there is none, using the longest matching Allow or Disallow rule, with * and $ wildcards.
Visible words, raw versus renderedA tiny word count in the raw HTML and a much larger one after rendering means your content depends on JavaScript.
Title, meta description and canonicalMissing values, or values that change after JavaScript runs, send crawlers mixed signals.
Canonical on a different hostPoints search engines at another site as the main version of this page.
H1s, links, images without alt, JSON-LD blocksShown side by side so you can see what only exists after rendering.

How to fix a page that only works with JavaScript

  1. 1Render important pages on the server or at build time. In Next.js, pages in the App Router are server components by default, so fetch data on the server instead of in a useEffect.
  2. 2For a Vite or plain React single-page app, pre-render your public pages (landing, pricing, blog, docs) to static HTML at build time, or move them to a framework that supports server rendering.
  3. 3Put the title, meta description and canonical link in the server response, not only in client-side code.
  4. 4Keep noindex out of the raw HTML of pages you want indexed. Do not rely on JavaScript to remove it.
  5. 5Re-run this check, then request indexing with URL Inspection in Search Console.

Check robots.txt from a terminal

You can read the same files we read. Only run this against a site you own or are allowed to test.

Terminal
curl -s https://your-app.example/robots.txt
curl -sI https://your-app.example/ | grep -i x-robots-tag

Frequently asked questions

Does this tool fetch my page as Googlebot?

No. We fetch and render with our own scanner user agent. Sites that treat Googlebot differently, or block unknown bots, can look different to Google. URL Inspection in Google Search Console shows what Google itself fetched.

Google renders JavaScript, so why does raw HTML matter?

Google does render JavaScript, but as a separate step after crawling, so JavaScript-only content can take longer to be processed. Many other crawlers only see the raw HTML. Vercel's 2024 analysis found that the major AI crawlers it measured, such as GPTBot, ClaudeBot and PerplexityBot, did not execute JavaScript, and link preview bots generally read only the raw HTML too.

What counts as visible words?

We parse the HTML in your browser, drop script, style, noscript and template elements, and count the words in the remaining text of the body. It is a rough measure of how much readable content each version has, not a copy of how Google measures content.

Why does the rendered side take so long?

We load the page in a real headless browser and wait for network activity to settle, up to 15 seconds. That usually takes 5 to 25 seconds. The raw HTML results appear as soon as they are ready.

Is the page's HTML run in my browser?

No. We parse the returned HTML with the browser's DOMParser, which does not run scripts, and show values as plain text only. Nothing from the checked page is injected into this page.

Can I check any site?

Only public web addresses. Private, local and internal hosts are blocked, site owners can opt out of our scanner, and there is a per-visitor rate limit. Only check sites you own or are allowed to test.

Sources

  1. 1.Google Search Central: Understand JavaScript SEO basics
  2. 2.Vercel: The rise of the AI crawler
  3. 3.Google Search Central: How Google interprets the robots.txt specification
  4. 4.Google Search Central: Robots meta tag and X-Robots-Tag
  5. 5.Search Console Help: URL Inspection tool
  6. 6.RFC 9309: Robots Exclusion Protocol