AEOscoreFree AEO score

Glossary / AI engines and crawlers

Googlebot

What is Googlebot?

Googlebot is Google's main search crawler, and in the AI era it has a second job: the content it crawls and indexes for Search is the same content Google's AI Overviews and AI Mode draw on. Google documents that the standard Search controls, noindex, nosnippet and friends, are exactly what govern those AI features too.

Last updated

In practice

Why it matters for AI visibility

Google chose not to create a separate crawler or opt-in for its AI answers: eligibility for AI Overviews rides on ordinary Search indexing, with, in Google's words, no additional technical requirements. That is convenient, because every year of normal SEO hygiene, crawlable pages, sensible snippets, an accurate sitemap, already qualifies you for the AI surfaces. It is also dangerous, because the same controls cut both ways: a nosnippet tag added years ago to protect content from scrapers now silently limits what AI Overviews can quote from you, and a noindex removes you from AI answers along with the results page. The practical work is an audit of old directives with new eyes: every snippet restriction and index exclusion on your site is now also an AI answers decision, and most of them were made before that was true.

Keep reading

Related terms

Go deeper on AEOscore

Sources

What's your AEO score?

Send your domain and a named expert will run it through all six engines, score the results and email your report within 1 to 2 working days. Free, no card, no signup.

  • ChatGPT
  • Claude
  • Gemini
  • Perplexity
  • Google AI Overviews
  • Google AI Mode