AEOscoreFree AEO score

Glossary / AI engines and crawlers

Claude-SearchBot

What is Claude-SearchBot?

Claude-SearchBot is the Anthropic crawler that analyses web content to improve the relevance and accuracy of Claude's search answers. Anthropic warns that blocking it may reduce your site's visibility and accuracy in user search results. Its sibling, Claude-User, fetches pages live when a Claude user's question requires them.

Last updated

In practice

Why it matters for AI visibility

Claude-SearchBot and Claude-User are the pair that determine whether Claude can find you and represent you correctly at answer time, which makes them the two Anthropic bots a customer-seeking business should be most careful never to block. Anthropic's own documentation spells out the cost in both directions: block the search bot and your visibility drops; block it and the accuracy of what Claude says about you can degrade too, because the engine falls back on older or second-hand sources. That accuracy angle is easy to underestimate: being described wrongly, old prices, discontinued services, a predecessor company's reputation, is often worse than being absent. Both bots are controlled by their own user agent tokens in robots.txt, so auditing your file takes minutes: confirm ClaudeBot reflects your training preference, and confirm Claude-SearchBot and Claude-User are allowed unless you have a genuinely good reason otherwise.

Keep reading

Related terms

What's your AEO score?

Send your domain and a named expert will run it through all six engines, score the results and email your report within 1 to 2 working days. Free, no card, no signup.

  • ChatGPT
  • Claude
  • Gemini
  • Perplexity
  • Google AI Overviews
  • Google AI Mode