Glossary / AI engines and crawlers
Claude-SearchBot
What is Claude-SearchBot?
Last updated
In practice
Why it matters for AI visibility
Claude-SearchBot and Claude-User are the pair that determine whether Claude can find you and represent you correctly at answer time, which makes them the two Anthropic bots a customer-seeking business should be most careful never to block. Anthropic's own documentation spells out the cost in both directions: block the search bot and your visibility drops; block it and the accuracy of what Claude says about you can degrade too, because the engine falls back on older or second-hand sources. That accuracy angle is easy to underestimate: being described wrongly, old prices, discontinued services, a predecessor company's reputation, is often worse than being absent. Both bots are controlled by their own user agent tokens in robots.txt, so auditing your file takes minutes: confirm ClaudeBot reflects your training preference, and confirm Claude-SearchBot and Claude-User are allowed unless you have a genuinely good reason otherwise.
Keep reading
Related terms
Quick tips that use this
Sources
- Anthropic: does Anthropic crawl data from the web? (accessed 7 August 2026)
What's your AEO score?
Send your domain and a named expert will run it through all six engines, score the results and email your report within 1 to 2 working days. Free, no card, no signup.
- ChatGPT
- Claude
- Gemini
- Perplexity
- Google AI Overviews
- Google AI Mode