Glossary / AI engines and crawlers
ClaudeBot
What is ClaudeBot?
Last updated
In practice
Why it matters for AI visibility
The same training-versus-search distinction that applies to OpenAI applies here, and Anthropic is unusually explicit about it: three named bots, three separate robots.txt decisions. Blocking ClaudeBot is a training opt-out only; it does not remove you from Claude's live answers, but what ClaudeBot reads today is a vote on what future Claude models will know about you by default. Claude's user base skews professional and technical, so B2B businesses in particular should think before excluding themselves from its training diet. Anthropic supports the Crawl-delay directive if server load is the worry, which is a gentler tool than a block, and publishes its crawler IPs at claude.com/crawling/bots.json for log verification. As with every AI bot decision, make it deliberately: the default posture many security tools ship, block everything unfamiliar, was not designed with your AI visibility in mind.
Keep reading
Related terms
Quick tips that use this
Sources
- Anthropic: does Anthropic crawl data from the web? (accessed 7 August 2026)
What's your AEO score?
Send your domain and a named expert will run it through all six engines, score the results and email your report within 1 to 2 working days. Free, no card, no signup.
- ChatGPT
- Claude
- Gemini
- Perplexity
- Google AI Overviews
- Google AI Mode