User agent

Exactness is the whole game with user agents. A robots.txt rule for GPT-Bot matches nothing, a log search for chatgpt misses ChatGPT-User's actual token, and a well-intentioned policy fails silently in both directions. The tokens also encode the distinctions that matter commercially: OpenAI alone operates GPTBot, OAI-SearchBot and ChatGPT-User for training, search and user actions respectively, and only by handling each token separately can your rules say yes to visibility and no to training. There is a trust caveat: user agents are self-declared, and scrapers routinely wear respectable bots' strings to slip through allowlists. The vendors answer with verification: OpenAI, Anthropic and others publish the IP ranges their bots genuinely use, so traffic claiming to be GPTBot can be checked against openai.com/gptbot.json before you trust or bill for it. Keep a current list of the tokens that matter to you, and audit both robots.txt and firewall rules against the spellings the vendors document.