Question
What is GPTBot?
GPTBot is OpenAI's crawler that collects public web content to train future models. It honors robots.txt and is separate from the crawlers that power ChatGPT search (OAI-SearchBot) and live page fetches (ChatGPT-User).
Key facts
- User-agent token:
GPTBot. - Purpose: model training data collection, not search indexing.
- Honors robots.txt
Disallowrules. - Blocking it does not remove you from ChatGPT search results.
Block GPTBot only
User-agent: GPTBot
Disallow: /
Related pages: should I allow GPTBot, what is OAI-SearchBot, OpenAI crawler guide.