Definition
GPTBot is the user-agent OpenAI uses for crawling web content that may be used to train its generative AI models. OpenAI documents it publicly, publishes the IP ranges it uses and states that site owners can control its access through robots.txt. It is one of several OpenAI crawlers, each with a different purpose.
For example, Bizbloom might decide it does not want its documentation used for model training and add a Disallow rule for GPTBot. That decision is separate from ChatGPT Search, which uses OAI-SearchBot, so Bizbloom can block one crawler and still allow the other.
Why it matters
GPTBot matters because blocking it is often confused with blocking ChatGPT entirely. It is a training decision: it affects whether future models may learn from your pages, not whether ChatGPT Search can find and cite them today. Brands should decide deliberately, weighing content protection against the chance that future models know less about them. Whatever you choose, check that your rules match your intent and that other crawlers are not caught by accident.
What to do
- Check robots.txt for rules naming GPTBot, and for wildcard rules that also affect it.
- Decide on training access separately from search access, and document the reason.
- Verify GPTBot requests in your logs against OpenAI's published IP ranges.
Common questions
Should I block GPTBot?
It depends on your priorities. Blocking GPTBot tells OpenAI not to use your pages for training future models, which some publishers prefer for content protection. Allowing it may help future models know more about your brand. Either way, it does not control whether ChatGPT Search can cite your pages.
What is the difference between GPTBot and OAI-SearchBot?
Both are OpenAI crawlers, with different purposes. GPTBot collects content that may be used to train models. OAI-SearchBot is used for search features, including ChatGPT Search results. They are controlled separately in robots.txt, so you can allow one and block the other.