Definition
OAI-SearchBot is the user-agent OpenAI uses to find and index web pages for its search features, including ChatGPT Search. OpenAI documents it separately from GPTBot, which collects content that may be used to train models. Site owners can control OAI-SearchBot through robots.txt rules that name it.
For example, Aarvi Skin might block GPTBot because it does not want its content used for training, while allowing OAI-SearchBot so its product pages can still appear as sources in ChatGPT Search. The two decisions are independent, and each needs its own robots.txt rule.
Why it matters
OAI-SearchBot is the crawler most directly linked to whether your pages can appear in ChatGPT Search results. Blocking it, deliberately or through a broad bot rule or firewall setting, can limit how your site shows up there. Because it is often confused with GPTBot, it is worth reviewing both by name and confirming your rules match your intent. Server logs show whether it is actually reaching your pages.
What to do
- Check robots.txt for rules naming OAI-SearchBot and for wildcard rules that also cover it.
- Confirm your CDN or firewall does not block OpenAI's published IP ranges for this crawler.
- Review logs to see which pages OAI-SearchBot requests and whether they return errors.
Common questions
Should I allow OAI-SearchBot?
If you want your pages to be eligible to appear as sources in ChatGPT Search results, OpenAI's documentation indicates OAI-SearchBot should be allowed. Allowing it does not guarantee your pages will be cited. It is a separate decision from whether to allow GPTBot for training.
How do I block GPTBot but allow OAI-SearchBot?
Add separate groups to robots.txt: one with User-agent: GPTBot and Disallow: /, and another with User-agent: OAI-SearchBot and Allow: /. Check that no firewall or wildcard rule overrides this, and confirm the result in your server logs after the change.