Getting cited by ChatGPT starts with the one lever OpenAI documents: allow OAI-SearchBot in robots.txt, because sites that opt out of it will not be shown in ChatGPT search answers. Everything past that is earned the same way other AI citations are earned — liftable passages, attributable entities, corroborated claims. This guide covers OpenAI’s three crawlers (they do different jobs), the access checklist, and the content work that converts access into citations.
OpenAI’s Three Crawlers
OpenAI documents three user agents with independent purposes: OAI-SearchBot surfaces sites in ChatGPT search results, GPTBot crawls content for model training, and ChatGPT-User fetches pages when a user’s request triggers a visit.
The settings are explicitly independent — OpenAI’s documentation gives the exact example of allowing OAI-SearchBot to appear in search while disallowing GPTBot to keep content out of training. ChatGPT-User is different again: it acts on user requests, so robots.txt rules may not apply to it, and OpenAI notes it is not used to decide search inclusion. The blanket "block all AI bots" advice some publishers followed in 2023–2024 therefore has a real cost today: blocking OAI-SearchBot removes you from an answer surface, without any training-related benefit that GPTBot rules already provide.
The Access Checklist
Four checks: robots.txt allows OAI-SearchBot, your CDN/bot protection does not block OpenAI’s published IP ranges, the content renders as text, and — after changes — you allow the ~24 hours OpenAI says its systems take to reflect robots.txt updates.
Our robots.txt explicitly welcomes AI search crawlers — the reasoning and syntax are in the llms.txt vs robots.txt vs sitemap guide. CDN bot protection is the silent killer here: many "we allowed the bot" sites are still serving OpenAI’s crawlers a challenge page. OpenAI publishes IP ranges for each agent (searchbot.json, gptbot.json, chatgpt-user.json) so you can verify real crawler traffic in your logs. The free AI Crawler Access Auditor checks your directives for exactly these misconfigurations.
Earning the Citation
Access makes you eligible; citations go to pages whose passages answer the question cleanly and whose source is worth naming — the same answer-first structure and entity clarity that win Google’s AI surfaces.
ChatGPT search composes an answer and attaches sources, so the passage mechanics from content formatting for AI extraction transfer directly: self-contained answers, real lists and tables, descriptive headings. So does the trust layer — a consistent entity, named authorship, and original information something a synthesized answer needs to reference rather than restate. Freshness earns weight for time-sensitive queries, and being the corroborated source for a specific claim — the long game of AI citation optimization — is what turns occasional mentions into a recurring presence.
Verifying and Tracking
Confirm crawler visits in your server logs against OpenAI’s published IPs, then track actual citations by querying ChatGPT on your priority topics and logging results — plus watching referral traffic from chatgpt.com in analytics.
ChatGPT citations are session-variable like every generative surface, so trends beat snapshots: run your 10–30 priority queries monthly, log whether you are cited and for which page, and watch the chatgpt.com referral segment in GA4 for the clicks citations send. The AI Citation Tracker structures the check across engines, and the same monthly loop from the AI Overviews tracking workflow extends naturally to ChatGPT.
- Allow OAI-SearchBot — OpenAI states opted-out sites are not shown in ChatGPT search answers.
- The three crawlers are independent: SearchBot = search visibility, GPTBot = training, ChatGPT-User = user-triggered fetches.
- Check CDN/bot protection against OpenAI’s published IP ranges — robots.txt alone proves nothing.
- Citations are earned by liftable passages + attributable entities — the same work as Google’s AI surfaces.
- Track monthly: logged citation checks + chatgpt.com referrals in analytics.
Frequently Asked Questions
Can I be in ChatGPT search but keep my content out of AI training?
Yes — that is OpenAI’s own documented example: allow OAI-SearchBot (search visibility) while disallowing GPTBot (training). The settings are independent, and OpenAI notes that when both are allowed it may reuse one crawl for both purposes.
How long until robots.txt changes affect ChatGPT search?
OpenAI states it can take about 24 hours from a robots.txt update for its systems to adjust. Citation changes follow from there as content is recrawled — verify actual crawler visits in your logs rather than assuming.
Why is ChatGPT citing my competitors but not me?
Run the same diagnosis as for AI Overviews: access first (OAI-SearchBot allowed, CDN not blocking), then passage quality (is there a self-contained answer?), then attribution (is your entity identifiable and your content original?). The workflow in our AI Overviews citation-diagnosis guide applies almost unchanged.
Does llms.txt help me get cited by ChatGPT?
OpenAI does not document llms.txt as a factor in ChatGPT search — robots.txt and OAI-SearchBot govern inclusion. llms.txt remains a low-cost, speculative courtesy file for AI systems generally; the full status is in our llms.txt guide.
The Bottom Line
ChatGPT citation strategy is refreshingly concrete at the access layer — one documented bot to allow, published IPs to verify, a ~24-hour propagation window — and familiar at the content layer: the answer-first, entity-clear work this whole GEO/AEO cluster teaches. Open the door, confirm the crawler actually gets through, give it passages worth quoting, and measure. Sites doing all four are showing up in a channel their competitors blocked by reflex two years ago.
Further reading & sources
- Overview of OpenAI crawlers (OAI-SearchBot, GPTBot, ChatGPT-User) — OpenAI
- Introduction to robots.txt — Google Search Central
See how your site actually shows up in AI search. An AI visibility audit maps where you’re cited, where you’re invisible, and what to fix first — in plain English.
Get your AI visibility auditTry the free SEO tools →
Prefer self-serve? The interactive checklists turn guides like this one into a working to-do list.
Keep reading in GEO / AEO
Get one email when something genuinely changes
AI search moves fast and most of it is noise. We send one short email when a real shift is worth your time. Unsubscribe anytime.
Published by Plain Intelligence — practical AI SEO, GEO, and technical SEO, documented in plain English. About Plain Intelligence →
↑ Back to GEO / AEO · Explore all articles · Free tools & resources · Glossary