What is llms.txt?
A proposed markdown file at /llms.txt that curates a site's key content for AI systems. Widely adopted, rarely fetched — know both facts.
Definition
llms.txt is a proposed standard (Jeremy Howard, 2024): a markdown file at your domain root listing your site's purpose and most important pages, so AI systems can find the good stuff without crawling everything. A companion, llms-full.txt, inlines full content.
Adoption is real — 8.7% of the top 1,000 sites publish one (Rankability, June 2026); ~10% across 300k domains (SE Ranking) — but consumption isn't: Ahrefs found 97% of llms.txt files received zero bot requests in May 2026, and no major LLM provider has committed to using it.
Why it matters
The honest case: llms.txt won't currently boost AI search citations — GPTBot, ClaudeBot, and PerplexityBot overwhelmingly skip it. But it costs minutes, carries no risk, and has one proven consumer: coding agents and IDE tools (Cursor, Claude Code, Copilot) fetch it on documentation sites. Treat it as cheap insurance plus agent-readable docs — and never let it substitute for fixing crawlability, which bots actually use.
Frequently asked
Should I create an llms.txt file?
Yes, if it takes ten minutes — Seeqly's free generator does exactly that. Just hold honest expectations: it's forward-compatible insurance and agent documentation, not a citation booster today.
What belongs in llms.txt?
A one-line site description, then curated markdown links with short annotations: docs, key product pages, pricing, canonical explainers. Curation is the point — it's a map, not a sitemap dump.
Related terms
- Large Language Model (LLM)The neural networks behind ChatGPT, Gemini, and Claude — trained on web-scale text, now mediating how buyers discover brands.
- AI CitationsThe linked sources an AI assistant credits when generating an answer. Being cited is the AI-era equivalent of ranking #1.
- AI CrawlerBots operated by AI companies — GPTBot, ClaudeBot, PerplexityBot, Google-Extended — that fetch web content for model training or live answer retrieval.
- AI OverviewsGoogle's AI-generated summaries shown above traditional results, now appearing on roughly half of searches and dramatically reducing clicks to websites.