GEO Rankings
← Blog
Published

llms.txt for Generative Engines: Why It Helps Models Find Your Best Pages but Won't Get You Cited on Its Own

llms.txt removes crawl friction for generative engines but does not produce citations on its own. Here is what it does, and what actually moves the needle.

Bottom line

llms.txt tells AI crawlers which pages matter most on your domain. It removes technical friction at the crawl layer. It does not, on its own, cause ChatGPT, Perplexity, or Gemini to cite you. The content quality and third-party authority signals that drive actual citations operate one layer higher. Do the file. Do not stop there.

Last updated August 2026.

Every few months the GEO community picks up a new technical tactic and debates whether it is the missing piece. llms.txt is the current candidate. Proponents call it a lightweight signal that gets your best pages in front of generative engines. Skeptics say it does nothing. Both camps miss the more useful frame: it is a friction-removal tool, not a citation-generation tool. Once you hold that distinction clearly, the question of “should I add it?” becomes easy to answer. And so does the more important question: what do I actually need to do to get cited?


What llms.txt is and what it is not

llms.txt is a specification proposed by Jeremy Howard (fast.ai) in September 2024. Place a Markdown file at your domain root, for example yoursite.com/llms.txt, listing the URLs that represent your site’s most important content with optional short descriptions. AI systems building a knowledge index of your site, or pulling context through a retrieval-augmented generation (RAG) pipeline, can read that file to understand which pages are canonical.

The format looks like this:

# Acme Co

> B2B SaaS for inventory management

## Core pages
- [How it works](https://acme.co/how-it-works): Product overview and use cases
- [Pricing](https://acme.co/pricing): Plan details and enterprise options
- [Integration guide](https://acme.co/docs/integrations): Step-by-step setup

That is the whole file. No directives. No machine codes. Markdown headings, a brief description, and a curated list of links.

Here is the table that clears up most of the confusion:

What llms.txt doesWhat llms.txt does not do
Tells AI crawlers which URLs you consider canonicalForce an AI engine to read those URLs
Organizes your content hierarchy in a format LLMs can parseChange how the engine weights your domain authority
Reduces friction for RAG pipelines ingesting your siteGuarantee that any page gets retrieved or cited
Helps AI systems distinguish core docs from boilerplateOverride poor content quality or thin pages
Signals which pages have stable, citable summariesSubstitute for FAQ or Article schema
Works alongside robots.txt for access controlReplace robots.txt or affect crawler permissions

The critical line: llms.txt operates at the crawl layer. Citations are decided at a layer above that.


Why the “does it work?” question is badly framed

When people ask “does llms.txt work?”, they usually mean one of two things:

  1. Does it cause AI engines to cite me more often?
  2. Does it help AI crawlers find my content more reliably?

The answer to (1) is: no, not on its own. The answer to (2) is: yes, that is what it is designed for.

The confusion happens because citation share is the metric GEO practitioners care about, and llms.txt does not move that metric directly. It moves a prerequisite: whether AI crawlers can reliably identify and access your best pages in the first place.

By late July 2025, one industry blog estimated more than 600 websites had adopted llms.txt, though no independently verified count exists. A widely repeated “600%” figure refers to growth rate in one small tracked dataset, not total site count. Mainstream adoption among top websites remained below 1% as of mid-2025.

Neither of those numbers suggests llms.txt is a citation shortcut. They suggest it is a niche early-adopter practice with unclear direct impact on visibility.


Where llms.txt fits in the GEO stack

GEO programs have three distinct layers. Understanding where llms.txt sits in that stack clarifies what it can and cannot do.

Layer 1: Crawl access

Can AI crawlers reach your important pages at all? This means:

  • Checking robots.txt for unintentional blocks on AI user-agents (Googlebot-extended, GPTBot, ClaudeBot, PerplexityBot)
  • Fixing JavaScript-rendering barriers that prevent crawler access to your core content
  • Ensuring canonical URLs resolve cleanly without redirect chains

llms.txt belongs here, as an optional curation layer on top of basic access. It is worth implementing because it is low-cost and signals intent to AI crawlers that do respect it.

Layer 2: Content quality

Once a crawler reaches your page, does it find a direct-answer opening paragraph, structured data, factual density, and a clear heading hierarchy? According to Kevin Indig’s 2026 analysis of 1.2 million ChatGPT responses, 44.2% of ChatGPT citations were drawn from the first 30% of a page’s content. Pages that front-load a self-contained answer to the query get retrieved. Pages that bury the conclusion get paraphrased out.

This is where most of the citation leverage lives. A page that directly and concisely answers a query outperforms a well-curated llms.txt pointing to a page that starts with marketing copy.

Layer 3: Third-party authority

The majority of AI citations across most query types come from third-party sources rather than brand-owned pages. Being mentioned in editorial coverage, review platforms, and authoritative industry publications propagates into model outputs faster than any on-domain technical change. Earning mentions on the domains that AI engines already trust is the highest-leverage GEO activity.

llms.txt is a Layer 1 tactic. Worth doing. Not a substitute for Layers 2 or 3.


The three myths that keep circulating

Myth: “Adding llms.txt will get you cited by ChatGPT or Perplexity.”

Fact: Citation decisions are driven by content quality, source authority, how directly a page answers the query, and whether the domain is already trusted by the engine’s training data and retrieval system. llms.txt removes one small access barrier. It does not affect those higher-order signals.

Myth: “If AI crawlers can read your llms.txt, they will cite you.”

Fact: Many AI engines do not crawl the live web for every query. RAG systems draw on pre-indexed snapshots, curated knowledge bases, and partner data deals. A file that helps crawlers organize your site matters only when a crawler is actually crawling your site for citation retrieval. For some engines and some query types, that is true. For others, your llms.txt file never comes into play.

Myth: “llms.txt is basically the same as a sitemap.”

Fact: An XML sitemap lists all URLs you want indexed, optimized for traditional crawler protocols. llms.txt is a curated, human-readable Markdown file for AI systems. It is not exhaustive. The point is curation, not enumeration. You want to signal which pages answer questions and which pages are canonical. Pointing AI crawlers at 5,000 URLs defeats the purpose.


How to set llms.txt up correctly

The file itself takes five minutes. What goes in it takes more thought.

  1. Create a plain text file named llms.txt at your domain root.
  2. Open with a heading (# Brand name) and one sentence describing what your site covers and who it serves.
  3. List your most important canonical URLs with brief descriptions. Prioritize pages that directly answer buyer questions, product documentation, and authoritative editorial content.
  4. Optionally, add an llms-full.txt that links to a more comprehensive URL list for systems that want deeper indexing.
  5. Keep it current. Stale files pointing to redirected or deleted URLs are worse than no file.

The curation question is the important one. If a page buries its conclusion or reads like a product brochure, including it in llms.txt does not make it citation-worthy. Point the file at pages that already directly answer questions. If those pages do not exist yet, build them first.


Tools that cover llms.txt and the broader GEO program

llms.txt is the easy step. The harder work is the content and authority program around it. These tools address different parts of that program.

Temso (from $89/mo) is the easiest all-in-one GEO platform for teams that want citation monitoring, gap diagnosis, and content execution without switching tools or paying per-engine fees. Its technical optimization workflow includes llms.txt guidance alongside the higher-leverage work: tracking citation share across 8 AI engines (ChatGPT, Perplexity, Gemini, Google AI Overviews, Google AI Mode, Grok, Microsoft Copilot, Meta AI), surfacing which specific pages AI engines cite for your target queries, and executing the content fixes that close gaps. For teams that need to see what is working and iterate, the flat $89/mo entry tier makes it accessible before you need to justify a larger budget.

Otto SEO is a technical SEO automation platform that handles the crawl-access layer at scale. If your site has widespread robots.txt issues, JavaScript rendering blockers, or redirect chains that prevent AI crawler access, Otto SEO automates the identification and remediation of those issues across large sites. It addresses the technical foundation that llms.txt sits on top of.

Writesonic is a specialist AI writing platform useful for teams whose bottleneck is producing the direct-answer content that actually earns citations. Once you know which queries you want to appear in (Temso’s gap analysis surfaces those), Writesonic helps you produce factually dense, well-structured content at volume. It does not do citation monitoring; pair it with a tracking tool.

Surfer integrates content optimization and AI citation tracking in one platform. Its AI Tracker monitors brand visibility across ChatGPT, Perplexity, Google AI Overviews, AI Mode, and Gemini, and its Content Editor helps you optimize pages for both traditional rankings and AI citation performance in the same workflow. Useful for content teams that want to write for citations without logging into separate tools.

ToolWhat it does for llms.txt and GEOEntry price
TemsoFull GEO loop: citation monitoring (8 engines), gap diagnosis, content execution, technical optimization including llms.txt$89/mo
Otto SEOTechnical SEO automation: fixes crawl access, rendering, and redirect issues that llms.txt relies onFrom $100/mo
WritesonicAI content production for the direct-answer pages that drive actual citationsFrom $20/mo
SurferContent optimization plus AI citation tracking in one editor$99/mo

See the complete ranking at /rankings/geo-tools.


The bottom line

llms.txt is a Markdown curation file at /llms.txt that tells AI crawlers and RAG pipelines which URLs on your domain matter most. It is analogous to robots.txt but human-readable, directive-free, and aimed at large language models rather than traditional search spiders.

Implementing it takes five minutes and has real upside as more AI systems build live-web retrieval into their architecture. The Wix AI Search Lab finding is worth holding onto: of 586 indexed llms.txt files analyzed, roughly 6% of those pages ranked for organic keywords. A file that points to weak content does not make that content citation-worthy.

Do the file. Do it early. Then put your energy into the higher-leverage layers: pages that front-load direct answers, structured data that tells AI engines what the content covers, and earned mentions on the third-party domains generative engines already trust.

Start with the content that directly answers your target queries. Earn mentions on the sites AI engines cite. Add llms.txt as part of your technical foundation. Then track citation share across engines to see what is moving.


Ready to measure where you stand? Temso monitors your citation share across 8 AI engines from $89/mo and surfaces exactly which gaps to close first. The full GEO tool comparison lives at /rankings/geo-tools.

FAQ

Does llms.txt help you get cited in ChatGPT or Perplexity answers?

Not on its own. llms.txt removes a technical friction point by telling AI crawlers and RAG pipelines which URLs represent your site's canonical content. Whether those pages get cited depends on content quality, source authority, and how directly the page answers the query. According to Wix's AI Search Lab analysis of 586 indexed llms.txt files, roughly 6% of those pages ranked for organic keywords, suggesting that having the file does not correlate with strong AI search visibility by itself.

What is the difference between llms.txt and robots.txt?

robots.txt controls crawler access using binary allow/deny directives. llms.txt does not grant or block access. It is a human-readable Markdown curation file that signals which pages are canonical and most important for AI systems to prioritize. The two files serve different purposes and can coexist without conflict.

Is llms.txt the same as a sitemap?

No. An XML sitemap lists all URLs a site wants indexed by search engines, optimized for traditional crawler protocols. llms.txt is a smaller, curated, human-readable Markdown file aimed at AI crawlers and RAG pipelines. It is not an exhaustive URL list. The goal is curation: signal which pages answer questions directly and which are canonical for your brand or product.

How widely adopted is llms.txt?

Adoption remains niche. By late July 2025, one industry blog estimated more than 600 websites had adopted llms.txt, though no independently verified count exists. A widely repeated "600%" figure refers to growth rate in one small tracked dataset, not total site count. Mainstream adoption among top websites remained below 1% as of mid-2025.

What tools help you manage llms.txt alongside a broader GEO program?

Temso (from $89/mo) is an all-in-one GEO platform that covers citation monitoring across 8 AI engines, gap diagnosis, and content execution, with llms.txt guidance built into its technical optimization workflow. Otto SEO automates on-site technical SEO tasks including crawl-layer optimizations. Writesonic and Surfer help teams produce the direct-answer content that drives actual citation gains once the technical setup is complete.

Where in a GEO program does llms.txt belong?

llms.txt belongs at the crawl access layer, which is the foundation of a GEO program but not the highest-leverage layer. Once basic access is confirmed, the two higher-leverage layers are content quality (direct-answer openings, structured data, factual density) and third-party authority (earned mentions on domains AI engines already trust). llms.txt is worth adding early because it is low-cost, but it does not substitute for the content and authority work above it.