llms.txt: What It Is, Who Actually Reads It, and Whether You Need One

llms.txt was proposed by Jeremy Howard on September 3, 2024, with a second version of the proposal in August 2026. The published proposal describes it as a way to provide information that helps agents use a website. The idea is simple: a web page full of navigation, scripts and layout is a poor input for a language model, so give models a short, clean map of what matters.

It lives at /llms.txt on your domain (or under a subpath, such as /docs/llms.txt, to cover that section). The proposal also suggests offering clean markdown versions of important pages at the same URL with .md added.

The format is ordinary markdown with a few conventions:

• An H1 with the site or project name. This is the only required part.
• An optional blockquote summarizing what the site is.
• Optional plain paragraphs with any extra context.
• H2 sections of links, each link followed by a short note on what it contains.
• An "Optional" section for pages a tool can skip when it is short on space.

A minimal example:

# High Salience

> Search intelligence for category leaders: AI search and SEO research, methodology and services.

## Research
- [AI Search Benchmark](https://highsalience.com/research/ai-search-benchmark/): method for measuring brand recommendations across AI surfaces
- [Payroll teardown](https://highsalience.com/research/payroll-teardown/): 100 payroll questions run through ChatGPT, with dataset

## Services
- [AI SEO](https://highsalience.com/ai-seo-agency/): combined Google and AI search program

## Optional
- [About](https://highsalience.com/about/)

Google settled the question for its own search engine in June 2026, when it added a note to its guide to generative AI features in Search. The guidance says llms.txt files are not needed for Google Search, including its AI features, and that publishing one will neither harm nor help your visibility or rankings, because Search ignores them. It also says maintaining one is fine if other services use it.

There is one wrinkle. Chrome's Lighthouse tool now includes an llms.txt check in its agentic browsing audits, describing the file as an emerging convention for AI agents. That is not a contradiction. The audit is about AI agents and browser tools, not Search rankings, and a site without the file is marked not applicable rather than failed. Only a server error when fetching the file counts as a failure.

Two independent studies published in summer 2026 looked at server logs rather than opinions.

EZY Research tracked 83 sites that publish llms.txt over 12 weeks, April 27 to July 19, 2026, and compared requests for llms.txt with requests for robots.txt:

OpenAI and Anthropic fetched robots.txt thousands of times and llms.txt in single digits; Perplexity never requested it at all. Meta was the only company requesting llms.txt more often than robots.txt. Source: EZY Research, July 27, 2026.

Ahrefs checked 137,210 domains in its web analytics data. 28 percent published an llms.txt file, and 97 percent of those files received no requests at all in May 2026. Of the requests that did arrive, the largest group, 21.7 percent, came from SEO audit tools, the same tools that often warn sites for not having the file. Source: Ahrefs, June 15, 2026.

Put plainly: the largest single source of llms.txt traffic is software checking whether you have one, not AI systems reading it.

It depends on who you expect to read it.

• Worth doing if you publish developer documentation, an API or a product that people use through coding assistants and AI agents. Those tools load documentation on purpose, and a clean map helps them.
• Fine but optional for most business sites. It costs little to publish and maintain, and it will not hurt you with Google.
• Not worth doing as an AI visibility tactic. If a vendor sells llms.txt as a way to rank in ChatGPT or AI Overviews, the evidence above says otherwise.

Do not let an audit tool's score push you into it. A missing llms.txt is not a technical SEO error.

• Write an H1 with your company name and a one-sentence blockquote describing what you do.
• Add two to four H2 sections that group your most useful pages: products, pricing, documentation, research.
• List each page as a markdown link with a short, factual note. Ten to thirty links is plenty for most sites.
• Put secondary pages under an "Optional" heading.
• Save it as plain text at /llms.txt and check that it returns a 200 status with no server error.
• Update it when you add or retire important pages. A stale map is worse than none.

Common mistakes: listing every URL on the site, stuffing descriptions with keywords, serving an HTML page at that address, and blocking the file in robots.txt by accident.

If your real goal is to show up in AI answers, spend the time on things the systems demonstrably use: allow OAI-SearchBot and the other search crawlers in robots.txt, fix the pages your buyers' questions point to, earn a place in the comparison and review pages that answers cite, and measure recommendations over a fixed set of questions. Our guide to AI search engine optimization walks through each of these, and our AI SEO program runs them for clients.

Is llms.txt an official standard?

No. It is a proposal published by Jeremy Howard in September 2024 and updated since. No standards body has adopted it, and no major AI search provider has committed to using it for ranking or retrieval.

Does ChatGPT use llms.txt?

OpenAI does not mention llms.txt in its crawler documentation, and in one 12-week server-log study OpenAI's crawler requested llms.txt 7 times against 3,990 requests for robots.txt. If you want to appear in ChatGPT search, what matters is allowing OAI-SearchBot in robots.txt.

Does llms.txt help SEO?

Not for Google. Google's documentation says Search ignores llms.txt files, so they neither help nor hurt visibility or rankings.

What is the difference between llms.txt and robots.txt?

robots.txt tells crawlers what they may and may not fetch, and every major crawler honors it. llms.txt is an optional reading list that points AI tools at your most useful pages. One controls access; the other is a suggestion.

Can an llms.txt file hurt my site?

Google says it will not affect rankings either way. The one practical risk is a server error on the file, which Chrome's Lighthouse agentic browsing audit flags as a failure. A missing file is simply marked not applicable.

What is llms-full.txt?

A convention some sites use to put the full text of their documentation in one file. It is not part of the core llms.txt proposal, and the same caveats apply: it is useful for developer tools that load it on purpose, not for search visibility.

• llms.txt proposal (Jeremy Howard)
• Google Search Central: guide to generative AI features in Search
• Chrome for Developers: Lighthouse llms.txt audit
• EZY Research: We put llms.txt on 83 websites
• Ahrefs: 97% of llms.txt files never get read
• OpenAI: Overview of OpenAI crawlers

Every High Salience engagement starts with the Category Salience Brief: your commercial questions run across Google and the AI surfaces, competitors side by side, and a ranked list of what to fix first.