What llms.txt Is and How It Helps AI Read You

If you know the robots.txt file, the idea behind llms.txt will feel familiar. It is a text file, placed at the root of your domain, meant to talk to artificial intelligence systems rather than to traditional search engines. The proposal came from Jeremy Howard in 2024, and it has been part of the conversation about preparing a website for ChatGPT, Gemini or Perplexity ever since.

It helps to start honestly. This is not an official standard, the major model makers do not endorse it in any binding way, and real adoption is still small. None of that makes it a bad idea, but it does mean treating it as a low cost experiment rather than a magic visibility lever.

What it actually is

llms.txt is a Markdown file that summarizes your site for an automated reader. The underlying idea is that language models work better with clean, well organized text than with pages full of menus, banners and scripts. Instead of forcing the machine to guess what matters, you tell it in a short, structured document.

The proposed format is deliberately simple. A heading with the project name, a brief paragraph explaining what it is about, and then lists of links to your most relevant pages, each with a line of context. It is often paired with a variant called llms-full.txt, which includes the full content in plain text for anyone who wants to serve it whole.

What a typical file contains

A shortened example would look like this. First the title with a hash, then a quote or summary paragraph, and then a bulleted list where each line points to a URL with its context. The point is curation. Do not dump the whole site, pick what really explains your business.

Where it goes

The file lives at the root of the domain, reachable at a path like yourdomain.com followed by a slash and llms.txt. It is the same logic as robots.txt or sitemap.xml. It should be served as plain text, never behind a login, and kept up to date when your offering changes. If you publish a full version, you usually link to it from the main file.

How to create one

1. Inventory your key pages: products, pricing, documentation, frequently asked questions and reference articles.
2. Write an honest summary of what your company does, with no filler adjectives.
3. Group the links by topic and give each one a sentence explaining what the reader will find.
4. Save the result as Markdown text and upload it to the domain root.
5. Review it periodically, just as you review a sitemap.

You need no special tools. A text editor is enough. Generators exist that build from your sitemap, but the value lies in the human decision about what to include.

What to really expect

Here we should lower the expectations. As of today there is no solid evidence that the main assistants read this file systematically to decide whom to cite. Some documentation platforms have adopted it and report that certain crawlers fetch it, but we are far from a widespread practice.

So is it worth it. For two reasons. First, the cost is almost zero and the risk is none. Second, the exercise of writing a clean summary of your site forces you to organize your thinking, which also helps with classic SEO and user experience. Think of llms.txt as a cheap bet in case the ecosystem consolidates it, not as the reason AI will start recommending you.

What does move the needle today is clear content, verifiable data and presence in the sources models consult. If you want to know whether those efforts turn into real mentions, the sensible thing is to measure how ChatGPT, Gemini and Perplexity name you over time. Tools like Bee LLM track that daily. The llms.txt file is a minor piece of a much larger board.

Keep reading: AI search visibility metrics and KPIs · tools comparison.