[email protected] 0330 122 6522
Abstract dark editorial graphic of a plain text file being read by a beam of light, illustrating llms.txt and how AI tools read a website

llms.txt Explained: Should Your Site Have One?

llms.txt is a plain Markdown file at the root of your website that lists your best pages for AI tools to read. It was proposed in September 2024 and is not a standard any major AI system has confirmed using. Google says you do not need it. Add one if it costs you an hour. Do not expect citations from it.

What is llms.txt?

It is a Markdown file called llms.txt, usually at yoursite.com/llms.txt, that gives AI tools a short, curated map of your site. Think of it as a reading list rather than a rulebook.

The idea comes from Jeremy Howard, who published the llms.txt proposal on 3 September 2024. The reasoning is sensible. A language model has a limited context window, and a typical web page is mostly navigation, scripts and cookie banners wrapped around a small amount of useful text. A file that points straight to the good stuff, ideally as clean Markdown, saves the model the trouble.

The format is deliberately simple. In order, the file holds:

  • A single H1 with the name of your site or project. This is the only required part.
  • A blockquote with a short summary of what you do.
  • Optional paragraphs or lists with extra context.
  • H2 sections containing lists of links, each with an optional note on what the page covers.
  • An optional section called “Optional”, for secondary links an agent can skip if it is short of space.

That is it. No schema, no special syntax, nothing to validate against. If you can write a README, you can write an llms.txt.

How is it different from robots.txt and sitemap.xml?

The three files do different jobs, and people mix them up constantly.

  • robots.txt sets access rules. It tells crawlers where they may go. It is the file to touch if you want to allow or block specific AI crawlers.
  • sitemap.xml lists every URL you want indexed, for search engines to work through.
  • llms.txt suggests a handful of pages worth reading first, in a form a language model can digest. It does not allow, block or index anything.

So llms.txt is not a replacement for either. It also does not stop a model from reading the rest of your site.

Does llms.txt help AI find and cite you?

On current evidence, no, or at least nobody has shown that it does. The honest position is that it is a reasonable idea with no confirmed take-up from the companies whose models you want to be cited by.

Here is what we can actually check.

Google has said it does not use it. John Mueller wrote on Bluesky in June 2025, as reported by Search Engine Roundtable: “FWIW no AI system currently uses llms.txt.” Google’s own guidance on AI features in Search goes further: “You don’t need to create new machine readable files, AI text files, or markup to appear in these features.” That page does not name llms.txt, but “AI text files” leaves little room for doubt.

Nobody has published proof that it moves citations. Several third-party studies have looked at whether sites with the file get cited more, and the reports we have seen conclude that there is no measurable effect. We have only read those through secondary coverage, so we are not going to quote their numbers here. The direction is consistent, though, and no one has put forward a credible study showing the opposite.

Things may change. The proposal is still being updated, and a standard can gain support after a quiet start. Mueller himself compared the file to the old keywords meta tag, which search engines abandoned because it was easy to game. That is a fair worry: a file the site owner writes for the machine, with no check against the page itself, invites exactly the sort of stuffing that makes engines ignore it.

Our view is simple. Anyone selling llms.txt as a way to get into ChatGPT is selling something the evidence does not support. It sits in the same bucket as a tidy robots.txt: sensible, cheap, not a strategy.

Where it does have a use

There is one place where the idea has real traction: documentation. If you run developer docs, an API reference or a product with a lot of technical pages, a clean index of Markdown links is genuinely handy for coding assistants and agents that a developer points at your site on purpose. That is a different situation from a general-purpose chatbot deciding whether to cite your plumbing business.

Should your site have an llms.txt file?

Probably yes if it is cheap, and no if it is going to eat a week. The downside is close to zero. The upside is small and unproven.

Use this as a rough guide:

  • Add one if you run documentation, a knowledge base or a content-heavy site, and your CMS or framework can generate it automatically.
  • Add one if you are already doing the harder work (structure, schema, crawlable pages) and want the box ticked.
  • Skip it for now if your site is a handful of service pages and the real problems are elsewhere. You will get more from fixing those first.
  • Do not pay for it as a standalone service. If a vendor lists llms.txt as the headline of an AI visibility package, ask what else is in there.

One caution on the downside. Your llms.txt is a public page, and it will say what you want a machine to believe about you. Keep it accurate, keep it consistent with the live site, and do not list pages you would be embarrassed for a customer to land on.

How do you add an llms.txt file?

Write a short Markdown file, save it as llms.txt, and publish it at the root of your domain. For most sites it takes well under an hour.

  1. List your core pages. Pick the ones that explain what you do, who it is for and how it works. Services, pricing, key guides, about, contact. Aim for the pages you would send a new colleague to on day one.
  2. Write the summary. One or two plain sentences in a blockquote. Say what the business does, where, and for whom. No slogans.
  3. Group the links under H2 headings. Each link gets a short note saying what is on the page.
  4. Put minor pages under “Optional”. Case studies, legal pages and older posts can live here.
  5. Publish it at the root so it loads at yoursite.com/llms.txt, and check it returns a normal page rather than a redirect or a 404.
  6. Keep it current. Add a line to your launch checklist. A stale llms.txt pointing at pages that have moved is worse than none.

Here is a small example for an imaginary site:

# Harbour Joinery

> Bespoke kitchens and fitted furniture, designed and built in Lancashire.
> We work with homeowners and developers across the north west.

## Services

- [Fitted kitchens](https://example.co.uk/kitchens): what is included, timescales and how we price
- [Bespoke furniture](https://example.co.uk/furniture): wardrobes, alcoves and media units

## Guides

- [How long does a new kitchen take?](https://example.co.uk/guides/kitchen-timescales): a week-by-week breakdown

## Optional

- [Case studies](https://example.co.uk/projects)
- [Privacy policy](https://example.co.uk/privacy)

If your site is built on a modern framework, you can generate the file from the same content you already publish, so it never drifts out of date. That is the sensible way to do it. Hand-maintaining a list of links is how files go stale.

What actually helps you get cited?

The things that matter are the unglamorous ones: pages that answer a question directly, structured data, content a crawler can read without running scripts, and a brand that exists in more places than your own website. None of them is a file you upload in ten minutes.

If you want a short list of where the effort pays back, start here:

  • Answer-first pages. Put the answer to a real buyer question in the first sentence or two under a heading that asks that question. We cover the method in how to get cited by AI.
  • Crawlable HTML. Most AI crawlers read the page as served and do not run JavaScript. A site that renders its content in the browser can look empty to them, whatever files sit next to it.
  • Clear structure and entities. Consistent naming, honest schema and a presence beyond your own domain. Our guide to AI search optimisation walks through the whole picture.
  • A fast, clean build. Sites that are quick and easy to parse tend to be easy for machines too. That is a big part of AI website optimisation.

llms.txt belongs at the end of that list, as a nice-to-have once the foundations are in place. If you are not sure where your foundations stand, that is a ten-minute question to answer, and we will answer it for free.

Free AI audit

Find out what is actually stopping AI from citing you

llms.txt is a small piece of a larger puzzle. Our free AI audit looks at the big pieces: whether ChatGPT, Perplexity, Gemini and Google mention you today, whether crawlers can read your pages, and what to fix first. You get a plain-English list, and if your site is already in good shape, we will say so.

  • Real prompt testing across the main AI engines
  • A check of crawler access, structure and schema
  • A short, ordered list of fixes, with no hard sell
Get my free AI audit

Frequently asked questions

What is llms.txt?

llms.txt is a plain Markdown file, placed at the root of your website, that gives AI tools a short curated guide to your most useful pages. It was proposed by Jeremy Howard in September 2024. It has a site name, a one-line summary and lists of links, so a model can find your key content without wading through navigation and scripts.

Does llms.txt help you get cited by ChatGPT or Google?

There is no good evidence that it does. Google's John Mueller said in June 2025 that no AI system currently uses llms.txt, and Google's documentation says you do not need new machine readable files or AI text files to appear in its AI features. Treat it as optional housekeeping, not a ranking lever.

Where do I put the llms.txt file?

At the root of your domain, so it loads at yoursite.com/llms.txt. The proposal also allows files in subdirectories, covering the URLs beneath that path, but most sites only need the one at the root. Serve it as plain text or Markdown, and make sure it returns a normal 200 response rather than a redirect or an error page.

Is llms.txt the same as robots.txt?

No. robots.txt tells crawlers what they may and may not access, and well-behaved bots follow it. llms.txt does not block or permit anything. It is a suggested reading list for AI tools. If you want to control which AI crawlers can reach your site, robots.txt is the file that does the job.

Is it worth adding llms.txt to a small business website?

If it takes under an hour and your platform makes it easy, it does no harm and may be useful later. Do not spend a week on it, and do not pay anyone to build one as a visibility fix. Clear answer-shaped pages, structured data and crawlable HTML will do far more for your chances of being cited.

Written by
Sam Wright · Founder, Aeonix

Sam Wright is the founder of Aeonix, an AI-first UK marketing agency. He writes about AEO, GEO and SEO, and what it takes to get found and cited now that buyers ask AI before they search Google. Less theory, more of what actually works.

Want this working for your business?

Start with a free audit, or talk to us about where to begin.