• llms.txt is a proposal from 2024, not an adopted web standard.
• It is not a control file. It does not allow or block anything, unlike robots.txt.
• Google has said it does not use it, and evidence of use by other major AI crawlers is thin.
• It is most useful for developer documentation that people feed to coding assistants.
• If you add one, keep it accurate and do not expect it to change AI visibility on its own.

llms.txt was proposed in September 2024 by Jeremy Howard of Answer.AI. The idea is simple. Websites are built for people, with navigation, scripts and styling that get in a model's way. A plain Markdown file listing the important pages, with short descriptions, would give a language model a clean starting point.

What it looks like

The file lives at /llms.txt and follows a loose Markdown structure:

 # Example Ltd

> Example Ltd makes scheduling software for dental practices in the UK.

Key facts: founded 2015, based in Leeds, used by over 2,000 practices.

## Product

- [Features](https://www.example.com/features.md): What the software does
- [Pricing](https://www.example.com/pricing.md): Plans and costs

## Guides

- [Getting started](https://www.example.com/docs/start.md): Setup in ten minutes

## Optional

- [Company history](https://www.example.com/about.md) 

• An H1 with the site or project name. This is the only required element.
• A blockquote with a short summary.
• Optional free-form detail.
• H2 sections containing lists of links, each with an optional note.
• A section titled "Optional" for links that can be skipped when context is limited.

The proposal also suggests offering a Markdown version of each page at the same URL with .md added, and some sites publish an llms-full.txt containing the full text of their documentation in one file.

What it is not

• Not robots.txt. It grants and denies nothing. Crawlers that ignore it lose nothing.
• Not a sitemap. It is a curated shortlist, not a complete inventory.
• Not an official standard. No standards body has adopted it.

Does anyone use it?

This is where expectations need managing.

• Google representatives have said plainly that Google Search does not use llms.txt, and have compared it to the long-ignored keywords meta tag.
• Log file studies across large numbers of sites have found that the main AI crawlers rarely request the file.
• No major AI search provider has announced that it uses llms.txt to decide what to retrieve or cite.
• Adoption is strongest among developer tools and documentation platforms. Several documentation hosts generate the file automatically, and it is useful when a developer pastes it into a coding assistant or when an agent is pointed at it deliberately.

So today it mostly works when a human or an agent chooses to load it, not as something AI search engines discover and act on by themselves.

Should you add one?

A reasonable position:

• Yes, if you run developer documentation, an API or a technical product where users work with AI coding tools.
• Maybe, if it can be generated automatically at no real cost and you would like to be ready should adoption grow.
• Not a priority, if it would take time away from things with proven impact such as crawl access, server-side rendering, content quality and brand mentions.

If you do create one

• Keep it short and curated. Link to your best, most current pages.
• Write descriptions as plain statements of fact.
• Make sure every linked URL works and is allowed in robots.txt.
• Keep it in step with the site. An out-of-date file is worse than none.
• Do not stuff it with keywords or promotional claims that do not appear on the site.
• Consider whether the Markdown versions should be noindexed or canonicalised, so they do not compete with your HTML pages in ordinary search.

Risks

• Markdown duplicates of pages can be indexed by search engines and shown to users in place of the designed page.
• Because the file is invisible to ordinary visitors, it is open to abuse, with content shown to models that differs from what people see. Expect platforms to treat mismatches with suspicion.
• Time spent here is time not spent on fundamentals.

How to test

• Check server logs for requests to /llms.txt and note which user agents ask for it.
• Fetch the file and validate that it is plain text or Markdown, returns 200 and that every link resolves.
• Paste it into an assistant and ask questions about your company to see whether it helps in that direct-use scenario.

The bottom line

llms.txt is a sensible idea with limited real-world uptake so far. Add it if it is easy, keep it honest, and keep your attention on making the actual pages accessible and worth citing. This is a fast-moving area, so it is worth rechecking provider documentation from time to time.

Add your title here

This is a paragraph. Writing in paragraphs lets visitors find what they are looking for quickly and easily. Make sure the title suits the content of this text.