AI search

llms.txt: What It Is, Who Reads It and Should You Bother?

An honest guide to llms.txt: what the file is, who actually reads it, how to write a good one in an hour, and why it isn't a fix for AI visibility.

FindSEO4 min read

Updated

A wooden card catalogue cabinet with rows of small drawers

Photo: MarkBuckawicki / Wikimedia Commons (CC0)

In this article
  1. What is llms.txt?
  2. Who actually reads it?
  3. What goes in the file?
  4. Services
  5. About
  6. Optional
  7. How to write a good one
  8. Where to put it
  9. Should you bother?
  10. What this means if you're hiring an agency

llms.txt is a plain Markdown file at the root of a website that gives AI agents a short description of the site and a hand-picked list of links to its most useful pages. It takes an hour or two to create, it does no harm, and it can help AI tools that look for it. But it isn't a ranking factor, Google says it doesn't need it, and it won't fix a site that AI crawlers can't reach.

Here's what the file is, who actually reads it, how to write one, and how much effort it deserves.

What is llms.txt?

The idea was proposed by Jeremy Howard in September 2024 and revised in 2026. Web pages are built for people, wrapped in menus, scripts, cookie banners and adverts. An AI agent trying to help someone has to dig through all of that, and it has a limited amount of space to hold what it reads. llms.txt gives it a shortcut: a concise overview and a map of where the important information lives.

Rows of small wooden card catalogue drawers with brass handles and label holders.
llms.txt is a hand-picked index, not a full catalogue. Photo: MarkBuckawicki, CC0, via Wikimedia Commons.

The proposal compares it to robots.txt and XML sitemaps, but it does a different job:

  • robots.txt tells crawlers what they're allowed to access.
  • An XML sitemap lists every page you want search engines to index.
  • llms.txt is a short, curated guide for an agent that needs to understand your site quickly.

It doesn't block or allow anything, and it doesn't replace either of the other two.

robots.txt sets what crawlers may access, an XML sitemap lists every page you want indexed, and llms.txt is a short curated guide for AI agents. llms.txt doesn't replace either of the other files.
Use all three. They do different jobs.

Who actually reads it?

This is where expectations need managing.

According to the proposal's author, thousands of sites now publish an llms.txt, documentation platforms generate one automatically, and the major AI companies publish llms.txt files for their own developer documentation. The heaviest use is in software documentation, where coding assistants follow the file to find the right reference page. Assistants with web access can also read it when they're helping someone with a task on your site.

What it isn't, at least today, is a signal that search engines use to rank pages. Google's documentation on AI features in Search says plainly that you don't need to create new machine-readable files or special text files to appear in AI Overviews or AI Mode. None of the main AI search tools has said llms.txt decides whether a site is cited.

So think of llms.txt as a courtesy to AI agents, not a lever for search visibility.

What goes in the file?

The format is deliberately simple. In order:

  1. An H1 with the name of your site or business. This is the only required part.
  2. A blockquote summary: one or two sentences on what you do and who for.
  3. Optional detail: paragraphs or bullet points with anything an assistant should always get right, such as your location, service area or opening hours. No headings in this part.
  4. Sections under H2 headings, each a list of links in the form - [Page name](URL): short note.
  5. An optional section called "Optional" for links an agent can skip when it needs a shorter overview, such as policies.

A small local business might end up with something like this:

# Example Plumbing

> Licensed plumbers serving Melbourne's eastern suburbs since 2009. Emergency call-outs 24/7.

We cover Box Hill, Doncaster, Ringwood and surrounding suburbs. Call-out fee $99, quoted before work starts.

## Services

- [Blocked drains](https://example.com/blocked-drains/): Same-day clearing and CCTV inspection
- [Hot water](https://example.com/hot-water/): Repairs and replacement, gas and electric

## About

- [About us](https://example.com/about/): Licences, insurance and team
- [Contact](https://example.com/contact/): Phone, email and service area

## Optional

- [Privacy policy](https://example.com/privacy/)

The proposal also suggests offering clean Markdown versions of important pages (for example the same URL with .md on the end) and linking to those from llms.txt. That's worth doing for documentation-heavy sites. For most small business sites, linking to the normal pages is fine.

How to write a good one

  • Keep it short. The point is to fit easily in an agent's working memory. A few dozen links is plenty for most businesses.
  • Lead with facts. The summary and detail lines are where you state what an assistant should never get wrong.
  • Write useful notes. "Blocked drains: same-day clearing and CCTV inspection" tells an agent far more than the page title alone.
  • Choose pages for a newcomer. Your main services, pricing, about and contact pages, and your most useful guides. Skip tag pages, thin blog posts and anything out of date.
  • Keep it current. An llms.txt that links to deleted pages or old prices is worse than none.
A good llms.txt keeps to a few dozen links, leads with key facts, gives each link a useful note and picks pages a newcomer needs. Leave out tag pages, thin posts, deleted pages, old prices, and any expectation that it improves rankings.
An out-of-date llms.txt is worse than none.

You can check an existing file, or get a starting draft built from your sitemap, with our free llms.txt Generator and Validator. Many website builders and WordPress SEO plugins can also generate and maintain the file for you.

Where to put it

Save the file as llms.txt at the root of your domain so it loads at https://yourdomain.com/llms.txt, served as plain text. If you only control part of a site, the proposal allows a file at a subpath (such as /docs/llms.txt) covering the pages beneath it.

Should you bother?

For most businesses, the honest answer is: yes, if it's quick, and only after the basics are done.

The things that decide whether AI tools find and cite you are the same things that matter for search: AI search crawlers allowed in robots.txt, content readable without JavaScript, clear pages that answer real questions, consistent facts across the web, and a good reputation. If those aren't in place, an llms.txt file won't make up for it. Check crawler access first with the AI Crawler Access Checker.

What this means if you're hiring an agency

Adding llms.txt is a sensible small task on an AI search checklist. If an agency makes it the centrepiece of an AI SEO proposal, or claims it will get you into Google's AI Overviews or ChatGPT answers, ask for evidence. Google's own guidance says AI text files aren't needed for its AI features, and no one can promise citations. The AI SEO glossary entry for llms.txt has a quick summary to share with your team.

FAQs

Does llms.txt help my Google rankings?

No. Google's documentation on AI features says you don't need new machine-readable or AI text files to appear in AI Overviews or AI Mode, and llms.txt isn't a ranking signal.

Is llms.txt the same as robots.txt?

No. robots.txt tells crawlers what they may access. llms.txt is a short guide to your most useful pages for AI agents. It doesn't block or allow anything.

What's the minimum a valid llms.txt needs?

Only an H1 with the name of your site. A blockquote summary and sections of links are recommended.

Where does the file go?

At the root of your domain, so it loads at /llms.txt, served as plain text.

Sources

Ready for help with SEO?

Compare SEO-only agencies near you, with client reviews checked by a person, or tell us what you need and get matched.

AI search

Does AI Content Hurt SEO? What Google Actually Says

Google doesn't penalise content for being made with AI. It penalises content made mainly to manipulate rankings. What's allowed, what's spam and how to publish safely.

FindSEO7 min read