In more detail
The llms.txt proposal was published by Jeremy Howard in September 2024 and updated in 2026. The idea is simple: web pages are built for people, full of menus, scripts and adverts, so give AI agents one plain, well-organised file that tells them what the site is and where the important information lives.
The proposal sets out a specific format, in this order:
- An H1 heading with the name of the site or project. This is the only required part.
- A blockquote with a short summary.
- Optional paragraphs or lists with more detail (no headings).
- Optional sections under H2 headings, each a list of links in the form
- [Name](url): optional notes. A section called "Optional" holds links an agent can skip.
It is different from robots.txt, which tells crawlers what they may access, and from an XML sitemap, which lists every indexable page. llms.txt is a short, hand-picked guide, mostly read on demand when an agent is helping someone with a task.
Adoption is real but uneven. Documentation platforms and some CMS plugins generate the file automatically, and it is widely used for software documentation. Google has not said it uses llms.txt for Search or AI Overviews, so treat it as a low-cost extra, not a fix for visibility.
Why it matters when hiring an agency
Adding an llms.txt file takes an hour or two and does no harm, so it's a reasonable item on an AI search checklist. It shouldn't be the headline of a proposal. If an agency presents llms.txt as the key to appearing in ChatGPT or Google's AI features, ask what evidence they have. The fundamentals (crawlable pages, clear facts, a good reputation) matter far more. You can check or draft a file with our free llms.txt generator and validator.