How to write an llms.txt file (with a copy-paste template)
If you have ever pasted a docs URL into ChatGPT or Claude and watched it summarize the cookie banner instead of the page, you already understand the problem llms.txt tries to solve.
Websites are built for browsers. Language models work better with a short, clean, text-only map of what matters. llms.txt is that map. It is a small Markdown file at the root of your site that tells AI tools what your site is, which pages are worth reading, and which ones they can skip.
This post walks through the format, gives you a template you can copy, and covers the mistakes I see most often.
What llms.txt is (and what it isn't)
The format was proposed by Jeremy Howard of Answer.AI in September 2024 and is documented at llmstxt.org. The idea is simple: put a Markdown file at /llms.txt that an AI agent can read in one go when it needs context about your site.
Two things it is not:
- It is not
robots.txt.robots.txtsays who may crawl what.llms.txtsays nothing about access. It only provides context. If a crawler is blocked, a perfectllms.txtwon't help. - It is not a sitemap. A sitemap lists every URL for indexing.
llms.txtis a curated shortlist with a sentence of context per link. The spec explicitly says it isn't meant to replacesitemap.xml.
The format, in order
The spec defines the sections in a fixed order:
- An H1 with the name of the site or project. This is the only required part.
- A blockquote (
>) with a short summary: the key facts someone needs to understand everything below it. - Optional plain Markdown (paragraphs, bullet lists) with extra detail. No headings in this part.
- Zero or more H2 sections, each containing a "file list". Every item is a Markdown link, optionally followed by a colon and a note:
- [Name](url): what this page covers. - An optional
## Optionalsection. By convention, links here are secondary. An agent short on context can skip them.
That's the whole format. No YAML, no special syntax, nothing a model has to learn.
A copy-paste template
Here's a template for a typical software product. Replace everything in angle brackets.
# <Product name>
> <Product name> is a <category> for <who it's for>. It helps them <main job> without <main pain>. Pricing starts at <price>; there is a free plan.
Key facts:
- Founded in <year>, based in <city/country>.
- Works with <main integrations>.
- Not a fit for <who should look elsewhere>.
## Product
- [Features](https://example.com/features): Overview of what the product does, grouped by use case.
- [Pricing](https://example.com/pricing): Plans, limits and what each plan includes.
- [Integrations](https://example.com/integrations): Supported tools and how the connections work.
## Docs
- [Getting started](https://example.com/docs/start): Setup in under ten minutes.
- [API reference](https://example.com/docs/api): Endpoints, authentication and rate limits.
## Guides
- [How to <main job>](https://example.com/blog/how-to-x): Step-by-step guide with examples.
- [<Product> vs <competitor>](https://example.com/blog/vs): Honest comparison table.
## Optional
- [Changelog](https://example.com/changelog): Release notes.
- [About](https://example.com/about): Team and company background.
Step by step
- Pick 10 to 30 pages, not 300. Start with the pages you would send a smart new colleague: what you do, what it costs, how it works, and the two or three guides that answer your most common questions. If a page wouldn't help someone answer a question about your product, leave it out.
- Write the blockquote like an encyclopedia entry. One or two sentences: what it is, who it's for, what makes it different. Skip the slogans. "The #1 platform for growth" tells a model nothing. "Invoicing software for freelancers in India, with GST-ready invoices and a free plan" tells it a lot.
- Group links by what people are trying to do. "Product", "Docs", "Guides" and "Comparisons" work for most sites. Ecommerce sites might use "Shop", "Shipping & returns" and "Size guides".
- Write a factual note for every link. The note after the colon is where most of the value is. Describe what the page contains, not how great it is: "Plans, limits and annual discount", not "Our amazing pricing".
- Push secondary pages into
## Optional. Changelogs, press pages and team bios are fine to include, but mark them as skippable. - Publish it at the root. Save it as a plain text file at
https://yourdomain.com/llms.txt. The spec also allows files in subpaths (like/docs/llms.txt) that describe only that section. If several apply, agents are meant to use the most specific one. - (Optional) Offer Markdown versions of key pages. The spec suggests serving a clean Markdown copy of important pages at the same URL with
.mdappended, and pointing to it with<link rel="alternate" type="text/markdown">. This is a bigger job, so do it for your docs first.
Mistakes to avoid
- Dumping your sitemap into it. If it has 2,000 links, it's no longer a map. Curate.
- Marketing copy in the notes. Models tend to pick up whatever wording you give them. Plain, factual descriptions are more useful.
- Linking to pages you keep out of search. Thank-you pages, gated content and thin tag pages don't belong here.
- Forgetting about crawler access. Check that AI crawlers can actually reach your pages. A firewall or CDN rule that blocks them makes the whole file pointless.
- Never updating it. Treat it like a sitemap: regenerate or review it when you launch or retire important pages.
Does anyone actually read it?
Be realistic. llms.txt is still a proposed standard, not something every AI engine has committed to. According to llmstxt.org, thousands of sites already publish one, documentation platforms generate them automatically, and several AI labs publish them for their own developer docs.
It is mostly designed for on-demand lookups, when an agent or coding assistant needs context right now, rather than for training. So don't expect it to move your rankings on its own. Do expect it to take about an hour, make your site easier for agents and coding tools to understand, and force you to write a crisp one-paragraph description of what you do, which is useful everywhere.
If you'd rather not start from a blank file, a free llms.txt generator can build a first draft from your sitemap and page titles, which you can then edit with the steps above.
Comments
No comments yet. Start the discussion.