llms.txt: what it is, why it matters and how to implement it

In December 2024, Jeremy Howard, the creator of fast.ai, proposed a simple standard: an llms.txt file placed in a domain's root directory, telling AI systems about the structure and content of the site. The standard is supported by Anthropic (Claude), Perplexity and a dozen or so other AI tools. Implementation cost: a few minutes. Potential effect: better understanding of your site by AI systems. What is llms.txt? llms.txt is a Markdown-format text file placed at yourdomain.com/llms.txt. Its job is to convey key information about the site to AI systems in a structured, easily processed form. The analogy: robots.txt tells search engines what they may index; llms.txt tells AI systems what is worth knowing about your site. The standard is backed by platforms including Reddit, Medium, Cloudflare, Akamai and Creative Commons. It is not mandatory - AI systems manage without it. But its presence signals that your company understands AI mechanisms and wants to shape actively what AI knows about it. Note: llms.txt does not replace good content on the site. An AI system will still index your domain and draw on the content of your pages. llms.txt is a pointer, not a substitute for valuable content. What does an llms.txt file contain? The file format is simple and human-readable. It consists of a few elements: - A title and company description - one sentence explaining what the company does and for whom - Links to key sections with descriptions - service pages, blog, about, case studies - Optional key facts - the data you want AI to know about your company - Optional guidance on content use - if you have preferences about citation An example llms.txt for a B2B company In short: An example llms.txt for a B2B service company: # AiVisible > AiVisible helps B2B companies measure and improve their visibility in AI answers. ## Services - AI Visibility Snapshot: a focused measurement of one category and market - Strategic AI Visibility Audit: complete answers, benchmark, sources and a roadmap - AI Recommendation Sprint: a backlog of recommendations and experiments - Continuous AI Visibility: ongoing measurement and further experiments ## Knowledge - Blog: articles on AI Visibility and GEO - Methodology: the measurement protocol and its limits ## Key facts - We work with B2B companies - The free stage qualifies the category and is not an audit How to implement llms.txt, step by step - Create the llms.txt file - open a text editor and write the content in Markdown, as in the example above. Save it as llms.txt. - Place the file in the domain root - it has to be reachable at yourdomain.com/llms.txt. In Next.js put it in the /public folder. In WordPress, in the server root directory. - Check availability - open yourdomain.com/llms.txt in a browser. You should see the text file with its content. - Optionally add llms-full.txt - the standard also allows an extended version containing the full content of key pages, for models that want to fetch everything at once. How to implement llms.txt in Next.js In Next.js the simplest approach is to place the file in the /public folder, where it is served automatically as a static asset. Alternatively you can generate llms.txt dynamically through a route handler at app/llms.txt/route.ts, which lets you update the content without rebuilding the project. Which AI systems read llms.txt? | Model or tool | llms.txt support | Notes | |---|---|---| | Claude (Anthropic) | Yes | Direct support for the standard | | Perplexity | Yes | Used when indexing sites | | ChatGPT (browsing) | Partial | Not officially, but indexes the file | | Google Gemini | Not officially | The standard is not confirmed | | Bing Copilot | No | No official support | Does llms.txt actually work? The standard is too new for hard data on its effect on AI visibility. Early experiments suggest that AI systems officially supporting it - Claude and Perplexity - do use llms.txt to build context about a site. For models that do not support it formally, the file is simply ignored and does no harm. Our conclusion: llms.txt is worth implementing on a cost-to-potential-benefit basis. Implementation takes 15 to 20 minutes. The risk is zero. The potential gain is better understanding of your site by the AI systems that support the standard. As the standard matures and adoption grows, companies that implemented it early will have nothing to catch up on. FAQ Is llms.txt mandatory? No, but it is an emerging standard that helps AI systems understand the structure and offer of your site correctly. Where should the llms.txt file go? The file should sit in the domain root, for example aivisible.pl/llms.txt, in the same way as robots.txt. Originally published at AiVisible. AiVisible measures how often companies and products appear in ChatGPT, Gemini, Claude and Perplexity answers. Top comments (0)