Optimizing for Agents with llms.txt

What Is llms.txt?

If you’ve spent any time poking around the AIE World’s Fair 2026 website, you may have come across the llms.md page. If you’ve clicked the link, you may have an idea of the page’s purpose already.

There is a distinct shift from energetic copy text, neat layouts, and advanced styling to focused, accurate details and well-labeled links. This whole page, while being a good resource for you, is not designed for you. It’s designed for AI.

It follows a standard proposal called the “/llms.txt file,” which you can read about at llmstxt.org. In fact, even though the AIEWF 2026 site has its page at llms.md, if you visit llms.txt instead, it handles it smoothly, redirecting to the llms.md page.

How It Works

The idea is that, rather than filling up an AI’s context window with footers, navs, sidebars, and styles, it is a focused and simple entry point for AIs visiting a site. The entry llms.txt (or .md) contains the minimum necessary information to use the site as well as clear links pointing to where the AI can get more information specifically about the topics it needs.

A common choice is to link to /llms-full.txt, which has much more detail (which the AIEWF 2026 site does). Another recommendation from the standard itself is to make all content pages mirror a simple text version with an appended .md suffix for the same reasons (e.g., posts.html becomes posts.html.md).

Recommended Layout

The recommended layout looks something like this:

• One H1 header with the overall site/project title
• A blockquote with a short summary of the most important information
• Zero or more markdown blocks with more details
• Followed by zero or more markdown sections with H2 headers containing lists of URLs pointing to further detail

Adoption and Criticism

To be clear, these minimal, AI-first text files are simply a proposal, not a standard, and there is mixed adoption. Most chat-based tools have not committed to looking for llms.txt by default.

Some people claim that they are “a solution in search of a problem.” This is reasonable, as the primary selling point is reduction of context clutter, and context efficiency is currently a popular research topic. (Look at how many talks at AIE 2026 are about context!)

However, these files are relatively painless to autogenerate - although most aren’t quite as pretty as ai.engineer’s - and coding tools like Cursor and others actually do claim to reference them, especially when looking up library documentation. Their use does seem to be trending upward.

Google even announced it as a new Lighthouse signal under the new Agentic Browsing category in May of this year, and it seems likely more tooling and standardization are coming in this area.

Important Caveats

Every source on the subject, however, seems to agree on one thing that is the most important: llms.txt files are not replacements for current standards like robots.txt or sitemap.xml files. They are best when used together.

A quick conversation with Gemini revealed that it was able to make use of the AIEWF 2026 llms.md file because it was already indexed by Google and showed up specifically in the search results. So, actions like listing it in your sitemap, ensuring crawlers can see it, and linking to it from landing pages will go a long way toward helping the AIs that use your site.