llms.txt
A proposed markdown file at your site root that lists your most important pages in a clean, structured form for AI systems to read.
In depth
What it really means
llms.txt was proposed by Jeremy Howard in September 2024. It is a markdown file listing your key URLs with short descriptions, so an AI system can find your best content without parsing navigation, banners and scripts. Think curated index rather than access control.
Adoption is the honest caveat. As of 2026 no major AI provider has publicly committed to reading it, and server logs at most sites show little to no traffic for it. It costs an hour to build and does no harm, so it is a reasonable low-cost bet. It is not a ranking lever, and anyone selling it as one is overstating the case.
How it works
- You publish /llms.txt as markdown at your root.
- It opens with an H1 site name and a blockquote summary.
- H2 sections group links by type, such as Docs, Blog, Product.
- Each entry is a markdown link with a one-line description. An optional /llms-full.txt holds the full text of those pages.
Pros & cons
Pros
- Cheap to create and maintain, with no downside risk.
- The exercise of choosing your 30 most important pages is useful on its own.
- If adoption arrives, you are already there.
Cons
- No confirmed major consumer as of 2026, so measurable return is close to zero today.
- It goes stale quietly, and a stale index is worse than none.
- It gets oversold, which crowds out work that actually moves the needle.
Common mistakes
- Treating it as a substitute for crawlability and clean HTML. It is not.
- Confusing it with robots.txt. llms.txt grants nothing and blocks nothing.
- Listing every URL on the site, which defeats the point of curation.
- Publishing once and never updating, so it points at dead and superseded pages.
Best practices
- List 20 to 50 genuinely important pages, not everything.
- Write a real one-line description for each, since that is the part AI systems would use.
- Group by section with H2 headings and keep the markdown clean.
- Put a review date in your quarterly content calendar.
- Spend the remaining effort on crawlable HTML and well-structured pages, which demonstrably work.
FAQs
What is llms.txt?
A proposed markdown file at your site root listing your most important pages with short descriptions, so AI systems can find your best content without parsing site chrome.
Is llms.txt an official standard?
No. It is a community proposal from September 2024. No major AI provider has publicly confirmed reading it, so treat it as a low-cost bet rather than a requirement.
How is llms.txt different from robots.txt?
robots.txt controls access and is universally respected. llms.txt controls nothing and is a curated index. Adding llms.txt does not grant or deny any crawler anything.
Should I add llms.txt?
It takes about an hour and carries no risk, so yes if you have the time. Do it after your content is crawlable in HTML and well structured, because those two demonstrably matter and this one does not yet.
Keep reading
Related on LymLyt
Beyond LymLyt
Further reading
Want this working on your site?
We build the content behind the term, ranked in search and cited by AI.
Book a 30-min call →