Definition
The llms.txt proposal, published by Jeremy Howard in 2024, suggests a Markdown file at /llms.txt with a site summary and links to key pages, optionally with fuller text versions. The idea is to help AI assistants find and understand a site's content without parsing complex web pages.
Adoption by site owners has grown, but evidence that AI crawlers actually request the file is thin. For most sites, readable page content, structured data and a robots.txt that welcomes the right agents matter more. scayl.ai publishes one at /llms.txt.