What is llms.txt?
llms.txt is a proposed markdown file at the root of a website that tells language models what the site is and lists the pages most worth reading. It was proposed at llmstxt.org in 2024 as a short, plain brief a model or agent can read in one fetch, instead of crawling and parsing a site's HTML. It is a proposal, not a standard: no engine is required to read it, and adding one is cheap.
What does an llms.txt file look like?
An llms.txt file is markdown with a fixed order: one H1 with the site's name, an optional blockquote summary, optional notes, then ## sections that each hold a list of links. Every link is written as a markdown link, optionally followed by a colon and a short note.
A section named Optional has a special meaning in the proposal: its links are the ones a reader short on context may skip. Here is a small example for a made-up company:
# Acme
> Acme makes invoicing software for small agencies: quotes, invoices,
> time tracking and payment reminders in one app.
Prices are per workspace. Acme does not offer payroll.
## Product
- [Features](https://acme.example/features): what the app does, with screenshots
- [Pricing](https://acme.example/pricing): plans and what each includes
## Docs
- [Getting started](https://acme.example/docs/start): set up a workspace in ten minutes
- [API reference](https://acme.example/docs/api)
## Optional
- [Blog](https://acme.example/blog): product news and guidesWho reads llms.txt today?
Some AI agents, coding assistants and developer tools fetch llms.txt when they are pointed at a site, because it is faster and cleaner than reading the HTML. Many documentation sites publish one for exactly that use.
The large answer engines are a different matter. None of the four engines Proofsource tracks has said it uses llms.txt to decide which brands to name or which pages to cite, and Google states that you do not need new machine-readable files, AI text files or markup to appear in AI Overviews or AI Mode (Google Search Central). Treat the file as documentation for agents, not as a ranking lever.
What does move AI answers is the pages engines cite. In the public Tesla example (4 Oct 2026), Google AI Overviews cited youtube.com 20 times and reddit.com 15 times, and tesla.com once. No file on a brand's own site changes that; earning a place on the pages engines already read does.
How is llms.txt different from robots.txt and sitemap.xml?
The three files do different jobs. robots.txt sets rules for crawlers, sitemap.xml lists URLs for search engines to crawl, and llms.txt describes the site in words for a model to read.
| File | Job | Format | Read by |
|---|---|---|---|
| robots.txt | Allow or disallow crawlers by user agent and path | Plain-text rules | Search and AI crawlers that honour it |
| sitemap.xml | List every URL you want crawled, with dates | XML | Search engine crawlers |
| llms.txt | Say what the site is and which pages matter most | Markdown | Some agents and developer tools |
What does a real llms.txt look like?
Ours is live at proofsource.co/llms.txt. It opens with an H1 and a one-paragraph summary of what Proofsource does, then a short brief: who it is for, what it measures, and what it is not, so a model does not confuse it with another product of a similar name.
Below the brief, every indexable page is grouped by intent: features, solutions, guides, docs, the glossary and the free tools. The newest blog posts sit under Optional, and the last line points to llms-full.txt, a companion file with the full text of every page for tools that want everything in one fetch.
How do you write and check an llms.txt?
Write it by hand or with the free llms.txt generator, which builds the file from a form in your browser and checks it as you type. Save it as plain text at the root of your site, so it answers at /llms.txt.
Then check the live copy with the llms.txt validator, and make sure the crawlers that would read it are not blocked with the AI crawler checker. A few rules keep the file useful:
- Keep the summary to one paragraph that says what the site is and who it is for.
- Link only the pages that answer real questions: product, pricing, docs, comparisons.
- Use absolute https:// URLs, so a model can open them from anywhere.
- Put long tails such as the blog under Optional, and keep the file short.
- Update it when key pages move; a stale link is worse than no link.
Common questions about llms.txt
Does llms.txt improve AI visibility?
There is no evidence that it does. None of the four engines Proofsource tracks has said it uses llms.txt for ranking or citations, and Google says no special AI text files are needed for AI Overviews. Readable pages and open crawler access matter far more.
Is llms.txt an official standard?
No. It is a proposal published at llmstxt.org in 2024. Nobody is required to read it or to publish it, and the format may change.
Does llms.txt replace robots.txt?
No. robots.txt is where crawler access is allowed or blocked, and llms.txt has no say over that. A site that blocks AI crawlers in robots.txt is not unblocked by publishing an llms.txt.
What is llms-full.txt?
A companion file with the full text of the pages llms.txt links to, in one markdown file, for tools that want everything in a single fetch. It is optional.
Related terms
- Answer engine optimization (AEO)
Answer engine optimization (AEO) is the work of getting your brand named, and your pages cited, in the answers AI assistants give.
- Generative engine optimization (GEO)
Generative engine optimization (GEO) is the work of improving how generative AI engines describe and recommend your brand, so you are named and cited in their answers.
- Google AI Overviews
Google AI Overviews are the AI-written summaries Google shows above some search results, with links to the pages they draw on.
- Citation share
Citation share is the share of all the citations in AI answers that point at a given website.
Worked example: Tesla, public AI answers, 4 Oct 2026: Proofsource's first scan of Tesla, 25 buyer questions and 5 brand questions, one answer per question on each of four engines. A small sample from one brand, shown as an example, not a benchmark.
See where AI leaves you out, and what to fix.
See which buyer questions name your brand, who gets named instead, and what to fix. 200 answers. 25 Prompts/Question. Top AI engines. 2 days. Free. No card.
Google AI OverviewsGoogle AI Mode
