What is llms.txt?

llms.txt is a proposed convention for a plain Markdown file at a website’s root that gives AI language models a short summary of the site and links to its most useful pages.

Want to know if your site gets llms.txt right?

I’ll check how AI search sees your business today and what would help it cite you, free.

No newsletter. Just what you asked for, from me. Privacy

In plain English

Jeremy Howard of Answer.AI proposed it in September 2024. The format is simple: a heading with the site name, a one-paragraph summary, then sections of links with short notes. Some sites also publish a longer companion file, often called llms-full.txt, with fuller content.

It’s a proposal, not a standard. Google has said it doesn’t use llms.txt, and no major AI provider has documented that it reads the file when choosing what to cite. It’s used most in developer documentation, where coding assistants can be pointed at it directly.

Why it matters for a small business

It’s cheap and harmless, so I add one to the sites I build. But I tell clients plainly not to expect rankings or citations from it. Crawlable HTML, clear pages and an accurate robots.txt matter far more.

An example

A Calgary architecture studio publishes an llms.txt with its name, a one-line summary (“Residential architecture studio in Calgary designing custom homes and renovations”) and three short sections linking to services, selected projects and the about page. Using the llms.txt generator, it takes about twenty minutes and sits alongside robots.txt and the sitemap. (More for architects.)

Common mistakes

Where I see llms.txt go wrong most often.

  1. Treating llms.txt as a way to block or allow AI crawlers, which it can’t do. That’s robots.txt’s job.

  2. Pasting the entire site or a list of keywords into it, which defeats its purpose as a short guide.

  3. Publishing it once and forgetting it, so it links to pages that were renamed or removed.

FAQ

llms.txt, answered

  1. Google has said it doesn’t. Google says AI Overviews and AI Mode draw on its normal search index, so regular SEO is what counts there.

  2. No. robots.txt tells crawlers what they may fetch and is widely respected. llms.txt is an optional summary that offers guidance but controls nothing.

  3. If it’s quick, sure. It does no harm and may help some tools. Just don’t spend money or time on it before your core pages are clear and crawlable.

Keep reading

Let’s begin

Let’s build something you’re proud to share.

Let’s Talk