llms.txt Generator

What is llms.txt?

llms.txt is a plain Markdown file at the root of a site that gives AI assistants a short, curated map of it: what the site is, and links to the pages that matter, each with a one-line note. Jeremy Howard of Answer.AI proposed it in September 2024 at llmstxt.org, and documentation sites, developer tools and SaaS products were quick to publish one.

Each file has its own job. robots.txt tells crawlers what they may fetch. An XML sitemap lists every URL you want indexed. llms.txt picks the pages worth reading and says what is on each, in a form a language model can take in at once.

What the llms.txt generator does

• Reads your site like a crawler: the home page, robots.txt and your XML sitemap, without running JavaScript.
• Picks up to 40 key pages: your navigation links first, then the sitemap from the top level down, at most 8 per section so a big blog does not fill the file. Login, cart, search and tag pages, noindex pages and pages whose canonical names another URL are left out.
• Writes the file in the llms.txt format, with each page's own title and meta description, grouped by section, and legal and contact pages under Optional.
• Writes an AI version: a clear summary, sections grouped by purpose and a one-line note for every page, using only what your pages say. Switch between the two, edit, copy or download.
• Checks what you have: the llms.txt your site already serves (title, summary, sections, link format), and which AI crawlers your robots.txt lets in.

The llms.txt format

The format is plain Markdown in a fixed order. Only the title is required; the rest makes the file useful:

A small but complete llms.txt looks like this:

# Acme

> Acme is project tracking for small teams: tasks, timelines and reports.

## Product

- [Features](https://acme.com/features): What Acme does, with screenshots.
- [Pricing](https://acme.com/pricing): Plans and what each one includes.

## Docs

- [Getting started](https://acme.com/docs/start): Set up a workspace.

## Optional

- [Privacy policy](https://acme.com/privacy): How Acme handles data.

Keep each note to one sentence that says what a reader finds on the page, not a list of keywords. Use absolute URLs, and give every link a real name instead of "click here".

How to add llms.txt to your site

Upload the file to the root of your site as plain text, so it opens at yoursite.com/llms.txt:

• Next.js, Astro, Vite and most static sites: put llms.txt in the public folder and deploy.
• WordPress: upload it next to wp-config.php with your host's file manager or SFTP, or let an SEO plugin that supports llms.txt publish it for you.
• Any server: serve it with the content type text/plain or text/markdown, in UTF-8.

Then open the address in a browser and run the generator again: the check under the file should now find it. Update the file when you add a key page; a sitemap keeps every URL current, llms.txt only needs the ones that explain your site. For the sitemap, the XML sitemap generator builds one from the same crawl.

Does llms.txt help SEO and AI search?

llms.txt is a guide for AI tools, not a ranking factor. Google has said it does not use llms.txt in Search. AI assistants, agents and coding tools read it when they are pointed at a site, and it costs you one small file, so it is worth having.

What decides whether AI answers cite your pages is the search index behind them. ChatGPT search and Copilot draw on Bing's index, and Google's AI answers on Google's. A page that is not indexed cannot be cited, however good your llms.txt is. Check that your key pages are on Google with the Google index checker, and read how ChatGPT and Perplexity indexing works.

IndexMachine gets those pages in on autopilot. It runs Google's own Request Indexing in your Search Console for every page Google hasn't indexed, with your full daily quota on every site, and sends every new URL to Bing, Yandex, Seznam and Naver through unlimited IndexNow. No link farms. Of 1,841 pages IndexMachine requested, 1,418 (77%) were on Google, typically by our first check about 3 days after the request. Sign up free to see which of your pages Google hasn't indexed, or compare plans on the pricing page.