How to Write and Deploy an LLMs.txt File (Complete Guide)

What Is LLMs.txt? Origin and Definition

LLMs.txt is a proposed web standard designed to give website owners a way to communicate directly with artificial intelligence systems about how their content should be used and interpreted. Introduced by Jeremy Howard of Answer.AI in September 2024, it functions similarly to robots.txt but is specifically tailored for AI applications rather than search engine crawlers. The file is written in Markdown format and placed at the root of a website’s domain, making it easily discoverable by AI systems that choose to respect it. The core problem LLMs.txt attempts to solve is the lack of standardized communication between content creators and AI platforms—website owners currently have no reliable way to specify how their content should be processed, cited, or prioritized, and no way to tell AI systems about that preference directly. This guide is the mechanics half of the story — what the file is and how to build one correctly. For the evidence on whether any of this actually changes what AI systems do, see our companion piece, The Truth About LLMs.txt: Overhyped or Essential?

The LLMs.txt Spec: Structure and Syntax

LLMs.txt is placed at the root directory of a website (e.g., example.com/llms.txt) and uses a structured Markdown format. The file typically includes an H1 title containing your site name, a blockquote summary of its purpose, and detailed sections organized with H2 headers that specify different content categories. Unlike robots.txt, which uses a simple text-based syntax with specific rules and directives, LLMs.txt leverages Markdown’s flexibility to allow more nuanced, human-readable descriptions. Individual pages are listed using the format [Title](URL): Description, keeping descriptions concise but informative — this is the atomic unit of the whole file, and getting it wrong (vague titles, missing descriptions, broken URLs) is the single most common implementation mistake.

Ready to Monitor Your AI Visibility?

Track how AI chatbots mention your brand across ChatGPT, Perplexity, and other platforms.

Get Started Contact

LLMs.txt vs Robots.txt vs XML Sitemaps

The three files are easy to confuse because they all live at your domain root and all talk to automated systems, but they solve different problems:

robots.txt tells crawlers where they can’t go. XML sitemaps tell search engines everything that exists. LLMs.txt does neither — it’s a hand-picked, annotated shortlist of the pages you’d want an AI system to read first if it only had time for a handful.

Step-by-Step: Writing Your LLMs.txt File

Building the file manually takes four steps:

1. Start with an H1 heading containing your site or company name, optionally followed by a one-line blockquote summarizing what your site does and why an AI system should care.
2. Organize content into H2 sections if your site has distinct areas — Documentation, Blog, API Reference, and so on — with a short description under each explaining what that section contains.
3. List individual pages under each section using [Title](URL): Description, one line per page.
4. Save the file as llms.txt (plural, not singular) and upload it to your domain root, so it’s reachable at yourdomain.com/llms.txt.

Here’s a basic example structure:

# Company Name

> Brief description of what your company does and why AI systems should care about your content

## Documentation
[Getting Started](https://example.com/docs/getting-started): Step-by-step guide for new users
[API Reference](https://example.com/docs/api): Complete API documentation with examples
[Best Practices](https://example.com/docs/best-practices): Proven patterns for using our platform

## Blog
[Why We Built This](https://example.com/blog/why-we-built-this): The problem we solved and how

What to include: evergreen content, well-structured pages, and pieces that demonstrate genuine expertise. What to avoid: your homepage (usually not valuable in isolation), every URL on your site (quality over quantity), and pages that don’t make sense without surrounding context. You can optionally add a section listing URLs to skip if shorter context is needed, though most implementations don’t require this level of granularity.

Stay Updated on AI Visibility Trends

Get the latest insights on AI mentions, brand monitoring, and optimization strategies.

Email address

Subscribe

llms.txt vs llms-full.txt: Which Do You Need?

Once you’ve built a standard llms.txt, you’ll run into its sibling format: llms-full.txt. Where llms.txt is a curated list of links with short descriptions, llms-full.txt inlines your entire documentation into one comprehensive file — sometimes 400,000+ words for large products. The tradeoff is simple: llms.txt asks an AI system to follow links and fetch pages as needed, while llms-full.txt hands over everything upfront so no follow-up requests are needed. Most sites only need llms.txt; llms-full.txt makes sense mainly for documentation-heavy products where you want zero-friction access to the complete corpus in a single fetch.

Generator Tools vs Manual Creation

Creating an LLMs.txt file is straightforward and requires no technical expertise beyond basic file creation, and you don’t have to write it by hand. Several popular SEO tools now offer built-in LLMs.txt generators, including AIOSEO, Rank Math, Yoast, and Squirrly, making the process a few clicks for website owners already using these platforms. Mintlify auto-generates the file for any documentation site it hosts, which is part of why adoption spiked so sharply after that feature shipped in November 2024. For developers who prefer scripting the process, the llms_txt2ctx CLI tool provides a command-line alternative for generating and managing LLMs.txt files, useful if you want the file regenerated automatically whenever your docs change. Manual creation is still the right call for smaller sites where you want full control over which pages are described and how — a generator will happily include pages that don’t deserve to be there.

Common Mistakes to Avoid

A handful of implementation errors show up repeatedly:

• Using the singular filename — it’s llms.txt, not llm.txt, and the wrong filename means the file simply won’t be found
• Listing every URL on the site — dumping your entire sitemap into llms.txt defeats the point of curation
• Vague or missing descriptions — a bare link with no description gives an AI system nothing to prioritize on
• Letting the file drift out of sync with your actual content as pages get updated, merged, or retired
• Describing content differently than it reads on the page — beyond being poor practice, this is the exact trust problem that makes the standard controversial in the first place

Does It Actually Work?

That’s a fair question, and not one this guide is trying to answer — it’s a separate discussion with its own evidence, adoption data, and skeptics’ case, and we’ve written it up in full as a dedicated essential practice debate: The Truth About LLMs.txt: Overhyped or Essential? The short version is that no major AI platform has confirmed using it, but implementation costs almost nothing, so most teams build it and move on rather than waiting for proof either way. What you can verify directly is whether your own file is correctly formatted and reachable — AmICited.com, our AI visibility monitoring platform, lets you track how AI systems actually cite and use your content across the web, which is the closest thing to real feedback on whether your llms.txt (and the rest of your content) is doing anything at all. That data point matters more than the file itself: no amount of llms.txt polish substitutes for quality content creation, and the monitoring is how you’d know the difference.