What Is llms.txt?
llms.txt is a file that tells AI search engines what your website is about. Learn what it is, how to create one, how it differs from robots.txt, and how it helps AI platforms like ChatGPT and Perplexity understand your content.
Jayson MundayUpdated March 202610 min readWhat is llms.txt?
llms.txt is a plain text file placed in the root directory of a website (accessible at yourdomain.com/llms.txt) that provides AI search engines and large language models with a structured summary of the website's content, purpose, and key pages.
llms.txt is a Markdown-formatted text file at your website root that tells AI models what your site is about, what pages are most important, and what topics you cover. It was proposed by Jeremy Howard (co-founder of fast.ai) as a standard for helping AI understand websites. Think of it as a "robots.txt for AI" that provides content context rather than access control.
The llms.txt standard was proposed by Jeremy Howard, co-founder of fast.ai and a leading figure in AI research. The specification is documented at llmstxt.org and is gaining adoption as businesses recognise the need to make their content accessible to AI platforms beyond traditional search engines.
While a traditional sitemap tells search engines "these pages exist" and robots.txt tells them "you can or cannot access these pages," llms.txt tells AI models "here is what this website contains and why it matters." It bridges the gap between machine-readable site structure and human-readable content context.
What does an llms.txt file look like?
An llms.txt file uses Markdown formatting. Here is an example for a plumbing business:
# Sydney Plumbing Co
> Sydney Plumbing Co is a licensed plumbing company serving
> Sydney's Eastern Suburbs, Northern Beaches, and Inner West.
> We provide emergency plumbing, hot water systems, blocked
> drain clearing, gas fitting, and bathroom renovations.
> Licensed by NSW Fair Trading. ABN 12 345 678 901.
## Pages
- [Home](https://sydneyplumbingco.com.au/): Overview of our plumbing services, service areas, and contact information
- [Emergency Plumbing](https://sydneyplumbingco.com.au/emergency/): 24/7 emergency plumbing with 45-minute average response time
- [Hot Water Systems](https://sydneyplumbingco.com.au/hot-water/): Installation, repair, and replacement of gas, electric, and solar hot water systems
- [Blocked Drains](https://sydneyplumbingco.com.au/blocked-drains/): CCTV drain inspection and high-pressure jet blasting for blocked drains
- [About Us](https://sydneyplumbingco.com.au/about/): Licensed since 2015, 3 qualified plumbers, based in Bondi Junction
- [Pricing](https://sydneyplumbingco.com.au/pricing/): Callout fees from $80, hourly rates, and fixed-price quotes
## Optional
- [FAQ](https://sydneyplumbingco.com.au/faq/): Common plumbing questions answered
- [Blog](https://sydneyplumbingco.com.au/blog/): Plumbing tips and maintenance guides for Sydney homeownersThe file is simple by design. It provides AI models with a clear, structured overview of what the business does, what pages contain, and where to find specific information. Brain Buddy AI's tools generate llms.txt files automatically by scanning your website content.
What is the difference between robots.txt and llms.txt?
robots.txt controls access (which pages crawlers can visit). llms.txt provides context (what your site is about). robots.txt says "you can or cannot crawl this page." llms.txt says "here is what this website contains and why it matters." They serve different purposes and should be used together.
What is the difference between llms.txt and llms-full.txt?
llms.txt is a concise summary (1-5 KB) listing your key pages with brief descriptions. llms-full.txt is an extended version (50-500+ KB) containing the full text content of your important pages. llms.txt is for quick site discovery. llms-full.txt gives AI models complete content without crawling each page. Most websites should have both.
How to make an llms.txt file
Creating an llms.txt file takes less than 30 minutes for most websites. Follow these 6 steps.
Create the llms.txt file
Create a plain text file named llms.txt using UTF-8 encoding. This file will be placed in the root directory of your website so it is accessible at yourdomain.com/llms.txt.
Write a title and site description
Start the file with a Markdown H1 heading (# Your Business Name) followed by a brief, factual description of your business. State what you do, who you serve, and where you operate. Keep it to 2-3 sentences.
List your key pages
Under a '## Pages' heading, list your most important pages using Markdown links: - [Page Title](URL): Brief description. Include your homepage, service pages, about page, key blog posts, and any pages you want AI models to prioritise.
Add topic sections (optional)
Include sections listing your areas of expertise, services offered, FAQs, and any other structured information that helps AI models understand your content coverage. Use clear headings and concise descriptions.
Create llms-full.txt (recommended)
For comprehensive coverage, create a second file called llms-full.txt that includes the full text content of your key pages. This gives AI models complete context without needing to crawl each page individually.
Upload and verify
Upload both files to your website's root directory. Verify they load correctly by visiting yourdomain.com/llms.txt in a browser. Optionally, reference your llms.txt in your robots.txt file to help AI crawlers discover it.
Brain Buddy AI generates llms.txt automatically. Our AI Citations tool scans your website and generates both llms.txt and llms-full.txt files customised for your content. Available as part of our managed SEO/AEO/GEO service. Get in touch to have yours generated.
Best practices for llms.txt
Use plain, factual language
Write your llms.txt like a factsheet, not a marketing brochure. AI models extract factual information. 'We are a plumbing company serving Sydney's Eastern Suburbs since 2015' is useful. 'We are the best plumbers in Sydney' is not.
Keep llms.txt concise
The standard llms.txt should be a concise summary (1-5 KB). Save comprehensive content for llms-full.txt. AI models use llms.txt for quick site understanding and llms-full.txt for deep content access.
Update regularly
Update your llms.txt when you add new pages, services, or content areas. Stale llms.txt files that do not reflect your current site content provide inaccurate signals to AI models.
Include your most authoritative content
Prioritise pages where you have the deepest expertise, strongest content, and most unique information. AI models will use these as primary citation candidates.
Use Markdown formatting consistently
llms.txt uses Markdown syntax. Use H1 (#) for your site name, H2 (##) for sections, and standard Markdown links for page listings. Consistent formatting helps AI parsers extract information accurately.
Reference in robots.txt
Add a line to your robots.txt pointing to your llms.txt file. This helps AI crawlers that check robots.txt discover your llms.txt automatically. Example: # AI content summary available at /llms.txt
How does llms.txt fit into AI search optimisation?
llms.txt is one layer of a comprehensive AI search optimisation strategy. It works alongside schema markup (structured data for machines), content optimisation (structured answers for AI extraction), and entity authority (consistent brand signals across the web). Together, these create multiple pathways for AI models to discover, understand, and cite your content.
Schema markup
JSON-LD structured data embedded in your HTML pages. Tells AI models about your business, services, and content structure at a page level.
llms.txt
A dedicated summary file at your domain root. Gives AI models a site-wide overview of your content, key pages, and areas of expertise.
Entity authority
Consistent brand information across your website, Google Business Profile, social media, and third-party sources. Verifies your business identity for AI models.
Content structure
Question-format headings, direct answer formatting, and FAQ sections within your pages. Makes individual passages extractable and citable by AI.
No single layer is sufficient on its own. The most effective AI search optimisation uses all four together. Brain Buddy AI's managed service implements all of these as part of every plan. Learn more about GEO.
Frequently asked questions about llms.txt
Written by
Jayson Munday
CEO & Founder, Brain Buddy AI
Jayson builds AI search optimisation tools including automated llms.txt generation for Brain Buddy AI's managed clients. His team has generated and deployed llms.txt files for hundreds of websites as part of their GEO service. Based at Stone and Chalk, Sydney Tech Central.
Want your llms.txt generated automatically?
Brain Buddy AI scans your website and generates both llms.txt and llms-full.txt as part of our managed AI search optimisation service. We also handle schema markup, content restructuring, and ongoing AI citation monitoring.
Included with every managed plan. No commitment required for a consultation.