What is llms.txt and should you add it? A note written for AI crawlers
What is llms.txt? In short, it is a text file placed in your site’s root folder that summarises, in Markdown, what your site is and which pages matter most, for AI tools. Jeremy Howard proposed it in September 2024. Adding one is cheap and harmless, but it will not earn you a ranking on Google or a place in an AI answer.
Most people do not know this: while Google’s 2026 guide states plainly that such files are not needed to appear in Google Search, Google’s own Chrome team started auditing llms.txt in Lighthouse 13.3 in May 2026. So the file matters not for search rankings but for AI agents moving around your site. Should you add it? Yes, as a five-minute job, without expecting miracles.
What exactly does an llms.txt file do?
llms.txt is a short guide telling AI tools “this is the site, and these are the important pages”. Instead of wandering through dozens of pages one by one, an agent reads this file and learns where to look. According to Chrome’s documentation, without it agents may spend more time crawling to understand a site’s overall structure. In short, the file saves time; it does not buy reputation.
An analogy
It is like handing a guest the floor plan at the door. The house is the same house, but nobody barges into the kitchen looking for the bathroom. The plan does not make the house bigger or make the guest like you more; they just get less tired.
How is it different from robots.txt and a sitemap?
All three sit in your site’s root folder but do different jobs. robots.txt is the doorman, saying who may enter. The sitemap is the address book, listing every page. llms.txt is the welcome leaflet, summarising the most important pages with a line of explanation each. None of them replaces another.
| File | Written for | What it says | Status |
|---|---|---|---|
| robots.txt | All crawlers | Which sections may or may not be entered | Established standard |
| sitemap.xml | Search engines | The full list of pages on the site | Established standard |
| llms.txt | AI tools and agents | A site summary and selected pages | Proposal, optional |
What does Google say, and what does Chrome do?
Google Search is clear: you do not need new machine-readable files, AI text files or special markup to appear in its AI features. Chrome, meanwhile, added an llms.txt audit to Lighthouse under the agentic browsing category. It looks like a contradiction, but the two teams answer different questions: one looks at rankings, the other at an agent getting work done on your site.
The search side: no ranking boost
Google’s guide says such files may be crawled but get no special treatment. Google’s John Mueller once compared the file to the old “keywords” meta tag, which no longer carries weight. Writing an llms.txt and expecting to climb on Google the next day is like cleaning the shop window and expecting turnover to double.
The Chrome side: errors get flagged
The Lighthouse audit checks whether the file exists and whether the server returns an error. If there is no file, the audit is marked “not applicable”, because the file is still optional. If the file exists but errors out, it is flagged. A badly set-up file looks worse than no file at all.
What goes into an llms.txt file?
The only required part is a heading carrying the site’s name. After that comes a one-line summary in quote form, an optional short description, and link lists under section headings. A section called “Optional” holds pages that can be skipped when time is short. Below is a sample for a hypothetical dental clinic in Beylikdüzü, Istanbul.
# Sample Dental Clinic
> Clinic in Beylikdüzü offering implants, orthodontics and children's dentistry.
## Services
- [Implant treatment](https://example.com/implant/): process, duration and price range
- [Clear aligners](https://example.com/clear-aligners/): who they suit, how many months
## Contact
- [Appointments and address](https://example.com/contact/): opening hours, directions
## Optional
- [Blog](https://example.com/blog/): oral health articles
Who should add it, and who should not rush?
Any business that can describe its services and pages clearly can add one; the cost is close to zero. The real question is order. If your site is closed to AI crawlers, your business details are written differently everywhere, or no page answers the customer’s question, llms.txt will not close that gap. Foundation first, note second.
Where it makes sense
- Multi-page service sites: clinics, law firms, consultancies
- Online stores with large product catalogues
- B2B companies with software, documentation or guide content
Who has other work first
If customers are not walking through the door, the problem is rarely a text file. Our why isn’t my website recommended by AI guide weighs the reasons step by step.
Where does llms.txt fit in the AI SEO campaign?
In Adapte’s AI SEO campaign, llms.txt is a small part of the “AI crawler settings” item. The real weight sits in entity setup, structured data and 108 articles across 6 question groups. The file is written last, because the pages you list in it need to exist first. The package is a fixed TRY 36,000 + VAT, delivered in 30 days.
Quick Summary
- llms.txt is a Markdown file that summarises your site for AI tools.
- It is not needed to appear in Google Search and brings no ranking.
- Chrome Lighthouse 13.3 has audited the file since May 2026.
- The only required part is a heading with the site’s name.
- Crawler access and business details first, llms.txt last.
- In the AI SEO campaign it is set up as part of the crawler settings.
Short Glossary
llms.txt: llms.txt is a Markdown file used to show AI tools a summary of a site and its important pages.
robots.txt: robots.txt is a rules file used to tell crawlers which sections of a site they may enter.
AI agent: An AI agent is software used to browse sites and gather information or take actions on a user’s behalf.
Next Step
The file is small; the foundation is the big job. To set up everything from crawler settings to 108 articles in one package, apply for the AI SEO campaign.
Frequently Asked Questions
Source: Chrome for Developers — Lighthouse llms.txt audit
Sık Sorulan Sorular
LLM stands for large language model: the text-writing AI behind tools like ChatGPT, Gemini and Claude. The file takes its name from there: a text file prepared for language models.
Markdown is a plain writing format. A # goes before a heading, a dash before a list item, and that is about it. Humans can read it, machines can read it, which is why AI tools like it.
No. Blocking or allowing an AI crawler is robots.txt’s job. llms.txt only guides whoever is already inside. How often crawlers visit your site is covered in how often AI crawlers read your site.
A sitemap treats every page as equal. Your salon’s price list and a promotion notice from three years ago sit on the same line. llms.txt says “look at these first” and explains in one sentence why they matter.
The official llms.txt site says major AI companies publish the file on their own sites. But no search engine has yet said “we look at this when we read your site”. The honest answer sits between the two.
The short note next to a link tells AI when to open that page. A line that says “price range” points to the page to open when a price question comes in.
Few and focused. Pages that answer your customers’ most frequent questions are enough. Dump the whole site in and the file turns into a copy of your sitemap and loses its point.
First, AI crawler access is checked in robots.txt. Then the business’s identity is made clear; the logic is in what is an entity in SEO. Last, llms.txt is written with selected pages and placed in the root folder.
The full package, the 30-day flow and the measurement setup are gathered in how to do AI SEO. To apply directly, go to the AI SEO campaign page.
Now you know: llms.txt is a welcome note written for AI tools. It brings no ranking on Google Search, and Chrome flags a broken one. It is worth writing, but it will not open the shutters for you.
No. Google states clearly that special files like llms.txt are not needed to appear in its AI features. The file is written to guide AI agents moving around the site.
Yes. In the fixed-price AI SEO campaign at TRY 36,000 + VAT, llms.txt and AI crawler settings are set up in 30 days together with entity setup, structured data and 108 articles.
Its effect on rankings is unknown, but Chrome Lighthouse flags a file that returns a server error. If there is no file, the audit counts as not applicable. So either set it up properly or leave it out.
Updated: October 2026