What is an llms.txt file, and does your business website need one? As more people use ChatGPT, Google AI Overviews and other AI assistants to research products and services, website owners are asking how to make their content easier for AI systems to discover and understand. An llms.txt file is one possible addition, but it is not a replacement for search engine optimisation.
This practical guide explains what llms.txt does, what it cannot do, how to enable it with Yoast SEO on WordPress and which improvements deserve your attention first. You will also find an example file, a website readiness checklist and a realistic plan for improving your visibility in AI-assisted search.
What is an llms.txt file?
An llms.txt file is a plain-text, Markdown-formatted document typically published at the root of a website, such as example.co.uk/llms.txt. Its purpose is to present a concise overview of the site and point to useful pages in a format that language-model-based tools can process easily.
Think of it as an optional, curated guide to your website. It can introduce your organisation, describe what you offer and link to the most useful pages. Unlike a conventional XML sitemap, which usually lists many URLs for search engines, an llms.txt file can be deliberately selective and descriptive.
The idea is straightforward: when a system chooses to read the file, it may find a useful starting point instead of having to infer your priorities from a large navigation menu. However, the important words are when a system chooses to read it. There is no universal requirement for AI services to discover, fetch or follow this file.
Why are businesses talking about it?
AI-assisted research is changing the kinds of questions people ask. Instead of typing “web designer Warwickshire”, someone might ask for help comparing a WordPress rebuild with a Shopify migration, or request an explanation of how a local business can automate its enquiries. Those questions reward accurate, detailed information, clear service descriptions and credible evidence.
A file that highlights your core pages may be useful to systems that support it. But the content of those pages, whether they are accessible, and how clearly your business is represented elsewhere remain far more important than simply publishing a new file.
Does llms.txt improve AI search rankings?
There is currently no reliable basis for promising that adding llms.txt will increase rankings, ChatGPT mentions or AI citations. Some vendors describe it as a way to make content easier to find, but implementation and adoption vary between AI products. You should treat it as an optional experiment, not a proven ranking factor.
Google is particularly explicit: its guidance says Google Search does not use llms.txt files to determine visibility in Search or its generative AI features. Google also says that no special AI-specific text file or schema is required to appear in AI Overviews or AI Mode. Read Google’s official guide to generative AI search for the current position.
llms.txt file can be useful documentation even when it delivers no measurable search benefit.For most businesses, the sensible approach is to make sure their core pages work well, then add and maintain llms.txt if it is easy to do. It should not consume the budget that would otherwise be spent improving service pages, product information, accessibility or site performance.
llms.txt vs robots.txt vs XML sitemaps
These files serve different purposes. They are not interchangeable, and creating one does not remove the need to maintain the others.
| File | Primary purpose | Who uses it? | What to remember |
|---|---|---|---|
llms.txt | Optional, curated Markdown guide to important content | Tools or workflows that choose to support it | Not a proven AI ranking factor; Google Search ignores it |
robots.txt | Instructions about which URLs compliant crawlers may request | Search and other compliant web crawlers | Not a reliable method of keeping private content secret |
sitemap_index.xml or XML sitemap | Lists URLs you want search engines to discover | Search engines and compatible crawlers | Helps discovery but does not guarantee indexing |
| Schema structured data | Machine-readable description of page entities and content | Search engines and other processors | Must accurately reflect visible page content |
A common mistake is to add llms.txt and assume search engines will automatically find everything on the website. Your sitemap, internal linking, indexing settings and canonical URLs still need attention. Equally, a permissive robots.txt file cannot guarantee that any particular AI service will visit your pages.
How to enable llms.txt in Yoast SEO on WordPress
If your website runs WordPress and already uses Yoast SEO, you may not need another plugin or a manually maintained text file. Yoast includes a feature that generates llms.txt and allows eligible sites to customise the selected pages.
According to Yoast’s official llms.txt setup instructions, the steps are:
- Open the WordPress dashboard. Sign in with an account that has permission to change Yoast settings.
- Navigate to Yoast SEO → Settings. Open the Site features area.
- Find AI tools → llms.txt. Switch the feature on, then save your changes.
- Review the generated file. Open
https://yourdomain.co.uk/llms.txtin a private browser window to confirm that it is publicly accessible. - Customise the pages if appropriate. Use the available customisation option and manual page selection to prioritise your key services, About page, important guides and other useful public pages.
- Check the result after updates. Confirm that the file still contains accurate links and reflects the pages you want visitors to find.
What does Yoast include automatically?
Yoast’s implementation can select recently updated posts, pages and relevant content types, with particular treatment for cornerstone content. Its technical documentation says the generated file is refreshed by a scheduled action, generally weekly. This is convenient, but automatic selection may not always match your commercial priorities. A frequently updated news post is not necessarily more important than your core service page.
If you use Yoast’s manual selection, choose pages that explain who you are, what you offer, where you operate and what makes your services credible. Avoid stuffing the file with dozens of low-value URLs simply to make it longer.
For the precise automatic selection rules and refresh behaviour, see Yoast’s developer specification for llms.txt. This is particularly useful if your site has custom post types or relies on cornerstone content.
What if the setting is missing or the file does not work?
First, update Yoast SEO and check your account permissions. Yoast’s documentation notes that its llms.txt feature is not available for WordPress multisite installations. File-generation problems can also be related to server filesystem permissions. If you see a 404 page, check the feature toggle, caching and whether a different plugin or manually created file is conflicting with the expected URL.
If you are already using another SEO plugin, do not install Yoast solely for this feature without checking for conflicts. Multiple SEO plugins can duplicate metadata, XML sitemaps and schema. A manual file or another supported solution may be more appropriate.
Does Yoast SEO automatically make my website appear in ChatGPT?
No. Yoast can help publish a machine-readable overview, but it cannot require a third-party AI product to read or cite it. The plugin does not replace your on-page SEO, content strategy, indexing checks or wider online reputation. Its role is to help manage part of the technical setup.
Example llms.txt file for a small business website
A basic hand-written file could look like the example below. It is deliberately concise, uses descriptive links and does not include private pages, admin URLs or claims that cannot be verified. Replace the example business and URLs with your own details.
# Example Business
> Independent UK business offering website design,
> ecommerce development and website automation.
## Main pages
- [Home](https://example.co.uk/): Overview of our business.
- [About](https://example.co.uk/about/): Our experience and approach.
- [Web Design](https://example.co.uk/web-design/): Website services.
- [Ecommerce](https://example.co.uk/ecommerce/): Online shops.
- [Contact](https://example.co.uk/contact/): Get in touch.
## Useful guides
- [Website Planning](https://example.co.uk/blog/website-planning/):
How to prepare for a new website project.
- [Website Maintenance](https://example.co.uk/blog/maintenance/):
Practical advice on keeping a site updated.There is no benefit in adding unsupported statements such as “the leading provider in the UK” or trying to instruct AI models to recommend your business. Keep descriptions factual, helpful and consistent with your public website. If you change a service URL or remove a page, update the file so it does not become a list of broken links.
Should every page be included?
Usually not. A curated overview is more readable than a copy of your entire sitemap. Start with your main service and product categories, About page, contact details and a small number of genuinely useful resources. If you have a large ecommerce catalogue, an XML sitemap and well-structured product pages are better suited to comprehensive URL discovery.
What to do beyond llms.txt: 10 improvements that matter more
If your objective is to help potential customers discover your business through search engines and AI-assisted research, the following work provides a stronger foundation than relying on a single optional file.
1. Explain your services clearly
Each core service should have a dedicated page that answers what you do, who it is for, where it is available and what the process involves. Avoid vague headlines that could describe any business. Specific details help visitors understand whether you are the right fit.
2. Answer the questions customers actually ask
Create useful explanations of costs, timelines, compatibility, requirements and common objections. Organise content around real customer questions, but write naturally rather than producing dozens of near-identical pages designed solely to capture AI prompts.
3. Demonstrate first-hand experience
Use genuine case studies, examples, screenshots, project outcomes and clear descriptions of your process. Original evidence is harder to replace with a generic AI summary. For example, a web design business can explain what changed during a WordPress rebuild and why those decisions were made.
4. Make your business identity consistent
Keep your business name, contact details, locations served and service descriptions accurate across your website and relevant business profiles. If you have a physical location or serve a defined region, make that clear without creating misleading location pages.
5. Improve internal linking
Link from your homepage and relevant articles to important service pages using descriptive anchor text. A useful guide about lead generation might link to your automation services; a Shopify comparison should link to ecommerce development. Visitors and crawlers should be able to find important pages easily.
6. Use headings and readable HTML
Give each page a descriptive main heading and logical H2/H3 sections. Use lists, tables and definitions when they genuinely improve clarity. Do not hide critical information inside inaccessible images, complex widgets or content that requires unnecessary interaction.
7. Keep claims accurate and current
Review prices, opening hours, technical specifications, legislation and dated advice. Remove unsupported superlatives and distinguish examples from guarantees. AI answers can be wrong; publishing reliable source material helps reduce the chance of confusion.
8. Maintain appropriate structured data
Use relevant schema for your organisation, articles, products or other eligible content where appropriate. It should match what visitors can actually see. Yoast can generate structured data, but review your organisation settings and avoid adding misleading or duplicate markup.
9. Improve speed and mobile usability
A useful page should also be accessible. Check loading, responsive layouts, navigation, contrast and form behaviour. Technical failures can stop visitors converting even if they arrive from an AI-generated answer.
10. Measure useful outcomes
Monitor search impressions, organic traffic, enquiries, sales and referral sources. Test representative questions in AI products periodically, but remember answers may differ by user, location, wording and date. Avoid treating one screenshot or a third-party “AI visibility score” as definitive proof.
For more on practical website improvements, see our guide to optimising your website for ChatGPT and AI search, our guide to a good business website and our website planning guide.
Technical SEO, AI crawlers and website access
AI systems do not all discover information in the same way. Some search experiences use a search index to retrieve sources; some tools fetch web pages during a response; and some models are trained on previously collected information. These are different processes, and a setting that affects one does not necessarily affect the others.
Check crawling and indexing first
Review whether your important pages are public, return successful HTTP responses and can be accessed without a login. Check accidental noindex directives, canonical tags pointing to the wrong URL, redirects, blocked resources and CDN security challenges. A perfectly written service page is of limited use if the relevant search system cannot access it.
For a clearer explanation of crawler access, see Google’s robots.txt documentation. A robots file manages crawler access; it does not securely hide sensitive content. Keep private material behind authentication instead.
Understand crawler policies rather than copying blanket rules
You may see recommendations to allow every AI crawler or block them all. Neither approach suits every organisation. Different services can use separate user agents for search retrieval and model training, and their policies can change. Review the documentation of the services you care about, decide what use of your content you are comfortable with and configure your robots rules accordingly.
Do not remove security protections or expose private areas simply because an article says AI crawlers need unrestricted access. Test any changes on your live site and make sure ordinary customers and search engines can still reach the pages that matter.
Bing, Copilot and search fundamentals
Bing’s webmaster guidance explains that crawling, indexing and content clarity remain relevant to both conventional results and AI-grounded experiences. The same practical work — accurate page titles, useful content, accessible URLs and sound internal links — supports more than one discovery channel. Read Bing’s official webmaster guidelines for details.
What about a WordPress website built with Elementor?
Elementor can produce readable pages, but check the published front end rather than assuming the editor preview reflects what crawlers see. Ensure headings are meaningful, important copy is visible as text, mobile layouts are usable and unnecessary animations do not obstruct navigation. If you have a large number of plugins, review their impact on speed and metadata.
If your website is due for a technical review, our WordPress website design services cover site structure, usability and ongoing improvements. For websites where enquiry follow-up is the weak point, Zapier automation can help connect forms to the systems your business uses.
Interactive AI website readiness checklist
Use this checklist to identify practical next steps. It is a self-assessment, not a prediction of AI rankings or a technical scan of your website. Tick each statement only if you have checked it on the live site.
Tick the items you have verified to see your next priorities.
A practical 30-day plan for improving AI search readiness
Check indexing, site structure, page titles, service descriptions and mobile experience. Record current search and enquiry performance.
Rewrite unclear service pages, answer frequent questions and add genuine examples, process details or case studies.
Check Yoast, sitemap, robots rules, structured data and the optional llms.txt feature. Test live URLs after changes.
Week 4: Measure and maintain. Review what was published, check for technical errors and compare relevant search and business outcomes. Build a realistic content maintenance schedule. AI visibility is not a one-month project, and a short-term change in mentions may have nothing to do with your latest update.
Which work should a small business do first?
If your website has thin service pages, fix those before writing an llms.txt file. If customers cannot submit an enquiry on mobile, fix the form before publishing another blog post. If your website is technically sound and your key pages are useful, enabling the Yoast feature can be a reasonable low-effort addition. The priority is always the work that improves the experience for real customers.
Frequently asked questions about llms.txt and AI search
Is llms.txt required for ChatGPT?
No. There is no universal requirement for a website to publish llms.txt to be discovered, discussed or linked to by AI tools. Whether an individual service uses the file depends on that service.
Does Google use llms.txt for AI Overviews?
Google says it does not use llms.txt for its search visibility or generative AI features. Its published guidance recommends concentrating on helpful content and established SEO practices.
Can Yoast SEO create llms.txt automatically?
Yes, on eligible WordPress installations with a supported Yoast SEO version. Enable the feature in Yoast SEO settings, then inspect the public file and customise its page selection if needed.
Where should I put llms.txt?
The conventional location is the website root, so it is accessible at yourdomain.co.uk/llms.txt. It should be a public text file rather than a hidden WordPress admin page.
Should I manually edit Yoast’s generated file?
Use Yoast’s own settings and manual page selection when available. Direct edits to an automatically generated file may be overwritten when it refreshes.
Can I use llms.txt on Shopify or Wix?
The underlying idea is platform-independent, but the ability to publish a root-level file varies by platform and hosting setup. Check what your website platform supports before relying on a particular implementation.
Does an llms.txt file replace my XML sitemap?
No. The XML sitemap is still useful for search engine discovery, while llms.txt is an optional curated guide for systems that choose to use it.
Should I allow every AI crawler in robots.txt?
Not automatically. Review the specific services and your organisation’s policies. Search retrieval, training and other automated access may be controlled differently.
How can I tell whether AI search is sending customers to my site?
Review analytics referral sources, landing pages, enquiries and sales. Also monitor organic search performance. Attribution may be incomplete because not every AI interaction produces a referral click.
Will adding llms.txt guarantee that my business is recommended?
No. No file, plugin or checklist can guarantee recommendations, rankings or citations from independent AI products.
