What llms.txt is and what it is not
llms.txt is an editorial index for AI retrieval. It is not a robots.txt, not a sitemap, and not a machine-generated list.
llms.txt is a proposed standard introduced by Jeremy Howard in September 2024. The idea is simple: a single-file markdown document at the root of your site that tells an AI retriever which pages are the canonical, most important representations of your content. Think of it as a human-curated reading list for AI crawlers, similar to how a table of contents helps a reader find the meaningful parts of a long book. It is not a legal permission file (that is robots.txt), it is not an exhaustive URL list (that is sitemap.xml), and it is not meant to be machine-generated from a template.
- robots.txt
- Legal permission — tells crawlers which URLs they are allowed to fetch. Required by every site.
- sitemap.xml
- Exhaustive URL index — lists every discoverable URL so crawlers can find and schedule them. Required for search engines.
- llms.txt
- Editorial curation — tells AI retrievers which 20-40 URLs on your site are the canonical, quotable, authoritative representations of your content. Optional but high-signal for GEO.
- llms-full.txt
- Extended version — same format as llms.txt but with longer descriptions and up to 200 URLs. Used when you want to include secondary resources without diluting the core curated list.
Which AI engines actually read llms.txt today
As of April 2026, Perplexity and Claude officially parse it. ChatGPT, Gemini, and Copilot use it as a signal without confirming the contract. Bing respects it opportunistically.
llms.txt adoption by AI engines is uneven and evolving. The public picture is that Perplexity and Anthropic's Claude explicitly document that their crawlers fetch /llms.txt on discovery and use it as a prioritization signal. OpenAI, Google, and Microsoft have not formally confirmed parsing of the file, but empirical evidence from server-log studies shows GPTBot, Google-Extended, and Bingbot all fetch /llms.txt when they discover a site — so the file is being read, even if the downstream weighting is not public.
Current state by engine
- Perplexity
- Explicit support. Uses llms.txt as a prioritization signal for which pages to crawl first when discovering your site.
- Claude (Anthropic)
- Explicit support documented. Claude's web retrieval layer reads llms.txt on site discovery.
- ChatGPT (OpenAI)
- Not confirmed but empirically read by GPTBot and OAI-SearchBot. Treat as a signal, not a contract.
- Google (AI Overviews, Gemini)
- Not confirmed. Google-Extended fetches /llms.txt but Google has not commented on how it is weighted.
- Copilot (Bing)
- Opportunistic. Bingbot fetches the file when discovered; weighting is unclear.
The llms.txt format, with a Shopify example
Markdown. H1 for your brand, optional blockquote, H2 for categories, bulleted link lists. That is the entire spec.
The format is deliberately minimal. H1 names your site. An optional blockquote (>) gives a one-sentence description. H2 headings group pages by category. Under each H2 is a bulleted list of markdown links with a short description after each link. The file lives at /llms.txt at the root of your domain.
# Kloira
> Kloira designs and handcrafts moissanite watches for men and women, based in the United States with a lifetime warranty on every timepiece.
## About
- [About Kloira](https://kloira.com/pages/about) — Our story, values, and the team behind the watches.
- [Our craftsmanship](https://kloira.com/pages/craftsmanship) — How every Kloira watch is made, from movement sourcing to final inspection.
- [Lifetime warranty](https://kloira.com/pages/warranty) — Terms of our lifetime warranty and how to claim it.
## Buyer Guides
- [How to choose a moissanite watch](https://kloira.com/blogs/guides/choose-moissanite-watch) — A 10-minute buyer's guide covering size, movement, and style fit.
- [Moissanite vs diamond in watches](https://kloira.com/blogs/guides/moissanite-vs-diamond) — The difference in sparkle, hardness, and price.
- [Watch size guide for men](https://kloira.com/blogs/guides/mens-watch-size-guide) — How to pick the right case size for your wrist.
## Hero Products
- [Kairos Chronograph](https://kloira.com/products/kairos-chronograph) — Men's 42mm moissanite chronograph with Swiss movement, from $399.
- [Nova 36 Ladies](https://kloira.com/products/nova-36-ladies) — Ladies' 36mm moissanite dress watch with sapphire crystal, from $329.
- [Aeris Automatic](https://kloira.com/products/aeris-automatic) — Unisex automatic moissanite timepiece with a 48-hour power reserve, from $599.
## Policies
- [Shipping policy](https://kloira.com/policies/shipping-policy) — Free US shipping over $100, international rates, delivery times.
- [Returns and refunds](https://kloira.com/policies/refund-policy) — 30-day returns on all products, refund timelines, condition requirements.
- [Contact us](https://kloira.com/pages/contact) — Email, phone, and live-chat support hours.That is roughly the size of a useful llms.txt. Under 20 entries and it is too sparse; over 50 entries and it loses the curation value that distinguishes it from a sitemap. 20-40 entries is the sweet spot.
What to include in a Shopify llms.txt — and what not to
Include: top products, top collections, best buyer guides, core policy pages. Exclude: every product, every blog post, auto-generated variants.
The curation decision is the whole point of the file. A well-chosen 25-page llms.txt outperforms a machine-generated 500-page one on every metric we have measured. The selection rules below are the ones we apply when we curate an llms.txt for a Shopify customer.
Include
- Your about page and contact page — these are E-E-A-T anchors for AI retrievers.
- Your 5-10 top-selling or hero products — not every product; the ones you would mention first if someone asked.
- Your 5-10 most-read buyer guides or blog posts — the ones that already earn external links.
- 3-5 hero collection pages, especially the ones that match common buyer-intent queries.
- Your shipping, returns, and warranty policy pages — AI retrievers get asked about these on almost every buyer query.
- Any original data or research you publish (studies, reports, proprietary benchmarks).
Exclude
- Every product — the point is to highlight the canonical ones, not to reprint the catalog.
- Tagged or filtered collection URLs — they dilute the collection signal.
- Blog posts older than 18 months that you would not stand behind today.
- Thank-you pages, cart pages, login pages, and other transactional UI.
- Duplicate content pages (e.g., print versions, AMP versions).
How to actually deploy llms.txt on a Shopify store
Three paths: a theme template, a Shopify app, or a reverse-proxy rewrite. Each has tradeoffs.
Shopify does not let you upload arbitrary files to your domain root the way a WordPress site does. That limitation is a design choice and you need to work around it. There are three reliable paths; pick the one that fits your team's technical capacity.
Path 1: Theme template with a static Liquid route
Create a new template at templates/page.llms.liquid that serves the markdown content as a text/plain response. Create a page in Shopify Admin at /pages/llms, assign the template, and put the markdown in the page body. Then redirect /llms.txt to /pages/llms at the CDN layer (Cloudflare) or via a Shopify redirect. This works but has two catches: Shopify content-type detection may serve it as text/html, and the URL is /pages/llms.txt rather than /llms.txt unless you set a rewrite.
Path 2: Shopify app
An app that runs a subdomain webhook or a theme extension can intercept /llms.txt at the edge and serve the correctly-curated content. This is how Surfient ships llms.txt — an app extension that renders the file dynamically from your catalog without requiring theme edits. This is the lowest-friction path if you are running an AI-indexing app already.
Path 3: Cloudflare Worker or reverse-proxy rewrite
If you run Shopify behind Cloudflare (most stores do), a small Cloudflare Worker can intercept /llms.txt requests and serve a static markdown response without touching the Shopify store. This is the most technically clean path but requires Worker deployment access and is brittle if the file content changes often.
// Cloudflare Worker that serves /llms.txt from KV storage
export default {
async fetch(request, env) {
const url = new URL(request.url);
if (url.pathname === '/llms.txt') {
const content = await env.LLMS.get('llms-txt-content');
return new Response(content, {
headers: { 'content-type': 'text/markdown; charset=utf-8' },
});
}
return fetch(request);
},
};Five common llms.txt mistakes on Shopify stores
The recurring failure modes we see on audits — each one is easy to fix, easy to catch with a validator.
- 1Auto-generating from the sitemap. Kills the signal value entirely. The file should be curated or it should not exist.
- 2Wrong content-type header. Shopify sometimes serves /pages/llms.txt as text/html instead of text/markdown or text/plain; engines may discard it. Verify with curl -I.
- 3Linking to URLs that do not resolve or redirect. A 301 or 404 in your curated list is a trust signal against you. Validate every URL before shipping.
- 4Including staging URLs or internal URLs. If your llms.txt lists preview.shopify.com URLs or localhost URLs, it leaks internal details and engines will weight it down as low-quality.
- 5Not updating it when your top products or top guides change. A file from October 2024 that still lists your Black Friday hero product from 2023 is worse than no file at all.
“The difference between a useful llms.txt and a useless one is the same as the difference between a chef's tasting menu and a list of everything in the kitchen. Curation is the entire product.”