A few weeks ago I shipped a guide arguing llms.txt is a cheap bet — twenty minutes of work to make your site legible to AI engines. I stand by that. I keep one on this very site.
So when people started messaging me "Google reversed course! They finally admit llms.txt works for AI SEO!" — I went and read the actual change.
It doesn't say that. Not even close.
What Google did is quieter, and honestly more interesting. A year ago they were openly mocking the file — "a stupid idea," "like the keywords meta tag." In June 2026 they slipped softer wording into a docs changelog: it's fine, it won't hurt. No announcement. No "we changed our minds." Just a tonal climbdown from ridicule to a shrug.
That's the story. Not a reversal — a face-saving retreat from mockery to indifference, done on the quiet. Here's the gap between what Google wrote and what got reported, and why the data still sides with Google over the headlines.
Key Takeaways
- Google's June 2026 update says llms.txt is "fine" to use but "won't harm (nor help)" your Google Search visibility — that's harmless, not helpful (Search Engine Journal, 2026).
- Google did not reverse its position. It's said the same thing since 2024: llms.txt is not a ranking or AI-visibility signal for Google Search.
- Ahrefs analysed 137,210 domains and found 97% of llms.txt files got zero bot requests in May 2026 (Ahrefs, June 2026).
- The softer wording isn't isolated — it rides Google's May 2026 "AEO and GEO are still SEO" guide and rising regulatory pressure on AI scraping. Context, not a coincidence.
- The file still earns its keep as a map for AI coding agents and a couple of engines — just don't expect Google to read it.
What did Google actually change?
In June 2026, Google updated its Search Central documentation with softer wording, not a new position. The new text reads: "It's completely fine if you decide to create and maintain LLMS.txt files… for other services or systems that use these files," adding that they "won't harm (nor help) your visibility or rankings in Google Search, as Google Search ignores them" (Search Engine Journal, June 2026).
Read that twice. "Won't harm nor help." "Google Search ignores them."
That's not an endorsement. That's Google saying do what you like, it makes no difference to us.
In June 2026, Google's documentation changelog filed this under "Clarifying guidance on llms.txt files" — a clarification, not a reversal (Search Engine Journal, June 2026). The only real shift: the old guidance discouraged the file broadly, and the update narrows "not needed" to Google Search specifically, while conceding other AI services exist that might use it.
So did Google reverse its position on llms.txt?
No. Google's stance has been remarkably consistent since 2024, and the June update didn't break it — it just lowered the volume. Over a year ago, Google's John Mueller compared llms.txt to the long-dead keywords meta tag, noted that no AI services requested the file, and called the idea of maintaining separate Markdown for bots "a stupid idea" (Search Engine Journal, 2026).
Then it got blunter. At Search Central Live Deep Dive Asia Pacific in 2025, Gary Illyes and Amir Taboul confirmed Google was not pursuing llms.txt (Search Engine Journal, 2026). And in May 2026, Google's AI-optimization guide listed the file among tactics you can skip for AI Overviews and AI Mode.
So the timeline is: not needed → not needed → not needed → "fine, but still ignored." The substance never moved. Only the phrasing got friendlier — and friendlier phrasing is what people mistook for a green light.
Mueller spelled out why it can't be a ranking signal, again, on the June 2026 Search Off the Record podcast. His take on a site declaring its own greatness in a text file: "It's basically you're telling these systems, like, I have the best website ever" (Let's Data Science, June 2026). Every site can write that. When everyone claims to be the best, the claim is worth nothing.
Self-reported metadata is exactly the kind of signal search engines learned to ignore a decade ago.
He did leave one narrow door open, though: once an AI agent is already on your site, llms.txt might help it find the right pages or finish a task. Navigation help for a visitor that already showed up — not a reason it'd show up in the first place. Hold that thought; it's the only real use case, and we'll get to it.
Google's June 2026 guidance says llms.txt files "won't harm (nor help) your visibility or rankings in Google Search, as Google Search ignores them." That is the opposite of an endorsement: it's a formal confirmation that Google Search does not read, weight, or reward the file in any of its surfaces, including AI Overviews and AI Mode.
Does llms.txt actually help AI SEO? Here's the data
In May 2026, 97% of the llms.txt files Ahrefs checked received zero requests — nothing fetched them at all. Ahrefs examined 137,210 domains that got traffic that month, found that 28% published a valid llms.txt file, and then watched what happened to those files (Ahrefs, "We Analyzed 137K Sites: 97% of llms.txt Files Never Get Read", June 2026).
The funnel falls apart fast.
Of the slim 3% that got any traffic, 96% of it came from bots, and only 19.5% of requests came from named AI tools — with GPTBot and Claude-Code leading the pack (Ahrefs, June 2026). The kicker: zero requests came in for llms.txt files that didn't exist. AI bots never go looking. They only fetch the file when something points them at it.
That last detail is the whole ballgame. A signal nobody requests can't be a ranking signal.
Then why does Google's own Chrome team check for it?
Because "Google" isn't one opinion — it's several teams that don't always agree. In May 2026, Chrome's Lighthouse 13.3 shipped an "Agentic Browsing" audit category that includes an llms.txt check, describing the file as a machine-readable summary for LLMs and AI agents (Search Engine Journal, 2026).
So one arm of Google audits your site for a file another arm says it ignores. Cool. Very clear. Thanks, everyone.
This isn't a contradiction once you separate the use cases. Search says: useless for ranking. Lighthouse says: useful for browser-agent readiness — agents that browse on a user's behalf, not the Google crawler that builds the index. Different jobs.
Same filename. Endless confusion.
And it's not the first time Google tripped over its own file. Not long ago, Google's own developer docs briefly sprouted an llms.txt — then yanked it within hours, with Mueller reacting "Hmmn :-/" when someone noticed (Search Engine Roundtable, 2026). Even Google can't keep its story straight internally. Which is exactly why you should read the guidance, not the vibes.
Why did Google soften the tone now?
Because the llms.txt line didn't ship alone — it landed inside a broader 2026 repositioning. On May 15, 2026, Google published its first AI-optimization guide, answered "is SEO still relevant for generative AI search?" with "in short, yes," and collapsed the whole AEO/GEO vocabulary back into plain SEO — listing llms.txt among the tactics you don't need (Search Engine Journal, May 2026). The softer llms.txt wording in June is a footnote to that bigger message: stop buying AI-specific snake oil.
Two things are happening at once. AI Mode is eating Google's own results — more answers synthesized on the page, more zero-click, AI Overviews "rooted in our core ranking systems" rather than some separate engine you can game. And regulators are circling. On June 5, 2026, the UK's Competition and Markets Authority forced Google — in what it called a "world first" — to let publishers opt out of AI scraping for search summaries, cite sources with clear links, and opt out of model fine-tuning (Fortune, June 2026).
The EU's Digital Markets Act has its own July 2026 deadline pointed at Google.
Google hasn't said these are connected. But the timing isn't subtle. When you're already fighting publishers and regulators over how you use everyone's content, openly mocking a harmless little community file is a needless extra fight. "It's fine" costs Google nothing, calms the noise, and keeps the actual message front and center: do good SEO, and the citations follow.
In May 2026 Google published its first AI-optimization guide, declaring that AEO and GEO are "still SEO" and that its AI features draw on the same core ranking systems as regular Search. The guide lists llms.txt, content chunking, and AI-specific rewriting as unnecessary — framing the later "it's fine to use llms.txt" note as a minor softening within a campaign against AI-specific optimization hacks, not an endorsement of any one of them.
And here's the distinction that keeps tripping people up: the file that actually controls AI access to your site is robots.txt (plus the Google-Extended token), not llms.txt. robots.txt is how you allow the retrieval bots that send traffic and block the training crawlers that don't. llms.txt is a suggested reading list nobody's obligated to open. One is a lever. The other is a sticky note. Don't confuse the file you use to opt out of scraping with the file that supposedly wins you AI citations — they're not the same, and only one of them does anything.
Should you bother with llms.txt in 2026?
Maybe — but for honest reasons, not because Google "approved" it. Google's position is that the file is neutral for Search, and the adoption data shows most files sit unread, so any case for llms.txt rests entirely on the non-Google agents that do fetch it: a handful of AI engines and, more reliably, AI coding tools parsing your docs.
Here's my actual decision rule, unchanged by the June update:
- You run developer docs or a tool/API. Worth it. Coding agents (Claude Code, Cursor, Copilot) genuinely use it as a routing layer. Real value, today.
- You want Perplexity and similar to grab clean summaries. A modest bet. Some engines fetch it; many don't. Low cost, low-but-nonzero upside.
- You think it'll lift your Google rankings or AI Overview citations. Don't. Google told you, twice, in plain English. Spend that hour on the foundational stuff that does move the needle.
If you want the build details — the spec, the /llms.txt route handler, the curation rules — I already wrote that up in the llms.txt definitive guide. This post is the reality check that sits next to it: do it if the non-Google payoff is worth twenty minutes, and ignore anyone selling it as a Google AI-SEO hack.
Because the boring truth is the same as it's always been. The way you get cited by AI search is by being a page worth citing — fast, crawlable, clearly written, well-structured. That's where AI genuinely helps SEO versus where it quietly wrecks it, and it's why I keep pointing people back at technical fundamentals that actually decide whether you get indexed instead of the file-of-the-month.
Frequently asked questions
Will an llms.txt file help your SEO?
No. Google's June 2026 guidance says llms.txt is "fine" to maintain for other services but "won't harm (nor help)" your Google Search visibility, because Search ignores the file (Search Engine Journal, 2026). "Fine to use" describes a file that's harmless, not effective.
Does llms.txt work?
Not as a search or ranking lever. It "works" only in a narrow sense: as a routing map for AI agents already visiting your site, mostly coding tools. Ahrefs found 97% of published llms.txt files got zero bot requests in May 2026 (Ahrefs, 2026), so for most sites it does nothing because nothing reads it.
Does Google use llms.txt?
No. Google Search does not read llms.txt for ranking, AI Overviews, or AI Mode, and states no special markup or Markdown is needed to appear in those features (Search Engine Land, June 2026). Confusingly, Chrome's Lighthouse audits for the file — but that's for browser agents, not Google Search.
How many sites' llms.txt files actually get read?
Very few. In a June 2026 study of 137,210 domains, Ahrefs found 28% published a valid llms.txt file, but 97% of those files received zero bot requests in May 2026 (Ahrefs, 2026). Of the 3% that saw any traffic, most was bots, and under a fifth came from named AI tools.
Who actually uses llms.txt then?
Mostly AI coding agents. Tools like GPTBot and Claude-Code lead the (small) pool of real requests, using the file as a routing map for documentation (Ahrefs, 2026). A few AI search engines such as Perplexity fetch it anecdotally. The Google crawler that builds Search does not.
Should I delete my llms.txt now that Google ignores it?
Not necessarily. If non-Google agents (coding tools, certain engines) benefit from it and it's already built, leave it — Google confirmed it won't hurt you. If you only created it expecting Google ranking lift, that expectation was always wrong, and the file costs you nothing to keep or drop.
The bottom line
Google didn't bless llms.txt for AI SEO. It said the file is harmless and that Search ignores it — then a clarification got reported as a conversion. The independent data lands on Google's side: most files are never read, and the bots that do read them are coding agents, not the engine ranking your pages.
Keep an llms.txt if a non-Google agent gets value from it. That's a real, narrow, twenty-minute reason. Just stop reading "fine" as "finally works," and put the hour you saved into the AI-SEO work that genuinely compounds.
Sources
- Search Engine Journal, "Google's Updated Guidance Now Says It's 'Fine' To Use LLMs.txt For AI SEO," retrieved 2026-06-19, https://www.searchenginejournal.com/googles-says-its-fine-to-use-llms-txt-for-ai-seo/579608/
- Search Engine Journal, "Google's llms.txt Guidance Depends On Which Product You Ask," retrieved 2026-06-19, https://www.searchenginejournal.com/googles-llms-txt-guidance-depends-on-which-product-you-ask/575431/
- Search Engine Land, "Google says llms.txt files won't harm or help your search rankings," published 2026-06-15, retrieved 2026-06-19, https://searchengineland.com/google-says-llms-txt-files-wont-harm-or-help-your-search-rankings-480264
- Let's Data Science, "Google's Mueller Says llms.txt Won't Guide LLM Recommendations," retrieved 2026-06-19, https://letsdatascience.com/news/googles-mueller-says-llmstxt-wont-guide-llm-recommendations-8afc3fa3
- Search Engine Journal, "Google's New AI Search Guide Calls AEO And GEO 'Still SEO'," published 2026-05, retrieved 2026-06-19, https://www.searchenginejournal.com/googles-new-ai-search-guide-calls-aeo-and-geo-still-seo/575026/
- Ahrefs, "We Analyzed 137K Sites: 97% of llms.txt Files Never Get Read," published 2026-06-15, retrieved 2026-06-19, https://ahrefs.com/blog/llmstxt-study/
- Fortune, "Google forced to allow news sites to opt out of AI scraping in 'world first,' UK watchdog says," published 2026-06-05, retrieved 2026-06-19, https://fortune.com/2026/06/05/auto-draft-3/
- Search Engine Roundtable, "Google Says No AI System Currently Uses LLMs.txt," retrieved 2026-06-19, https://www.seroundtable.com/google-ai-llms-txt-39607.html