🔍 Read the full analysis: The AI Training Opt-Out From Four Companies: A Setup Guide on ThorstenMeyerAI.com
Get the latest gadgets delivered free with Prime
- Fast, free delivery on millions of items
- Prime Video, Amazon Music and more included
- Member-only deals all year
TL;DR
OpenAI, Google, Microsoft and Anthropic have announced plans to honor a shared robots.txt signal asking them not to use a website’s content to train generative AI models. The signal is voluntary, does not block AI answers based on live web content, and will be adopted on different schedules; Meta expressed support but made no implementation commitment.
OpenAI, Google, Microsoft and Anthropic said on October 30 that they plan to honor a shared robots.txt signal asking that a website’s content not be used to train generative AI models. The proposal could spare publishers from maintaining separate training opt-outs for each company, but it is a voluntary request, not a technical block, and the firms have set different implementation timelines.
The signal is defined in an Internet-Draft titled “Machine-Readable Training Preference Signal”, published by Anthropic standards lead Kevin Kelley. A site that wants to express the preference would add a rule group to its robots.txt file using User-Agent: __usage_ai-training followed by Disallow: /. The draft was published on October 30 and has no formal standing as an IETF standard.
The companies’ plans differ. Anthropic says it has started implementing support, though it expects the change to take time across its services. OpenAI aims to implement the signal “in the coming months.” Google says it aims to support it in 2027, with its crawler documentation specifying early 2027. Microsoft says it will honor the shared training preference from February 1, 2027, as it crawls websites.
Microsoft also announced a separate page-level preference for AI Grounding, which concerns use of website content to inform answers in Bing, Copilot and Microsoft 365. Microsoft says both new preferences take effect on February 1, 2027. Its existing NoArchive and NoCache tags will stop counting as preferences for training or grounding after March 1, 2027. Meta spokesperson Andy Stone expressed support for a common signal but did not commit the company to honoring it.
Infographic 01 · Who backs it, and when
One line in robots.txt, four companies, four timelines.
A shared signal against AI training. Only Anthropic says it has started; the others name dates or months. The standard itself is still a draft.
- 30 Oct 2026Draft published, four companies announce supportdoneIndividual Internet-Draft by Kevin Kelley (Anthropic)
- 30 Oct 2026Anthropic: “has already started implementing support”startedIts standards lead on the IETF list: “It’ll take us some time to add support across our services.”
- Coming monthsOpenAI aims to implement itplanExisting GPTBot and OAI-SearchBot rules keep applying
- 16 + 19 Nov 2026Working group meets at IETF 127, San FranciscoopenOn 21 Oct a chair said no consensus has been declared yet
- 1 Feb 2027Microsoft honors the new preferences as it crawlsplanTraining in robots.txt or per page; grounding per page
- 1 Mar 2027Bing stops reading NoArchive / NoCache as AI preferencesdeadlinePublish the new settings before this date
- 2027Google plans support; its docs say “early 2027”planA “no” maps to Google-Extended unless you set that token yourself
- 3 May 2027Draft version 00 expires unless updatedInternet-Drafts are valid for at most six months
Binding? No, voluntary
- OpenAI calls it a “voluntary effort”.
- The vocabulary draft does not “ensure that preferences are followed”.
- The draft “has no formal standing in the IETF standards process”.
- Private agreements may override it, the draft says.
Who else?
- Meta supports a common signal, but did not say whether it will honor it.
- Apple, Amazon, xAI, Perplexity: no statement in the sources reviewed for this article.
- The draft’s acknowledgments list contributors “for identification only”, not supporters.
Sources: IETF draft, 30 Oct 2026 · Anthropic on the IETF list · AIPREF minutes, 21 Oct 2026 · Google, 30 Oct 2026 · Google crawler docs · OpenAI, 30 Oct 2026 · Microsoft Bing, Oct 2026 · Meta (Andy Stone on X). Company plans are company statements.
Powered by Thorsten Meyer AI
One Rule, Uneven Publisher Control
A shared signal could make it simpler for website operators to communicate a training preference across multiple AI companies. Without it, publishers have had to use company-specific controls, such as rules for OpenAI’s GPTBot, Google-Extended and Anthropic’s ClaudeBot. Microsoft has used separate page tags. The shared proposal offers a common option, but sites may still need existing controls while companies roll out support.
The change also clarifies that training and AI-generated answers are separate choices. A publisher may object to having material used to train a model while still allowing it to appear in search or be used to answer a question using current web content. Microsoft’s separate grounding preference makes that distinction explicit for its services. The shared training rule, on its own, does not control that use.
For publishers, the practical effect depends on whether each company implements the signal and how it applies it. For readers and site owners, the announcement is not a guarantee that opting out will remove material already incorporated into existing models, prevent all use of a site, or result in payment. Microsoft said its preferences do not guarantee inclusion, traffic or payment.
Products Worth Considering
Logitech MX Brio Ultra HD 4K Webcam, Graphite
Samsung 27-inch Odyssey OLED G6 QD-OLED QHD Gaming Monitor
DJI Mini 5 Pro Fly More Combo Plus with DJI RC 2
As an affiliate, we earn on qualifying purchases.
The proposal builds on the AI Preferences working group at the Internet Engineering Task Force, an organization that develops internet standards. Kelley published the individual draft on October 30, and the companies’ announcements refer to the same document. Its author’s affiliation is Anthropic; the companies’ stated plans are separate commitments and should not be confused with formal adoption of an internet standard.
The draft remains at an early stage. At a working-group meeting on October 21, one of the group’s chairs said, “We haven’t declared consensus on anything yet.” The draft’s existence and the companies’ public announcements are confirmed, but the signal has not been approved as an official IETF standard.
Existing controls remain relevant during the rollout. Google says that if a site uses the new token but has no Google-Extended rule, it plans to interpret the new signal as a disallow for Google-Extended. OpenAI says its current crawler controls will continue to apply. Microsoft says its changes do not alter eligibility for Bing search results; Google says Google-Extended does not affect Google Search. These distinctions mean a training opt-out is not automatically a general search opt-out.
“Anthropic “has already started implementing support for this draft.””
— Anthropic standards lead Kevin Kelley
Adoption and Coverage Remain Open
Whether every company will follow the signal remains uncertain. Meta has expressed support without making an implementation commitment. The source material reports no statement from Apple, Amazon, xAI or Perplexity about honoring it. The draft is voluntary and has not received formal IETF standing, so its wording and status could still change.
The precise rollout and application may also vary. Anthropic says implementation has begun but that support across its services will take time; OpenAI has not provided a firm date beyond “in the coming months.” Google and Microsoft have given 2027 dates, but the signal does not itself specify enforcement or what happens when a crawler does not comply. It is also not a way to remove content from models already trained, and it does not cover AI answers that draw on live web content unless a separate control applies.
Watch the 2027 Rollout Dates
Website operators can add the shared training rule if they want to express the preference, but should check each company’s documentation and keep existing company-specific rules until the relevant rollout is in place. Microsoft users relying on NoArchive or NoCache for training or grounding preferences will need to review those settings before the tags stop counting for those purposes after March 1, 2027.
The next milestones are Anthropic’s expansion of support across its services, OpenAI’s planned implementation in the coming months, and the 2027 launches from Google and Microsoft. Further statements from Meta and other AI companies, along with the IETF working group’s consideration of the draft, will show whether the proposal becomes a broadly observed convention. Until then, publishers’ choices remain requests whose effect depends on each company’s implementation.
Key Questions
What does the shared AI training opt-out do?
It lets a website state in robots.txt that it does not want participating companies to use its content to train generative AI models. It is a preference signal, not a technical barrier.
Which companies have said they will honor it?
OpenAI, Google, Microsoft and Anthropic have announced plans to honor the signal, with different rollout dates. Meta expressed support but did not say whether it would implement it.
Does the rule block AI answers or remove content from existing models?
No. The shared rule addresses training. It does not remove material already used in trained models or, by itself, control AI answers based on live web content.
Is the signal an official internet standard?
No. It is an individual Internet-Draft connected to the IETF’s AI Preferences working group, and it has not been formally adopted as an IETF standard.
Should publishers remove their existing crawler rules?
Not yet. Companies are adopting the shared signal on different schedules, and existing controls remain in effect. Publishers should keep company-specific rules until each relevant company’s implementation and documentation are clear.
Source: ThorstenMeyerAI.com
