Microsoft joins major AI firms in backing a shared publisher opt-out

Microsoft has joined Google, OpenAI and Anthropic in backing a shared way for publishers to signal that their websites should not be used to train generative AI models. The proposed robots.txt instruction could simplify settings across providers, but it is voluntary, not yet an adopted internet standard, and does not itself block crawling. Key takeaways […]

Server towers beneath a shared glowing blue digital gateway

In This Article

Share this

Microsoft has joined Google, OpenAI and Anthropic in backing a shared way for publishers to signal that their websites should not be used to train generative AI models. The proposed robots.txt instruction could simplify settings across providers, but it is voluntary, not yet an adopted internet standard, and does not itself block crawling.

Key takeaways

The announcement gives website owners a common preference to communicate, while leaving important differences in timing and scope between providers. For enterprise teams, the distinction between training, crawling and AI-generated answers will shape how the change affects content strategy.

  • The shared robots.txt token is __usage_ai-training.
  • Microsoft plans to honor it from February 1, 2027; Google targets early 2027, while OpenAI says implementation is coming in the coming months.
  • Anthropic says it has begun implementing support, but has not set a completion date.
  • The signal does not automatically prevent crawling, guarantee payment or undo prior data collection.

One instruction for participating providers

Publishers currently need separate crawler rules for different AI companies. Under the proposal, they could use a single policy token in a robots.txt group to express whether content may be used for AI training, including setting different preferences for selected website paths. Microsoft, Google and OpenAI announced plans to support the signal on October 30, 2026; Anthropic shared its commitment through the standards working group.

The mechanism is based on an individual Internet-Draft submitted to the Internet Engineering Task Force’s AI Preferences working group. It has no formal standing as an IETF standard and may change. The draft also specifies that a general wildcard rule such as User-agent: * does not count as expressing a preference through this new signal.

Training is not crawling or grounding

The token expresses a preference about using content to change a generative model’s learned parameters. It does not instruct every crawler to stay away, nor does it replace provider-specific access rules. Publishers seeking to prevent crawling still need to manage those rules separately, and the proposal cannot reverse collection that has already occurred.

Microsoft is also introducing separate page-level controls for AI grounding—the use of web content to inform generated answers. Its training and grounding preferences are distinct, while Google says it will interpret the shared training opt-out as a Google-Extended disallow. Google-Extended also covers grounding for Gemini services, so Google’s handling may affect answer-time use as well as training. Microsoft says restrictions do not affect traditional Bing search eligibility; neither company guarantees visibility in AI answers.

What publishers and enterprise teams should watch

Microsoft’s transition includes a December 15, 2026, opening date for temporary grounding preferences in Bing Webmaster Tools for eligible sites using its older NoArchive or NoCache controls. Microsoft plans to stop treating those tags as AI preferences after March 1, 2027. Website owners will need to review their settings and timing, particularly because preferences are applied as Microsoft recrawls pages.

For enterprises, the announcement underscores the value of clear content governance: teams need to know which material is published publicly, how it may be used, and which controls apply across platforms. That is separate from governing internal analytics data. Spargent helps organizations build and manage Microsoft Fabric and Power BI data platforms, supporting reliable reporting and decision-making as digital policies evolve. The shared signal may reduce repetitive website configuration, but it does not settle copyright disputes, create licensing terms or establish compensation.

Spargent Analytics Logo Microsoft Fabric Consulting services

Spargent Analytics

Microsoft Fabric consulting, implementation, analytics modernization, and long-term support for enterprise data teams.

Microsoft Fabric
Project Review

Free Expert Session
Need help turning this insight into a Microsoft Fabric roadmap?

Spargent Analytics can help you design, implement, migrate, and optimize Microsoft Fabric solutions that bring your data, analytics, AI, and business intelligence into one secure and scalable platform.

More insights

Continue with related Microsoft Fabric articles.

Eventstreams vs Data Pipelines vs Mirroring in Microsoft Fabric

Choosing the wrong ingestion method in Microsoft Fabric creates pain fast. You either overbuild for a simple load, or you

KQL Database vs Lakehouse for Operational Analytics

Operational teams need to know immediately when a production line begins to drift, inventory runs short, or transactions start failing.

Microsoft’s Xbox Patent Sketches AI-Driven Stories and In-Game Ads

Microsoft has outlined two separate patent concepts that could change how games respond to players: one uses generative AI to

Start a Conversation

We will get back to you within 24 hours with proposal to set up intro call.