If you run a website, you may soon be able to say “don’t train AI on my content” once, in one robots.txt group, instead of writing separate rules for every AI company’s crawler. On October 30, 2026, Anthropic’s Kevin Kelley published an IETF draft for a shared training-preference signal, and Google, Microsoft, and OpenAI said the same day that they plan to honor it. None of them support it yet.

Coffee summary

  • FACT: The signal is a new robots.txt “policy token,” __usage_ai-training, used with ordinary Allow and Disallow lines.
  • FACT: It states a preference about AI training. It does not block crawling, and it does not affect search rankings.
  • FACT: Timelines differ: OpenAI says “in the coming months,” Google says early 2027, and Microsoft says February 1, 2027. Anthropic says it has started work.
  • FACT: The draft is an individual submission. The IETF has not endorsed it.
  • OPINION: Publishers should plan for it now, but keep their existing crawler rules in place.

What happened

The draft, titled “draft-kelley-aipref-training-preference-signal-00,” came out of the IETF’s AI Preferences (AIPREF) working group discussions. It adds no new robots.txt directive or grammar. Instead, a site writes a normal rule group whose User-Agent line names the policy token rather than a crawler.

The draft’s own example opts the whole site out of AI training except a press kit folder:

User-Agent: __usage_ai-training
Disallow: /
Allow: /press-kit/

“AI training” here means using content to change a generative AI model’s learned parameters, according to the working group’s vocabulary draft. Non-generative uses such as classification or ranking are outside that definition.

Why it matters

Today, a site that wants to stay out of AI training must track crawler names such as GPTBot, Google-Extended, and ClaudeBot, and keep adding rules as new ones appear. A shared token could replace that list with a single group, at least for the companies that honor it.

Microsoft framed it as a choice publishers should not have to make: being discoverable in Bing versus deciding how generative models use their work.

What changed: who supports it and when

Company Plan Notes
OpenAI “In the coming months” Existing crawler controls such as GPTBot still apply.
Google 2027 (crawler docs say early 2027) If you opt out with the token and do not mention Google-Extended, Google plans to treat it as a Google-Extended block.
Microsoft Honors new preferences from Feb 1, 2027 Replaces NoArchive/NoCache for training and grounding; old tags stop counting for those uses after March 1, 2027.
Anthropic Work has started Kelley said support across services will take time.

Microsoft is also adding a separate, page-level AI Grounding preference. Grounding is when an AI answer draws on a live web page. A DisallowExtended option allows grounding in Microsoft’s own products, such as Copilot, while blocking the grounding services Microsoft sells to other businesses. Microsoft says these preferences do not change a page’s place in Bing search results.

Who should care

  • Publishers and bloggers who want a simpler way to opt out of AI training.
  • SEO teams, because Google ties the token to Google-Extended, which also governs grounding in Gemini Apps.
  • Sites using NoArchive or NoCache to limit Copilot. Microsoft’s change affects you first.

Limitations

  • Nothing works yet. Based on each company’s timeline, none of the three supports the token today.
  • No legal force. It is a stated preference, not a technical block or a license. Microsoft said it does not create licensing or payment arrangements.
  • Wildcards don’t count. A User-Agent: * group never matches the token, and a robots.txt file without a __usage_ai-training group expresses no preference.
  • Not the same everywhere. At Google, training and Gemini Apps grounding sit under one control. At Microsoft, they are separate settings.
  • Private deals can override it, according to the draft as reported by The Next Web.

What to do next

  1. Keep your current rules. Do not remove GPTBot, Google-Extended, or ClaudeBot lines yet.
  2. Decide your policy. Do you want to block training only, or grounding too? Blocking grounding can reduce your visibility in AI answers, Microsoft warned.
  3. If you use NoArchive/NoCache for Bing, read Microsoft’s guidance. Search Engine Journal reports a temporary Bing Webmaster Tools setting opens December 15 for sites still using those tags.
  4. Add the token when you’re ready. It does not change crawling, so adding it early should not affect search indexing. Re-check each company’s docs as support goes live.

AIImpish Take

This is the first time the biggest AI companies have lined up behind one training opt-out, and that alone simplifies a messy corner of the web. But it is a preference signal with staggered start dates and different meanings at each company. Publishers should add it to their plans, not treat it as a finished shield.

Related on AIImpish

*Sources: OpenAI, “Making it easier for websites to express AI training preferences,” Oct 30, 2026; IETF draft-kelley-aipref-training-preference-signal-00; Google crawler documentation; Search Engine Journal, Oct 31, 2026; The Next Web, Oct 31, 2026.*