Training is not grounding, and as of this week the difference has dates on it. Google, Microsoft, OpenAI and Anthropic have backed a shared robots.txt signal, drafted at the Internet Engineering Task Force, that lets a site owner answer two questions separately: may my pages be used to train the next model, and may my pages be retrieved and cited inside an answer today. Microsoft starts honoring training opt-outs on February 1, 2027. Google targets support sometime in 2027. The file being edited is one line long. The decisions it records are among the largest distribution decisions most site owners will make this decade, and they are two decisions, with opposite economics, that three years of blanket bot-blocking have been answering as one.
The facts, as reported in this morning's roundup: the draft is the AIPREF initiative's Machine-Readable Training Preference Signal, its token written in the collected announcements as usage_ai-training, and it can apply to a whole site or to named paths. Google says a training opt-out will count as a Google-Extended disallow unless Google-Extended is set separately, which tells you how Google has been reading silence until now: as consent to train. Microsoft describes training and grounding as two separate preferences and sets March 1, 2027 as the transition deadline after which its older NoArchive and NoCache signals stop doing this work. OpenAI has committed to honoring a shared signal. Anthropic describes active work to support the draft in its crawlers. The standard is still a draft, its syntax can still change, and no provider honors a final form today. What exists today is a direction, four signatures, and two dates.
Until now the owner answered in four dialects at once: a token per vendor, a crawler list per vendor, semantics per vendor, and no guarantee that a refusal written for one company's bot meant anything to another's. The shared switch ends the translation problem. It does not touch the judgment problem, which was always the actual work.
This is the home-galaxy item, because it draws the line Chris' clients will live on for years: training is not grounding. Training is whether your pages help build the next model. Grounding is whether your pages can be retrieved and cited inside an answer today. A business that blocks both by reflex protects its archive and deletes its storefront. A business that allows both by reflex gives away the archive to keep the storefront. The work for every site owner now, before the 2027 dates arrive, is to decide those two answers separately, in writing, section by section where the economics differ. One switch is simpler than four vendor dialects. Two decisions are still two decisions.
The economics differ because the goods differ. Training consumes the archive wholesale and pays, when it pays at all, in a currency that arrives late and spread thin: a slightly better model answering everybody's customers, including the ones who used to visit you. Grounding is retail and immediate. A page that can be retrieved can be cited, a page that can be cited can be chosen, and the citation is the new shelf position, re-decided at every question asked. Opting out of training costs most businesses nothing this quarter and keeps the corpus an asset with an owner. Opting out of grounding, by accident or by blanket rule, costs the storefront this quarter. The reflexes are symmetrical and wrong in both directions: the publisher who blocks everything to protect the archive has protected it out of the answer, and the merchant who allows everything to stay visible has donated the archive that made the store worth citing.
So the unit of decision is the section, not the site. Product and service pages exist to be found and quoted; they are grounding assets first, and their training answer follows the owner's licensing posture. The archive, the reporting and research and long backlist, is where the training answer earns its keep, because that is the material a model would otherwise absorb without a byline. Help and reference pages sit between the two: written to be cited verbatim, they do their work in grounding and lose little in training. The policy that results fits on an index card per section: train, yes or no; ground, yes or no; who decided; when it gets reviewed. What it cannot be is unwritten. A preference that lives in a meeting lives nowhere a crawler can read, and the crawlers are literal readers. This practice has said for a decade that if the words are not on the page, the page does not come up for them. The same law now runs in reverse: if the preference is not in the file, the preference does not exist.
Two decisions, then, before two dates. Decide training and grounding separately, in writing, section by section, before Microsoft's February 1, 2027 start and Google's 2027 support make the shared switch the way the answer engines read every site they visit. The switch settles the syntax, and the syntax was the easy part. The judgment, what your archive is worth and what your storefront needs, stays yours, and it was the work all along.