| Training | Pages used to build the next model |
|---|---|
| Grounding | Pages retrieved and cited inside an answer today |
| Control surface | robots.txt signals, moving toward a shared standard |
| Source post | Training is not grounding (November 10, 2026) |
Training is the use of a site's pages to build the next model. Grounding is the retrieval of a site's pages at answer time, so an AI answer can cite them today. The weblog states the distinction as a rule for site owners: training is not grounding, and the two decisions carry opposite economics. A shared robots.txt signal drafted at the Internet Engineering Task Force, from the AIPREF initiative, lets an owner answer the two questions separately for a whole site or for named paths. Google, Microsoft, OpenAI, and Anthropic have each backed the shared signal, with honoring dates arriving during 2027.
The two economics
Training consumes an archive wholesale. The return arrives late and spread thin: a slightly better model answering everybody's customers, including customers who used to visit the source site. Grounding works at retail speed and pays immediately. A page available for retrieval can be cited, a cited page can be chosen, and the citation is re-decided at every question asked. An owner who blocks both by reflex protects the archive and closes the storefront in the same motion. An owner who allows both by reflex keeps the storefront and gives the archive away with it. The post training is not grounding works through the announcements, the dates, and the per-vendor signals the shared standard replaces.
Where the signals stand
Microsoft describes training and grounding as two separate preferences and starts honoring training opt-outs on February 1, 2027, with March 1, 2027 as the transition deadline after which its older NoArchive and NoCache signals stop doing this work. Google targets support sometime in 2027 and reads a training opt-out as a Google-Extended disallow unless Google-Extended is set separately. OpenAI has committed to honoring a shared signal. Anthropic describes active work to support the draft in its crawlers. The standard remains a draft, its syntax can still change, and the practical work for an owner today is a written decision, section by section where the economics differ, recorded before the 2027 dates arrive. Crawler controls and robots practice for this site are maintained at Static weblog operations.
Before the shared signal
Until the shared draft arrived, an owner answered the two questions in four vendor dialects at once: a token per vendor, a crawler list per vendor, and semantics per vendor, with no guarantee that a refusal written for one company's bot meant anything to another's. The shared switch ends the translation problem. The judgment problem stays with the owner, and it was always the actual work: deciding, in writing, which sections of a site train, which sections ground, and where the economics of a particular section argue for a different answer from the rest of the site.