A site I am auditing went up with about eleven hundred pages in three months. Google has indexed sixteen of them. The other thousand-odd pages load fast, return clean 200 codes, and sit in the sitemap where they were put. Google crawls them, reads them, and declines to keep them.
How the press works
Programmatic publishing is a printing press for web pages. You design one template and keep a database of variables: a service, a city, a product, a price band. The press stamps the template once for every combination in the database. Each stamped page aims at a long-tail query pattern, the kind of specific phrase a buyer types when he is close to spending money. A human writer needs a week for fifty honest pages. The press needs an afternoon for eleven hundred. That arithmetic is the whole sales pitch, and the arithmetic is real.
Why Google keeps sixteen
Google reads every page it crawls and compares it against the rest of the index, including the other pages on the same site. Page 843 of this site differs from page 12 in the swapped variables and in nothing else. Same structure, same paragraphs, same claims, new nouns in the slots. The index keeps a representative copy of a page like that and files the rest under two coverage labels worth learning: “Duplicate without user-selected canonical” and “Crawled – currently not indexed.” The second label is a refusal in polite language. Google spent the crawl, weighed the page, and decided it adds nothing the index lacks.
A page earns its place by carrying something the neighboring pages do not: a local fact, a real price, an inventory record, an answer with a name and a date on it. Swapped variables carry none of that. Multiply the thin page by eleven hundred and the site teaches Google a lesson about itself: this domain ships volume. Crawl visits thin out. New pages from the domain get a harder read. The sixteen indexed pages are the ones with something on them, a link from outside, real text, a reason to exist that survives comparison.
Google also publishes its verdict on the method. Its spam policies define scaled content abuse as “many pages generated for the primary purpose of manipulating search rankings and not helping users,” built from “large amounts of unoriginal content that provides little to no value to users, no matter how it’s created.” The last clause is the one to underline. The policy is indifferent to whether the press was a script, a spreadsheet, or an AI. It cares what landed on the page.
I have said for years that Google is a literal machine. It is no thesaurus and it grades no intentions. A page gets indexed and ranked for the words and facts actually on it. The owner of this site imagines eleven hundred doors into his business. Google sees one room described eleven hundred times and keeps the best description.
What a diagnosis examines
The coverage report comes first. The exclusion buckets tell you which failure you have, and in what proportion, before anyone theorizes. A site heavy in “Crawled – currently not indexed” has a value problem. A site heavy in soft 404s and redirect errors has a plumbing problem. The buckets leave the diagnosis half finished before the crawl starts.
Then the dumb accidents, because they are cheap to rule out: robots.txt, meta robots tags, canonical tags pointing in circles, a sitemap advertising pages the site itself tells crawlers to ignore. Every veteran of this work has lost a week to a staging noindex that rode into production. You check the locks before you blame the neighborhood.
Internal linking comes next. Pages need to hang in a structure a crawler can walk: sections, subsections, links with anchors that name their targets. Eleven hundred pages behind one flat listing are a pile, and a page nothing links to is a rumor Google is free to disbelieve.
The backlink profile finishes the picture. The profile on this audit carries spam links in the thousands against honest links in single digits. That tells me how the site was promoted, and whether a disavow file is worth the argument. It also tells me the promotion bought the appearance of authority in place of the substance, and Google prices those two things very differently.
Last, the side-by-side test. Print two of the stamped pages, black out the swapped words, and read what remains. What remains is exactly what Google indexed, once, out of eleven hundred tries.
The repair
The repair runs against the instinct that built the site. Fewer pages. Each survivor carrying facts only it can carry, written for the one query it can honestly answer. Consolidate the duplicates behind the strongest version, point the canonicals at it, and give the press a rest until every page it prints earns its slot. Sixteen pages that answer real questions are a business. Eleven hundred that answer none are a printing bill.