Half the Readers Were Google

A view counter makes a claim about who arrives, and the claim is only as good as the instrument behind it. This site was built on the premise that its future readers include machines: models, and the agents that fetch pages for them, as distinct from the search crawlers every public site already has. The premise shaped the build: three tiers, the lightest of them plain HTML built for crawlers, a content negotiation layer that answers a request for markdown with markdown, an llms.txt, an MCP server and an API. The counter under every post was written with an AI column, and it reports two numbers, total views and the number of them it attributes to AI. A July review of the site read those numbers back across the top ten pages of the past year and found 112 human views against 110 AI views. The review called that result the founding thesis confirmed with original data, and it recommended a post and a live counter. This is the post. The exact rows behind the two numbers were never read again: a retention rule deleted them before anyone looked. What survives is the same counter's rows in two preserved datasets, a database export covering its first four months and a read of the live table in late August, and neither dataset supports the sentence the review drew from its own.

What the rows contain

The counter works the way most analytics on small sites work. A script in each page fires a request to the API when the page loads. The API stores the page, the visitor's address, the user agent string and a timestamp, and it sets a flag when the user agent matches one of a short list of patterns (GPTBot, ClaudeBot, Googlebot, Applebot and nine more). Every AI share the site has ever displayed is a count of that flag. A database export from early July holds 564 rows, the counter's first day in February through the ninth of June, and is the only surviving copy of them. Group those rows by the user agent that produced them and the picture changes. The full breakdown by family and by month is in the attached investigation.

Of the rows the flag marked as AI, all but a handful came from two crawlers, Googlebot and Applebot, with five from Bingbot and three from the crawler that Meta documents as gathering material to train its models. The agents the thesis is about are the ones that fetch a page for a model, whether to train it, to build its search index, or because a person using it asked. Apart from Meta's three rows, they appear zero times: no GPTBot, no ClaudeBot, no ChatGPT-User, no Claude-User, no PerplexityBot, no Common Crawl, and none of Cohere's either. The flag was true for 58 percent of the rows, more than the review's even split because the review summed only the ten most viewed pages while these 564 rows span every page.

The export's own top ten pages sum to the review's 112 exactly, against roughly a hundred flagged as AI, when they are ranked the way the review's endpoint ranks them, by rows the flag called human. Four pages tie for tenth place, and the machine count moves by ten rows depending on which is taken, while the 112 holds under every choice. The review's human count matches these rows exactly, which may be coincidence, since the review read the table forty-six days after the export's last row and the rows it read are gone. Its machine count does not match, by eight to eighteen rows depending on the tie, which the export cannot supply and cannot place: they may be later rows, a different tenth page, or a difference the two readings do not share. The review drew one sentence from its even split: half of the readership is not human. What that sentence means can be read only against rows that still exist, and in the export the machines the flag counted were, all but a handful, the crawlers of two search engines, fetching pages the way search engines fetch them. The review's own rows are gone, and nothing in its method could have told those crawlers apart from a model reading for a person. A user agent string does not record what Google and Apple do with those bytes afterwards, whether they feed an index, a training set or a grounding call for a model. A flag that cannot tell those apart, and files all of them beside a model reading for a person, is not measuring an audience of any kind. The pattern list named Googlebot, Bingbot and Applebot as AI from its first day, so counting them was a choice rather than a slip, and the strongest case for the choice is that Google's and Apple's crawls do feed AI products now. The answer this post gives is that a column which cannot separate that from a model reading for a person measures nothing the thesis asked about.

The rows the flag marked as human divide in half. One half carries ordinary desktop and phone browser strings from a few dozen distinct addresses, concentrated in the site's first two months (plausibly the author and the people he showed it to), with none at all in April or May, a gap the investigation cannot tell from a beacon fault. The other half, 118 of 237, carries the string Google documents as GoogleOther, its generic crawler for product teams fetching public content, in the phone form for all but two rows. The pattern list does not include that string, so the counter filed it as a person. Half the humans were Google, and every one of those rows arrived from an address Google publishes as belonging to its crawlers. The export's own top ten, which sums to the review's 112, carries that string on about a third of its human-flagged rows. Whether the review's own 112 did is a question the export cannot answer, because those rows are gone. Counting Google under both flags, one company's crawlers account for a majority of every row the export holds. Move GoogleOther to the side of the ledger it belongs on, and the machine share of the corpus rises from the 58 percent the flag reported to roughly four fifths.

What the instrument cannot see

For the OpenAI, Anthropic, Perplexity and Common Crawl agents, the zero is a fact about the beacon before it is a fact about the agents. Googlebot renders pages in a headless Chromium and executes their scripts, though Google also warns that its renderer may skip requests that do not contribute to page content and that client-side analytics can under-count it, so the beacon may have missed some of what Googlebot rendered, and a row records a request that reached the endpoint rather than a rendering. Apple says only that Applebot may render in a Safari-shaped browser. The rows narrow what Apple leaves open, because a hundred and twenty Applebot rows in the export were written by an endpoint that only the page script calls, so whatever carried Apple's name either ran the script or reached the endpoint some other way. Every one of those addresses resolves to Apple's own crawler hostnames, and those hostnames resolve back to the same addresses. Every Googlebot and GoogleOther address in the spring window sits inside the ranges Google publishes for them, so the spring rows are Google's and Apple's own, whatever they did on the page. The investigation checks the Bing and Meta rows against the routes their publishers announce, and every crawler-labelled row in the spring window passes. The summer rows were not checked the same way. That check needs a read of the live table this investigation did not make, and the purge removes each row ninety days after it was written, so it is already impossible for the oldest summer rows and stays possible for the newest into November. Until it is made, the summer names stay as they declared them.

Vercel published a study of crawler traffic across its network in December 2024. It found that none of the major AI crawlers rendered JavaScript, while Googlebot and Applebot did. The crawlers it named include GPTBot, ChatGPT-User, ClaudeBot, Bytespider, PerplexityBot and Meta's crawler. A crawler that fetches HTML and never runs the script inside it leaves no row unless something in its pipeline calls the endpoint the script calls. Vercel listed Meta's crawler among the ones that did not render, yet it wrote three rows in the spring and forty-nine in the summer, and ByteDance's wrote four. Either those two ran the page script at least sometimes, twenty months after the study, or something carrying their names called the endpoint directly. A row records a request that reached the endpoint, not a rendering, so the table cannot separate those two readings, and neither of them contradicts what the study observed.

Meta's own documentation of its crawlers says nothing about scripts either way. What the study licenses is a narrower claim than the one the counter's design assumed. The counter records nothing when a crawler makes an ordinary fetch and never runs the script. Its AI column filled with search engines instead, which the pattern list accounts for on its own, since it named Googlebot, Bingbot and Applebot as AI from the first day. A collector that runs inside the page would tilt the same way, but which route these particular requests took to the endpoint is not something the rows record. A crawler that never renders still reaches the table if something in its pipeline calls the endpoint directly, and for the two families above the table does not record which of the two happened. For the families that never appeared, a zero in its table cannot say whether they stayed away or fetched without running the script. Only a record of requests at the edge could say whether GPTBot or ClaudeBot has ever fetched a page from this site. This investigation had no such record to consult: the site's edge analytics are aggregate counts, and no request log covering these pages was kept, so the question stays open.

The same blindness applies with more force to the surfaces built specifically for machines. The site advertises seven of them on its developer page: an MCP server, an OpenAPI description, a tool surface for browser agents inside the page, an llms.txt file, a JSON feed, an RSS feed and a hosted GPT. On the app tier's protocol rail the RSS feed gives way to the endpoint that agents write to. Every one of them was fetched on the twentieth of August, the write endpoint on its read side, along with the markdown a reader gets by asking the API for it. None of the returned bytes contains the view beacon, which appears only in the HTML pages a person reads. The markdown arrives as thirteen kilobytes of prose with no script in it, the llms.txt arrives as plain text, and the JSON feed and the OpenAPI description arrive as JSON. A read of those scriptless formats leaves no row anywhere the counter can see. The two surfaces that do link scripts, the developer page and the hosted GPT, were not exercised, so only their returned bytes are known to be free of the beacon. The instrument collects on its own only from the HTML pages that carry the beacon, which a model reader has the least reason to fetch when the same words are offered as markdown, a feed and an API. The traffic that would best support the thesis is precisely the traffic the site cannot count, if it exists at all.

What the second window shows

The July figures cannot be derived again, because they no longer exist. A retention rule in the API's nightly cleanup deletes view rows older than ninety days. That cleanup began running on the eleventh of August, after the removal of a dead statement that had aborted its first run. A request for the past year now returns the same rows as a request for the past quarter. What survives is a second window. Read on the twentieth of August, the live table held 556 rows reaching back to the twenty-fourth of May. The two spans overlap for seventeen calendar days, in which the export holds twenty-eight rows and the live table held at most twenty-four, fewer for the same June days than the export, a shortfall the investigation records and does not resolve. At most two dozen rows can sit in both windows. If they do, the second window's machine share moves by less than a point in either direction, whichever rows they are, and nothing this post concludes turns on that.

Read the same way as the export, the second window agrees with the first on every point this post rests on. Search crawlers account for a clear majority of it, most of them Google's. GoogleOther is still filed as a person, and its seventy-six rows are nearly as many as the eighty from every plausible browser together. The machine share is roughly six sevenths, against the spring's four fifths. The changed crawler mix explains that gap as readily as any trend would, since Meta's training crawler alone grew from three rows to forty-nine. Every crawler from OpenAI, Anthropic, Perplexity, Cohere and Common Crawl is again absent. Set the crawlers aside, along with eighteen rows from a headless Chrome that is almost certainly the author's own browser sessions. What remains under the human flag is a readership of a little under one page view a day. Those views come from clients presenting an ordinary browser string, which is the most the table can say about a person.

The second window adds one row the first did not have, and the counter filed it as a person. On the twenty-ninth of July a request arrived carrying the bare token Google-NotebookLM, which Google documents as the fetcher that its Gemini Notebook product sends when a user hands the model a page as a source. Google says nothing about whether that fetcher runs a page's scripts, and the row is the only hint that anything under that name ran them, since the endpoint that wrote it is called by the page script and by nothing else the site ships, though a direct call to that endpoint would leave the same row. Because the token matches nothing on the pattern list, the row went into the human column. Two more followed in the ten days after the snapshot was taken, on a later read that travels with the post as well. Three rows in three months is not an audience, and a fetch made because a user supplied the address as a source is not a model choosing to read. It is still the first trace on the ledger of a model reading for a person. It is there because something under that name reached an endpoint only the page script calls, which is the whole of what the ledger can say; the alternative the training crawler rows raise, a call made directly, is open here too. The counter answered the one model it could see by filing it as a person.

Both windows now live in aggregate files. The spring rows are gone from the database, and the summer rows are following them out at the purge's pace. The spring rows were counted from a database dump taken ahead of a migration in July, a file the repository's own ignore rules keep out of version control, so it exists on the machine that took it and nowhere else. The summer rows were counted from a snapshot taken ahead of the purge, which reached the oldest of them two days after the snapshot. Every number in this post's arithmetic is checkable against the aggregate files published beside the investigation, and the July figures only against the review that reported them. Here the artifact had to be cut from the source before the source destroyed itself.

The statement that deletes the rows was written in July as a housekeeping trim, ninety days of analytics for a table with no natural lifecycle, and for five weeks nothing ran it. What made it run was a schedule added for a different table. The site's survey instrument promises the people who fill it in that their old rows are deleted, and that promise had gone unenforced. The trigger added in August to enforce it fired the whole cleanup pass, the page views trim included. The change that added the schedule is about the survey instrument from its first line to its last, and does not mention page views.

What a counter should count

A counter that answers the thesis has to measure at the edge instead of in the page, and it has to sort visitors into more bins than two while covering the surfaces a machine actually reads. Measured at the edge, every request to any tier is a row, including the markdown fetches and the llms.txt reads that carry no script. The display then separates four populations and a residual, as far as a user agent string can tell them apart. The four are people, search crawlers, model crawlers gathering pages for training or for a model's own search index, and agents fetching on behalf of a user in a live conversation. The labels are what the crawlers call themselves, and the display says so, since a search crawler's bytes may feed a model all the same. That is why the bins sit side by side and are never summed into an audience. Where the publisher of a crawler lists the addresses it fetches from, the edge can check the claim at the moment of the request and discard the address once the family is known, which is the one job an address has in this design. Whatever fits none of those goes to a residual machine bin, which the display shows and never adds to the audience. GoogleOther and the author's own headless sessions land there. The four named labels mean four different things for a site that wants machines to read it, and collapsing them into one AI column is how a Google crawl came to be reported as an audience. A table aggregated by day and by family before storage needs no address at all, provided the unit it counts is the request and not the visitor. The current table keeps raw addresses for ninety days only to collapse repeat hits within an hour. Once that job is gone, the purge that erased the July numbers becomes unnecessary, because nothing left in the rows is worth purging.

The narrow repair is a patch of an afternoon, and it has been written and run over the table. It extends the pattern list with the strings it lacks: GoogleOther and the notebook fetcher first, then the fetchers that OpenAI, Anthropic, Perplexity, Amazon and DuckDuckGo each document. It then sorts the stored rows into bins by what their strings say when they are read, so the flag that was set when they were written no longer decides anything. Until the measurement moves to the edge, it produces an honest version of the existing widget: the model agent bin holds the handful of notebook fetches and nothing else, and the training crawler bin holds Meta and ByteDance. Publishing a bin that is nearly empty is more informative than the even split it replaces.

One more finding sits under the repair, and the counter would never have surfaced it. The llms.txt this site offers to models is served from the raw subdomain, which the proposal allows, and the front page links it there. A request to the main domain for the conventional path gets a 404 page instead. Whether a model has ever made that request went unrecorded, for the same reason the rest of this went unrecorded: the counting happens in the page and not at the request that asked for it. It stays unanswered until that moves.