Executive Summary

Key Numbers

The first three figures give the size of the domain gap and the speed at which it opened. The last is the limit that needs the most care when this study gets quoted.

Sources: Pew Research Center chart source data and methodology (2026-08-20)

Pew drew a random sample of 10,000 English-language webpages from each of the 49 crawls Common Crawl produced between January 2021 and July 2026. The 490,000 pages went through a detection model, and the results were split by top-level domain and plotted as six-month averages. That time series is the most valuable part of the study. Plenty of outlets have already quoted the share at a single point in time, but the order in which those shares came apart exists only in the series.

In the July 2022 crawl, the last one before ChatGPT went public, the readings were .com 1.07%, .org 0.63%, .edu 0.64% and .gov 0.25%. Four lines pressed against the floor, nearly on top of one another. From the first half of 2023, .com pulls away first, .org follows at roughly half the pace, and .edu and .gov drift around 1% for four straight years.

Pew makes no claim about what caused the gap. What the study observes is correlation, and why only .edu and .gov stayed low falls outside its scope. One observable difference between the four does stand out, though. Addresses under .edu and .gov are not open to anyone who wants to register one, and documents posted there usually come attached to an affiliation, a named author and an institutional sign-off. The channels with more friction before publication are the ones where the accumulation stayed slow, and that much the data shows directly.

The instrument was editlens_Llama-3.2-3B, an open-weight detection model released by Pangram. Feed it the body text of a page and it returns a score between 0 and 1, and Pew treated anything at 0.2 or above as a meaningful sign of AI authorship or editing. The threshold was not set at random. It was calibrated to line the open model up with the commercial one, according to the methodology. Across 62,370 pages from seven crawls, Open Pangram and the flagship commercial model Pangram 3.3 agreed in 96% of cases, with a Cohen's kappa of 0.61. Disagreements remained at the level of individual pages, but the aggregate shares the two models produced per crawl landed close together, and the shape of the upward trend was the same either way.

The model never saw the full HTML of each page. Pew used the body text file that Common Crawl stores separately from the raw markup, the WET record that contains just the text of the page with the HTML code, images and other media stripped out. That is also the file most teams pick up first when they turn a web corpus into training data. The layer of text this study measured is the same one that reaches a training run.

The widely quoted 35% comes with a condition attached. Only 10% to 15% of pages in a given crawl sample carry a publication date field in their HTML. The 35% is the share within that subset whose date falls after ChatGPT's release. Pew did not bury this. The methodology says plainly that "the subset of pages with a detectable publication date is not a random subset of the web, so our post-ChatGPT estimate reflects the prevalence of AI authorship among dated content rather than the web as a whole." The caveat fell off in the headlines and the number traveled alone.

A caveat is not the same as a debunking. In the same document Pew notes that an independent study of newly published sites, built on Internet Archive data and a different methodology, arrived at the same 35%. Both figures describe documents that are verifiably new, and within that scope they support each other. The trouble starts when the value is carried over and quoted as a share of the web.

The same document holds one more caveat that matters even more for reading the domain gap. Open Pangram has a higher false-positive rate on pages created before AI use was common, and on the 2021 and 2022 crawls the model returned estimates of around 1%. Run the commercial model on those same pages and the values come out far lower. Do not read that as 1% of the pre-ChatGPT web having been written by AI. That value is the height of this model's false-positive floor.

2.1The Web's Prose Moved Along With It

Separately from the detector scores, Pew counted the sampled text a second time, tallying how often certain marks and words appear per 10,000 words. Looking only at pages with publication dates after ChatGPT's release, and comparing the January 2023 point with the January 2026 point, all four measures climb.

Uses per 10,000 words, plotted as six-month averages. The vocabulary list is a fixed set of 27 items Pew defined in advance, including "delve," "interplay," "testament" and "bolstered." Negative parallelism means a comparison structured as "it's not just X, it's Y." Source: Pew Research Center chart source data

Pew did not count these marks only on the AI-written pages. It counted them across the whole sample, human-written pages included, so the average sentence on the web has drifted in that direction. The curves are not simple either. Em dash use actually sank into the low 4s through the second half of 2023 and into 2024 before turning steep in 2025 and doubling. Negative parallelism nearly tripled yet remains rare at 2.36 uses per 10,000 words, and Pew is explicit that no single one of these markers proves AI authorship on its own.

The style checklist Pebblous uses to review Korean drafts flags the same two habits, dash overuse and the "not simply X, but Y" construction. Seeing them confirmed statistically across the English-language web is useful, but the direction is what matters in practice. The more common these markers become, the less any judgment resting on them can carry, and the more weight shifts to the information sitting outside the sentence. Which address the document came from is one such signal.

The source data Pew made downloadable next to the chart also carries per-domain sample counts. In the January 2026 window the four domains account for 49,719 pages, and 41,116 of them are .com. The rest is .org with 6,363, .edu with 1,571 and .gov with 669. More than eight in ten pages come from one domain. The table counts only pages that fell into those four top-level domains, so .net and country-code addresses are absent, but among the four, .edu and .gov together hold just 4.5%.

That composition sends you back to the two earlier numbers. Across the full 10,000-page sample from July 2026, 10% of pages showed signs of AI authorship, and .com on its own came in at 9.35%. The two are measured over different windows. The 10% is a single snapshot from one July 2026 crawl, and the 9.35% is a six-month average at the January 2026 point, so they are not values you can add or subtract. Still, their sitting side by side is no accident. The largest block in the sample is .com, which makes the figure everyone calls the web average close to a restatement of .com. However clean .edu and .gov may be, they carry no weight to pull the average down.

What the sample misses tilts the same way. To make it into Common Crawl, a page has to be publicly accessible. Pew's methodology notes that sites with paywalls or a login requirement are likely underrepresented in these samples. Subscription journalism, closed academic archives and documents behind institutional accounts were never counted. This study measured the open portion of the web, and any team building training data from the crawl inherits exactly the same portion.

Pretraining corpora derived from Common Crawl inherit that composition too. Contamination arrives shaped like a domain instead of settling over a corpus as an even fog, and how much of each source a corpus holds decides how contaminated that corpus actually is. Pebblous has written before about the problem of "clean data" being asserted without proof, and this dataset points to one minimum unit of that proof. Not the total, but the distribution across sources.

That composition already spells out what happens when you design a filter around the single figure of a 10% web average. Sweep the whole corpus at one uniform strength on the assumption that contamination is spread evenly, and most of the .com documents where it actually collected survive, while institutional documents that had no problem to begin with get cut at the same rate. What remains is a corpus with roughly the same contamination rate as before, only smaller.

The point where practice can change sits earlier in the pipeline than most teams assume. If you keep the original URL alongside the text when you clean crawl data, the top-level domain is information you already hold. Stratifying by source before scoring each document, then applying different thresholds and different sampling rates per stratum, removes more contamination for the same budget. It also cuts the cost of running the detector, since there is no reason to spend the same compute on a stratum that holds almost nothing.

4.1Four Things to Check in Your Pipeline Today

Pew's study design leaves behind a few working rules for anyone handling a corpus.

• Carry the source field through to the derived datasets. Once the URL or the top-level domain drops out at an intermediate stage, stratified filtering stops being possible from that moment on.
• Keep detector scores as continuous values instead of freezing them into binary labels. Pew's 0.2 was itself chosen to calibrate against a commercial model, and moving the threshold changes the picture.
• Record the sample count for each stratum. A share drawn from a stratum of a few hundred documents swings on a handful of cases, so those strata are safer with sampled review attached than with automated rules.
• Measure your detector's false-positive floor first. Run documents from before AI use was common through the same model, treat the resulting value as that stratum's floor, and stop reading anything below it as signal.

That last rule applies anywhere a detector is used. The study Pebblous covered earlier, in which 21% of ICLR 2026 reviews showed signs of AI, used a detection model from the same vendor, and there too the confidence came from aggregate counts rather than from any individual document. A verdict on a single document is the output of a probabilistic tool, and the moment a pipeline hardens it into fact, a false positive becomes a label.

Research Reports

• 1.Bestvater, S., Smith, A., TerBush, C., Baronavski, C., & Chavda, J. (2026). "How Much of the Internet Is Written With AI?" Pew Research Center, Aug 20, 2026.
• 2.Pew Research Center. (2026). "How Much of the Internet Is Written With AI? Methodology." Aug 20, 2026.
• 3.Dolezal, J. et al. (2026). "The Impact of AI-Generated Text on the Internet." Independent study based on Internet Archive data, cited by Pew's methodology as a comparison point.

Tools and Data

• 4.Pangram. "editlens_Llama-3.2-3B." Hugging Face (checked Aug 26, 2026). The open-weight AI detection model used in this study.
• 5.Pangram. "Introducing Open Pangram." (checked Aug 26, 2026).
• 6.Common Crawl. "Common Crawl." (checked Aug 26, 2026). The web archive the sample was drawn from.