Executive Summary

Before we dig in, let four numbers sketch the shape of the problem. How many times over dataset-level deletion drags innocent data out along with the target, how many existing tools ever let you revoke a single author, how much a precise deletion target lifts machine unlearning quality, and how much the pipeline slows down to buy that precision.

For years, machine unlearning research has concentrated on one question: given a batch of data to erase (the forget set), how do you make a trained model forget its traces? Algorithms like NPO, RMU, and Gradient Ascent are all attempts to answer it. Flagship benchmarks such as TOFU and MUSE also start with the forget set already fixed. They hand you a synthetic author and a predefined corpus and say, "now go unlearn this."

But real deletion requests do not arrive in that shape. They come not as "forget these files" but as "forget that person." GDPR's right to be forgotten, a copyright holder's withdrawal of training data, the AI-training lawsuits now being fought across jurisdictions — all take the same form. They name a specific author and demand that exactly their share be removed. Which means the first job is to find where "that person's share" is scattered across the corpus — and the benchmarks skip precisely that upstream step.

The problem is that virtually no tool supports that upstream step. Line up the version-control and lineage-tracking tools common in data pipelines against the criterion of author revocation, and the gap is stark.

The ability to single out and revoke one author's contribution simply did not exist in any tool until now. And this gap is more than one paper's claim. A separate survey that systematically reviewed 95 works from the past decade reaches the same conclusion: there is still no general method for removing data that was wrongly absorbed into a trained language model, and the right to be forgotten is growing heavier in the LLM domain specifically. On a data infrastructure that was designed without that right in mind, the whole industry is now scrambling to catch up with the problem.

Why dataset-level deletion is so expensive becomes obvious the moment you look at Wikipedia. A single Wikipedia article is a collaborative work that hundreds of people have edited together, so it cannot be split by file. One person's contribution is spread thinly across the whole article, which makes file-level deletion and dataset-level deletion effectively the same thing. There is no middle option — throw out every article that person touched, or leave them all in place.

The OriginBlame team took 219,555 Chinese Wikipedia articles (about 480,000 real contributors, CC-BY-SA-4.0) and compared line-level deletion against dataset-level deletion for four authors with differing contribution shares. The result: the smaller the share, the steeper the over-deletion.

For a contributor with a 79.5% share, wiping the whole dataset wastes only 1.3× — most of those articles are theirs anyway. At the other end, handling HuangQQ's 1%-share request at the dataset level means that to remove the 99 lines that truly must go, up to 9,999 lines disappear alongside them. The remaining 99% is the work of hundreds of other people, entirely unrelated to the request. Erasing one small voice ends up burning the community's whole record.

In case this pattern was peculiar to wikis, the team cross-checked it against data of an entirely different character. On the Linux kernel source (6,964 contributors by git blame, roughly 44,000 files), file-level deletion again removed far more lines than necessary. The ratio eases as scale grows, but it does not converge to zero. Without line-level precision, over-deletion is a structural loss that persists regardless of domain or scale.

▲ Original Pebblous diagram (reinterpreted from OriginBlame's measured table — log scale)

To excise precisely, you need to know which data came from whom. Here the direction OriginBlame chose matters. It is not a tool that tries to discover authorship anew; it is one that propagates the author information already present in the source all the way to the end of the training data. MediaWiki revision histories and GitHub commits already record who wrote what. The trouble is that this information is lost during tokenization and packing, so by the time the training corpus reaches the trainer, the link between an individual training line and its original author has been severed.

So the tool pins that link into a hierarchy that joins author, section, and document with SHA-256 hashes. At the top author layer sit an identifier derived from name and email, and a single revocation flag called revoked. That field is the single source of truth for the whole system. Below it hangs file-level copyright information, and below that, line-level records. When an author requests a takedown, you change only the tags at whatever granularity you want — the entire author, a specific section, or specific lines — and actual physical deletion is separated into a distinct operation that is permitted only after revocation has been set, a built-in safeguard.

Even capturing things down to the line, one last gate remains. Once tokenization and packing are done, the line boundaries themselves vanish, because discarding author metadata at the tokenization step is the default behavior of data-processing pipelines. OriginBlame adds one more independent token-index layer at this point. It reconstructs the token range each document occupies from the cumulative token count of the documents before it, and reconnects that range to the author chain. The output is a bitmask that can be fed directly to an unlearning algorithm. A 1 standing at each revoked token's position — that mask is the precise forget set.

▲ Original Pebblous diagram (reinterpreted from OriginBlame's architecture — author→file→line→token-index propagation)

All of this tracing comes at a price. Because it adds one recording point to an existing pipeline, the integration itself is lightweight, but training throughput slows by anywhere from 1.3% to 19% depending on the integration mode, and storage grows by 1.2–1.3×. That cost, however, is paid once when the data is created, not repeated every time a deletion request arrives. The reverse is true, in fact: without provenance, every single request forces you to rerun an expensive after-the-fact computation to figure out what to delete.

That a precise forget set is not merely an elegant theory, the team showed with an actual unlearning experiment. After fine-tuning a Qwen3-1.7B model on Wikipedia-based QA data, they compared two forget sets for two authors of similar share. One was that author's data pulled precisely by line-level provenance; the other was a random sample of the same size. The amount to erase was identical; only the choice of what to target differed.

With the NPO algorithm, the precise forget set beat random on all four metrics. How thoroughly the target content was forgotten was on average 42% better, and how well the knowledge that should be kept was preserved was 23% better too. Same algorithm, same volume — yet the outcomes diverged for the sole reason that the target was chosen accurately. This is where the experiment's message comes from: what you erase governs the result more than how you erase it.

The effect of precision shows up more dramatically the cruder the algorithm. Gradient Ascent failed to avoid collapsing the target knowledge under both forget sets, but the precise one held a fivefold edge over random in preserving the knowledge that should stay. An accurately aimed deletion produces a local collapse even under a destructive algorithm — driving the target deep while leaving the surroundings less disturbed. RMU, on the other hand, was fundamentally incompatible with a 4-bit quantized setup and effectively did not work: quantization's adjustment step is far coarser than the fine hidden-state shifts RMU needs, so the signal was buried in rounding and vanished.

▲ Original Pebblous diagram (reinterpreted from OriginBlame's Qwen3-1.7B unlearning experiment)

Why Pebblous is watching this

The angle from which this blog has covered training-data provenance has mostly been copyright auditing — checking who made something and whether its license holds at the point of ingestion. This topic points to the step after that: the point of removal. What the right to be forgotten, copyright takedowns, and the training-data lawsuits spreading across jurisdictions all demand comes down to one thing: cut out exactly this person's share. Dataset-level provenance can answer that only with "delete everything" or "delete nothing," and the price is up to 101× of innocent loss.

So the condition for AI-ready data widens by one axis. Until now that condition has been tuned to quality at the point of ingestion — stripping duplicates, fixing labels, checking for bias. To that, precision at the point of removal is added. Can you, when needed, pull out one specific author's contribution alone? Does that deletion avoid harming the rest? The ability to put data in well and the ability to take it out precisely are now two sides of the same quality.

• 1.Xue, H. (2026). "OriginBlame: Record- and Token-Level Data Provenance for AI Training Datasets." arXiv:2607.13037.
• 2.Hohensinner, R., Mutlu, B., Mendoza Estrada, I. G., Vukovic, M., Kopeinik, S., & Kern, R. (2026). "Tracing the Data Trail: A Survey of Data Provenance, Transparency and Traceability in LLMs." arXiv:2601.14311.
• 3.D'Angelo, A., Gullo, F., & Stilo, G. (2025). "The forget-set identification problem." Machine Learning, 114(247).
• 4.Maini, P., Feng, Z., Schwarzschild, A., Lipton, Z. C., & Kolter, J. Z. (2024). "TOFU: A Task of Fictitious Unlearning for LLMs." First Conference on Language Modeling (COLM 2024).
• 5.Zhang, R., Lin, L., Bai, Y., & Mei, S. (2024). "Negative Preference Optimization: From Catastrophic Collapse to Effective Unlearning." arXiv:2404.05868.