Executive Summary

A study released this month reports that the checks used to confirm an AI model's claim of having deleted training data are easy to fool. It is "Rethinking the Verification of Machine Unlearning," by Lei Zhou, Youwen Zhu, and Rongke Liu. Most verifiers deployable in practice today infer deletion from the predictions and probabilities a model emits, and the authors built an attack that leaves the model's internals untouched while dressing up only those answers, then fed it into the checks.

The disguise passed existing verifiers 97% of the time. The signal those verifiers read comes from a layer the operator can control at response time. RUV, the alternative the authors propose, compares intermediate representations in the original model and the model said to have unlearned, and it rejected 99% of those disguised submissions. For RUV to run, though, the verifying side has to be able to receive internal representations, and the experiments are confined to image classification models.

Key Figures

The rates the paper reports fall into three pairs: the share the attack broke through, the share existing checks blocked, and the share the new check blocked. The pairs measure different things, so they have to be read side by side.

Source: the abstract and body of arXiv:2611.18658 (submitted October 19, 2026; listed as new on November 17, 2026).

97.0 / 97.7%

Share of fakes that passed the check

Means for class-level and for sample-level requests

5.1 / 3.6%

Highest rejection rate among output-based checks

The best any of the three existing verifiers managed, on each of the two request types

99.5 / 99.8%

Share of fakes RUV sent back

On the same two request types, by comparing intermediate representations

4.37 s

One RUV verification

On CIFAR-10 with ResNet18. White-box TruVRF takes 91.2 seconds on the same setup

1

The Law Says Delete, but Who Checks That It Happened?

The right to demand that your personal data be deleted is already written into law. Article 17 of the European General Data Protection Regulation (GDPR) establishes a right to erasure, and Article 36 of Korea's Personal Information Protection Act lets a data subject require deletion of their own personal data. For a row in a database the right is enforced cleanly. Delete the row, write the log entry, done.

A trained AI model is a different matter. One person's data does not live in a file of its own. It is scattered across hundreds of millions of weights. The technique for rolling back that influence alone is called machine unlearning, and it comes in two broad kinds. Exact unlearning retrains from scratch with the requested data removed. Approximate unlearning skips retraining and edits the weights to cancel the influence. The first is expensive, and the second is fast but offers a weak guarantee that anything was truly removed.

Regulators acknowledge the difficulty. The Guide to Personal Data Processing for the Development and Use of Generative AI, published by Korea's Personal Information Protection Commission on August 6, 2025, states that where the scale or structure of a training dataset makes rights such as access, correction, and deletion hard to guarantee, a controller may inform the data subject of that fact and the reason behind it and offer alternative means. The same guide treats machine unlearning as a technology trend for responding to deletion requests.

One more layer stands between the right and the technology: verification, the step where someone outside the company checks whether a claim of deletion is true. The paper this article reads aims squarely at that layer.

2

Today's Verifiers Read What the Model Answers

Verification was not set up from the start to read a model's answers. One family of methods plants a traceable signal before training begins and later checks whether it is gone. Another asks the provider to hand over records produced during the deletion itself. The paper groups these together as process-bound methods, which "either insert traceable signals before training or require the provider to expose detailed artifacts of the unlearning process." Ordinary model services rarely meet either requirement. A model that has finished training has no signal left to plant, and a running service seldom opens its internal records. That left post hoc verification, performed from the outside after training, as the family that could actually be deployed at scale.

What does a post hoc verifier look at? It feeds the data that was supposed to be deleted into the model and checks whether the answers look like those of a model that never saw it. The paper's four baselines each read a different signal. Accuracy checks whether performance on the requested data has dropped, LiRA uses membership inference to estimate whether that data was part of training, and TAPE measures the posterior difference between the original and the unlearned model. All three are black box: answers come back, and nothing else does.

The fourth baseline, TruVRF, works differently. It examines how gradients change between the original and the unlearned model, which calls for white-box access to parameters and a backward pass. It looks deeper, and it costs more. On CIFAR-10 with ResNet18 a single verdict takes 91.2 seconds and 2,528 MB of memory.

The paper's objection targets the premise the three black-box verifiers share. In the words of the abstract, these verifiers infer unlearning from "the final model response that remains under the provider's control." The provider under audit supplies the input to the audit. Accuracy and probability distributions are not internal states. They are what the model has chosen to emit, and what it emits can be edited at serving time.

The complaint that output alone cannot prove deletion is not new in itself. In the robot demonstration unlearning audit Pebblous covered in August, an edited policy that had recovered its task success rate was still indistinguishable, under a membership audit, from a policy that had erased nothing. That case was about an honest attempt at deletion failing to produce proof. This paper turns the situation around and asks what happens when a provider never intended to delete and simply wants to pass the check.

3

Two Ways to Fake the Answer and Keep the Memory

The attack the authors built is called ODR, short for Output-Level Deceptive Response. The principle is in the name. The state in which the model still remembers the data stays untouched, and only the answer the verifier reads is dressed up to look like an unlearned model's answer. Two forms cover the two units a deletion request can arrive in.

3.1ODR-Head: Rewiring Only the Classifier

This form handles class-level requests, where an entire category is to be removed. The backbone that produces representations is left alone, and only the classifier at the very end is retrained. The target fake output is built by mixing a uniform distribution with a perturbed copy of the original model's output. The model then answers that category as though it knew nothing about it, while the internal representation that recognized the category has not moved an inch. Because the classifier is retrained jointly to preserve accuracy on the remaining data, from the outside it looks like a healthy model that forgot exactly one category.

3.2ODR-Gate: Intercepting the Query

Sample-level requests, where individual records are to be removed, get a form that never touches a parameter. When one of the registered queries arrives, it is intercepted, and the posterior for a non-member sample from the same class is returned instead. To the verifier this reads as a model that never trained on that record. Since the weights were not edited, accuracy on every other input holds.

The diagram below lays the model out from input to output and marks where each attack intervenes and where each check looks.

Where the attack intervenes, where each check looks ODR-Head Tuned to a fake output ODR-Gate Swaps only listed queries Input Audit queries Internal representation Where the memory remains Classifier The final layer Output Predictions, probabilities RUV Compares the two models Accuracy · LiRA · TAPE Black box: answers only ODR-Head edits only the classifier, ODR-Gate changes no parameters, so both leave the internal representation intact TruVRF, a white-box verifier reading parameters and gradients, takes the whole model row as its target Drawn from the description in arXiv:2611.18658
▲ Original Pebblous diagram. The attack points the paper describes and each verifier's observation point, placed on the model's flow

The experiments cover exact unlearning (Retraining, SISA) and approximate unlearning (Amnesiac, SCRUB, SalUn, SSD). The datasets are CIFAR-10, CIFAR-100, and FaceScrub, a collection of face images. ResNet18, 34, and 50 run on the first two, and ResNet50 runs on FaceScrub. Deletion requests arrive in two shapes: class-level requests that remove one category or one identity outright, and sample-level requests that use stratified sampling to pick 10% of the training data. Across that range, ODR's mean deception success rate was 97.0% for class-level requests and 97.7% for sample-level ones.

The number on the other side is the more striking one. It is the share of these disguised submissions that the verifiers sent back. Every submission here is a fake, so how many did the checks return? A check that works should send back nearly all of them.

Verifier Where it looks ODR-Head rejection ODR-Gate rejection
Accuracy · LiRA · TAPE Predictions and probabilities the model emits (black box) at most 5.1% at most 3.6%
TruVRF Change in parameters and gradients (white box) 15.7% 79.6%
RUV Intermediate forward representations (gray box) 99.5% 99.8%

▲ Rejection rates as the body of the paper gives them. For the three black-box verifiers the paper states an upper bound rather than individual figures

The three black-box verifiers sent back at most about one submission in twenty. TruVRF, the white-box verifier, rejected 79.6% of the sample-level fakes, but fell to 15.7% against the class-level fake that edits only the classifier. One shared property explains both results. Both forms leave untouched the internal state in which the model still holds the data, and a check that never looks at that state has no way to notice that nothing has changed.

4

RUV Measures Four Traces Inside the Model

RUV, or Representation Unlearning Verification, moves the audit target. Rather than what the model returns, it reads what has changed inside the model. Concretely, the same data goes into the original model and into the model said to have unlearned it, and the representations formed along the intermediate layers are pulled out of both and laid against each other. Parameters, gradients, and training logs are not required — only the intermediate representations from a forward pass — which is why the method is classified as gray box.

What does the verifying side have to receive? The paper writes it out plainly: the original and unlearned models, the list of data to be forgotten, a matched retained control of the same size, and a reference set to compare against. One more condition comes with it. Both models have to run in an environment that exposes intermediate representations. For anyone designing deletion-request handling as a procedure, that list is the specification of what to demand from the provider.

The comparison splits four ways. It measures whether the route the forgotten data used to take through the model has been reorganized (Activation Route Shift), whether the feature vector for that data has been pushed off its original position (Representation Shift), whether the neighborhood of similar data around it has come apart (Neighborhood Persistence Gap), and whether the structure supporting that data inside the learned manifold has collapsed (Manifold Rank Drop).

Taking the four signals at face value produces misjudgments, because editing a model shakes the whole thing a little and not only the data someone asked to have removed. RUV pairs in retained data that was never meant to be deleted as a control, which separates a target-specific response from drift across the model as a whole. The four pieces of evidence then go through joint permutation calibration to produce one decision.

Verification accuracy on genuine deletions came to 98.9% for class-level requests and 94.9% for sample-level ones. Why is the sample figure lower? Removing an entire category leaves a block-shaped hole inside the model, whereas the trace of a handful of individual samples is buried among other samples of the same class, and the signal grows much fainter.

False submissions are not the only thing RUV screens. The authors added three controls that change a model without carrying out the requested deletion: a benign drift control that simply keeps training on the full data, a wrong-target unlearning control that unlearns an equally sized disjoint set while the audit asks about the original target, and a same-target fine-tuning control that runs ten epochs of correct-label fine-tuning on the forgotten set, moving the representation while reinforcing the knowledge. All three look like deletion in the sense that the model visibly changed. RUV rejected every one of them at 96.8% or better for class-level requests and 92.9% or better for sample-level ones. Deletions aimed at the wrong target and deletions that changed only the surface get caught by the same check as disguised ones.

A submission that stopped partway through gets a graded reading. When a model has deleted only part of what was requested, the scores climb with the share deleted. On CIFAR-10 with ResNet18, a submission that deleted a quarter of the request was accepted 6.8% of the time for class-level and 3.7% for sample-level requests, and at three quarters acceptance rose to 84.2% and 64.7%. The paper reads this scale as evidence that RUV tracks how far execution went. Read the same table from the auditor's side and a different number stands out: a submission that deleted only three quarters is accepted most of the time. Where execution starts counting as compliance is not something the verifier settles, and it stays with policy.

Whether this performance depends on one model architecture was tested separately. Running the same procedure over seven CNN and Transformer backbones on CIFAR-10, verification accuracy stayed between 96.3% and 99.4% for class-level requests and between 91.1% and 96.0% for sample-level ones. MobileNetV2, the lightweight model, marked the bottom of both ranges, and ViT-S/16, the Transformer, came in at 97.8% and 93.3%.

Cost is part of the case as well. On CIFAR-10 with ResNet18, RUV used 4.37 seconds and 1,684 MB, while TruVRF, which needs a backward pass, used 91.2 seconds and 2,528 MB on the same setup. The gap comes from comparing forward representations without backpropagation.

The paper's conclusion fits in one sentence. Verifying machine unlearning takes internal evidence, not output that looks like deletion. Pushing the boundary of the audit down to the forward representations is the practical starting point the authors offer.

5

What the Amazon Orders Left Blank

Orders to delete not merely data but the models built from it already exist. On May 31, 2023, the US Federal Trade Commission (FTC) and the Department of Justice charged that Amazon had held children's Alexa voice recordings indefinitely despite parents' deletion requests, and proposed a $25 million settlement. That order bars the company from using geolocation and voice information for which deletion was requested to create or improve data products.

The Ring case announced the same day goes a step further. Alongside $5.8 million in consumer refunds, it required deletion of customer videos and face embeddings collected before 2018, as well as data, models, and algorithms derived from videos the company had reviewed unlawfully. The business guidance post the FTC published afterward tied the two together: "With Ring and Alexa, as well as Kurbo, Cambridge Analytica, and Everalbum, the FTC has obtained numerous orders requiring companies to delete data and delete or refrain from generating data products, like algorithms, models, and other tools derived from ill-gotten data."

The sentence in the orders is clear enough. The box after it is empty. By what procedure does a regulator confirm from the outside that the models and algorithms were in fact deleted? Nowhere do the orders say. Compliance ends up confirmed through the reports a company files. And the technical evidence available to attach to those reports has been, until now, the output-based checks from Section 2 of this article.

Placed in that context, what ODR demonstrated carries more weight. As long as a verification procedure reads output alone, a company that honestly retrained its model and a company that edited a single classifier layer hand in submissions that look identical. However strongly a deletion order is worded, its force comes from where the means of verification stands.

6

Why Pebblous Is Watching Machine Unlearning Verification

6.1A Signal the Provider Controls Is Weak Evidence

What this paper showed with the number 97% is not a flaw in one algorithm. It is a structural fact. When the verification signal is drawn from a layer the audited company controls, no amount of sophistication earns that signal the standing of evidence. ODR is less a new attack technique than a demonstration of where the verification signal was drawn from.

RUV does not escape the problem entirely. The intermediate representations are still handed over by the company that holds the model. A design that asks for neither parameters nor training logs widens the room for a business to accept it, but the material used for verification still passes through the provider's hands. The attack the paper built stays at the output layer, and whether the same kind of disguise works against the representation layer has not been tested yet. The experiments are also confined to image classification models, and the authors leave extension to generative and multimodal models as the next task.

The counterargument from the paper's side deserves a line as well. Fooling the representation layer means changing four signals at once while keeping the rest of the performance intact, and doing it so that the change lands only on the deletion target. That is a different order of work from editing one classifier layer or intercepting one registered query. The authors aimed to push the cost of faking up toward the cost of actually deleting. If the purpose of verification is not to make cheating impossible but to make lying expensive, this design meets that condition.

6.2Proving Deletion Starts with the Data

Proof of deletion is not a problem that gets solved on the model side alone. Someone has to identify what is to be deleted first. Token-level provenance tracking, which Pebblous looked at in July, dealt with the way dataset-level provenance erases up to 101 times more innocent data alongside one author's share, and the deletion problem in agent memory pointed out that a record written without a label gives a deletion request nothing to reach. Verifying a claim of deletion comes after those two questions. If you do not know what went in, you cannot confirm what came out.

What Pebblous does with DataClinic belongs to that lineage. It spreads data out in a high-dimensional space and diagnoses where things cluster and where the space is empty, and the diagnostic method is registered as US patent 12,481,720. Holding the distribution of the training data as coordinates also means holding a baseline against which to ask whether the spot a given record occupied is genuinely empty now.

For an organization that handles data, this research narrows to one sentence. When a company answers that it deleted something, if the evidence behind that answer is evidence the company gets to choose, verification has not begun. What the designers of a deletion-request procedure have to settle first is not which unlearning technique to adopt but what will serve as proof that deletion happened. Further reading: the Data Greenhouse strategy, which links data diagnosis and synthesis into one loop, and the ISO/IEC 5259 roadmap, which writes data quality in the language of an international standard.

R

References

Academic Paper

Official Document

Industry Coverage