Why Machine Unlearning Can't Certify GDPR Erasure

Machine unlearning cannot currently certify GDPR erasure, because no shared evaluation protocol exists to prove that an unlearning operator actually removed a person’s data from a model’s weights. A consolidated survey of generative-model unlearning, arXiv:2507.19894, confirms the field’s evaluation leg is fragmented across incompatible targets, mechanisms, and metrics. For teams answering Article 17 deletion requests, the defensible path today is retrain-and-attest, not a vendor’s unlearning claim.

What does the unlearning survey actually consolidate?

The survey organizes generative-model unlearning into three components: the target event you want removed, the operator that removes it, and the evaluation protocol that checks whether removal worked. Formally, the survey frames GenMU as target-constrained distributional projection: given a target event, an unlearning operator transforms the model’s generative distribution to suppress target-related outputs while preserving useful behavior and controlling the operator’s cost.

That formulation is worth sitting with, because it quietly defines the limits of what unlearning can promise. The goal is to suppress outputs related to a target while keeping everything else intact. It is not to prove that information is gone. The operator’s success criterion is behavioral: does the model stop producing target-related content? Whether the underlying capability or memorized fact persists in the weights, recoverable under a different prompt, a different decoding strategy, or a fine-tuning pass, is a separate question the formulation does not answer.

The survey’s consolidation itself is a contribution. Before it, the literature on machine unlearning for generative models was scattered across target definitions (a document, a fact, a style, a person’s identity), operator families (gradient ascent on forget sets, knowledge editing, output filtering, retraining variants), and evaluation approaches that rarely overlapped. By mapping the space along those three axes, the survey gives practitioners a shared vocabulary. What it cannot give them, as its abstract admits when it calls objective comparison “difficult,” is a way to compare results across studies.

One caveat on the source itself. arXiv is a moderated preprint repository, not a peer-reviewed venue: submissions are approved after moderation but carry no formal review weight. The survey is a useful map of the field, not a standards document. Treat its taxonomy as the current state of research conversation, and its findings as well-organized expert opinion rather than ratified consensus. (The repository hosting it, incidentally, split from Cornell into an independent nonprofit on July 1, 2026, which changes nothing about its review model but is worth knowing if your compliance documentation cites it.)

Why can’t unlearning evaluation certify removal?

Because the field’s evaluation protocols are fragmented enough that objective comparison across studies is impossible, and a number that cannot be compared cannot be certified. The survey’s central finding is that existing unlearning studies remain fragmented in their target definitions, unlearning mechanisms, and evaluation protocols, which makes objective comparison difficult across models, modalities, and applications.

Fragmentation is not a cosmetic problem. It means that when a vendor reports a hypothetical “99% unlearning success,” that number is measured against a target definition the vendor chose, with an operator the vendor built, evaluated with a protocol the vendor selected or designed. There is no shared benchmark of what counts as removed, no agreed probe set that distinguishes suppressed outputs from deleted knowledge, and no standard for how much collateral damage to retained capabilities is acceptable. Two papers can both report near-complete unlearning while measuring entirely different things.

This matters more for generative models than it did for classifiers. A classifier that unlearns a training example can be probed with membership-inference attacks, and while those have their own limitations, the evaluation question is at least well-posed: is this example’s influence detectable? A generative model that has “unlearned” a person’s data faces a harder question, because the same information can surface paraphrased, split across tokens, or elicited only after multi-turn priming. Suppressing the direct output is cheap. Demonstrating that no prompt recovers the target is a statement about an infinite input space, and current evaluation protocols sample a thin slice of it.

The practical consequence: an unlearning evaluation today answers “did the operator do what its authors intended under their test conditions.” It does not answer the question a regulator asks, which is “is the personal data gone.” Until a shared protocol exists that operationalizes the second question, every unlearning metric is an internal engineering signal, not external proof.

What does GDPR Article 17 actually oblige, and what does failure cost?

Article 17 gives individuals a right to erasure: when someone no longer wants their personal data processed and there is no legitimate reason to keep it, the data must be deleted. The regulation, adopted on 14 April 2016 and effective since 25 May 2018, defines personal data broadly: information relating to an identified or identifiable individual, including people who are identifiable indirectly by means reasonably likely to be used.

Three properties of the GDPR turn this from a policy footnote into an operational constraint. First, extraterritorial scope: the regulation applies to organisations outside the EU that process personal data of individuals located inside the EU, so a US-based model provider cannot opt out by geography. Second, the fine structure: penalties run in two tiers, topping out at €20 million or 4% of global revenue, whichever is higher, and data subjects can separately seek compensation for damages. Third, the US has a parallel regime: the CCPA, adopted 28 June 2018, carries many of the same deletion mechanics, making deletion-request handling a two-jurisdiction problem at minimum.

The “identifiable indirectly” clause is where model weights get uncomfortable. If a fine-tuned model can reproduce a person’s data, or facts from which that person is identifiable by means reasonably likely to be used, there is a plausible argument that the weights themselves constitute processing of personal data. No enforcement action has settled that question, and nothing in the evidence base for this article includes a supervisory authority position on unlearning as erasure. But the argument does not need to be settled to change your risk calculus. It needs to be plausible enough that a regulator, an auditor, or opposing counsel can make it.

That is the asymmetry driving this whole analysis. The cost of answering a deletion request with an unverifiable unlearning claim is bounded below by an awkward audit and above by a percentage of global revenue. The cost of answering it conservatively is compute and documentation. Compliance teams do not need to resolve the legal theory to see which side of that trade they want to be on.

Exact versus approximate unlearning: what can each honestly claim?

Exact unlearning, retraining on data with the target removed, can honestly claim that the resulting weights were never exposed to the deleted data; approximate unlearning can only claim that outputs were suppressed, which is a weaker statement no current protocol can upgrade to proof of removal. The distinction is the load-bearing one in this entire debate, and the survey’s taxonomy makes it cleanly: the operator’s cost and the strength of its guarantee trade off against each other, with retraining at the expensive, high-assurance end.

Exact unlearning has one decisive property: it reduces the erasure question to a data-governance question you already know how to answer. If your training set is versioned, your lineage records show which examples were included, and the deletion target is provably absent from the final training corpus, then the resulting model’s weights are clean by construction. The attestation is about your data pipeline, not about probing the model. Auditors understand data pipelines.

Approximate unlearning inverts the epistemics. Instead of proving the data was never there, you intervene on a trained model, via gradient-based forgetting, weight editing, or output suppression, and then attempt to verify the intervention worked. The verification is exactly where the survey says the field is weakest. You are asserting a negative (the model no longer contains recoverable information about this person) using fragmented protocols that cannot objectively compare even positive results.

The table hides one uncomfortable fact in the cost row: for a large fine-tuned model, exact unlearning per deletion request is expensive enough that the industry has strong incentives to make approximate methods work. That incentive is legitimate. Approximate unlearning is a real cost lever and an active research area, and for internal purposes, reducing exposure between retraining cycles, it has genuine value. The failure mode is not using it. The failure mode is forwarding it to a regulator as if it carried the weight of a retrain.

What does a defensible deletion-response pipeline look like today?

Retrain-and-attest: handle each deletion request by excluding the subject’s data from a retraining run, documenting the exclusion through lineage records, and purging every other artifact where the data lives, including caches, embeddings, and fine-tuning checkpoints. This is conservative practice, not settled law, but it is the only pipeline where every link in the chain produces evidence an auditor already knows how to read.

The pipeline has four components, and most teams that get this wrong fail on the third.

First, intake and scoping. Map the deletion request to actual data instances: training examples, fine-tuning examples, retrieved documents, cached interactions. This requires knowing where personal data entered your system, which is a lineage problem you should have solved before the request arrived.

Second, exact unlearning on a schedule. Pure per-request retraining is uneconomic for most deployments, so the practical pattern is batched: deletion requests accumulate in a queue, and a filtered retraining run executes on a defined cadence, with each run’s corpus exclusion documented. The queue depth and cadence become compliance parameters you can defend and tune.

Third, the artifact purge, which is where unlearning claims most often silently fail. A model’s weights are not the only place personal data persists. Embedding indexes built over user documents, retrieval caches, prompt logs, fine-tuning checkpoints from before the deletion, and evaluation sets all carry the same data in more legible form. Purging these is cheap, deterministic, and auditable. Skipping them while unlearning the weights is locking the vault and leaving the windows open.

Fourth, attestation. Produce a written record per deletion: what was excluded, from which corpus version, which artifacts were purged, when the retrained model shipped. This document is the actual deliverable. The retraining is just what makes it true.

Where does the deletion obligation actually bind: data or weights?

The obligation binds wherever personal data is processed, and under the GDPR’s broad definition, that almost certainly includes systems derived from the data, not just the raw records. Personal data covers information about anyone identifiable directly or indirectly by means reasonably likely to be used, which sweeps in databases and embedding indexes trivially, and model weights plausibly.

The data-versus-weights distinction is the conceptual error at the center of most vendor unlearning marketing. Deleting a row from a database is data-level deletion: fast, verifiable, complete. Removing that row’s influence from trained weights is a different operation with no equivalent verification story. Vendors conflate the two because data-level deletion is easy to prove and weights-level removal is easy to claim, and the conflation lets them sell the proof of one as evidence of the other.

A defensible posture separates the two explicitly. For data stores, indexes, caches, and logs, deletion is straightforward and should be immediate on request; there is no excuse for latency here. For weights, the honest answer is the retraining schedule, plus whatever approximate suppression you apply in the interim, described accurately. If your deletion-response documentation does not distinguish these two scopes, an auditor eventually will, and the discovery will not improve the conversation.

There is a harder version of this question lurking behind it: whether a base model you did not train, fine-tuned on your data, puts you in a position where the deletion obligation binds to weights you cannot modify. Nothing in the current evidence resolves that, and this article will not pretend to. But it is the question your procurement team should be asking foundation-model providers now, because the answer determines whether your deletion pipeline is a pipeline or a fiction.

What should you tell your regulator, your auditors, and your vendors?

Tell your regulator that deleted data was excluded from training by documented process and purged from all derived artifacts; tell your auditors to trace lineage records rather than probe models; and tell your vendors that unlearning claims without a shared certification protocol are engineering notes, not compliance evidence. That is the practical verdict, and it holds until either evaluation science or enforcement guidance changes the ground underneath it.

The survey’s own finding is the strongest argument for this posture. Fragmentation across target definitions, mechanisms, and evaluation protocols means no unlearning operator today can produce a certificate that transfers: a number your auditor accepts because it was measured against a standard they recognize. Retraining sidesteps the entire problem by making the claim about data governance, a domain with mature audit practice. Approximate unlearning remains valuable, but in its proper place, as a cost and exposure lever inside your own risk model, priced accordingly.

Now the limits, stated plainly. Every legal claim in this piece rests on general GDPR references: the regulation’s text and scope, the erasure right as summarized by EUR-Lex, and the published penalty tiers. None of it comes from Article 17 enforcement guidance or case law addressing model weights, because no such position appears in the evidence. The sufficiency of any unlearning method, exact or approximate, for legal erasure is untested. Retrain-and-attest is the conservative reading of an unresolved question, chosen because the downside of a fine tier reaching 4% of global revenue dwarfs the cost of extra compute.

The other live risk is the moving target. The certification gap is a finding about the field as of the survey’s consolidation, observed in August 2026. A shared evaluation protocol, a membership-inference standard for generative models, or supervisory guidance on weights-as-processing could obsolete the “no certification exists” claim within months. Watch the evaluation literature, not the operator literature; the operators are already good enough to be tempting. What is missing is the science that would let you prove what they did.

Frequently Asked Questions

Does the retrain-and-attest pipeline apply to base models provided by third parties?

No, because you cannot retrain a base model you do not own. The obligation binds to weights you cannot modify, making the deletion pipeline a fiction unless the provider offers a certified retraining service or you fine-tune on a local copy. Procurement teams must verify if the provider supports exact unlearning via filtered retraining, as approximate suppression on a black-box base model is not a valid erasure claim.

How does the cost of exact unlearning compare to approximate methods for large models?

Exact unlearning requires full or partial retraining compute, which scales linearly with model size and is expensive per request. Approximate unlearning uses gradient-based forgetting or weight editing, which is far cheaper per request but lacks certifiability. For large fine-tuned models, the cost difference is significant enough that industry incentives favor approximate methods, but only for internal exposure reduction, not regulator-facing erasure.

What specific artifacts must be purged in addition to model weights?

Embedding indexes, retrieval caches, prompt logs, fine-tuning checkpoints, and evaluation sets must all be purged. These artifacts often contain personal data in more legible form than the weights themselves. Skipping these purges while unlearning weights is a common failure mode that leaves data recoverable, undermining the erasure claim even if the weights are clean.

Can a vendor’s ‘99% unlearning success’ metric be used as compliance evidence?

No, because the metric is measured against self-selected targets and protocols with no shared benchmark. The survey confirms that evaluation protocols are fragmented, making objective comparison impossible. A ‘successfully unlearned’ report only indicates the operator passed its own test suite, not that the data is unrecoverable. Treat these claims as internal engineering signals, not external proof for supervisory authorities.

What is the primary risk of relying on approximate unlearning for GDPR erasure?

The primary risk is that the claim is unverifiable and legally untested. No shared evaluation protocol exists to certify that weights-level removal actually happened, and no regulator has accepted or rejected unlearning as satisfying Article 17. The cost of an unverifiable claim is bounded below by an awkward audit and above by a fine tier reaching 4% of global revenue, making conservative retrain-and-attest the safer choice.