How to Attribute AI-Generated Images Without Retraining: CLIP vs DINOv2

If your trust-and-safety or forensics team has to name the generator behind a suspicious image, you do not need to retrain a classifier every time a new diffusion model ships. The practical answer from a July 2026 controlled study is a training-free retrieval pipeline: encode queries and per-generator reference images with intermediate layers of CLIP or DINOv2, then attribute by nearest-reference match. Two coupled settings determine how well that works before you write a line of training code: which layer you probe (accuracy peaks at normalized depths of 0.34–0.58 across six CLIP variants, self-reported in RefPrint, arXiv:2607.12052) and how you select the reference images for each generator. The consequence is worth stating plainly: the representation choice caps your accuracy before reference curation even begins, so a silent default in the pipeline code can cost you more than weeks of careful curation will recover.

Everything below is conditional on one caveat: the core evidence is a single group’s self-reported analysis on lab datasets, with no independent replication and no testing against re-encoding chains, JPEG AI neural compression, or adversarially evasive images. Ordinary JPEG compression was checked, with a cautionary result covered below. Treat every accuracy number here as a starting hypothesis for your own pilot, not a specification.

Why retraining stopped being the default assumption

Classical supervised attribution learns each generator’s fingerprint in a closed set: train on labeled samples from N generators, and the classifier can only ever name those N. That works until the generator set changes, which it now does constantly. As the RefPrint authors put it, supervised fingerprint learning is effective in closed-set settings but hard to scale when new generators continuously emerge, proprietary models expose only limited samples, and retraining a global model each time is impractical.

Training-free reference-based attribution inverts the economics. Instead of labeled training data and a retraining run per generator, you maintain a curated set of source-specific reference images per generator. A new model onboards by registering references, no gradient updates required. The same retrieval framing has been demonstrated in the video domain, where a training-free retrieval approach (arXiv:2607.28955) reports a self-reported 84.6% Rank-1 accuracy at 100-shot and argues the paradigm is “highly scalable, as newly emerging generators can be incorporated by simply registering new samples without retraining the model.” A second benefit matters for casework: retrieved references double as explainable evidence. When a match goes to a reviewer, an analyst, or a legal process, “this image is closest to these registered outputs of Generator X” is more defensible than a bare softmax score.

Do not mistake “training-free” for “work-free.” The burden moves from data labeling and GPU training runs to reference curation, and the study shows arbitrary curation measurably underperforms. That trade is the real subject of the next sections.

How the pipeline actually works

The mechanism is straightforward. Encode the query image with a pretrained vision encoder (CLIP or DINOv2, frozen, no fine-tuning). Encode each registered reference the same way. Attribute the query to the generator whose references are nearest in that feature space. There is no task-specific model at all, which is why onboarding a generator is an indexing operation rather than a training job.

The RefPrint study ran a controlled factorial analysis over three variables: encoder layer, reference-selection strategy, and per-generator reference budget M. Evaluation used three datasets (all dataset figures self-reported by the authors): FaceResyn, an external face-only benchmark from prior work covering 10 generators with 1,000 head-and-shoulder portrait queries drawn from 100 character descriptions and 10,000 query-specific resynthesis references; BC-Attr-6, a controlled multi-category set constructed for the study, spanning ten text-to-image generators across six semantic categories; and COCO-Attr, also constructed for the study, which replaces predefined categories with MSCOCO-derived captions (1,000 queries, a 39,429-image reference pool) so that curation is tested against heterogeneous content. Code and datasets are released at github.com/meiling-fdu/RefPrint, which at least makes the factorial design reproducible by other groups.

A related precedent exists in the style-attribution literature: IntroStyle (arXiv:2412.14432) showed that internal features of a diffusion model alone can support training-free attribution, with ablations over probe location (timestep, block index) that mirror the layer-selection knob. Its authors also flag the compute cost of running attribution through a large Stable Diffusion UNet, which is one reason probing a pretrained encoder like CLIP is the cheaper operational choice.

Knob one: the representation, and why the middle layers win

The study uses CLIP as its main representation and DINOv2 to check whether the findings are encoder-specific. The layer-wise behavior is not encoder-specific, though absolute accuracy is: “Although CLIP outperforms DINOv2 overall, both exhibit similar layer-wise behavior” (arXiv:2607.12052, self-reported). The clearest finding: “Both encoders generally perform best at intermediate layers, with lower accuracy at shallow and final layers” (same source, self-reported). Across six CLIP variants, peak attribution accuracy occurred at normalized depths of 0.34–0.58, again self-reported by the same study.

The authors’ own diagnostic backs the mid-network choice: diagnostic linear classifiers trained on frozen CLIP features show semantic separability increasing toward deeper layers while generator separability peaks at intermediate layers before declining (Fig. 4, self-reported, arXiv:2607.12052). Why that pattern holds is interpretation rather than a measured result: shallow layers plausibly encode low-level texture, where generator fingerprints live but so does irrelevant noise, while final layers encode strong semantic abstraction, where the model has largely decided what the image depicts and discarded how it was synthesized. Intermediate layers retain source-discriminative cues “before strong semantic abstraction dominates.” So if you switch encoders tomorrow, do not default to the final embedding layer, which is what most off-the-shelf feature-extraction code gives you. Sweep the middle third of the network and validate.

One caution against reading this as “CLIP wins, done.” In an adjacent task, EKILA (arXiv:2304.04639) found that a purpose-built visual matcher can “outperform CLIP and common perceptual metrics (LPIPS, SIFID) at the attribution task.” That was training-data attribution rather than generator attribution, a different question, but it demonstrates that a task-matched representation can beat general-purpose CLIP features when someone bothers to build one. CLIP’s advantage here is availability and zero training cost, not proven superiority against every alternative.

Knob two: how you pick the references

The second finding is less intuitive and more consequential for operations. Intermediate features are not semantically neutral. Quoting the study directly: “intermediate representations are not completely semantically neutral, making reference selection critical: semantically constrained references reduce query-reference mismatch and improve attribution, especially under limited reference budgets.”

The authors tested three reference-selection strategies per generator, in increasing order of semantic constraint:

• Arbitrary: sample references however they come. This baseline “generally yield[s] the lowest accuracy” (self-reported, arXiv:2607.12052).
• Semantically aligned: select references whose content matches your expected query distribution.
• Resynthesis-based: take an actual query or query-like prompt and regenerate it through the candidate generator, producing a reference semantically near-matched to that query by construction.

The budget interaction is the operational heart of the result: resynthesis references are most effective when you have very few references per generator, while semantically aligned selection becomes stronger as the per-generator budget M grows (self-reported, same source). In other words, the strategy ranking is not fixed; it flips with your curation budget. A team that picks one strategy and hard-codes it has made an accuracy decision without realizing it.

COCO-Attr, the study’s caption-derived dataset with no predefined categories, supplies the most operational version of this result, because it removes the excuse that semantic alignment needs a taxonomy. The authors select each source’s top-M references by image-only semantic retrieval, with CLIP’s final layer ranking the pre-generated pool, and at L12 that reaches 93.20% accuracy against 75.80% for arbitrary sampling and 82.80% for resynthesis (self-reported, arXiv:2607.12052). Semantic alignment can be computed from the reference pool you already own, with no category labels and no test-time generator access.

Putting the two knobs together

The two knobs interact: the residual semantic content in intermediate features is exactly why reference semantics matter, and probing a different layer changes how much reference semantics help. The table below consolidates the self-reported findings from the RefPrint study into the decision a pipeline owner actually faces.

The CLIP-versus-DINOv2 framing in this article’s title is, honestly, the least interesting axis in the study. My reading of the evidence is that probe depth is the more consequential knob, because the intermediate-layer trend held for both encoders even though CLIP outperformed DINOv2 overall; encoder choice still moves absolute accuracy. Treat encoder selection as an engineering decision (licensing, inference cost, existing infrastructure) and spend your validation effort on probe depth and reference strategy.

Onboarding checklist for a new generator

When a new generator appears in your traffic, the evidence supports this sequence:

1. Register before you classify. Collect initial samples of the new generator and add them as source-specific references. This is the scaling property the study identifies as the paradigm’s core advantage: no retraining, so onboarding is a curation and indexing step rather than a labeling-and-training cycle.
2. Probe intermediate layers, and re-probe under compression. If you have not validated a layer for your encoder, sweep the middle third of the network; the 0.34–0.58 normalized-depth band (self-reported, arXiv:2607.12052) is a starting range, not a guarantee for an untested backbone. Under JPEG 70 the study’s own optimum moved to L17, outside that band, so run the sweep on degraded images too.
3. Match references to traffic, not to convenience. Under a small budget, prefer resynthesis against representative queries. As the budget grows, build semantically aligned sets covering the content categories you actually see in casework.
4. Keep the retrieved references. They are your evidence trail; the video-domain retrieval work makes the interpretability argument explicitly.
5. Validate against your own distribution shift. The lab check covers JPEG 70, resizing, and Gaussian blur on one dataset; your traffic brings re-encoding chains and platform codecs the study did not simulate. Run the pipeline on that traffic before trusting any lab number.

When retraining is still the right call

The case for training is robustness, and it is currently the stronger-evidenced case on that axis. OmniDFA (arXiv:2509.25682), a trained few-shot model, “maintains robust performance within the augmentation range and exhibits strong resilience to JPEG compression even beyond the augmentation range used in training,” while “other methods suffer significant performance degradation from the outset” (self-reported by that paper’s authors). Its authors also report accuracy rising rapidly from 1-shot to 10-shot and then improving gradually, calling 10-shot the favorable performance-compute trade-off (self-reported).

The open-set objection to training, that a classifier cannot name generators it has never seen, has a trained answer too. Open-set metric learning (arXiv:2308.11557) learns transferable embeddings that discriminate generators unseen during training, and its authors found pretraining on camera-model identification improved transferability, a sensible trick since camera identification is the classical version of this problem. That study also JPEG-compressed training and evaluation images at quality factors 75 through 95 to mimic real distribution shift, building compression robustness into training rather than measuring it afterward.

So the decision rule I would apply: if your generator set is stable, labeled data is abundant, and your traffic is heavily re-encoded, a trained model with compression augmentation is the better-evidenced choice. If generators appear faster than you can label, training-free retrieval with mid-layer features and curated references is the economical default, with the explicit understanding that its measured JPEG-70 robustness came with a steep accuracy drop and a shifted optimal layer, and that JPEG AI compression is untested.

What the lab numbers do not cover

Three gaps should shape any deployment decision. The first two are tested only partially in the study, and the partial results are themselves cautionary.

Compression and re-encoding. Ordinary JPEG is tested. The study’s robustness check (Fig. 5, arXiv:2607.12052) evaluates attribution under JPEG compression at quality factor 70, resizing, and Gaussian blur; intermediate CLIP layers remain preferable, but the optimum moves. Under JPEG 70, accuracy “peaks at L17 (50.25%), compared with 40.83% at L12 and 27.33% at L24” (self-reported), and the authors conclude that “the optimal representation depth depends on image degradation rather than remaining fixed.” On the same BC-Attr-6 protocol with semantically aligned references, clean-data accuracy at L12 was 85.33%; JPEG 70 leaves it at 40.83%, and even the shifted L17 peak reaches only 50.25%. Ordinary re-encoding therefore both costs accuracy and invalidates a layer choice tuned on clean images. What is genuinely untested is neural compression: research on JPEG AI neural compression (arXiv:2412.03261) finds that a strong second neural compression may cancel forensic traces left by a previous compression step, and its authors call for forensic techniques robust to neural compression artifacts. The training-free pipeline has never been tested against this. If your traffic passes through social platforms or modern codecs, lab accuracies may not survive contact with them.

Unseen generators. Retrieval attribution handles new generators by registration, but genuinely unregistered generators are a different matter: the system will return its nearest registered reference, which may be wrong with high confidence. The study does measure this. In a 10-fold leave-one-generator-out evaluation that treats the held-out generator as unknown and sweeps a rejection threshold over the maximum enrolled-source similarity, AUROC at L12 and M=10 runs from 66.17% with arbitrary references to 73.74% with semantically aligned ones (self-reported, arXiv:2607.12052), and the authors themselves call the exercise “an initial open-set stress test, rather than comprehensive open-world attribution.” Rejection is therefore tested, but only preliminarily: AUROC between 66% and 74% leaves room for both missed unknowns and false rejections at any operating threshold, and the nearest-registered-reference failure mode has not gone away. The trained open-set approach remains the design built around this problem.

Adversarial evasion. The study does not test whether someone who knows the pipeline can perturb an image to shift its intermediate-layer features toward a different generator’s references. Given that the pipeline relies on publicly available encoders, assume evasion is possible until tested.

Provenance is a complement, not a substitute

An October 2026 paper on partial C2PA Content Credentials (arXiv:2610.38639) shows that even incomplete credentials can be used to reconstruct the last signing tool’s processing and adapt a forensic classifier accordingly, under explicit assumptions that the credentials are trusted and the signing key is uncompromised. C2PA provenance and forensic matching answer different questions: credentials tell you what a trusted tool says happened; retrieval attribution tells you what the pixels say when no credential exists or none can be trusted. EKILA combined both routes for generative art, and that architecture (provenance where available, forensic matching where it is not) is the reasonable end state for a production pipeline.

For teams that must attribute AI images without retraining, the evidence supports a concrete build: frozen CLIP or DINOv2 features probed at intermediate layers (self-reported peak band 0.34–0.58 normalized depth, arXiv:2607.12052), per-generator references curated semantically rather than arbitrarily, resynthesis when the budget is small and semantically aligned sets as it grows. In this evidence the two knobs are coupled rather than ranked: both move accuracy by double digits, and arbitrary references measurably underperform.

Two verification steps belong on your roadmap before deployment, because the published evidence cannot settle them for your traffic: measure the pipeline on your own re-encoded and platform-compressed images, where a single JPEG-70 check cannot stand in for your chain of codecs, and calibrate its rejection behavior when the true generator is not in the registry, beyond the study’s preliminary open-set stress test. Both are properties of exactly the traffic a forensic pipeline exists to handle.

Frequently Asked Questions

Which encoder layers should be used for attribution?

Across six CLIP variants, peak attribution accuracy occurred at normalized depths of 0.34–0.58, again self-reported by the same study.

How does JPEG compression affect attribution accuracy?

Under JPEG 70, accuracy “peaks at L17 (50.25%), compared with 40.83% at L12 and 27.33% at L24” (self-reported), and the authors conclude that “the optimal representation depth depends on image degradation rather than remaining fixed.”

Which reference selection strategy is best for small budgets?

The budget interaction is the operational heart of the result: resynthesis references are most effective when you have very few references per generator, while semantically aligned selection becomes stronger as the per-generator budget M grows (self-reported, same source).