Synthetic AI training data: Define the task, check the result

Epoch projects that public human-generated training text could be fully used between 2026 and 2032, depending on scaling and overtraining. That conditional window raises a practical question for developers: which skills can generated examples teach, and how will the training pipeline detect their mistakes? (Epoch AI. “Will we run out of data to train large language models?”)

Synthetic AI training data: Define the task, check the result

Generated data can enter a model through pretraining examples, instruction responses, reasoning traces, preference comparisons, or examples selected during reinforcement learning. Those stages have different purposes and evaluation needs. A coding exercise with an executable answer offers a different checking surface from a natural-language preference judgment.

Epoch’s forecast depends on assumptions about scaling, reuse and data selection. Read the projected constraint alongside those assumptions when deciding whether it applies to a particular training plan. (Epoch AI. “Will we run out of data to train large language models?”)

The forecast also leaves a separate measurement question unanswered: what fraction of all frontier training is synthetic today? Answering that requires comparable data covering the developers and training stages being described. A collection of successful releases cannot supply that denominator. (Epoch AI. “Will we run out of data to train large language models?”)

Start with a bounded success case

Microsoft’s Textbooks Are All You Need describes phi-1, a small code model trained with selected web-code material and GPT-3.5-generated textbooks and exercises. Its coding evaluations make this a concrete starting point for examining a generation recipe: inspect the selected material, the exercise stage and the tests together. (full Phi-1 study)

The Phi-4 report describes creating post-training data for supervised fine-tuning (SFT) and direct preference optimization (DPO) to mitigate hallucinations in simple settings. It also describes an improved decontamination process intended to reduce benchmark leakage. These are separate interventions: record what the generated examples teach and how the evaluation data is protected when comparing a recipe. (Microsoft Research. “Phi-4 Technical Report.” arXiv:2412.08905)

Geometry offers a more explicit checking mechanism. AlphaGeometry combines generated training problems with symbolic deduction. Its project explanation describes how the specialized geometry system joins a language model’s suggestions to a deduction engine. A derivation checked against a formal system provides a different quality signal from a fluent answer. (Trinh, Trieu et al. “Solving olympiad geometry without human demonstrations.” Nature, January 2024; Google DeepMind’s AlphaGeometry explanation)

AlphaGeometry2 extends this geometry setting and evaluates its own system. Its results make generation, deduction and the tested problem distribution the relevant unit of comparison. (Google DeepMind. “AlphaGeometry2.” arXiv:2502.03544)

The practical question is which part of an example can be checked. In this setting, formal deductions offer a defined checking boundary. A useful experiment in another domain needs to specify its own equivalent, whether that is an executable test, an independently checked answer, or an evaluation that can expose mistakes in the explanation. (Google DeepMind. “AlphaGeometry2.” arXiv:2502.03544)

Keep model reports attached to their training stage

Choose a training stage before comparing releases. The Llama 3 report documents generation and selection within its training and post-training process. (Meta AI. “The Llama 3 Herd of Models.” arXiv:2407.21783) DeepSeek-R1’s distillation work has a more specific teacher and student relationship: it fine-tunes named base models on curated R1-generated samples. Compare the teacher, selected examples, student and evaluations when assessing another distillation attempt. (DeepSeek-AI. “DeepSeek-R1.” arXiv:2501.12948)

Keep other interventions visible when asking what generated data contributed. Qwen3’s report describes a complete set of training and post-training choices. (Qwen Team. “Qwen3 Technical Report.” arXiv:2505.09388) The study Synthetic Data Generation & Multi-Step RL for Reasoning & Tool Use combines generation and reinforcement learning; a comparison needs to retain both rather than attributing the full outcome to generation alone. (“Synthetic Data Generation & Multi-Step RL for Reasoning & Tool Use.” arXiv:2504.04736)

Model collapse depends on the feedback process

Shumailov and colleagues study the loss of information when generated outputs feed subsequent generations of training. Their full manuscript describes the recursive process and the distributions in which collapse occurs. This is a reason to record what happens to the original data as generated examples enter later rounds. (Shumailov, Ilia et al. “AI models collapse when trained on recursively generated data.” Nature, July 2024; full recursive-training study)

Feng, Subramonian and Kempe’s Strong Model Collapse focuses its theoretical analysis on regression. Its experimental validation also includes toy settings and small language models, including GPT-2 models trained on BabiStories. Keep the theoretical assumptions and each experiment’s model and dataset distinct when applying its feedback and scaling questions to a new pipeline. (Feng, Subramonian and Kempe. “Strong Model Collapse.” arXiv:2410.04840)

Gerstgrasser and colleagues change the data lifecycle: they accumulate generated examples alongside retained real data instead of replacing the old dataset. Their experiments avoid collapse in the tested settings. That difference makes retention and replacement substantive design choices to record before comparing two synthetic-data results. (Gerstgrasser, Matthias et al. “Is Model Collapse Inevitable?” arXiv:2404.01413)

Choose a verifier with a known boundary

Beyond Model Collapse studies verification through analytical models and two practical tasks: computing matrix eigenvalues with transformers and news summarization with LLMs. It examines when a verifier can select generated examples that help prevent collapse, including the use of imperfect verifiers. Applying that result elsewhere requires examining what the checker can detect and how its errors relate to the generated data. (Verification and synthesized-data study)

These are proposed review checks derived from the studies. They describe the evidence needed to assess an experiment; this article supplies no tested training recipe.

Related discussions of manifest validation and synthetic clinical notes concern different checking surfaces. A checker for format or syntax leaves broader task requirements to be evaluated separately.

The useful comparison is between complete pipelines: what generated the data, what selected it, what real evidence remained, and how the final model was evaluated. Keep those records and held-out tests when borrowing a successful recipe. They are how an experiment can reveal what the generator or verifier misses.