In traditional software development, the Product Requirements Document (PRD) has been the canonical artifact: a meticulous blueprint that dictates what the product should do, how it should behave, and what success looks like. As machine learning becomes a larger share of what ships, a slogan has been making the rounds in AI product circles: “Data is Your PRD.”
The phrase is frequently attached to Andrew Ng, whose data-centric AI work argues that systematic dataset engineering deserves the same attention as model engineering. Ng is a coauthor of DataPerf, a benchmark suite designed to make that work measurable. The slogan itself has the texture of folk attribution: it circulates with Ng’s name on it, but it is not a named framework in his published courses, and there is no single primary source showing that he coined it. The underlying argument is consistent with the broader data-centric AI literature, which is probably why the attribution sticks. Either way, the claim the slogan compresses is worth engaging with on its own terms: when behavior is learned from data, the dataset becomes part of the specification.
Why the Traditional PRD Breaks
For decades, software development followed a predictable pattern: product managers wrote detailed PRDs, engineers implemented the specifications, and QA teams verified the outputs matched the requirements. This worked because traditional software is deterministic. Given the same inputs, the same code produces the same outputs every time.
AI systems break this contract.
Large language models and other ML systems are probabilistic. The same prompt can yield different responses. Edge cases that deterministic testing would catch multiply. And the part that determines system behavior is no longer written by engineers; it is learned from data.
Andrej Karpathy named this transition “Software 2.0” in a 2017 essay. In Software 1.0, humans write explicit instructions. In Software 2.0, humans specify objectives and supply data, and the network learns the implementation. The data, in that framing, is where the desired behavior actually lives. The essay predated the generative-AI boom by several years, but it remains the cleanest statement of why a prose PRD stops being the only requirements artifact once the system is learned rather than coded.
What “Data is Your PRD” Actually Means
The argument, whoever first compressed it into the slogan, rests on three principles that AI product managers have to internalize.
1. Specifications Become Data Collections
In traditional product management, you define what the product should do through written specifications. In AI product management, you define it through curated datasets.
Instead of writing “the chatbot should provide helpful customer-service responses,” you collect thousands of examples of helpful customer-service interactions. Instead of specifying “the code review tool should identify security vulnerabilities,” you assemble datasets of code with and without security issues.
The data becomes the specification, and its quality, coverage, and relevance cap what the product can do.
2. Success Metrics Shift from Features to Performance
Traditional PRDs measure success through feature completion: “Did we build the login system?” “Does the dashboard display analytics?” AI PRDs measure success through performance on representative tasks: “Does the model hit our accuracy target on the held-out evaluation set?” “Do raters score a target share of outputs as acceptable?” The thresholds are set by the team, not discovered; the discipline is defining a fixed benchmark and measuring against it, rather than declaring a feature done.
This requires product managers to become fluent in evaluation metrics that were once the domain of ML engineers. Accuracy, precision, recall, F1, perplexity, BLEU, and human-evaluation protocols become the language of product requirements. DataPerf’s benchmark design is useful here because it scores the data-development process itself rather than treating the training dataset as a fixed input.
3. Iteration Cycles Accelerate Dramatically
Traditional software releases happen in weeks or months. AI systems can iterate in hours or days, given the right data infrastructure. When your PRD is a dataset, “updating requirements” means collecting new examples, not rewriting documents.
This acceleration cuts both ways. Product managers must develop new muscles for rapid experimentation, online evaluation, and continuous deployment of model improvements.
Concrete Examples: Data as PRD in Practice
Three scenarios where traditional PRDs fail and data-driven requirements hold up.
Example 1: Customer Support Chatbots
Traditional PRD approach:
- Write detailed conversation flows
- Define keyword triggers
- Specify response templates
- Result: brittle systems that break when users deviate from the expected path
Data-as-PRD approach:
- Collect a large corpus of real customer-support conversations
- Label ideal responses and common failure modes
- Define an evaluation target on a fixed rubric (e.g., a target share of responses rated helpful by human raters)
- Continuously collect new conversations and retrain
- Result: adaptive systems that handle unexpected queries
Example 2: Code Generation Assistants
Traditional PRD approach:
- Specify supported programming languages
- Define code style guidelines
- Document API integrations
- Result: limited to explicitly programmed capabilities
Data-as-PRD approach:
- Curate a large body of high-quality code, weighted toward the languages and domains the product targets
- Label code by functionality, quality, and security characteristics
- Define an evaluation target: generated code passes a held-out test suite at a target rate
- Result: systems that produce solutions not seen verbatim in training
Example 3: Document Analysis Tools
Traditional PRD approach:
- Define document types to support
- Specify fields to extract
- Write extraction rules
- Result: fragile parsers that fail on format variations
Data-as-PRD approach:
- Collect diverse document examples with human-annotated extractions
- Define an evaluation target: a target field-extraction accuracy across document variations
- Continuously expand the dataset with new document types
- Result: robust systems that generalize to unseen formats
The Infrastructure Required
Embracing “Data is Your PRD” requires infrastructure that traditional software teams rarely need.
Data Collection Pipelines
You need systematic ways to collect, clean, and annotate data. This often means human-in-the-loop systems where domain experts label examples, review model outputs, and provide feedback that becomes training data.
Evaluation Frameworks
You need rigorous evaluation protocols that measure performance on tasks that matter to users. This includes automated metrics (accuracy, F1, and the rest) and human-evaluation protocols that capture subjective quality.
Version Control for Data
Code has version control; datasets need it too. DVC, Weights & Biases, and MLflow can track dataset changes alongside model performance and experimental results. The exact tool matters less than preserving the mapping from dataset version to evaluation result and deployed model.
Continuous Training Pipelines
You need infrastructure to retrain models as new data arrives: data validation, training, evaluation, and deployment pipelines that run automatically or on demand.
Challenges and Pitfalls
The framework is powerful but not free.
Data Quality vs. Quantity
More data is not always better. Poor-quality data produces poor-quality models. Product managers have to develop taste for what counts as good training data, which usually requires real domain expertise.
The Cold Start Problem
New AI products face a chicken-and-egg problem: you need data to build the product, but you need the product to collect data. Successful teams usually bootstrap with synthetic data, open datasets, or manual data collection before launch.
Bias and Fairness
Your data reflects the world as it is, biases included. If your training data encodes historical hiring decisions, your model will reproduce historical biases. Identifying and mitigating that is active work, not a checkbox.
Regulatory Compliance
As AI regulation tightens, documenting what data trained a model is becoming a compliance requirement, not merely a best practice. The EU AI Act entered into force on 1 August 2024 and phases in through 2027, with documentation and data-governance obligations for covered high-risk and general-purpose AI systems. Various U.S. state laws layer on top. For teams under these regimes, treating data as a versioned requirements artifact can also produce part of the audit trail regulators ask for.
What the Slogan Leaves Out
Data cannot replace the entire PRD. It can specify learned behavior, but it cannot by itself define latency budgets, privacy boundaries, escalation rules, accessibility requirements, unit economics, or the consequences of a wrong answer. Those remain prose, policy, and system-design requirements. A production AI product therefore needs three linked artifacts:
- A behavior dataset containing representative inputs, desired outputs, difficult boundary cases, and explicitly disallowed outcomes.
- An evaluation contract defining metrics, slices, human-review rubrics, acceptable thresholds, and the regression budget for every release.
- A conventional product specification defining everything outside model behavior: permissions, retention, latency, cost, rollback, observability, and incident ownership.
The data-centric framing is most useful when it changes how these artifacts relate. A requirement such as “the assistant must not disclose another customer’s account data” should exist as policy text, as adversarial examples in the evaluation set, and as a deterministic authorization boundary outside the model. If it exists in only one place, the requirement is fragile. A prose-only rule may never be tested; a dataset-only rule may be learned imperfectly; a guardrail without an evaluation may silently regress.
This is also why a larger training corpus is not automatically a better PRD. The dataset must make product choices visible. Teams should record why examples were included, which user population each slice represents, who labeled ambiguous cases, and what changed between versions. Data-centric AI research emphasizes systematic data engineering, not indiscriminate accumulation. The useful unit is a documented, testable behavior decision.
A Practical Data-PRD Review
Before an AI feature ships, a product manager should be able to answer a short set of questions:
- Which real user tasks and failure modes appear in the evaluation set?
- Which important cohorts or languages are underrepresented?
- Are training, validation, and production feedback separated well enough to prevent leakage?
- Does each metric have a threshold and an owner who can stop the release?
- Can the team reproduce the result from a named dataset and model version?
- What deterministic system control catches failures that the model cannot reliably learn away?
- How will newly observed failures become reviewed examples without poisoning the dataset?
That review turns “data is your PRD” from a slogan into an operating method. It also makes the limits legible: the dataset expresses examples of desired behavior, the evaluation harness measures them, and the surrounding product specification controls the system in the cases where probabilistic behavior is not acceptable.
The Role of the AI Product Manager
In this model, the AI product manager’s job evolves in three directions.
Data Strategist
AI PMs have to think hard about what data to collect, how to collect it ethically, and how to maintain its quality over time. They become stewards of the organization’s most valuable AI asset: its datasets.
Evaluation Designer
Instead of writing feature specifications, AI PMs design evaluation protocols. They define what “good” looks like, build test sets that represent real user needs, and establish metrics that correlate with user satisfaction.
Iteration Orchestrator
AI PMs orchestrate rapid iteration cycles. They decide when to collect more data, when to retrain, when to run experiments, and when to ship. They trade off model performance against latency, cost, and user experience.
Where This Goes Next
As AI capabilities advance, the framework is likely to evolve in a few directions.
Automated Data Generation
Synthetic data generation, producing training examples through models rather than human collection, keeps getting more capable. It reduces the manual-collection burden while raising fresh questions about quality, diversity, and model collapse.
Real-Time Adaptation
Future systems may adapt in real time based on user interactions, updating their behavior without explicit retraining. This raises a technical problem (catastrophic forgetting) and a product problem (predictable behavior).
Multi-Modal Requirements
As systems handle text, images, audio, and video, the “data” in “data is your PRD” gets more complex. Product managers have to think in terms of multi-modal datasets and cross-modal evaluation.
Strip the slogan of its branding and the operational claim is straightforward. Once a system’s behavior is learned from data, the dataset is where the requirements live, the evaluation harness is where they are checked, and the prose PRD becomes a secondary artifact. The job of the product manager shifts accordingly: less specification, more dataset curation and evaluation design.
That is the part worth keeping, and it holds whether or not Andrew Ng ever said the words on a stage. The teams that build the data flywheel, define honest evaluations, and treat the dataset as a versioned engineering artifact will ship better ML products than the teams still writing ten-page specs for systems whose behavior no one can fully specify in prose. The teams that do not will keep being surprised when the model does something the document did not predict.
