Robotics teams treat data quality as a virtue: a general goodness you want more of, funded by whatever survives the compute and collection budgets. We think quality is a curve. Each class of training-data defect (noisy actions, dropped frames, desynchronized cameras, corrupted normalization statistics, mislabeled joint orders, and inconsistent demonstrators) has a price in task success, the prices differ by orders of magnitude, and few teams can name theirs.
The public evidence says the prices are large. Moving behavior cloning from one proficient demonstrator's data to a mixed-skill pool cut task success from 92% to 12% in the RoboMimic study (Mandlekar et al., CoRL 2021; arXiv 2108.03298). Perturbing camera viewpoints and initial states at evaluation dropped leading Vision-Language-Action (VLA) models (policies that map camera images and language instructions to robot actions) from 95% to below 30% (arXiv 2510.13626). Common sensor artifacts such as dead pixels and lens contamination pushed VLAs from roughly 90% success to as low as 2% at inference (arXiv 2602.01158). These results bracket the problem from the sides: mixed quality on one end, corrupted inputs at test time on the other. The curve itself, the function from a defect's dose in the training set to the points of success it costs, has never been published for modern robot policies.
Those numbers are worth separating carefully, because not one of them is the curve. Each measures something adjacent to it.
- RoboMimic, mixed-skill operator pool, 92% to 12%. Observed (CoRL 2021). A whole-dataset swap, not a dose series: no intermediate mixing ratios were tested, so it bounds the damage without pricing it.
- Viewpoint and initial-state perturbation, 95% to below 30%. Observed, but applied at evaluation rather than in the training set. It measures inference-time brittleness, not what a defective training hour costs.
- Sensor artifacts, roughly 90% to 2%. Observed, also at inference. It shows how little degradation a trained policy tolerates, not how much a training set can absorb.
- MNIST label dilution, above 90% accuracy held at 100:1. Observed, in supervised image classification. Robot policies act on their own predictions, so this transfers as a ceiling on optimism rather than a guarantee.
- Constant-count poisoning, roughly 250 documents from 600 million to 13 billion parameters. Observed, in language models, and adversarial by construction. Reading it as evidence about accidental contamination in robot fleets is our inference, not a robotics result.
- Demonstration-consistency metrics, predicting 70.2% and 88.6% of task success. Observed, correlational, across user studies of 24 and 30 participants. Predictive of success, not demonstrated to cause it, and small-n.
Read the second sentence of each entry and the gap is the point. Every result is either a test-time perturbation, a different learning setting, or a single all-or-nothing swap. Not one is a training-side dose-response curve for a modern robot policy. That absence is the subject of this post.
This post lays out the shape of that curve as we see it from inside production pipelines: three regimes of defect, why teams systematically misprice them, and how we are measuring the prices directly. The regime map is a hypothesis with strong priors, not a measurement, and we have tried to mark throughout where the evidence ends and our inference begins.
Quality gets priced as a virtue and paid as a curve
We have argued that cost per hour is the wrong way to buy robot data and that data, not architecture, now decides who ships. This season's research sharpened the stakes: the strongest 2026 papers are workarounds for data the field cannot collect, which leaves the data that does get collected carrying more weight per hour, defects included. The quality question has the same structure. QA is usually budgeted as a flat tax: some percentage of episodes reviewed, some percentage of labels audited, uniformly across defect types. Uniform review implicitly prices every defect the same. They are not the same. Three regimes cover the failures we encounter, and their prices differ by orders of magnitude.
Regime one: zero-mean noise washes out
Unbiased noise is the cheapest defect class, because averaging is on your side. The reference result comes from supervised learning: Rolnick et al. showed that an image classifier can hold above 90% accuracy on MNIST even when every correctly labeled example is diluted by 100 randomly labeled ones, provided the dataset grows enough to compensate (arXiv 1705.10694). The robot analog is jittery teleoperation: small, unbiased errors in recorded actions that scatter around the true signal. With enough demonstrations, the policy regresses toward the mean, and the mean is right.
Two conditions hide in that sentence, and both cost money. The noise must be unbiased. And the compensation is volume: noisy data behaves like a smaller dataset, so you pay for the same signal in extra hours of the scarcest data in AI. Robot policies are also less forgiving than classifiers, because they act on their own predictions: imitation learning compounds mistakes, since one small error carries the robot into states no demonstration covered (Ross, Gordon, and Bagnell, AISTATS 2011; arXiv 1011.0686). Survivable does not mean free. It means the curve slopes gently.
Regime two: structural defects never average out
Averaging removes variance. It does not remove bias. A camera mounted 3 degrees off spec, a teleoperator who releases the gripper early on every episode, a joint reported in the wrong order: these are systematic errors, and more data repeats them rather than canceling them.
Demonstrator skill is the best-documented case. RoboMimic's collapse from 92% to 12% came from exactly this: not corrupted files, but a pool of operators whose differing strategies and skill levels produced conflicting supervision. The effect is measurable before training: Sakr et al. found that consistency metrics computed on the demonstrations themselves collectively predicted 70.2% and 88.6% of task success rate across two user studies (arXiv 2412.14309).
The second property of this regime is that dose is not always a percentage. A 2025 study of poisoning attacks found that roughly 250 poisoned documents were enough to backdoor language models from 600 million to 13 billion parameters: across that scale-up, the poisoned fraction shrank to a few ten-thousandths of a percent, and the attack kept working (arXiv 2510.07192). The lesson transfers uncomfortably well to robot fleets. One bad collection session or one miscalibrated rig can be a constant-count contaminant that scale does not wash away.
Regime three: the silent classes
The most expensive defects produce no symptom during training. Loss falls, gradients behave, checkpoints save, and the policy is broken.
Normalization statistics are the canonical case. Policies scale observations and actions by per-dimension means and standard deviations computed over the dataset. If those statistics do not match the data (recomputed on a subset, carried over from another robot, corrupted in a merge), the model trains successfully on garbage scaling. The public issue trackers of open VLA stacks record fine-tunes that converge normally and then evaluate near 1% success, traced to mismatched normalization statistics. The broader phenomenon is documented at scale: silent data corruption in large training runs can look like benign numerical noise while degrading or derailing the model (arXiv 2502.12340).
Synchronization belongs here too. A VLA training example is a supervised pair: what the robot saw, and what it did at that moment. Offset the camera stream from the action stream by 50 milliseconds and every pair in the affected episodes is quietly rewritten, while every individual frame still looks fine to a reviewer. We walked through the mechanics in our analysis of the VLA data problem: temporal alignment is part of the label, and no loss curve audits it.
What we see in production pipelines
These are field notes from our own data operations and quality assurance work, not benchmarks. Two patterns repeat.
First, QA effort follows visibility, not toxicity. Review queues fill with the defects a human can see in a video scrub: motion blur, a dropped object, an obviously botched demonstration. Those live mostly in the gentle regimes. The classes that dominate the price list (desynchronized streams, stale normalization statistics, and schema drift) are invisible in a video player, so uniform review spends the budget where the curve is flattest.
Second, silent defects are born at boundaries. A format conversion that resamples timestamps. A firmware update that reorders joint indices. Statistics recomputed after an episode purge, but only for one of three dataset shards. Each tool in the chain is correct in isolation; the defect lives in the seam between them.
This is why our pipeline is built the way it is. Every episode is gated at ingest against a defect taxonomy (synchronization, calibration, schema, statistics, and demonstration-quality checks) before it can reach a training set, and First-Pass Yield, the share of episodes that clear that gate without rework, is the health metric we track per source and per rig. Every label carries glass-box provenance: a traceable record of the source frames, calibration state, and reviewer decisions behind it, so a deployment failure resolves to specific data in minutes rather than archaeology. And the loop is closed: a traced failure reprioritizes what gets collected and reviewed next, instead of filing into a postmortem.
The strongest objection: scale digests dirty data
Frontier language models train on lightly filtered internet text, and robot co-training mixes already dilute any single source: Physical Intelligence reported that 97.6% of π0.5's training mix came from sources other than the target robot (arXiv 2504.16054). If robot foundation models follow the same trajectory, will scale not digest the defects?
For one regime, yes. That is what regime one says, and it is why the dilution result exists. The objection fails on the other two. Structural bias survives averaging at any scale. Constant-count contamination survived a 20x scale-up in the poisoning study. And robot data lacks the dilution reservoir language had, with roughly 300,000 hours of robot manipulation data in existence against a billion hours of internet video (the gap we audited in the data-economics piece). Scarcity cuts both ways: each defective hour is a larger fraction of everything there is.
Closed loops sharpen the point. The field is converging on flywheels in which auto-labeling (machine-generated candidate labels for humans to verify) is trained on yesterday's data to produce tomorrow's. A defect that slips into that cycle stops being an error in one dataset and becomes an error in the ground-truth generator. Flywheels compound whatever you feed them, including the defects.
Where the argument is weakest is worth saying plainly. Early on, before you have a defect distribution to budget against, uniform review is probably both cheaper and more informative than a taxonomy you have not earned; toxicity-weighted QA should start paying once you are running enough sources, rigs, or a closed loop for the defect classes to separate. We are deliberately not putting an episode count or a rig count on that crossover, because we cannot: it is a hypothesis from our own operations, not a measured threshold, and locating it is one of the things the study below is meant to do. Note also that the regime where toxicity weighting wins is exactly the regime our own product serves, so weigh the argument accordingly.
Measuring the curve directly
Field notes are not a curve, so we are running the measurement. Fix open VLA models (SmolVLA at 450 million parameters, and a π0.5-class model) on a standard simulated manipulation benchmark (LIBERO), establish clean-data baselines, then inject one defect class at a time into the fine-tuning data at controlled doses and measure task success per condition. The outputs we intend to publish: per-defect dose-response curves, a toxicity ranking of which classes are survivable at 10% contamination and which are fatal at 1%, a catalog of the silent failures where training metrics stayed healthy, and the injection toolkit with corrupted dataset variants, released openly so the results can be checked and extended.
The honest caveats, before the results exist. No published training-side dose-response study covers modern VLAs, which is the reason to run one, but it also means the regime map above leans on adjacent evidence: a CoRL 2021 study, user studies with 24 and 30 participants, supervised-learning results, and language-model poisoning. Theory on learning from corrupted demonstrations exists (arXiv 2201.12594) but gives practitioners no thresholds. Our own field observations are informal and unaudited. Treat the regimes as a hypothesis with strong priors, and the study as the test.
Budget by toxicity
If quality is a curve, the operational consequences are concrete. QA budgets should allocate by toxicity, not uniformly: screen the fatal classes (schema, statistics, synchronization, and calibration) with automated gates on 100% of episodes, and sample the gentle classes at rates the curve justifies. Data contracts should specify defect classes and acceptable doses, not adjectives. And provenance stops being compliance paperwork: it is the mechanism that makes a defect traceable, and a defect you can trace is a defect you can price.
The teams that ship will be the ones that know their prices. When the curves are measured, we will publish them here, along with everything needed to reproduce them.
Related: why data, not architecture, decides who ships, what robot data costs, and where budgets leak, and the 2026 papers engineering around missing data.
