Physical Intelligence has published 13 deeply technical blog posts since October 2024. Flow matching, action tokenization, knowledge insulation, embodied memory, reinforcement learning methodologies. The technical depth is impressive.
Safety gets one passing mention across all 13 posts.
This isn't a criticism of PI specifically. It's the whole field. Google DeepMind published a Robot Constitution focused on runtime constraints. Agility Robotics announced passing an OSHA field inspection. Figure AI discusses safety in terms of collision avoidance and force limits. Everyone treats safety as something you add at deployment time: guardrails, emergency stops, geofenced operating zones.
But here's what I keep thinking about: if a robot does something dangerous, can you trace that behavior back to the training data that caused it?
If you can't answer that question, you don't have safety. You have compliance theater.
The deployment-time illusion
The standard playbook is train the model, deploy the robot, then add safety layers: force limits, collision detection, geofencing, human supervisors, emergency stops. These measures are necessary. They're also insufficient, because they can only react to dangerous behavior. They can't prevent the model from learning dangerous patterns in the first place. If your training data contains systematic biases, edge case gaps, or corrupted labels, those problems are baked into the model weights, and no runtime guardrail fully compensates for a model that learned the wrong things.
Consider a manipulation policy trained on demonstrations from a controlled lab: consistent lighting, standardized objects, no obstacles. The model learns that these conditions are normal. Deploy it in a real warehouse, with variable lighting, damaged packaging, and human coworkers moving through the space, and it meets situations its training data never prepared it for.
The dangerous failures aren't the obvious malfunctions that trigger an emergency stop. They're the ones where the robot confidently executes a learned behavior that's wrong for the situation: reaching through a space where a human hand might be, applying force calibrated for rigid objects to something fragile, moving at lab speeds through a crowded space. Runtime systems catch some of these. But they're fighting a rearguard action against problems that originated in the training data.
Without lineage, debugging is guesswork
When a deployed robot misbehaves, a good team can usually find the proximate cause: a perception error, a planning failure, an unexpected edge case. What's almost always missing is the ability to trace backward from the failure to the training data that caused it. Which demonstrations taught the model this behavior? How were similar situations labeled? Was this edge case represented in training at all?
Without that lineage, you can observe that the model does something wrong, but you can't systematically understand why. You can add training data to address the failure, but you can't verify the new data corrects the underlying problem rather than introducing new ones.
The autonomous driving industry learned this through painful experience. Early AV systems trained on data from multiple vendors, collected under different conditions, labeled with different conventions, stored in different formats. When something went wrong, there was no reliable way to trace the failure back through the pipeline. The companies operating driverless fleets today all invested heavily in traceability: every prediction traceable to the training examples that influenced it, every label with provenance, every preprocessing step logged and versioned.
Physical AI is relearning this lesson. Most teams still run fragmented pipelines, with demonstration data in one system, annotations in another, and training runs in a third, and no unified lineage connecting them.
What regulators actually require
The EU AI Act takes effect in August 2027 for high-risk AI systems, and autonomous robots operating in workplaces, public spaces, or homes almost certainly qualify. Four requirements land directly on data infrastructure:
Risk management throughout the lifecycle. Article 9 requires continuous, iterative risk management across the AI system's lifecycle, not just at deployment. Training data risks must be identified and mitigated.
Data governance. Article 10 requires that training, validation, and testing datasets be "relevant, sufficiently representative, and to the best extent possible, free of errors and complete." It also requires documented data characteristics, potential biases, and gaps. You can't demonstrate any of that without knowing what's in your training data.
Record-keeping and traceability. Article 12 requires automatic logging of risk-relevant events and the ability to trace back through the AI system's operation. For Physical AI, that means connecting robot behavior to the training data that produced it.
Technical documentation. Article 11 requires detailed documentation of training processes, data used, evaluation metrics, and design choices, maintained and updated throughout the system's lifecycle.
The common thread is traceability. Regulators aren't asking whether your robot is safe. They're asking whether you can demonstrate how you know it's safe and trace any behavior back to its causes.
Most Physical AI teams cannot currently meet these requirements. Not because they're negligent, but because their infrastructure was never designed for it. Demonstration data was collected to train models, not to satisfy audits. Annotations were created to improve performance, not to document provenance. The pipeline optimized for ML metrics, not compliance.
Coverage is a safety property
Safety isn't just labeling quality. It's coverage: does your training data represent the situations your robot will encounter, including the dangerous edge cases? Edge cases are, by definition, rare, so a dataset collected in normal operating conditions underrepresents exactly the situations where safety matters most.
Autonomous driving built systematic answers. Shadow mode deployment found situations where the system would have erred. Simulation synthesized rare conditions. Active learning directed collection toward regions where the model was uncertain. Physical AI needs the same machinery, adapted for manipulation. The edge cases for a robot folding laundry: fabric that's wet, tangled, or damaged; unexpected items mixed in with the clothes; human hands reaching into the workspace; pets or children entering the operating zone; sensors degrading over time.
Each of these needs enough representation to learn robust behavior, labeled in ways that capture safety-relevant distinctions. A slightly damp t-shirt is not a soaking one. A hand entering the workspace slowly is not one moving quickly. Without systematic tracking of which edge cases are represented and how they're labeled, you can't assess gaps, can't prioritize collection, and can't demonstrate to regulators that you've addressed foreseeable risks.
The correction loop most teams don't have
The most valuable safety signal in Physical AI is the near-miss: the dangerous behavior a runtime system caught, or nearly had to. Physical Intelligence's RECAP methodology shows one way to use it: corrections from human coaches during deployment become training data, labeled with what went wrong and how to fix it.
This is powerful, but it demands infrastructure: capturing corrections in a training-compatible format, integrating them without corrupting version history, and tracking which corrections landed in which model versions. Most teams handle it ad hoc. A coach intervenes, the intervention is logged somewhere, and weeks or months later someone manually folds it into training data. The connection between failure and fix is lost, and nothing verifies the correction worked.
A safety-focused pipeline treats the correction loop as a first-class concern: near-misses flow into a triage queue, labels capture whether the failure was perception, planning, or control, and corrections propagate to training with full provenance. That doesn't just improve safety. It demonstrates safety. When a regulator asks how you handle failures, you show the process.
Demonstrator inconsistency is an invisible risk
One safety issue rarely gets discussed at all: the humans providing the demonstrations. Vision-Language-Action (VLA) models learn from human demonstrations, and humans are inconsistent. Different demonstrators have different styles, risk tolerances, and skill levels, and the same demonstrator varies with fatigue and distraction. A demonstrator who's rushing takes shortcuts that are usually fine and occasionally dangerous. Those demonstrations get mixed in with good ones, and the model learns from all of them.
Autonomous driving handled this with demonstrator quality control: monitoring behavior, flagging anomalies, down-weighting problematic segments. Doing the same for manipulation means tracking which demonstrator created which trajectories, scoring demonstrations on quality metrics (task completion time, motion smoothness, force profile appropriateness), and filtering or weighting on those scores, all while maintaining traceability so the decisions can be audited later. Without that, demonstrator inconsistency is a risk you can't assess, can't address, and can't show regulators you've controlled for.
What safety-first data infrastructure looks like
The through-line in all of these challenges is infrastructure. Not runtime guardrails, not deployment checklists, but the data infrastructure that makes training-time safety possible. Five characteristics:
End-to-end traceability. Every prediction traces to the training examples that influenced it. Every example traces to its raw sensor data, annotations, preprocessing steps, and demonstrator. Every annotation traces to its annotator and the labeling guidelines in effect at the time.
Coverage tracking. Edge cases are explicitly catalogued, gaps are identified and prioritized for collection, and coverage metrics are tracked over time.
Quality control. Demonstrations are scored, annotations are validated, and outliers are flagged. The system distinguishes between "in the training set" and "verified safe to train on."
Correction integration. Near-misses and failures flow automatically into the pipeline, labeled with safety-relevant context and incorporated with full provenance.
Audit support. Documentation is generated automatically, data characteristics and gaps are recorded, and version history is preserved. Regulators get systematic answers, not reconstructed narratives.
None of this produces impressive demos or benchmark-topping results. It's the difference between a robot that seems safe in controlled conditions and one that can be demonstrated safe under regulatory scrutiny.
The window is closing
Most Physical AI companies are not ready for the EU AI Act's August 2027 deadline, because their data infrastructure was built for ML performance, not compliance. And retrofitting traceability into an existing pipeline is much harder than building it in from the start. Every team collecting demonstrations with ad hoc tooling, annotating with inconsistent schemas, and storing training sets without version control is accumulating debt that comes due when regulators ask hard questions.
The robots you're training today will still be operating in 2027, and the data decisions you make now determine whether you can demonstrate they're safe. Data infrastructure where traceability is foundational rather than an afterthought isn't optional. Not because regulators require it, though they will. Because you can't make a robot safe if you can't trace its behavior back to the training data that caused it.
Safety isn't a deployment checklist. It's a property of the entire pipeline, from sensor to model weights. The companies that understand this will build robots that are safe. The ones that don't will build robots that seem safe until they aren't.
