Executive Summary
A demonstration recorded as a success can still carry the traces of an operator who wandered, and the robot learns those too. The finding comes from an arXiv paper posted on November 17, 2026, in which a team that has collected more than 200,000 hours of real-robot data turned the analysis on its own collection process. One paper holds a kinematic study of 25 dual-arm robots, teleoperation experiments run across different control schemes, and a real-robot evaluation of policies fine-tuned on the resulting data.
The number that stands out is 90%. In a dual-arm block transfer task, raising the share of demonstrations containing an operator detour from 0% to 80% stretched the mean wrist path of the demonstrations by 0.215 m, and the path of the policy trained on them by 0.194 m. About 90% of the added distance moved into the robot's own motion. Every one of those demonstrations completed the task, and the label attached to each of them reads success.
Sections 1 through 4 follow the experiments and the numbers the paper reports. Section 5, which reads those results as an AI-Ready Data problem, is this article's own interpretation.
Key Numbers
Source: Hidden Variables Behind Real-Robot Data Quality (arXiv:2611.22388, 2026-11-17)
The first three cards measure the same channel — collected data turning into robot behavior — from three different angles. The first asks how much of an operator's inefficiency transfers. The second asks how success rates split depending on whether well-calibrated demonstrations survive in the mix. The third asks what happens when the zero offset drifts at inference time. The fourth card belongs to the other side of the channel, the control scheme that produces the data.
about 90%
Detour distance kept by the policy
Demo path grew 0.215 m; the learned path grew 0.194 m
66.7% → 26.7%
With and without the aligned 25%
Test-tube handover and insertion, 30 trials per group
0 of 30 trials
Miscalibration at inference time
Same outcome even when the training data is fully clean
43–53%
Cut in task completion time
When body parts are controlled in parallel
Training Does Not Filter Out the Wasted Path
Operators collecting robot data by teleoperation do not move cleanly. The workspace is tight and the arm catches on something, the control device is stiff, the view on the screen arrives a beat late. So the motion stops mid-way, backs up to a point it has already passed, and resets its posture. The object still ends up where it belongs. The task is a success and the record says so.
The researchers tested directly whether this successful inefficiency survives training. In a dual-arm block transfer task they raised the share of demonstrations containing a detour segment through 0%, 20%, 50%, and 80%, training a policy on each mixture under identical conditions. Going from 0% to 80%, the mean wrist path of the demonstrations grew by 0.215 m. The path of the policy trained on them grew by 0.194 m. The robot took on about 90% of the extra distance the human had walked.
What those injected demonstrations look like is on the record too. Part of a 100-demonstration set was swapped for trajectories containing pauses and backtracks. Against the clean demonstrations, those trajectories run wrist paths that are 33% longer on average, carry about 1.8 times the joint jerk, and raise single-arm pauses longer than 0.3 s from 1.1 to 1.6 per episode. On video they look like slightly sluggish demonstrations and nothing more.
Not every defect transferred at the same rate, though. Only about 45% of the increase in joint jerk stayed in the policy. Wasted path carries over almost intact while roughness of motion gets filtered out by half. Which means a trained robot that moves smoothly is no evidence that the data behind it was clean.
Time went the same way. As the detour share rose from 0% to 80%, the span from the robot starting to move to the object being placed stretched from 25.3 s to 32.6 s, about 29% longer. More data produced a slower robot, and nothing in a quality check gives you grounds to screen that data out. All of it succeeded.
This is where the paper asks for a different acceptance criterion. Deciding whether a demonstration belongs in training should weigh path efficiency, motion continuity, and task duration together. There is a recommendation about ordering attached to it. Corrective motions that repeat because of joint limits, awkward control mappings, or control lag should be fixed on the collection system first, before anyone tries to filter them out of the training set. Filtering comes second.
A success label records the outcome only. The route the operator took to get there falls outside the label. Inefficiency created during collection is never screened out, and it becomes the model's default motion. Put this beside Robot Demos That Run Clean With the Wrong Instruction, which deals with demonstrations where the instruction and the trajectory disagree, and the grain differs. There the description is wrong; here the description and the outcome are both right and the process is messy.
The 25% That Keeps Miscalibrated Data Usable
Calibration is the work of aligning the reference by which a robot reads its own body as numbers. The paper lists joint zero positions, kinematic parameters, torque sensors, dynamics, and camera intrinsics and extrinsics. Among these the experiments focus on joint zero positions, because they bear most directly on the state representation. If one robot reads a given elbow angle as 0 degrees and another reads the same angle as 2 degrees, the demonstrations the two leave behind are records with identical characters and different meanings.
Any team that has collected for long enough has a pile of such records. The reference shifts a little every time new hardware arrives or the calibration procedure changes, and whether the old data has to be thrown out wholesale is a question with money attached to it on the floor. The researchers split the training mixture four ways for a test-tube handover and insertion task and ran 30 trials on each.
| Group | Training data | Success rate |
|---|---|---|
| A | All demonstrations well calibrated | 63.3% |
| A′ | 25 well-calibrated episodes + 75 episodes offset in the same direction | 66.7% |
| B | No well-calibrated episodes, offsets scattered in both directions | 20% |
| B′ | No well-calibrated episodes, offsets biased in one direction | 26.7% |
▲ Test-tube handover and insertion task, 30 trials per group. 'Well calibrated' means aligned with the zero of the robot used at inference. | Source: arXiv:2611.22388 §4.2
This table is easy to read in the wrong direction. The pair that matters is A′ and B′. Both hold offset demonstrations of the same kind, and the only difference between them is whether 25 well-calibrated episodes are in the mix. With that 25% present the policy reaches 66.7%, which puts it on essentially the same footing as the fully clean A at 63.3%. With that 25% gone it drops to 26.7%.
B is a separate value to read. Its offsets are not biased but scattered in both directions, so averaged across the dataset the errors cancel out to near zero. Success still lands at 20%, the lowest of the four groups. Errors averaging to nothing and demonstrations sharing one reference are two different things.
So what separated the performance was not the proportion of offset data. Even with 75% of it offset, the policy holds up as long as a few demonstrations still match the reference of the robot that will run it. The paper's recommendation has the same shape. A history of mixed calibration need not be discarded; instead, keep a subset of baseline trajectories aligned with the inference robot as an anchor.
A Different Zero on the Inference Robot Fails Every Trial
The same experiment includes the opposite condition. The training data is entirely well calibrated, but the joint zero of the robot the policy runs on differs from the one used during collection. All 30 trials failed. The success rate did not fall; nothing succeeded at all.
Set sections 2 and 3 side by side and the asymmetry shows. On the training side, offsets mixed in up to 75% are survivable as long as the anchor remains. On the inference side, one offset ends it. The chain of data quality does not stop at collection; it runs through to the calibration of each individual robot. Collecting data well and aligning the robot that will use it are the front and back of one job.
The paper nails one procedure down here. Checking that the zero of the collection robot and the zero of the inference robot agree should be a mandatory step before a model is deployed. The method is written out as well: record calibration versions, verify joint readings at a standard pose, and compensate each joint's zero offset individually. All three belong to facilities management more than to data science, and in this experiment they are what stood between 0 of 30 trials and 63.3%.
The Body and the Controller Decide Quality Before Collection
The first half of the paper goes back upstream, to before the data exists. Which robot you collect with, and through which control device, settles the quality of the demonstrations in advance.
4.1Reaching a Point Is Not the Same as Working There
The researchers took 25 dual-arm robots and, at 10,075 workspace positions each, counted 64 directions the tool axis could point. On position alone, a humanoid form and a wheeled lift-arm form reach 55% and 53%. Add the requirement that the orientation match as well and those drop to 41% and 33%. Tabletop-mounted forms sit at 91% and 75% by the same measure. The fingertip arriving at a location and the tool being holdable there at the angle the job needs are two different facts. A metric that counts only position inflates the range of work a robot can actually do.
The fix turned out to be small. Allowing the shoulder to move within a sphere of 15 cm radius expanded the reachable cross-sectional area to 3.4–5.5 times that of a fixed shoulder. It corresponds to the slight lean of the torso a person makes when picking something up. Without making the arm longer, a little freedom at the trunk changes the range of tasks you can collect.
4.2The Control Interface Reshapes the Demonstration
In the control experiments, letting the operator drive body parts such as the two arms and the waist in parallel cut completion time by 43–53%. Picking an object off the floor fell from 35.7 s to 16.7 s, and a wiping task from 27.0 s to 15.3 s. On contact-rich tasks, compliant control succeeded 100% of the time across all four tasks, while position control reached 100% only on block stacking and stopped at 83.3%, 80%, and 86.7% on door opening, wiping, and box moving. Peak contact forces were lower under compliant control as well.
The result on vision is more direct. In a task where a block sits outside the robot's initial field of view and the nearer hand has to be chosen, a policy trained on demonstrations that turn the head to look at the block before selecting an arm reached 100% arm-selection accuracy. A policy trained on demonstrations without that look-around stayed near 50%, which is chance. The operator had already seen the block with their own eyes, but the robot camera had not, and that is what the record preserved. When the scene that grounds a decision is absent from the demonstration, you get a policy that imitates the outcome alone.
Latency belongs in the same place. The researchers deliberately added delay to the control device and measured as they went: on a 2 cm target, raising the added latency from 0 to 200 ms doubled mean completion time from about 13 s to 26 s, and at 800 ms it climbed to about 62 s. Each additional 100 ms of gripper latency added roughly 200 ms to a single pick-and-place. In the other direction, once total latency drops below about 100 ms the operator can barely perceive the lag, so cutting it further gains little. What latency does not stop at making the operator slower. The corrective motions a slowed operator produces become the paths of section 1 and stay in the data.
All four results point the same way. Data quality is a question of which body and which control device you choose, and most of it is settled at that stage, well before anyone sorts through what was collected. If We Mixed in Flawless Data, and the Robot Got Worse showed how hard the sorting is, this paper looks at the step before it.
Why Pebblous Is Watching This Paper
The goal you hear most often from people collecting robot data is measured in hours. We have tens of thousands, and next year we fill hundreds of thousands. That is the metric this paper shakes. A team with 200,000 hours behind it took apart its own data and concluded that the design of the collection system sets what those hours are worth.
The awkward part is that every defect at issue here hides inside a success. The detour is a success, the demonstration with a 2-degree offset at the zero is a success, and the arm chosen while the robot camera saw nothing is a success. Surface metadata — the pass/fail label, the task name, the collection timestamp — tells none of the three apart. If AI-Ready Data means data a model can use as it stands, then for robot data that state takes more than a correct label: the trajectory itself has to be clean.
A team preparing for Physical AI can put three questions to the data it is accumulating.
- Are you looking at the distribution of path length and duration across your successful demonstrations? How widely the demonstrations of one task scatter is where inefficiency at the teleoperation stage becomes visible. Tally only pass and fail and it stays hidden.
- Does the data still hold demonstrations calibrated to the robot you are running now? Before discarding an old batch, the thing to check is not the proportion that drifted but whether the anchor is there.
- Is the scene that grounded each action choice present in the record? When what the operator saw and judged leaves no trace in the robot's sensors, the policy mimics outcomes with nothing underneath.
The paper does not leave the questions open either. Before scaling collection, it names five things to measure: effective workspace, control efficiency, observation completeness, calibration alignment, and motion quality. The first two get measured while choosing hardware and control devices; the last three can be measured on data you already hold. What the five share is that all of them lie outside the pass/fail label.
One distinction deserves to be stated plainly. The extra motion inside a demonstration divides into the kind worth keeping and the kind worth removing. Motions that arise from the task itself — observing a target, avoiding an obstacle — should be kept along with the decision basis behind them, the paper writes. What should be removed is the corrective motion that repeats because of joint limits, awkward mappings, or control lag. The active search of section 4 and the detour of section 1 both lengthen the path, and they are not the same thing. One is the robot acquiring information it will use to decide; the other is the trace of a person covering for a flaw in the equipment.
All three questions ask what the data you hold is teaching, rather than whether to collect more of it. The means of measuring are already available. Path length, pause counts, and joint zero values are all numbers already in the logs. What is still rare is a team with a procedure for looking at those numbers before scaling collection up.
Editor's Note
The figures in this article were checked against the body of the paper as posted on arXiv. It went up days ago, has not been peer reviewed, and no other team has yet reproduced the experiments. The 90% transfer rate comes from a single dual-arm block transfer task, and the calibration experiment carries the authors' own qualifier that it holds in the configurations they tested. The limitations the paper states for itself are not narrow. The data-quality experiments hang on particular tasks, a single policy model, and one action representation, and some of them rest on artificially injected errors and a single training run. The kinematic figures in section 4 were not measured on physical hardware either; they rely on URDF models and discretized inverse kinematics. Factor in collisions, dynamics, and hardware limits and the real workspace narrows further. Change the task or the robot and the numbers will change. The direction is clear all the same.