Executive Summary

Robots Need More than VLA and World Models (arXiv:2606.06556), posted on 4 June 2026, is not an experimental paper. Its abstract opens with "In this position paper, we argue…" — it is an argument, not a result. Nine authors signed it, led by Elis Karcini, with affiliations at Stanford, ETH Zürich, TU Darmstadt, IIT, and UCL. The argument fits in one sentence: the central bottleneck is not policy learning alone but the absence of machinery that converts the world's abundant unstructured behavioral data into grounded robot supervision.

It matters where that sentence is aimed. Debate about robot foundation models has mostly been a debate about model shape — whether to push vision, language, and action through one network, or to predict the next frame of the world and plan against it. The paper does not take a side. It points one floor below the argument. Whichever model you pick, the supervision you feed it has to be manufactured somewhere, and the machinery that manufactures it does not exist.

1.1Four Translators

The paper splits that machinery into four interfaces: a data interface that labels unstructured behavior automatically, an embodiment interface that carries human motion into robot action, a world-model interface that handles physics-grounded 3D reasoning, and a reward interface that infers task progress and success from video and language. One word covers all four: translators. On one side sits video of people accomplishing things; on the other sits supervision a robot can consume. Getting from one to the other is the whole job.

1.2Did the Field Actually Reach the Same Diagnosis?

Answering that precisely requires splitting the diagnosis in two. The broad claim — that the bottleneck in robot learning sits in the data layer rather than in model architecture — has plenty of independent support. A survey of VLA data and benchmarks led by the University of Maryland (arXiv:2604.23001) reached almost the same sentence six weeks earlier, in April, with no shared authors and no citation between them: the next advance in VLAs depends less on architecture than on co-designing high-fidelity data engines with structured evaluation protocols. A peer-reviewed JAIR survey (arXiv:2404.19664) had formalized the same problem two years before that as a chicken-and-egg loop: robots are not capable enough to collect the data, and without the data they do not get more capable.

The narrow claim — that exactly these four interfaces form a single stack — is largely this paper's own. No prior literature adopts the same four-way split, and no industry player uses it as a roadmap. In August 2025, Science Robotics ran an Oxford-style debate on the motion "data will solve robotics." That the room divided between a data-first camp and a models-and-first-principles camp is itself evidence that this was a live dispute until very recently. So this report avoids the phrase "industry consensus." A widely shared diagnosis, with one paper's distinctive prescription on top of it, is the accurate description.

1.3The Side With the Surplus Has No Labels

Put the two data axes side by side and the need for a translator becomes obvious. HumanScale (arXiv:2606.20521), a June 2026 study of large-scale egocentric video pretraining, tabulates the major public datasets by hours and summarizes both axes directly in a figure caption. Egocentric human video is accessible at roughly a million hours, cheap to collect, and open-ended in its diversity — but all it supplies is pseudo action labels obtained through hand retargeting. Teleoperated robot data carries precise action labels, but the public pool stops at roughly ten thousand hours, collection is expensive, and scene diversity is narrow. The roughly hundredfold gap is the authors' own figure, not our arithmetic.

The two axes are exactly complementary. One has scale and no labels; the other has labels and no scale. With a translator in between, the million hours enter training. Without one, they sit where they are.

The two data axes are complementary: scale on the left, labels on the right. The hour counts are the authors' own, stated in the Figure 1 caption of HumanScale (arXiv:2606.20521).

This report belongs with two recent Pebblous reports on robot data. What Do You Feed a Robot? asked which sources to feed a policy; We Mixed in Flawless Data, and the Robot Got Worse asked how to curate what you collected. This one takes the question that sits between them: how do you translate data a robot cannot read in the first place? The four sections that follow open each interface in turn and check, against evidence from outside the paper, how much of it is automated and where human hands come back.

A cooking video on YouTube does not record the angle at which the hand gripped the pan or how much force it applied. The data interface exists to have a machine estimate what was never written down. The template came from games, not robotics. In 2022, OpenAI's VPT paid humans to annotate controller inputs on 1,962 hours of Minecraft footage, trained an inverse dynamics model on those labels, and used it to fill in pseudo labels across 70,000 unlabeled hours. One hour of human labeling bought thirty-six hours of labeled data.

Moving to robots changed the approach once. Instead of imitating true action labels, these methods invent discrete latent actions from the change between one frame and the next. A policy is pretrained on those latent actions, and only at the end does a small amount of real robot data train a decoder that maps them onto actual joint commands. LAPA (arXiv:2410.11758) established the structure in 2024. UniVLA (arXiv:2505.06111) attacked its weak point in 2025 by extracting language-conditioned, task-centric latent actions in feature space rather than pixel space, so that background motion contaminates them less. NVIDIA put a latent action codebook into GR00T N1, which is the idea reaching an industrial-scale foundation model. By 2026 the research focus had moved again: Motion-Focused Latent Action (arXiv:2606.18955) separates motion from background using physical masks. The question has shifted from whether you can learn without labels to how you strip the noise out of the labels you invented.

Original Pebblous diagram (Fig. 2, reinterpreted) — reconstructing the latent-action pipeline shared across the LAPA/UniVLA family. The demonstration count needed at the decoding stage (~50) reflects the most recent 2026 method, discussed in §2.3.

2.1Translation Pays Only When Supervision Is Scarce

The most honest accounting of what this translation buys you is a table in the May 2026 JOPAT paper (arXiv:2605.23856). The design is simple. On LIBERO-Long tasks, vary the number of action-labeled demonstrations available for downstream fine-tuning — ten, twenty-five, fifty — and measure success. Pretraining masks the action branch entirely, so not a single robot action label enters it. The column worth studying in the table below is not the leftmost one; it is how the gaps change as you move left to right.

Source: JOPAT (arXiv:2605.23856) Table 6, original caption "Action-free video pretraining on LIBERO-Long. SR are reported." Columns are the number of action-labeled demonstrations used in downstream fine-tuning; values are LIBERO-Long success rates.

At ten demonstrations, pretraining on unlabeled video lifts success from 11.9% to 64.2% — a gain of 52.3 percentage points. At fifty demonstrations, the difference between in-domain robot video (96.4%) and entirely unrelated web video (95.1%) has effectively vanished; what you pretrained on stops mattering. Read across the columns and a second thing appears. A policy that was pretrained and fine-tuned on ten demonstrations (64.2%) lands roughly where a policy with no pretraining and fifty demonstrations lands (66.1%). That is about a fivefold saving in demonstration budget — a multiple we computed by comparing columns in the same table, not one the authors state.

2.2Automatic Labels Still Lose to Real Ones

So are automatically inferred labels as good as human ones? Compare across different pretraining corpora and the automatic side sometimes wins: UniVLA, pretrained on human video alone, averaged 88.7% on LIBERO, ahead of OpenVLA's 76.5% from robot teleoperation data. But that comparison changes two things at once. Hold the domain fixed and vary only the kind of label, and the result flips. The LAPA paper ran exactly that head-to-head on the Language Table benchmark.

Source: LAPA (arXiv:2410.11758) Table 1, Language Table benchmark success rates (%).

Automatic labels beat doing nothing by a wide margin — 15.6 to 62.0, close to fourfold. They still lose to real action labels: by 15.0 points in the same environment, and by 31.2 points once the environment changes. The doubling of the gap under environment shift is the part that matters. It suggests that what latent actions capture is closer to what moved and how in this particular scene than to the meaning of an action that would carry into another one.

Performance is not the only axis, though. LAPA's pretraining consumed 272 H100-hours; OpenVLA, its comparison, consumed 21,500 A100-hours. While the performance gap sits in the low tens of points, the cost gap runs from twenty to eighty times. The right frame for judging this interface follows: automatic labeling competes on performance per dollar, not on performance. It is not a technology for beating real labels. It is a technology for deciding how far you can get on a budget that cannot buy them.

2.3Where the Human Hands Come Back

Anyone entering this field expecting full automation meets human hands again at four points. First, the final decoding step that maps latent actions onto real joint commands always requires real robot trajectories; even 2026's newest methods need roughly fifty demonstrations. Second, latent actions collapse when the background contains distractors. An ICML 2025 study (arXiv:2502.00379) demonstrated that collapse and then reported that mixing ground-truth action labels into just 2.5% of the data improves downstream performance by 4.2× on average. The conclusion is that the usual pipeline order — learn latent actions first, add real labels later — is itself wrong.

Third, there is now a theoretical account of why the collapse happens. A 2026 study from Seoul National University and Microsoft Research (arXiv:2605.20223) showed that an objective trained to reconstruct the next frame pushes exogenous, action-irrelevant information into the latent action itself. Fourth, cutting unstructured video into the units training requires is still manual work. A survey of human-video-to-robot-manipulation methods (arXiv:2606.00054) puts it plainly: web-scale human video is "rarely packaged into episodes at the supervision granularity robot learning requires." Deciding where a task begins and ends, and what to call the segment, is still someone's job.

Industry produced the most dramatic version of this interface. In September 2025, Figure AI announced Project Go-Big, reporting that a humanoid had learned end-to-end — images and language in, low-level velocity commands out — from egocentric video passively collected while people went about their lives. The announcement's phrasing is that no robot-specific data or training was required. What the announcement does not contain is a single success rate, training time, or benchmark score. The evidence released was video. That asymmetry comes back later in this report.

Manufacturing action labels is not the end of it. Something has to tell the policy whether the behavior accomplished the task, or learning has nothing to move toward. Having a person watch and grade every rollout does not scale, so the reward interface hands that verdict to a vision-language model reading the video alongside the instruction. Before reading any number in this section, though, the ceiling has to go up first, because the baseline is not 100%.

SuccessVQA (arXiv:2303.07280), the 2023 Google DeepMind work that recast success detection as a visual question-answering problem, reported that agreement among human raters watching the same footage sits at roughly 88%. On the question of what counts as success, people disagree with each other about one time in eight. Every automatic number that follows should be read against that ceiling, not against a perfect score.

3.1Serviceable In-Distribution, Broken on New Tasks

Fine-tuned detectors in that same study land in the mid-80s inside the training distribution: 83.4% on unseen episodes, 85.0% on unseen behaviors. Move to an unseen task and the number falls to 59.3%. Another dataset shows the same shape, with 99% accuracy at training time becoming 62% at test time. A detector is good at re-recognizing the shape of success it was taught, and close to a coin flip in front of a shape of success it has never seen.

Precision matters more than accuracy here. When GVL (arXiv:2411.04549), a DeepMind and University of Pennsylvania study of progress estimation, reproduced the standard success-detection setup, accuracy came out at 0.62 while precision came out at 0.33 — two out of every three declared successes were false. The gap between those two numbers describes what this interface actually is. Overall it is right more often than not; restricted to the moments when the policy is actually rewarded, which is to say the moments it calls success, it is mostly wrong.

Original Pebblous diagram (Fig. 3, reinterpreted) — visualizing the accuracy 0.62 / precision 0.33 result from the GVL reproduction (arXiv:2411.04549) as a case ratio. The actual distribution is not exactly 1:2; this is an approximation meant to make the precision figure intuitive.

Several 2026 studies benchmarked the interface head-on. RoboReward, from Stanford and Berkeley, set 22 frontier VLMs up as robot reward models, graded them on human-verified robot episodes, and concluded that even the newest models are not yet reliable reward models; the top model's mean absolute error against five-point progress labels is 0.665. Robometer, presented at RSS 2026 by USC and collaborators, changed the axis — asking which of two trajectories is better instead of scoring one in isolation — and reported a failure-detection F1 of 0.81. Another line of work reports 87.4% accuracy on trajectory preference judgments. These numbers, however, count different things. Absolute error, failure-detection F1, and preference accuracy cannot be lined up and read as a curve of improvement. A survey of progress reward modeling (arXiv:2607.21655) makes exactly that fragmentation its complaint.

3.2The Moment the Judge Becomes the Objective

Worse than a judge that makes mistakes is the structure in which a policy trains against that judge. Christiano and colleagues' 2017 work on learning from human preferences contains a scene still cited today. A robot arm trained to grasp an object learned instead to position itself between the camera and the object so that it looked, to the human rater, like a successful grasp. The rater pressed approve, and the policy reinforced the behavior. It is the canonical case of reward hacking.

If human raters can be fooled that way, a VLM judge is easier to fool. There is theory behind that intuition too. A Berkeley study of reward misspecification at ICLR 2022 (arXiv:2201.03544) showed that as an agent grows more capable, proxy reward keeps rising while true reward falls, and that past a certain threshold the switch is abrupt. A judge is by definition a misspecified proxy, and a policy optimizer is trained to find the directions in which that proxy is weakest. This is where Pebblous's earlier work on proxy metrics and reward hacking and on the provenance of reward data in RL reappears in robotics.

3.3Reward Models Are Already in the Open Stack

The interface the paper described as missing had, in fact, entered the open robot data stack around the time the paper was posted. TOPReward (arXiv:2602.19313), from the University of Washington and NVIDIA-affiliated researchers, feeds a trajectory video and an instruction into a prompt that asks the model to judge whether the statement is true, then reads the probability the model assigns to the "True" token directly as reward. There is no fine-tuning. That method now sits in Hugging Face's official LeRobot documentation as a supported option you switch on with one line of config.

The documentation is honest about its own limits: the port is inference-only, cannot be trained, and "inherits the biases of the underlying VLM." Which is to say the open stack adopted the reward interface without also adopting a field for how often that interface is wrong.

The third interface does something different from the first two. Where they attach labels to data, this one supplies physical knowledge the policy can query. How heavy is this cup? If I set that object down here, does it topple? Is my finger touching the object right now? None of this needs teaching for a person. For today's models it does.

The largest measurement of that gap is PhysBench (arXiv:2501.16411), published at ICLR 2025, which built ten thousand multiple-choice questions about physical-world understanding and gave them to humans and models alike. On PhysBench, humans scored 95.87% and the best model evaluated scored 49.49% on a test where random guessing yields 25.00%. The 46.4-point gap means the best model barely clears half a human score. Because that figure rests on a single benchmark, it is more accurate to name the test whenever the number is quoted.

Original Pebblous diagram (Fig. 4, reinterpreted) — reconstructing PhysBench (arXiv:2501.16411) figures as proportional bars.

Tests built in entirely different ways reach the same conclusion. Stack blocks and ask only whether the tower will fall — the ShapeStacks-style stability judgments in arXiv:2508.10770 — and the best open-source model scores 0.542 on a task where a coin flip scores 0.5. The best commercial model reached 0.641; humans reached 0.779. Generation looks similar. On VideoPhy-2, where human raters score whether generated scenes satisfy both physical law and the prompt, the top score on the hard subset sits around 22%. On PhyWorldBench, which deliberately asks for anti-physics scenarios and checks whether the model produces them correctly, that rate falls to 0.011.

Four tests, four different teams, four different methods. Multiple choice, binary judgment, and human scoring of generated output do not add together, and they cannot be strung into a curve of progress. Read each row's best score alongside that test's own baseline, though, and the four lines point the same way.

Tests built by different teams in different ways converge on the same band. Physically sound discrimination and generation stall in the 20–40% range, and collapse into single digits as difficulty rises.

One more observation belongs here. No study yet measures contact, mass, friction, and support relations — the very targets the paper assigns to this interface — inside a single unified 3D benchmark. Each team measures its own fragment from its own angle. That fragmentation is itself a statement about the interface's maturity, because it means the field has not agreed on what to measure.

Yet this judging capability is already at work outside the lab. NVIDIA's synthetic data recipe generates multiple rollouts per prompt, has Cosmos Reason score physical plausibility on a five-point scale, and keeps only the 4s and 5s for the training set — where 1 means the clip obeys no physical law and 5 means it obeys them fully. A physics judge is running as a pipeline filter, not a research benchmark. What the document does not report is what share was rejected, or how downstream performance differed with the filter on and off. The process is public; the numbers that would say whether the process is right are not.

The problem of verifying generated video against physics has its own Pebblous report, Is Generated Video Physically Correct? What differs here is position, not method. This section is not concerned with verification after the fact but with how far a policy can trust the physical knowledge it queries during training.

This is the most intuitive of the four. A person reaches for something; map that onto the robot's joint angles and you are done. Except that when the bodies differ, some motion does not survive the trip. Two studies measure what is left behind.

H2O (arXiv:2403.04436), from Carnegie Mellon, documented each stage of moving 13,000 sequences from a large human motion database onto one particular humanoid. Roughly 10,000 survived retargeting; roughly 8,500 remained after filtering for what the simulation could actually reproduce. About 35% drops out in transit. On top of that, even a privileged upper-bound policy inside the simulator — one with access to all state information — could not reproduce 14.5% of the human motions. The irreducible residue created by differences in body exists already at the data stage.

Original Pebblous diagram (Fig. 5, reinterpreted) — reconstructing the stage-by-stage retargeting attrition documented by H2O (arXiv:2403.04436) as a funnel. About 35% is filtered out in transit because of body differences. Even within these 8,500, a privileged upper-bound policy inside the simulator failed to reproduce 14.5%.

The more operational observation comes from a 2025 Stanford study (arXiv:2510.02252). Holding the motion data fixed and varying only the retargeting method used to move it, the resulting policies diverged sharply: across 21 motions, the open-source method produced a 0% success rate on three of them, while the commercial method produced 100% on all 21. Same data, same robot — and the quality of the translation alone splits policy success between zero and one hundred.

Open the four interfaces one at a time and their technical difficulties look nothing alike. Look at the shape of their failures, though, and they converge on one question: what makes an automatically generated supervision signal trustworthy? Label, reward, or physics verdict, the structure is the same — a signal no human checked enters training. And on what happens when that signal is wrong, separate studies keep finding the same thing: the damage depends heavily on which direction the error runs.

6.1The Direction of the Error Matters

The most direct experiment appears in a 2026 systematic dissection of hierarchical VLA agents (arXiv:2606.10267). In an architecture where a success detector serves as the termination condition for each command, the authors injected false-positive and false-negative rates independently and measured average success. A false positive calls a failure a success; a false negative misses a success. At identical error rates, the results split like this.

Source: arXiv:2606.10267 Appendix B. Average success rate across the MuJoCo Aloha suite.

Two things stand out. First, performance does not degrade at all up to a 10% error rate — it even ticks up slightly, which is evidence that an imperfect detector is still worth having. Second, at 50% the direction decides the outcome: false positives cost 14.9 points while false negatives cost 3.1. Falsely declaring success is nearly five times more expensive than missing one. The authors explain why: a false completion signal makes the upper module treat unfinished work as finished and move on to the next command. Their own caveat is worth carrying over as well — the injected errors were independent across states, which may understate the damage from false negatives. Real detectors make correlated errors on consecutive frames, so one miss can leave a command running for a long time.

The same asymmetry turns up independently in two other places. A University of Melbourne study of reward models confirmed in oracle experiments that false positives damage training scores more than false negatives, and reported environments where an agent trained on VLM-supplied reward did worse than one given no reward at all. The precision figure of 0.33 from the previous section says the same thing in different words: if two of every three declared successes are false, most of what the policy is reinforcing is not success.

One could object that bad signals can simply be filtered out afterward. The limits of that filtering have been measured too. A June 2026 study of automated curation (arXiv:2606.05588) planted two kinds of defect in demonstration data and tested whether quality metrics caught them. Fine-grained defects — correlated action noise, jitter, truncated trajectories — were caught well enough that multivariate outlier detection fully recovered the performance gap. Structural defects, where a wrong action enters at the decisive moment, went undetected by metrics that look only at action values, and two metrics actually rated the defective demonstrations as high quality. Even the best metric, one that also reads the state trajectory, recovered just a third of the gap. The layer that manufactures supervision automatically and the layer that inspects it automatically share the same blind spot: both can see how much a value jumps, and neither knows whether that moment was the moment that mattered. This one is an unreviewed preprint, so it is better read as a pointer to a direction than as a settled magnitude.

6.2Three of the Four Had Already Shipped

The paper diagnosed all four translators as missing. Around the time it was posted, three of them were already inside products and infrastructure. Put the findings from the preceding sections in one place and the common feature is visible: they shipped, and none of them says how often it is wrong.

All three disclosed the process and withheld the accuracy of that process. Academia measures the same objects in numbers: a 46.4-point gap in physical understanding, a judge precision of 0.33, −14.9 points at a 50% false-positive rate.

This is where the report arrives. It does not mean the paper's diagnosis was wrong. The accurate statement is that within two months of the paper declaring the machinery absent, the machinery appeared — and exposed the next gap, the one the paper did not address. The race to collect more data has already become a race to translate it. The seat after that, the one where somebody measures whether the translation was right, is still empty.

It is also worth noting where the paper's symmetry diverges from the actual thickness of the evidence. The space this report gave each interface was not an editorial preference; it followed the independent evidence we could obtain.

Section lengths in this report follow this table rather than the paper's four-way symmetry. Where the evidence was thick, we could publish tables; where it was thin, we stopped at saying so. The empty cells near the bottom are where the next research belongs.

6.3Where Korea Stands

Korea entered this area at scale in 2026. The Ministry of Science and ICT is establishing data training centers to build Physical AI training data: central hubs in Seoul and elsewhere producing primitive-motion data — reaching, grasping, twisting, pulling — along with synthetic data, and regional hubs in North Jeolla, South Gyeongsang, and other provinces gathering field data from manufacturing, agriculture, logistics, and services. The inter-ministerial K-Physical AI full-stack strategy describes the data plan as five specialized regional hubs that mass-produce real and synthetic behavioral data using teleoperation in physical space and digital-twin simulators in virtual space. Press reporting puts KRW 1.4131 trillion (about $975M) behind Physical AI industrial-transformation R&D in South Gyeongsang and North Jeolla from 2026 through 2030, and KRW 402.2 billion (about $277M) behind the Ministry of Trade, Industry and Energy's Physical AI development program.

Tag those plans against the four interfaces and one thing stands out. Primitive-motion collection, synthetic augmentation across lighting and viewpoint, teleoperation and digital-twin hubs — all of it is collection and generation. The layer that manufactures labels automatically, the layer that judges success and reward, and the layer that verifies physical plausibility do not appear in the published plans. That is not a criticism. Without collection there is nothing to translate, so it has to come first. But once the collection hubs are standing, the next question arrives immediately: who manufactures the labels for all that data, and who certifies that the labels are right?

The previous section reduces to one sentence: the translators exist, the gauges do not. That is also where we kept stopping while reading the paper. Of the four interfaces it names, the data interface and the reward interface are robotic in name only. What they actually do is manufacture labels automatically and verify whether those labels are right — which is precisely the work DataClinic and AI-Ready Data have been doing. The embodiment and world-model interfaces sit next to PebbloSim and our Physical AI line. Read from where we sit, the paper is close to a restatement of Pebblous's business definition in robotics vocabulary.

Automatically Generated Labels Have No Inspection Process

From a data-quality standpoint the four interfaces share exactly one property: all of them manufacture supervision automatically. Automatically manufactured labels and rewards can be wrong, and a wrong supervision signal quietly distorts a policy's internal representations. The numbers in this report describe the size of that distortion. A 50% false-positive rate costs 14.9 points of success. Automatically inferred action labels fall 31.2 points behind real ones once the environment changes. A detector's precision drops to 0.33. Human-annotated labels come with an inspection process; automatically generated labels have no such process established yet. Having a translator and knowing the translation is correct are different problems, and what this report shows is that the second one is vacant.

Three Questions for Practitioners

For an organization working with robot or manufacturing data, this report reduces to three decisions. First, where do you start? If you hold a few dozen demonstrations or fewer, effort on a translation interface pays back overwhelmingly; if you already hold several hundred, collection and cleaning are the better spend. Second, what makes an automatic label trustworthy? Do not collapse false positives and false negatives into a single accuracy number. If a false success is nearly five times as expensive as a false failure, the metric that belongs in a contract for a detector is precision, not accuracy. Third, have you asked the vendor for the tool's error rate? The translation and judging tools reaching the market do not publish theirs. This report is the ground on which to demand it.

The fiercer the competition among robot foundation models becomes, the more valuable the layer that manufactures their supervision and independently certifies its quality. Pebblous does not compete with the robot models, and it no longer competes with the translators either. In a market where translators already ship, we take the position of the instrument that measures translation quality. The clearest thing this report established is that the position is still open.

The robot data series has run from what to feed a policy, to how to curate what you feed it, to how to translate it. The next installment takes up how to measure the quality of translated supervision, and how to design that instrument.

Pebblous Data Communication Team
August 11, 2026

Primary Paper · Secondary Commentary

• 1.Karcini, E., Mehrban, F., Nguyen, Q., Schwager, M., Ajoudani, A., Cadena, C., Peters, J., Hutter, M., & Bou-Ammar, H. (2026). "Robots Need More than VLA and World Models." arXiv:2606.06556 [cs.RO], 2026-06-04. Position paper — the primary source this report tests. Origin of the four interfaces (data / embodiment / world-model / reward).
• 2.Engineeringuniverse | Robot Engineer. (2026). "Behavioral data is everywhere — so why can't robots learn? | Robots Need More Than VLAs & World Models" [Video]. YouTube. Third-party Korean-language commentary on the paper (secondary source) — the video that started this report. Every factual claim in the body was re-verified against primary sources.

Academic — Independence of the Diagnosis, and the Dispute

• 3.Wang, Z. et al. (2026). "Vision-Language-Action in Robotics: A Survey of Datasets, Benchmarks, and Data Engines." arXiv:2604.23001. Independent evidence: the same diagnosis six weeks earlier, with no citation link.
• 4.McCarthy, R. et al. (2025). "Towards Generalist Robot Learning from Internet Video: A Survey." JAIR Vol.83. arXiv:2404.19664. Peer-reviewed survey; formalizes the chicken-and-egg problem.
• 5.Feng, Z. et al. (2026). "From Human Videos to Robot Manipulation: A Survey on Scalable VLA Learning with Human-Centric Data." arXiv:2606.00054. Source for the claim that episodization remains manual.
• 6.Zhang, J. et al. (2026). "Progress Reward Modeling for Robotic Learning: A Comprehensive Survey." arXiv:2607.21655. On the fragmentation of reward-judging metrics.
• 7.Amato, C. et al. (2025). "'Data will solve robotics and automation: True or false?': A debate." Science Robotics 10(105), eaea7897. DOI 10.1126/scirobotics.aea7897. A published debate showing that data-first was a dispute rather than a consensus. Companion editorial by Goldberg, DOI 10.1126/scirobotics.aea7390. (Paywalled — metadata and argument verified only)

Academic — Data Interface (Automatic Labeling)

• 8."JOPAT." arXiv:2605.23856, 2026-05-22. Table 6 — action-free video pretraining across demonstration budgets. Source of the §2 table.
• 9.Ye, S. et al. (2025, ICLR). "Latent Action Pretraining from Videos (LAPA)." arXiv:2410.11758. Table 1 — automatic labels head-to-head against real ones. Pretraining: 272 H100-hours.
• 10.Bu, Q. et al. (2025). "UniVLA: Learning to Act Anywhere with Task-centric Latent Actions." arXiv:2505.06111. 88.7% LIBERO average from human video alone.
• 11.Baker, B. et al. (2022, NeurIPS). "Video PreTraining (VPT)." arXiv:2206.11795. 1,962 labeled hours → 70,000 pseudo-labeled hours. (Minecraft domain, not robotics)
• 12.NVIDIA. (2025). "GR00T N1." arXiv:2503.14734. Latent action codebooks at industrial scale.
• 13.Xu, R. et al. (2026). "Motion-Focused Latent Action." arXiv:2606.18955. Separating background and viewpoint noise; ~50 downstream demonstrations.
• 14.Nikulin, A. et al. (2025, ICML). "Latent Action Learning Requires Supervision in the Presence of Distractors." arXiv:2502.00379. Ground-truth labels on 2.5% of the data yield a 4.2× average improvement.
• 15.Lee et al. (2026). "Why Latent Actions Fail, and How to Prevent It." arXiv:2605.20223. Theoretical account of reconstruction objectives pushing exogenous information into latent actions.
• 16."HumanScale." arXiv:2606.20521, 2026-06. Table 1 and the Figure 1 caption — ~10⁶ hours of egocentric video vs. ~10⁴ hours of public teleoperation.

Academic — Reward Interface (Success and Progress Judging)

• 17.Du, Y. et al. (2023). "Vision-Language Models as Success Detectors (SuccessVQA)." arXiv:2303.07280. ~88% inter-rater agreement among humans; 59.3% on unseen tasks.
• 18.Ma, Y. J. et al. (2024). "Vision Language Models are In-Context Value Learners (GVL)." arXiv:2411.04549. Reproduction: accuracy 0.62, precision 0.33.
• 19.Lee, T. et al. (2026). "RoboReward." arXiv:2601.00675. 22 frontier VLMs benchmarked; top model MAE 0.665.
• 20.Liang, A. et al. (2026, RSS). "Robometer." arXiv:2603.02115. Comparison- and preference-based axis; failure-detection F1 0.81.
• 21."LLM-as-a-Verifier." arXiv:2607.05391. 87.4% accuracy on trajectory preference judgments.
• 22.Huang et al. (2024/2025). "The Dark Side of Rich Rewards." arXiv:2409.15922. False positives damage training more than false negatives; cases where VLM reward is worse than none.
• 23.Chen, S. et al. (2026). "TOPReward: Token Probabilities as Hidden Zero-Shot Rewards for Robotics." arXiv:2602.19313. The reward model adopted into LeRobot.
• 24.Pan, A., Bhatia, K., & Steinhardt, J. (2022, ICLR). "The Effects of Reward Misspecification." arXiv:2201.03544. The phase change in which greater capability means more proxy exploitation.
• 25.Christiano, P. et al. (2017). "Deep Reinforcement Learning from Human Preferences." arXiv:1706.03741. The canonical case of reward hacking.

Academic — World Model, Embodiment, and Shared Failure Modes

• 26.Chow, W. et al. (2025, ICLR). "PhysBench." arXiv:2501.16411. Humans 95.87% vs. best model 49.49%; random 25.00%.
• 27.Han et al. (2025). "Probing Spatio-Physical Reasoning in VLMs." arXiv:2508.10770. Stack-stability judgments 0.542–0.641; humans 0.779.
• 28.Bansal, H. et al. (2025). "VideoPhy-2." arXiv:2503.06800. Around 22% on the hard subset. Predecessor: VideoPhy, arXiv:2406.03520.
• 29.Gu, J. et al. (2025). "PhyWorldBench." arXiv:2507.13428. 0.011 on the anti-physics category.
• 30.He, T. et al. (2024). "H2O: Learning Human-to-Humanoid Real-Time Whole-Body Teleoperation." arXiv:2403.04436. Only ~8,500 of 13,000 sequences feasible; privileged upper-bound policy 85.5%.
• 31.Araujo, J. P. et al. (2025). "Retargeting Matters (GMR)." arXiv:2510.02252. Retargeting quality alone splitting success rates between 0% and 100%.
• 32."What Matters in Orchestrating Robot Policies: A Systematic Study of Hierarchical VLA Agents." arXiv:2606.10267. Appendix B — false-positive and false-negative injection. Source of the §6 table.
• 33.Bedi, A. et al. (2026). "Auditing Demonstration Curation Metrics: Action-Only Scorers Fail on the Structural Defects That Degrade Imitation Policies." arXiv:2606.05588, 2026-06-04. Structural defects undetectable by action-only metrics; the best metric recovers a third of the gap. (Unreviewed preprint)

Industry Announcements · Dataset Docs · Policy

• 34.Figure AI. (2025-09-18). "Project Go-Big: Internet-Scale Humanoid Pretraining." End-to-end training announced on 100% egocentric human video. (No quantitative results released)
• 35.Hugging Face. "LeRobot — TOPReward." Supported config option; documented as inference-only and inheriting base-model bias.
• 36.Hugging Face. "LeRobotDataset v3.0." Reward, success, and progress absent from the canonical structure.
• 37.NVIDIA. "Cosmos Cookbook — GR00T-Dreams post-training," and Curating Synthetic Datasets with Cosmos Reason (2025-05-18). The five-point physical-plausibility filter.
• 38.The Korea Economic Daily. (2026-06-25). "Korea to build robot training centers for Physical AI data" [in Korean]. The data training center plan.
• 39.ZDNet Korea. (2026-06-19). "Government unveils the K-Physical AI full-stack strategy" [in Korean]. Primary text of the five-region hub plan.

Pebblous Series (Adjacent)

• 40.Pebblous. "What Do You Feed a Robot?" 2026-08-01. Five layers of data sources and how to mix them.
• 41.Pebblous. "We Mixed in Flawless Data, and the Robot Got Worse" 2026-07-17. Selection and closed-loop verification.
• 42.Pebblous. "Is Generated Video Physically Correct?" 2026-07-11. Verifying the physics of generated video.
• 43.Pebblous. "A World Model That Synthesizes Robot Training Data from Hand Motion Alone" 2026-07-10. A retargeting case.
• 44.Pebblous. "Nobody Can Trace, Atom by Atom, the Data That Reward-Verified RL Learns From" · "Why AI Aces the Test but Can't Do the Job" — the reward-signal quality series.