Executive Summary

Start with the conclusion: pixel-based world models break physics not by accident but by structural necessity. These models watch enormous amounts of video and learn "what the next frame should look like to be statistically plausible." Nowhere in that training objective are the physical states of mass, force, velocity, or contact made explicit. The model does not know how the world moves; it knows how pixels should be arranged to look natural. So it calmly generates scenes where objects pass through walls, cups vanish and reappear, and a falling ball forgets about gravity.

These failures recur in a few familiar types: interpenetration, where one object passes through another; object-permanence violations, where something that left the frame returns looking different; mass-conservation violations, where liquids fail to preserve their volume; and momentum-conservation violations, where objects rebound with the wrong momentum after a collision. Frame by frame it looks plausible to the human eye, but held to the ruler of physics it goes wrong everywhere.

1.1How common are the flaws?

These flaws are not the exception but the majority. The Physion-Eval benchmark had humans directly inspect generated video and count physics flaws: 83.3% of third-person (external-view) videos and 93.5% of first-person (egocentric) videos contained at least one identifiable physics flaw. Eight or nine out of every ten videos broke physics somewhere. That the flaw rate is even higher for egocentric video, where the camera moves along with the scene, suggests the problem grows worse in real applications like robotics and autonomous driving, where the observer is in motion.

The stricter the evaluation, the more steeply the pass rate falls. Below are three measurements of the same "physics compliance" at different levels of strictness. A pass rate already around 40% on a benchmark demanding both semantic and physical correctness (VideoPhy) drops to 22% on VideoPhy-2, which narrows to harder physics scenarios, and the flaw rate climbs into the 90s when humans are asked to find any flaw at all (Physion-Eval).

World models handle physics along three broad paths. We laid out this lineage itself in an earlier survey, so here we rebuild it on a single axis: verifiability. How each path handles physics is directly tied to whether you can confirm after the fact that its physics is correct.

Pixel-based (Sora, Cosmos, Genie) dissolves physics implicitly into pixel statistics. With no explicit physical state, the only way to check the physics of the output is to extract physical values back out of the pixels — the inverse-extraction pipeline of Section 3. Physics-based (simulators, explicit state) has a physics engine compute trajectory, force, and collision at every step, so the physical values exist as ground truth alongside the output and need no verification. Understanding-type JEPA (V-JEPA 2, Yann LeCun) abandons pixel reconstruction and predicts the next state in latent space, giving a compact representation — but whether that latent representation actually captures real physics still has to be confirmed by external tests.

The hope that "if it's understanding-type, surely it knows physics" is premature. V-JEPA 2 too falls significantly short of humans on intuitive-physics tests like IntPhys 2, which deliberately insert physics violations. Predicting well in latent space is not the same as understanding physics accurately. In other words, none of the three paths can declare itself "physically correct without verification." The only exception is the simulator, which holds the physics answer key from the start — a contrast we meet again in Section 6.

Pixel-based output has no physical state, so to verify it you have to read the physics back out of the pixels. Over the past two years a de facto standard assembly line has emerged for this work. Separate the object (segmentation), follow where it moved (trajectory tracking), measure how it was oriented (6-DoF pose), measure how far it sat from the camera (metric depth), and finally differentiate these values to obtain velocity and acceleration and check them against physical law (physical-quantity computation). Each stage is handled by a well-trained foundation model.

Each tool's role, representative accuracy, and structural limits are gathered in the table below. Individual scores are impressive, in the 90s, but the moment the limits in the last column are chained together, the errors multiply.

3.1Segmentation — what counts as an object? (SAM2)

Every physical measurement begins with "what do we treat as a single object?" Meta AI's SAM2 is a foundation segmentation model that separates and tracks objects frame by frame across an entire video, scoring J&F 90.7% on the DAVIS benchmark. But when objects of similar color or shape overlap or move quickly, the mask leaks into its neighbor, and that boundary error shakes the entire baseline for the trajectory, pose, and depth measurements that follow.

3.2Trajectory tracking — where did the object go? (CoTracker3 · SpatialTrackerV2)

This stage follows where the separated object went over time. CoTracker3 densely tracks the 2D path of points on screen, but when an object is occluded by another, its accuracy (AJ) drops from 55.8 to 46.7. Lose the tracking point during even a brief occlusion, and the velocity over that stretch becomes wholly unestimable. SpatialTrackerV2 extends this to 3D to recover the spatial trajectory, but with monocular (single-camera) video it cannot, in principle, fix the absolute scale of "how big this object really is." When the scale wobbles, so does the absolute value of the velocity.

3.36-DoF pose — how was it oriented? (FoundationPose)

The 6-DoF pose, combining position (3) and rotation (3), is essential for judging rotational motion, contact, and collision. FoundationPose became the standard for novel-object pose estimation with ADD-S 91.5% on the YCB dataset. But that precision leans on a CAD model of the object or a few reference images, so it wobbles at its very foundation when faced with the never-before-seen objects common in generated video, or with deforming bodies and fluids that change shape.

3.4Metric depth — how far from the camera? (Depth Anything V2)

To return a 2D trajectory to real physical space, you must know how many meters each pixel sits from the camera. Depth Anything V2 delivers an excellent relative error (AbsRel) around 0.05 on indoor benchmarks, but that strength collapses sharply as distance grows. The accuracy falls almost in proportion to distance: 89.1% at 2m drops to 70.8% at 6m. Under the far, underwater, and outdoor conditions of robotics and autonomous driving, relative error degrades to 0.27. (V2 does not always beat V1; the original paper notes it is "on par with V1.")

3.5Computing physics and the price of scale (Finite-diff · Morpheus · ObjectForesight)

Once trajectory, pose, and depth are gathered, the final step differentiates with respect to time (finite differences) to obtain velocity and acceleration, and checks them against physical laws like the conservation of momentum and energy. The problem is that differentiation amplifies the small errors from earlier stages. Morpheus grades against roughly 80 videos (130 in the revised edition) of real physics experiments, using only conserved-quantity metrics like an Acceleration Score and a horizontal Momentum Conservation Score. Because its design asks only "were the conservation laws upheld?" without a separate answer video, it applies as-is even to generated output that has, in principle, no ground truth. ObjectForesight takes a different route. With a SAM2 → SpatialTrackerV2 → TRELLIS → FoundationPose chain, it extracts over two million trajectories from EPIC kitchen videos, bundling them into a 0.84 TiB dataset.

But those two million trajectories were not "well extracted" — they are what was "left after filtering." ObjectForesight could trust only about 112K of roughly 229K raw trajectories, half, and discarded the rest. Even the survivors needed local re-registration whenever their projected IoU fell below 0.1, so the correction that patches the losses takes up a substantial part of the pipeline. Morpheus, too, matches scale by assuming a substantial discard rate. Post-hoc inverse extraction is inherently the work of picking out ground truth — it scales only by accepting massive loss. Only after the untrustworthy trajectories are discarded do the usable ones remain.

Physics-IQ is the attempt to measure not an individual tool but a whole model's physical understanding head-on. Its method is simple and clever. Sixty-six real-world scenarios (a glass tipping over, paint spreading in water, dominoes falling) are filmed from three viewpoints, twice each, producing 396 real videos. A model is then shown the first 3 seconds and asked to predict the next 5, and the future it draws is compared to the real continuation via IoU and MSE. The 100-point standard is not a "correct answer video" but the natural variability of real-to-real video pairs of the same scene. Even two real takes of the same scene are never identical to a human eye, so that natural difference is set as the 100. The subjects are not skewed toward one phenomenon: paint dispersing in water (fluid dynamics), reflection and refraction of light (optics), dominoes and collisions (rigid-body mechanics), magnetism, and thermodynamics — five branches of physics evenly spread, so a model cannot inflate its score by mimicking just one or two easy ones.

The result is sobering. Even the strongest generative model reached only 24.1 on this baseline, reproducing about a quarter of the physical continuity a human expects. The gap holds under the other ruler of pixel fidelity too: the best model's prediction error (MSE) was 0.010, five times the 0.002 of real-to-real pairs. Measured by either surface quality or physical continuity, generated output falls far short of the real. More intriguing is the relationship with visual realism. Sora, rated the most visually plausible, actually fell to the bottom (8.7) on physical understanding. Both data sources note that the correlation between realism and physical understanding is low or negative. The intuition that a better-looking video knows physics better does not hold.

4.1Who verifies the verifier? — Physics-IQ Verified

When a benchmark gains authority, the moment comes to ask how trustworthy the benchmark itself is. Physics-IQ Verified, released in June 2026, precisely audited the original benchmark and found that 57.6% of samples were contaminated by artifacts unrelated to the physics being measured (lighting flicker, camera shake, compression noise), and 34.8% of prompts were flawed. Decisively, when the contamination was removed and the measurement rerun, model rankings actually reshuffled. The correlation between original and re-measured rankings was only Kendall τ=0.46 — "gentle, but rankings genuinely shift."

Industry too has begun to admit the limits of automatic grading. NVIDIA Cosmos 3 (Nano 16B, Super 64B), unveiled at GTC Taipei in June 2026, is paired with the Cosmos Evaluator/HUE announced alongside it. When existing leaderboards saturated and could no longer distinguish between models, NVIDIA changed its grading method. Instead of scoring a generated video whole, it breaks the task into Yes/No questions about single facts — "atomic binary verification." This evaluator is not a standalone tool but the final stage of a pipeline running from curation (Cosmos Curator) → domain adaptation (Cosmos-Transfer) → quality assessment (Cosmos-Reason/Evaluator), with NVIDIA OSMO orchestrating the whole and covering seven Physical AI domains including robotics and autonomous driving.

The decomposed questions bundle into four axes: does the scene match the prompt's meaning (semantic alignment), does it obey physical law (physical law), are space and geometry consistent (geometric reasoning), and is it visually complete (visual completeness). Grading one video by aggregating binary questions like "did the ball bounce after hitting the floor?" is better at pinpointing what went wrong than a vague total score. NVIDIA reported that this grading correlates highly with human evaluation (ρ=0.96).

The direction is clearly right. But two limits remain. First, the structure has the vendor that generated the video grade it too. When generation and evaluation sit in the same hands, the independence of that evaluation is in principle limited (the ρ=0.96 is also NVIDIA's own report, and third-party verification is limited). Second, the concrete scoring formula is not disclosed, making reproduction and audit difficult. And all of this automatic verification still stands on top of the inverse-extraction weaknesses seen in Section 3 — occlusion, distance, monocular scale ambiguity, and unverified deforming bodies and fluids.

Stand everything so far on a single axis and it resolves like this. There are three paths to obtaining physics data, and the three differ fundamentally in their relationship to the physics ground truth: the path that embeds physics at the moment of generation (RoboScape), the path that reads physics back after generating (this report's inverse-extraction verification), and the path that starts with the physics ground truth in hand (the simulator).

Both the embedded and post-hoc-audit approaches are ultimately approximations. RoboScape embedded physics into the generation process and reached 91% — near real data's 92% — with just 200 synthetic clips, but it still had to embed the physics. How large a share that physics carries in the data is shown by an ablation: removing physical components like depth and keypoints from RoboScape collapsed performance by 40.6 percentage points. Physics is not decoration laid on top of data but a condition for the data to work at all. Conversely, the physics-compliance rate of ordinary unverified generated output sits at 22% by the VideoPhy-2 measure. The gap between 91% and 22% is exactly the difference made by "how physics is put into the data." The simulator, however, sidesteps the question entirely. The trajectory, force, mass, and collision events its physics engine computes at every step remain, alongside the video, as a ground-truth event ledger. Nothing to read back, nothing to embed.

This contrast interlocks precisely with the three dimensions of AI-Ready Data: accuracy, consistency, and provenance. Generated video has neither provenance nor ground truth, so a separate verification infrastructure must be built — and even that infrastructure can be 57.6% contaminated. Simulator data comes with ground-truth labels at the moment of generation, so it inherently satisfies all three dimensions. The harder a problem is to verify, the more the ability to mass-produce ground truth that needs no verification becomes the data moat.

For further reading, we recommend the companion piece on embedding physics at the moment of generation, the RoboScape edition, along with the broad survey of the world-model lineage, the World Model Survey.

Pebblous Data Communication Team
July 11, 2026

References

Benchmarks · Physical Understanding

• 1.Motamed, Culp, Swersky, Jaini, Geirhos. "Do generative video models understand physical principles?" (Physics-IQ), arXiv:2501.09038 (ICCV 2025).
• 2.Rädsch, Asano, Kuehne, Bauer, Jaini, Geirhos, Lüth. "Physics-IQ Verified," arXiv:2606.18943 (2026).
• 3.Zhang et al. "Morpheus: Benchmarking Physical Reasoning of Video Generative Models," arXiv:2504.02918.
• 4.Bansal et al. "VideoPhy," arXiv:2406.03520 · "VideoPhy-2," arXiv:2503.06800.
• 5."Physion-Eval," arXiv:2603.19607.
• 6."ObjectForesight: Predicting Future 3D Object Trajectories from Human Videos," arXiv:2601.05237.

Inverse-Extraction Tools

• 7.Ravi et al. (Meta AI). "SAM 2: Segment Anything in Images and Videos," arXiv:2408.00714.
• 8.Karaev et al. "CoTracker3," arXiv:2410.11831 · "SpatialTrackerV2," arXiv:2507.12462.
• 9.Wen et al. "FoundationPose," arXiv:2312.08344 (CVPR 2024).
• 10.Yang et al. "Depth Anything V2," arXiv:2406.09414.

World Models · Industry

• 11.NVIDIA. "Cosmos 3: Omnimodal World Models for Physical AI," arXiv:2606.02800 (2026) · NVIDIA/cosmos-evaluator (GitHub).
• 12.Assran et al. (Meta AI). "V-JEPA 2," arXiv:2506.09985.
• 13."RoboScape," arXiv:2506.23135 (NeurIPS 2025).
• 14.google-deepmind/physics-iq-benchmark (GitHub) · INSAIT announcement.