Executive Summary

Key numbers

The first three are measured on one scale for how far a model retraced the flaws people had marked. The last one is the material that scale was built from.

Source: arXiv:2610.30866 (October 22, 2026)

Detectors for AI-generated video have long been judged by a single number, balanced accuracy. It takes the share of real videos called real and the share of fake videos called fake, works the two out separately and averages them, so that a lopsided mix of clips cannot inflate the result. Models are easy to line up against each other that way, and most detection papers rank their field by it.

The Verus-Insight paper set a second column beside it. That column is called Finding F1, and it measures whether a model also pointed at the places a person watching the video had marked as wrong. Put the two side by side and the ranking moves.

The two columns do not come from the same evaluation set. The verdict score is balanced accuracy on a set called RewardData, and the evidence score can only come out of material where a person marked each flaw by hand. Carrying the name of the column along with the number is the safer habit. Even so, read one model's row across and a fact stays put. VideoVeritas, the specialized detector with the highest verdict score, sits only just above the general-purpose models on evidence.

People make the answer key behind the evidence score. An annotator watches a video, finds a spot that looks off, writes down a category and a description, and marks where on the screen and roughly when it shows up. A finger that gains a joint for one frame, a shadow that drifts away from the object casting it. There are 18 categories in all.

Those marks are collected in an evaluation set called Verus-Bench. It holds 192 videos, half of them filmed and half of them generated, 96 each. Attached to them are 326 flaws that people found and wrote down. The evidence score is graded against those 326.

Grading pairs the marks with whatever the model pointed out. The pairing runs on meaning rather than on wording, and the share of marked flaws the model found is folded together with the share of the model's remarks that landed on a real flaw to give an F1 value. A video carries somewhere between one and five flaws, so pointing anywhere and collecting credit is not an option.

In its opening figure the paper laid out two ways an existing detector comes apart. It answers real or fake wrongly, or it answers correctly while the flaw it offers as evidence sits somewhere else. One video with a mangled face, shown to four models, puts both of those in one place.

The verdict and the handling of the flaw are carried over directly from the paper's Figure 2.

The row that catches the eye here is BusterX++. It got the real-or-fake verdict right but failed to find the central flaw. On a scorecard with only a verdict column, that model is written down as "correct" in exactly the same shape as Verus-Insight, which actually recovered the flaw. VideoVeritas got the verdict itself wrong on this clip. The reason was not that it failed to reach the spot, but that it reached the spot and judged it to be fine. Both missed the flaw, but the way they missed it, and the score each one paid for it, are not the same.

Verus-Insight does not say real or fake the moment a video arrives. It first sets out what to check and where, then goes item by item to see whether the flaw is actually there, and only at the end gathers what it confirmed into a verdict. With the verdict resting on a record of checks, the order is hard to run backward, settling the answer first and attaching an explanation afterward.

Training was arranged to teach that order. The model was first drilled on following regions through a video and drawing up the checks to run, and then on reproducing the whole stretch from description to checks to verification result to final call. Both of those stages are supervised learning, where it works from answers people had written out in advance.

That was apparently not enough. For the last stage of training the researchers mixed the evidence score straight into the reward. When the model is tuned with reinforcement learning, the reward does not stop at whether the verdict was right; on a generated video it is multiplied by how far the evidence lined up with the flaws people had marked. Inventing a flaw that is not there in a real video costs points. The shortcut of answering correctly and leaving the evidence blank was closed off while the model was still learning.

The material was gathered in the same direction. There are 8,411 training instances that people worked on, carrying 36,309 rubric checks, of which 11,745 confirm a flaw and 24,359 confirm that nothing is wrong. The clean side outnumbers the other by more than two to one. Train a model only to go find something odd and it will start producing flaws in footage that has none.

Set against the backbone, the result is sharp. Qwen3-VL-8B, the model Verus-Insight is built on, stayed at 3.38 on evidence while working the problem alone, and a model of the same size came out at 35.42 after going through this process. Against 18.08 from GPT-5.5, the best figure on the proprietary side, that is close to double.

The figure that remains deserves the same look. A score of 35.42 is only the highest in this comparison. On a scale where the model's flaws and the people's marks overlapping completely would read 100, it stands a little past one in three. On Verus-Bench, the set that carries the human annotations, the verdict score is 89.06, the same value GPT-5.5 reached. The contribution here sits closer to a scale that asks whether evidence was read at all than to a model that reads evidence well.

Video detection is not what Pebblous does. Still, the hole this paper opened has the shape of one that turns up daily wherever data quality is handled. A data validation pipeline also tends to be reported as a single pass rate. The line that reads "97% passed" is tidy, and nothing inside that number says which column and which pattern the decision rested on.

What 91.12 and 8.47 say together is that the two can move apart. A correct verdict carries no guarantee that the checking was done, and a score that counts verdicts alone keeps no record of the difference. Three things are worth settling whenever a scorecard for an in-house model or validation tool arrives.

• Whether a passing verdict comes with evidence a person can open. It only counts as evidence once it goes down to which record and which value tripped the check
• Whether an answer key exists to check that evidence against. Something like the 326 annotations in Verus-Bench, a set marked by people first, has to be there even in small form before evidence can be graded
• Whether the accuracy of the evidence enters the criteria for choosing a model. A metric that lands in a report and never reaches a decision does not survive long

The third is the part most worth taking from this paper. The researchers did not stop at measuring the evidence score; they put it into the training reward. Where the measuring and the choosing sit apart, a metric stays decorative.

Thanks for reading this far. Every figure in this article was checked against the comparison tables in the paper posted to arXiv on October 22, 2026, and with the paper two days old there is no outside reporting to cross-check it against. If you share data validation results inside your own organization, we would be glad to hear what evidence you write down beside a passing verdict.

Pebblous Data Communication Team
October 24, 2026

• 1.Zhang, Z., Shi, Y., Wang, Q. et al. (2026). "Verus-Insight: Towards Explainable Artifact Detection in AI-Generated Videos." arXiv:2610.30866.