Executive Summary

GRPO came out of DeepSeekMath in 2024. To optimize a policy without a critic (a value function), it has a model solve one problem many times and lets those responses score each other. The next year, researchers at Sea AI Lab in Singapore and NUS argued that dropping the division by the standard deviation makes learning cleaner, and proposed Dr.GRPO. Around the same time, ByteDance Seed released DAPO, a large-scale system that filters problems with a standard deviation of zero out of the batch entirely.

Each name carried its own paper, its own institution, its own story. So from a practitioner's seat, "should I use GRPO, switch to Dr.GRPO, or is DAPO better" becomes a natural question. The July 2026 paper by Bay and Yearick unsettles the premise of that question. The three methods only turn one knob in different ways, and the knob's name is standard deviation.

At the center of the paper is one short theorem. Under a binary reward that scores 1 for a correct answer and 0 for a wrong one, the size of the GRPO update for a problem is exactly equal to the standard deviation of the responses to that problem. If the group size is $G$ and $k$ of them are correct, the standard deviation works out as follows.

This expression is zero when $k$ is 0 (all wrong) or $G$ (all correct), and reaches its maximum when the number of correct answers is half the group. In the authors' phrasing, the number that had long sat in the denominator to normalize the advantage was actually the length of the gradient itself. The value pulled out for the sake of dividing turned out to be a direct measure of how hard the problem pushes the model.

The intuition is simple. If the model solves a problem eight times and gets four right and four wrong, a clear contrast forms between the right and wrong answers, and that contrast makes a "go this way" signal. If it gets all eight right, there is no wrong answer to compare against and no basis for setting a direction. For learning to happen, the responses have to split, and the size of that split is the size of the learning.

Figure 1. Standard deviation across correct/wrong mixes at group size 8. σ = 0 (silent) when all correct or all wrong; σ peaks at an exact even split, where the learning signal is largest. Original Pebblous diagram (reinterpretation of Fig. 1).

A group with a standard deviation of zero, meaning every response is correct or every response is wrong, is what the paper calls silent. A silent group contributes nothing to the update. It spent compute generating several responses yet added nothing to learning, a literally quiet batch.

The probability that a problem goes silent is set entirely by its accuracy $p$ and the group size $G$. It is the probability of getting them all right plus the probability of getting them all wrong.

Put this expression on real data and the scale becomes tangible. Applied to the Big-Math dataset of 215,608 problems at a group size of 8, the prediction that about 44% of all groups go silent matched the measured value to within two percentage points. That means nearly half of a training batch can pass through without contributing anything to learning. Problems too easy to ever miss and problems too hard to ever solve fall silent for opposite reasons in equal measure.

Figure 2. Silent probability P(silent) = p⁸+(1−p)⁸ versus problem accuracy p at group size 8. It spikes as accuracy nears the extremes (0 or 1) and bottoms out near p=0.5, where the learning signal is largest. Big-Math's 210,000-problem average is about 44%. Original Pebblous diagram (reinterpretation of Fig. 2).

Given the same number, the choices the three methods make diverge in their learning outcomes. That divergence shows up in two places. One is whether each method holds on to the hardest problems all the way; the other is how much extra compute it pays to do so. The difference between dividing by one standard deviation, keeping it, and dropping it widens into a visible performance gap along these two axes.

Figure 3. GRPO divides by σ and amplifies extreme difficulty, Dr.GRPO skips the division and leaves weight uniform, and DAPO cuts the σ=0 silent zones and resamples. Original Pebblous diagram (reinterpretation of Fig. 3).

4.1GRPO — Divides

GRPO divides the update by the standard deviation. That attaches a difficulty weight to each problem, and for a problem with accuracy $p$ the weight takes the form $w(p) = 1/\sqrt{p(1-p)}$.

This curve spikes at both ends, on the very easy and the very hard problems. It loads two to five times more weight on extreme-difficulty problems. That is how GRPO lifted accuracy on the hardest quartile to 0.99. What looked like a side effect left behind by the division was in fact a device that keeps hard problems from being dropped.

4.2Dr.GRPO — Does Not Divide

Dr.GRPO removes that division. With the difficulty weight gone, learning follows a problem's raw success rate $p$ directly. It is an unbiased, clean signal, but it does not give extreme difficulty any special attention. In the same experiment, accuracy on the hardest quartile stopped at 0.88 and rose no further.

4.3DAPO — Drops

DAPO strips silent groups out of the batch and resamples. Because it spends no resources on silent problems, it learns fastest. In return it pays an oversampling cost, needing roughly 3.5 times more sampling to filter out the silent groups and fill an effective batch.

The identity that the standard deviation is the update size ends up confirming, inside the training equation, the old question of which data teaches a model. Problems where right and wrong answers split, the ones the model does not yet know for sure, lead the learning. Problems it already knows completely or cannot solve at all stay silent even if you swap the algorithm from GRPO to DAPO. A problem with nothing to learn from stays unlearnable no matter which optimizer you reach for.

So the order flips. It is easy to worry first about which of the three algorithms to use, but the decision that comes before it is which difficulty of problem to put in the training set in the first place. A selection strategy that measures accuracy in advance, prioritizes problems near 50-50, and handles the extremes separately meets the paper's mathematical conclusion exactly. The algorithm only decides how to handle the standard deviation; putting problems likely to have a standard deviation into the data has to be finished a step earlier.

This identity does not settle everything, of course. A later analysis points out that an implicit difficulty weight still remains in Dr.GRPO and DAPO as well. Even so, the direction is clear. Saying that good data is informative data is not rhetoric bolted on from outside training; it is already true inside the equation that says an update happens only as much as the standard deviation.

This is exactly the point Pebblous means by the information content of data. Between what a model knows for sure and what it does not know at all, in the band where responses split, is where learning concentrates. Picking out that band is a matter of data quality, and it comes before the choice of algorithm.

• 1.Bay, Y. Y., & Yearick, K. A. (2026). "GRPO, Dr. GRPO, and DAPO Are Three Operations on One Number: The Group-Standard-Deviation Identity." arXiv:2607.00152.
• 2.Shao, Z., et al. (2024). "DeepSeekMath: Pushing the Limits of Mathematical Reasoning in Open Language Models." arXiv:2402.03300. (Original GRPO paper)
• 3.Liu, Z., et al. (2025). "Understanding R1-Zero-Like Training: A Critical Perspective." arXiv:2503.20783. (Original Dr.GRPO paper)
• 4.Yu, Q., et al. (2025). "DAPO: An Open-Source LLM Reinforcement Learning System at Scale." arXiv:2503.14476. (Original DAPO paper)
• 5.Albalak, A., et al. (2025). "Big-Math: A Large-Scale, High-Quality Math Dataset for Reinforcement Learning in Language Models." arXiv:2502.17387.
• 6.Anonymous. (2026). "Your Group-Relative Advantage Is Biased." arXiv:2601.08521. (Follow-up analysis noting the residual difficulty weight)