Executive Summary

Key Numbers

Source: Zongye Lyu, Same Answer, New Box (arXiv:2610.20639, October 2, 2026).

The paper opens on a single ARC-Challenge item. Anna ran 100 meters in 20 seconds, so what was her average speed, with 2 m/s, 5 m/s, 20 m/s and 25 m/s on offer and B) 5 m/s as the key. The base model Qwen2.5-7B ends its output with "The correct answer is B) 5 m/s." The output of the model that SimpleRL-Zoo trained from it by reinforcement learning ends with "Anna's average speed is \boxed{B}." The two answers are the same and both are correct.

The difference is on the scoring side. An extractor that reads only inside the answer box \boxed{} finds no box in the base model's output, abstains, and the item goes down as wrong. It reads the box the RL model filled and counts it as correct. Only one of the two scores.

What that difference comes to over a stack of items is shown by scoring twice. Over the 500 ARC-Challenge items, the box-only extractor reports that reinforcement learning raised accuracy by 30.6 points. Score the same outputs with a fixed extractor that takes a stated answer whether it sits in the box or in the text and the figure is 9.8 points. The 20.8-point difference is what the author calls the format gain, and on this set of items it is 68% of the reported gain.

One term settled here makes the rest easier. The author calls an extractor lifted straight out of the evaluation code of a public repository a harness extractor, and calls the gain that such an extractor produces a reported gain. It is not a number some research team printed in a paper. It is a number the author obtained by running the released checkpoints, generating outputs, and applying that code to them.

The method is simple. Score the same outputs twice. The weights are never touched and the reinforcement learning is never re-run. One pass uses a harness extractor and the other uses the fixed extractor the author wrote.

The fixed extractor has one job: to count the same answer the same way for both models. So it reads an answer only where one is explicitly stated. Inside \boxed{}, after "the answer is" or "Answer:", on a line of its own, or after "option" or "choice". Where it finds none it abstains, and the item counts as wrong for either model. What it does not do is drop the item. An extractor allowed to skip items could raise its own accuracy by skipping the hard ones.

Whether that rule falls evenly on both models is itself reported as a number. On the comparisons behind the 82%, the fixed extractor reads an answer in 87% of the base models' outputs and 88% of the RL models'. The arithmetic of the split sits on top of that. The difference between RL and base as a harness extractor scores them is the reported gain, and the difference as the fixed extractor scores them is the gain in the answers themselves. What lies between the two is the format gain, and the format gain as a fraction of the reported gain is the format share.

The material is seven base and RL pairs drawn from three public releases. SimpleRL-Zoo, SimpleRL-Zero from the same group, and Open-Reasoner-Zero. Four of them clear both conditions: the Zoo and Zero successors of Qwen2.5-Math-7B, and the Zoo successors of Qwen2.5-7B and Mistral-7B-v0.1. The DeepSeek-Math-7B pair had no gain to split in the first place, and with Qwen2.5-1.5B almost every output of the base model ran to the token limit, leaving the fixed extractor nothing to work with.

The test bed is MMLU, MMLU-Pro and ARC-Challenge, three out-of-domain multiple-choice benchmarks. The harness extractors come from six public repositories, and the four of them that handle multiple choice look in different places. The MATH evaluation code and Open-Reasoner-Zero read the last \boxed{} and nothing else, Qwen2.5-Math's reads the box first and then takes a letter from the text when the box yields none, and simple-evals' looks for a line in the form "Answer: X". Both prompts are run, the one that asks for the box and the one that makes no request about format.

Across the three benchmarks the average gain a harness extractor reports comes to 22.5 points for each comparison. Score the same outputs with the fixed extractor and 4.0 points are left. Pool all 46 comparisons and the format share comes out at 82.2%, with the 95% interval running from 77.4 to 87.2%. Cut it up and it does not lean one way. By benchmark it lands between 81% and 84%, and by Qwen pair between 76% and 87%.

Of the four pairs, the Mistral one is a different animal. The harness extractors report a gain of 7.7 points, and the fixed extractor reads a loss of 0.7. The answers themselves did not improve, which is why the format share for this pair runs above 100%.

That the 82% rests on a set of choices is not something the paper hides. So the choices were changed one at a time and the count redone. The fixed extractor's rules were loosened and then tightened, one pair or one benchmark or one harness extractor was left out, one prompt was used instead of both, and the comparisons whose share runs above 100% were dropped. In every case the share stays between 75% and 87%. What moves it most is which comparisons are pooled, and pooling every comparison with a non-zero gain gives the floor of 75%.

3.1Readers That Share No Code Reach a Similar Figure

The possibility that the fixed extractor is producing the result on its own still stands. So two language-model readers, sharing no code with it, each read 2,720 responses sampled from the outputs behind the 82%, with a third reader settling their disagreements. A reader sees the question, the options and one response, and never learns which model wrote it or what the key is. Scored that way the format share is 77%.

The two verdicts almost coincide. In 98.6% of the responses where the extractor finds an answer the readers name the same one, and they part company only where the extractor abstains. The answers the readers recover there are more often correct for the RL models, so the readers credit RL with a slightly larger gain. That is where the gap between 82% and 77% comes from. The sample is too small to say whether the two really differ, the author writes.

3.2Asking for the Box in Advance Does Not Make It Go Away

Change the prompt and the conclusion holds. Under a prompt that spells out that the final answer must go inside \boxed{} as a single letter the share is 79%, and under a prompt that makes no request about format it is 86%. The direction is exactly what the mechanism predicts. Ask for the box and the base model uses it more often, so fewer of its answers are hidden from an extractor that reads only the box. The RL model still uses it more, and most of the gain is still format.

The paper also logs a case where the prompt was pushed much harder. Both of the study's prompts keep the chat template's default system message, and for Qwen2.5-Math-7B that message itself asks for the box. In a pilot that swapped in a request of the author's own, the MMLU share for the Qwen2.5-Math-7B and SimpleRL-Zoo pair falls to 31%. The same pair gives 88% and 86% under the study's two prompts. Under that prompt the base model stated no answer the fixed extractor could find in 45% of its outputs, so much of the gain measured there was a gain in stating an answer at all rather than in answering better. The share, the author concludes, is a property of the pair, the prompt and the extractors together.

That the reported gain swings when only the extractor changes is on the record too. On the outputs of Figure 1 the two extractors that look only in the box report 30.6 points, Qwen2.5-Math's extractor, which also takes letters from the text, reports 1.0, and simple-evals' extractor, which hunts for the "Answer:" line, reports a loss of 1.8. Even the sign splits. Over 112 combinations of pair, benchmark and prompt, the largest and smallest gains the harness extractors report on the same comparison differ by a median of 15.1 points. Count without simple-evals' two extractors, which look for the "Answer:" line their own prompt asks for, and it falls to 10.

Why this happens comes out when you count where the answers sit. Among the out-of-domain multiple-choice outputs in which the fixed extractor reads an answer, the share containing a \boxed{} is 26.6% for the base models and 58.3% for the RL models. Nearly all of the base models' remaining answers are stated in the text after "the answer is" or "Answer:", with no box anywhere in the output. A box-only extractor misses most of the base models' answers and fewer of the RL models'. That difference is the format gain.

4.1In Domain, There Is No Gap

Put the same models on in-domain free-form mathematics and the picture changes. On MATH-500 and GSM8K, 96 to 97% of the outputs the fixed extractor can score contain a \boxed{}, and that share is practically the same for base and RL. Both of them box almost every answer, so a box-only extractor and the fixed extractor end up counting the same answers, and the format gain approaches zero. What reinforcement learning taught is the habit of writing the answer in the box, and the habit follows the model into multiple-choice questions the reward never touched.

The explanation that the box is merely empty because the token limit cut the answer off is ruled out separately. Take only the items where neither model reached the limit and the gap does not narrow but widens. The share containing a box is 34.7% for base and 81.9% for RL.

That rewarding format raises scores was already known. Shao et al. showed that reinforcement learning with a reward for merely producing a \boxed{} expression, correct answer or not, raises Qwen2.5-Math-7B's MATH-500 accuracy by 13.8 points, about half of what the ground-truth reward gives. That answer format shakes reported numbers had been pointed out as well. Chandak et al. found that the base-model accuracies seven RL papers report sit below the official ones in most cases, named answer format and parsing among the causes, and stopped at asking for sample-level outputs to be released. The new part is the counting: how much of that share sits inside the gains actually reported on released checkpoints.

4.2On the 32B Pair This Does Not Happen

Format gain is not something every run of reinforcement learning produces. On the Qwen2.5-32B pair trained with DAPO the format share is 2.1% over 9 comparisons. The 95% interval runs from −18.5% to 19.9%, which is wide. The reported gain barely shrinks when the fixed extractor does the counting.

The author set this pair apart before settling on the four, and declared before any output was generated that a share below 40% on it would make the result a property of models of 7B and below. The reason is symmetric. Ask for the box and the 32B base model already writes nearly every answer in it, and without that request the DAPO-trained model rarely uses one. The two models do not diverge in where they write their answers, so the choice of extractor cannot move the score. One pair, the author adds, cannot say whether the difference comes from scale, from the DAPO recipe or from the base model.

4.3The Lines the Author Draws

What is not being claimed is set down plainly in the paper as well. There are four.

• • No published number is decomposed. What was split is the gain obtained by running the checkpoints, generating outputs, and applying the author's own choice of prompts and extractors to them. SimpleRL-Zoo produced its own numbers with its own prompt, sampling at temperature 1.0, and a script that also reads letters in the text, and the author did not reproduce that evaluation. The 82% is not an estimate for those published numbers
• • The share is only stable on out-of-domain multiple choice. On free-form BBH or on in-domain benchmarks it moves by more than 50 points with the rule for which comparisons are pooled, so it cannot be quoted. Whether the format gain goes with the out-of-domain property or with the multiple-choice answer form, this study cannot tell
• • It rests on extractors that read the box first. Nearly all of the gain behind the 82% is reported by extractors that look in \boxed{} first. Qwen2.5-Math's extractor, which also reads letters in the text, reports 56% on the comparisons it scores
• • One release family, one size. All four pairs come from the SimpleRL releases and rest on three 7B base models, and the Qwen base models carry nearly all of the reported gain. Whether the same share holds in other model families is open

When model performance gets reported, people usually ask about the number first. From what to what, and whether the difference could be chance. This study points at the cell before that one. Who read the number out. The good news is that checking costs almost nothing. Neither the reinforcement learning nor the inference has to run again, because the stored outputs can simply be re-read by a second extractor. All three of the practices in the paper's conclusion live in that spot.

• • Name the extractor behind every number you report. If the gain read out of the same outputs moves by a median of 15.1 points with the extractor, a score with no extractor named is half a number
• • Report the rate at which the extractor reads an answer out of the base model. It can be measured before any RL model exists, and in this study the extractor that read the most of the base models' outputs reported the least format gain
• • Re-score the same outputs with a second extractor that reads both models' stated answers. The difference between the two results is the format gain

Why this is not somebody else's problem shows up in a second number the study counted. Of 18 recent RLVR and reasoning-model releases surveyed, 17 carried out-of-domain multiple-choice results, and 13 of those read the answer out of generated text. One scored log-likelihoods and three do not say how. Move that to an in-house evaluation and the question turns into this. What does our evaluation script accept as a correct answer? Change that rule and what is left of last quarter's improvement report? Answering requires that the outputs the model produced were kept rather than discarded for the scores alone, and that it is written down which checkpoint and which prompt each output came from. With both, the re-scoring finishes on one laptop. Without either, that question can never be answered.

Reading this result as proof that reinforcement learning teaches nothing goes too far. Even under the fixed extractor 4.0 points remain, in domain the format gain is near zero, and on the 32B pair the reported gain survives almost intact. What this study claims is narrow. On out-of-domain multiple choice a large part of the reported gain is a property of the program that produced the number rather than of the model, and that can be confirmed from the stored outputs alone.

Thanks for reading this far. The facts in this article were checked against the full text of the paper on arXiv. The author published the generation and scoring code, the frozen evaluation items with their hashes, and the independent readers' responses as ancillary files, along with the SHA-256 hash of each of the 264 generation files. If your team already writes down which extractor produced a number when it reports a model evaluation, or if you have ever re-scored stored outputs under a different rule, we would like to hear about it.

Pebblous Data Communication Team
October 19, 2026