Executive Summary

When an AI is confidently wrong, we usually blame the data — too little of it, or too messy. A team at KAIST offers a different answer. The root of overconfidence, they argue, lies not in the data but in how a network is set up before training even begins: the random initialization that has hardened into standard practice. A network that has seen no data at all already holds strong confidence in particular answers, and that bias survives all the way through training into the final model.

The remedy is surprisingly simple. Before showing the model any real data, you let it warm up on meaningless random noise for a while. It learns nothing in this phase — its accuracy stays at chance level. Yet a model that has gone through this apparently pointless warm-up shows a statistically significant drop in calibration error during real training (P<10⁻³), and gains the ability to lower its confidence on unfamiliar inputs: the metacognition to say "I don't know what I don't know."

What Pebblous finds compelling here is the shift in framing. Is trustworthy AI purely a matter of cleaner data, or is it a matter of the training recipe? Seeing calibration as a question of training order rather than data reminds us that data quality and training design are, in the end, two halves of the same body.

Key Figures

Four numbers compress the finding: the statistical evidence that overconfidence exists before any data is seen, how far that bias fades after the noise warm-up, the accuracy that reveals the warm-up's paradox, and the calibration gain confirmed in real training.

Sources: Nature Machine Intelligence (2026), arXiv:2412.17411

P<10⁻³

Overconfidence before training

Class bias in a network that has seen no data (one-way ANOVA)

P=0.429

Bias dissolved

Class bias disappears statistically after the noise warm-up

Chance level

The warm-up's paradox

Loss falls during noise training, yet accuracy stays at chance

P<10⁻³

ECE & OOD gains

Lower calibration error and better detection of unfamiliar inputs, both significant

1

Confidently Wrong

Anyone who has worked with AI knows the scene. The model returns an answer that is plainly wrong, and the confidence score attached to it reads 99%. It misreads a medical image with conviction, mistakes an unseen road situation for a familiar one, treats an unfamiliar pattern as if it knows it. The problem is not being wrong; it is being wrong without lowering its confidence. What we really want is not an AI that never errs, but one that tells us when it doesn't know.

The measure of this mismatch is calibration. A well-calibrated model should be right about 80% of the time among all the predictions it labels "80% confident." How badly that promise is broken, summed into a single number, is the Expected Calibration Error (ECE). A large ECE means the gap between stated confidence and actual accuracy is wide — and that gap usually leans toward overconfidence.

Accuracy and calibration are different abilities. Accuracy measures how often a model is right; calibration measures how honest its confidence is. However accurate a model is, if its confidence is dishonest, the decisions we make trusting that confidence become dangerous. A trustworthy AI is, in the end, one that knows its own ignorance.

2

The Culprit Was Initialization, Not Data

So where does overconfidence come from? The usual answer is the data: biased data, insufficient data, dirty data taught the model wrong. But before suspecting the data, the authors looked one step earlier — at random initialization, the stage where a network's weights are filled with random numbers before any learning starts. Nearly every deep learning model today begins here, and we take the practice so much for granted that we never think to question it.

What they found was unexpected. When they examined the output of a freshly initialized network that had seen no data at all, confidence was already skewed toward particular classes. With nothing learned, the network still favored some answers over others. When the authors tested this output distribution with a one-way ANOVA, the bias between classes was statistically clear (P<10⁻³). Confidence that already exists at zero bits of knowledge — this was the seed of overconfidence.

Confidence distribution before seeing any data Random init only skewed P<10⁻³ · clear class bias After noise warm-up even P=0.429 · bias gone Overconfidence was etched into initialization before training; the noise warm-up erases it (Nature Machine Intelligence, 2026)
▲ A network's confidence bias before training, and its change after the noise warm-up — Pebblous original diagram | Source: Cheon & Paik (2026)

More importantly, this seed does not simply vanish. As training proceeds the model learns from data, but the tilt of confidence etched into its starting point is rarely corrected and survives into the final model. That is why even a well-trained model can be groundlessly confident. If we see overconfidence only as a data problem, this starting-line bias stays invisible, and we end up cleaning the wrong place.

3

A Noise Warm-Up Before Training

If the starting line is the problem, the remedy has to live at the starting line too. The authors' prescription is to run the network through meaningless random noise once, before training on any real data begins. The warm-up is simple, in five steps. The inputs are random images made of Gaussian noise (mean 0, standard deviation 1); the targets are labels drawn at random each time; and there is no rule connecting input to target. Noise and label are resampled at every iteration, so no pattern exists for the model to memorize. On top of that, it simply minimizes the usual cross-entropy loss.

Noise warm-up random input · random label Loss ↓ · accuracy at chance learns nothing, but bias erased Train on real data starts from lower loss One warm-up, before training begins The paradox: a stage that learns nothing erases the starting-line overconfidence, lifting calibration later. Source: Cheon & Paik, Nature Machine Intelligence (2026)
▲ The flow of the noise pretraining protocol — Pebblous original diagram | Source: Cheon & Paik (2026)

Here lies the paradox of the method. During the noise phase, loss falls steadily, but accuracy stays at chance level to the end. Of course it does — there is no rule to learn. The model learns nothing. And yet that very waste of effort scatters the tilt of confidence etched into initialization. Test the class bias again after the warm-up and you get P=0.429: real training begins from a state where the bias has statistically vanished.

This is where the approach differs fundamentally from existing calibration methods. Most widely used methods correct confidence after the fact — once training is over, or at the moment of inference. They fix the result while leaving the bias at the starting line untouched. The noise warm-up, by contrast, erases the bias before training begins, right where the cause is born.

Method When it acts Trait and cost
Temperature scaling After training (post-hoc) Divides logits by a temperature to soften confidence. Needs a separate validation set and a hyperparameter.
Label smoothing During training Softens the target labels. Set the strength wrong and it tips into underconfidence instead.
MC dropout At inference Runs inference several times to estimate uncertainty. Multiplies the inference cost.
Noise pretraining At initialization Once, before training starts. No post-processing; just one added step in the training routine.
4

What the Numbers Confirmed

However plausible an idea sounds, without numbers behind it, it stays a story. The authors ran experiments on a subset of CIFAR-10 (4,000 images) with a six-layer network. Models that went through the noise warm-up saw a statistically significant drop in calibration error (Wilcoxon rank-sum test, P<10⁻³), and the effect held consistently across depths from two to six layers and across dataset sizes. In other words, it was not a lucky result in one particular setup.

Metacognition showed up most clearly when the model met unfamiliar input. In an out-of-distribution (OOD) test — training on CIFAR-10 and then showing the model entirely different SVHN images — the model without a warm-up kept high confidence even on inputs it had never seen. The model that had gone through the warm-up, by contrast, dropped its confidence to chance level on such unfamiliar inputs (P<10⁻³). AUROC, which quantifies the ability to tell the known from the unknown, rose significantly as well. In a two-dimensional toy model, drawing a confidence map makes the difference visible: what had been confidence clustered in particular regions before the warm-up spread out into an even, low confidence across the whole input space afterward — the model grew humble where it did not know.

The first question a practitioner usually asks is whether this costs accuracy. It did not. Models that went through the warm-up started training from a lower loss, and their final accuracy was equal or even higher. The familiar dilemma of trading calibration against accuracy did not appear here. That making confidence honest did not eat into performance is exactly what makes this method attractive.

The limits are clear too. The experiments are confined to relatively simple image classification and small networks, and whether the same effect holds in larger systems such as large language models has not yet been directly verified, owing to computational cost. The authors expect the principle to generalize to any model that outputs a probability distribution, but that remains an unverified hypothesis. The size of the finding is matched by the size of what is left to verify.

5

What the Brain Did Before Birth

Where did this idea come from? The authors drew their inspiration from an unexpected place: the fetal brain. The biological brain begins to act on its own long before it opens its eyes to the world, from a fetal stage when it has had no sensory experience at all. This fetal spontaneous neural activity, observed in the visual and auditory cortices, statistically resembles random noise. That seemingly meaningless firing turns out to be a warm-up that tunes the neural circuits in advance, before any sense arrives.

Noise pretraining in an artificial network mimics exactly this fetal warm-up. The authors see this resemblance as more than a metaphor: it may be one explanation for how the biological brain develops the ability to tell the known from the unknown, that is, metacognition. Perhaps the brain, by first playing on its own before it ever sees the world, sets aside a place for the sense of not knowing.

Editor's Note. What Pebblous keeps its eyes on in this study is the shift in framing. When we talk about AI overconfidence, our first instinct is usually to gather cleaner, more abundant data. Yet this study shows that with the same data, simply changing the order of training can change how honest a model is. Data quality and the training recipe are not competing options but a pair that travels together. To build trustworthy AI, we have to ask not only what we feed the model, but in what order we feed it.

We have long read the confidently wrong AI as a fault of the data. This study moves that gaze to the starting line before training, a place no one had thought to examine. The next time a model answers you with self-assurance, it is worth asking once whether that confidence really came from learning, or was a habit it carried from birth. Thank you for reading to the end.

Pebblous Data Communication Team
July 8, 2026

R

References

R.1Academic Papers

R.2Background