Executive Summary

Generative AI builds well only where its training data is thick, among compositions that are already plausible. The unexplored regions where new materials might hide are empty of data, so the model assigns them low probability and keeps steering around them. Hyunsoo Park and Aron Walsh of Imperial College London filled that gap not with more data but with a reward. This article looks at how they did it and what came of it.

The core move was to attach to a crystal-structure generative model a verifiable reward that scores creativity, stability, and diversity at once. As a result, the share of structures that are simultaneously metastable, unique, and novel (mSUN) rose from 15.9% to 61.3%, overtaking even MatterGen (41.0%), the leader until then. One diversity metric dipped slightly, a trace of a design that balances several objectives rather than maximizing a single number.

The question this study raises runs well beyond materials science. To make a model explore where good data simply does not exist, what fills the absence of data? This paper's answer is reward design — defining the goal as if it were data.

15.9→61.3%

mSUN surge

Roughly 4× from the same base model, adding only the RL reward

41.0%

Best prior mark passed

MatterGen, the leader until then, was overtaken

≈0%

Collapse of prior guidance

Push the band gap with CFG and mSUN falls to near zero

97.5%

Novelty

Up from 62.3% — aimed at structures absent from the data

1

Why Generative Models Build Only Where Probability Is High

Generative AI such as diffusion models has learned to produce chemically plausible compositions and structures. But at bottom these models are optimized to reproduce whatever is high-probability in the training distribution. High probability means plenty of data nearby, and plenty of data nearby means many already-known or closely related compounds.

The trouble is that the places where new materials might live are exactly where the data is thin. No one has been there yet, so the training set is empty and the model assigns those regions low probability. Sampling by likelihood and the real goal of searching where no one has gone pull in opposite directions. The paper calls this mismatch an objective misalignment.

Force the model toward novelty and it hits a different wall. The harder you steer toward diversity, the more it pours out structures that are thermodynamically unstable or physically nonsensical. Prioritize stability alone and it circles the compositions it already knows. This is the dilemma in which novelty and stability trip each other up. The concept diagram below summarizes that terrain.

The likelihood landscape of chemical space — why new materials get skipped Probability (likelihood) Chemical space (composition · structure) Where data is thick = already-known compositions Unexplored novel-materials region Thin data · low probability The reward pushes this way Follow likelihood and you stay on the left peak; the right-hand valley keeps getting skipped.

Figure 1. The likelihood landscape of chemical space. Generative samples pile up on the data-thick peak, while the valley where new materials hide is skipped for its low probability. Original Pebblous diagram.

2

Redefining the Goal as a Reward

The researchers did not start by gathering more data. If you can score what counts as a good result by rule even where there is no data, that scoring function stands in for the missing data. So they attached a verifiable reward to the crystal-structure generative model Chemeleon2 and refined it with reinforcement learning.

The algorithm they used is GRPO (Group Relative Policy Optimisation), a technique from reinforcement learning for large language models, carried over to crystal-structure generation. The point is that it needs neither human-labeled preference data nor a separately trained reward model. What makes a good structure is verified directly against physical and chemical rules, and those scores are compared relatively within a group before being fed back into the policy.

The reward looks at three axes at once: how stable, how novel, and how little the structures overlap one another. Stability is measured by the energy above the convex hull (E_hull), but the bar is a relaxed metastable one (roughly E_hull ≤ 0.1 eV/atom) rather than fully stable (E_hull ≤ 0), so that structures worth attempting in the lab are included. Novelty is scored by whether a structure overlaps anything already in the reference database used for training, and diversity by whether the generated structures within a batch duplicate one another.

The metric that ties these three axes together is mSUN, the paper's evaluation criterion. It denotes the share of structures that are simultaneously Metastable, Unique, and Novel, in effect a relaxed version of SUN, the earlier standard metric that demanded full stability. The diagram below lays out what the reward is built from and how it merges into a single signal.

A verifiable multi-objective reward — three axes into one signal Stability E_hull ≤ 0.1 eV/atom (metastable) Novelty Not in the reference DB Diversity No duplicates within a batch mSUN reward metastable · unique · novel at once GRPO policy update no trained reward model verified directly by rule Not data but a scoring rule defines "what a good structure is."

Figure 2. Composition of the verifiable multi-objective reward. Stability, novelty, and diversity are scored by rule, merged into the mSUN signal, and GRPO updates the policy. Original Pebblous diagram.

3

Reward Alone Lifted mSUN From 15.9% to 61.3%

The effect was large. Generating 10,000 structures on the Alex-MP-20 benchmark and re-measuring, the same base model's mSUN jumped from 15.9% to 61.3%. The architecture was not changed; reward-based reinforcement learning was simply layered on top. What deserves attention is the starting point. Before reinforcement learning, Chemeleon2's 15.9% was lower than its predecessor Chemeleon1 (32.3%) and lower still than MatterGen (41.0%), the leader until then. This was not the best-performing model being tuned; a model that started from the bottom climbed past both benchmarks to 61.3% on the reward alone. The chart below places all four models' mSUN side by side.

mSUN comparison — Alex-MP-20, 10,000 generated structures 0% 41.0% MatterGen prior best baseline 32.3% Chemeleon1 15.9% Chemeleon2 before RL (base) 61.3% Chemeleon2+RL this paper Figures per the arXiv:2511.07158 supplementary table (cross-checked on the web). Minor rounding possible.

Figure 3. mSUN comparison on the Alex-MP-20 benchmark. Chemeleon2+RL, with reward-based reinforcement learning layered on, surpasses even the prior best baseline MatterGen. Source: reconstructed by Pebblous from Park & Walsh (arXiv:2511.07158).

Break the numbers apart and the balancing act shows. After reinforcement learning, uniqueness, a measure of how little the structures overlap, slipped from 99.4% to 88.7%. In exchange, novelty rose from 62.3% to 97.5% and metastability from 51.2% to 72.1%. Maximizing a single metric would not produce a trade-off like this. It is the signature of multi-objective optimization pushing several goals up together.

What matters is the source of the gain. Not a bigger model, not more data. Just a reward, attached to the same base model, that scores what makes a good structure. What lifted performance was not the volume of data but the definition of the goal.

4

Order a 3 eV Band Gap and the Structure Still Holds

A more practical question is this: can you specify a desired function and have materials made to order? The researchers had the model generate structures aimed at a specific property, a band gap of 3 eV. In semiconductor and photonic design the band gap is the value that governs a device's character, so this kind of inverse design is a flagship task for materials AI.

The comparison was against CFG (Classifier-Free Guidance), the standard conditional-guidance technique for diffusion models. Computing the band gaps of 512 generated structures and plotting the distribution, the CFG family spread broadly from 0 to 5 eV and formed only a weak cluster near 3 eV. It failed to aim precisely at the target. Reward-based reinforcement learning built a distinct peak around 3 eV.

The more decisive difference came down to structural validity. Applying CFG alone collapsed mSUN to near zero, and even layering on LoRA to preserve the pretraining prior improved it by less than 3%. The harder it was pushed in the desired direction, the more it fabricated nonsense. Reward-based reinforcement learning, by contrast, hit the target property while also keeping chemically sensible, stable structures intact.

Ordering a 3 eV band gap — outcome by guidance method Prior method · CFG target 3 eV spreads broadly, 0–5 eV mSUN ≈ 0 (structures collapse) Reward-based RL target 3 eV distinct peak at 3 eV hits target + keeps validity

Figure 4. Comparison of 3 eV band-gap target design. CFG spreads its distribution wide and its structures collapse, while reward-based reinforcement learning builds a peak on the target and preserves validity. Source: reconstructed by Pebblous from Park & Walsh (arXiv:2511.07158).

5

Filling the Absence of Data with Reward Design

The problem this study addressed sits at the opposite extreme of the usual data-quality conversation. Ordinarily we clean and validate data before feeding it to a model. Here the question was how to handle a region where there is no data to feed in the first place. The answer was not to gather more data but to define what counts as good through verifiable rules.

That way of thinking is not confined to materials science. In any domain that has to explore where good training data is thin or absent (drug-candidate discovery, say, or R&D areas where cases have not yet accumulated), the same question returns. What fills the absence of data? This paper's answer is reward design: defining the goal as if it were data.

A piece on the Pebblous blog treats the same root from a different angle. The provenance problem of the data that verifiable-reward reinforcement learning learns from was about how no one can trace, atom by atom, the origin of what RL learns. This article looks at the other side: how a reward fills a place where there is no data at all. Seen more broadly, it connects to a survey of how AI is changing the questions science asks. The three are different vertices of the same triangle — generative AI, reinforcement learning, and data.

Editor's Note

Pebblous is a company that has worked on AI-Ready Data, making data clean and validated enough to train on. What this paper shows is the mirror image of that problem. When data exists, you weigh its quality and lineage; when data is absent, you define the goal by rule and let that stand in its place. Writing down what counts as a good result in a verifiable form ends up using the same muscle as handling data. This paragraph is background from the editor's perspective and should be read apart from the analysis in the main text.

R

References

Academic papers

  • 1.Park, H., & Walsh, A. (2026). "Guiding Generative Models to Uncover Diverse and Novel Crystals via Reinforcement Learning." Nature Machine Intelligence. (published 2026-07-06) — primary source
  • 2.Park, H., & Walsh, A. (2025). "Guiding Generative Models to Uncover Diverse and Novel Crystals via Reinforcement Learning." arXiv:2511.07158. — preprint (supplementary table S1 figures)
  • 3.Park, H., Onwuli, A., & Walsh, A. (2025). "Chemeleon: A text-guided generative model for crystal structures." Nature Communications. — predecessor (Chemeleon1)

Code & datasets

  • 4.Park, H. Chemeleon2 — VAE encoding · latent diffusion · RL fine-tuning pipeline. github.com/hspark1212/chemeleon2.
  • 5.Zeni, C., et al. (2025). "MatterGen: a generative model for inorganic materials design." (comparison baseline)
  • 6.Alex-MP-20 / MP-20, benchmark sets built on the Materials Project. materialsproject.org.

Pebblous adjacent