Executive Summary

Key Figures

The first two describe the size of the board this benchmark lays out; the last two are the scores that came off it.

Source: On the Selectivity of Generative Models in Structure-Based Drug Design (arXiv:2610.24124, October 16, 2026)

Structure-based drug design means looking into the pocket of a protein that drives a disease and sketching a small molecule that fits it. It is closer to studying the keyhole first and then cutting the key. These days a generative model does the cutting. Feed it the three-dimensional shape of a protein pocket and it draws, atom by atom, a molecule that might sit in there.

The trouble is that the key opens other doors too. A human body holds tens of thousands of proteins, and some of them have pockets shaped much like the one being aimed at. If the key that was cut also slides into those lookalike pockets, the drug does what it was asked to do and then some. A good share of what we call side effects starts there.

Side effects are not a minor blemish. Nine out of ten drug candidates that enter clinical trials never reach approval, and a 2022 review of clinical trial data from 2010 to 2017 attributes roughly 30% of those failures to toxicity that could not be managed. That is the second largest share, behind the 40% to 50% that fall short on efficacy. Whether a molecule stays off everything else matters about as much as how well it binds to its target.

Yet generative models have rarely been asked that second question. The paper's diagnosis is plain: most generative models still optimize binding to a single target and disregard selectivity, because training data is limited and off-target interactions are unknown. Models get good at what they are asked for. Nothing has asked them for selectivity.

SELECTBENCH turned that blank into a task. It split selectivity, one lump of a concept, into three situations with different characters, and gave each its own ruler. The three are not arranged by difficulty; each takes on a different scene that comes up in practice.

▲ The three evaluation tasks in SELECTBENCH. Source: adapted from arXiv:2610.24124

The five pairs in the first task are not merely lookalike proteins gathered together. The paper's table records, for each pair, which side effect follows. Next to the cancer target PARP1 sits PARP2 and hematological toxicity; next to the antiviral target BMP2K sits TTK, with hematological toxicity and neutropenia; CDK2, which both the cancer targets CDK9 and MAPK1 keep as a neighbor, brings hematological toxicity and gastrointestinal problems; next to the neuropsychiatric target DRD2 sits HRH1, with hypertension and neutropenia. The side effects of Section 1 turn here into specific places with protein names on them.

The number 85 in the second task has a story behind it as well. The base is Brennan-77, a public panel linking drug targets to adverse events. Some targets have to be examined both when blocked and when switched on, so holding 77 targets took 85 protein structures. Of those, 22 structures correspond to 17 targets graded high-risk, and the scoreboard the paper puts in its main text is based on that priority set. On the other side stand five on-targets: PARP1, FA7, 5HT1B, BRAF and JAK2.

The difficulty distribution of the third task is worth a look on its own. Of the 9,900 pairs, 9,834 are easy, 38 are medium and 28 are hard. The ruler is the Z-score from a tool called ProBiS, which measures how much two binding sites overlap, with higher values meaning more alike and therefore harder. The places that are genuinely hard to tell apart make up less than 1% of the whole. Scaling the set up brings easy problems along with it. Below, those three slices are drawn at their true proportions.

This picture should not be read as the makeup of the scoreboard, though. The paper did not run all 9,834 easy pairs. A footnote records that, for reasons of compute, it sampled five non-overlapping subsets of 20 pairs, 100 pairs in total. The 38 medium and 28 hard pairs were all run. Easy problems were kept by hand from dragging the average down by sheer headcount, which read the other way means that of the 9,834 easy pairs collected, only 100 were put to use.

The researchers set up two judges. One axis is AutoDock Vina, a docking program that fits a molecule into a protein by computer and returns a score; the other is Boltz-2, a model that predicts the folded shape of a protein and molecule together and reports a binding probability. The design is meant to show whether two judges of different temperament point the same way. The metrics build on the Δ-score, the gap between the on-target and off-target scores, and from there the paper reads the selectivity rate, which is the fraction of molecules whose gap clears a set margin; the top-20% ratio, the fraction of molecules whose on-target score ranks in the top fifth of all scores across the target and the panel; and a stricter worst-case rate, the fraction of molecules that clear the margin against every off-target in the panel at once.

The scores do not enter those metrics raw. Each pocket produces docking scores on its own range, so the researchers first docked a reference set of 128 drug-like ligands into each target, measured that pocket's mean and spread, and converted the scores onto that footing. Without the conversion, the mean Δ-score fails to pick up even the direction of selectivity, and only picks it up afterward, which the paper demonstrated by feeding in molecules already known to be selective. A warning that the choice of ruler can flip the conclusion is built into the benchmark itself.

Nine models took the test. Five belong to the diffusion and flow family, which blurs a molecule and then draws it back by reversing the blur; two are autoregressive, adding one atom at a time; and two were trained with selectivity in their objective from the start. The first seven were never built with selectivity in mind, so there is nothing strange about them doing poorly. The last two are a different matter. How far those two land from the rest is the question this benchmark exists to ask.

▲ The nine generative models evaluated by SELECTBENCH. Source: adapted from arXiv:2610.24124

Start with the contest between families and the result is fairly clear. The paper concludes that diffusion and flow models generally outperform autoregressive approaches in selectivity. The reason it offers is molecular size. Pocket2Mol and PocketFlow, which place atoms one at a time, consistently produce smaller molecules, and a smaller molecule slips into this pocket and that one alike. A habit that was no flaw when binding was all that counted turns into a liability once the request is to bind selectively.

The answer is less crisp than one might hope. The paper reports that the methods explicitly designed for selectivity show mixed results: they sometimes improve on the standard baselines, but performance varies widely from target to target. Taken across all regimes, the sentence that selectivity-focused methods are largely on par with standard baselines with no consistent improvement sits right there in the findings. The average is not the whole picture, though. The same passage notes that highly selective candidates occasionally appear among the top-10. They are not steady, but they are not absent either.

Where the models do well and where they fall apart also split. In settings where the target and the off-targets sit far from each other, such as the safety panel of 85 adverse-reaction proteins, selectivity training helped. Facing a pair like PARP1 and PARP2, relatives whose pockets are nearly identical, that help vanished. The places where human designers struggle most and a model's help would count for the most are exactly the places where the scores collapse.

Part of why the safety panel looks better is that its bar is looser. The selectivity rate used there averages a molecule against the panel as a whole, so it does not demand that every listed protein be dodged. By that measure several on-targets clear half, among them PARP1, BRAF and 5HT1B. Switch the condition to missing not a single protein in the panel, and in the paper's own words success rates remain near zero for all methods. The second question is the one safety screening actually asks in practice.

Retraining is expensive. So practitioners often leave a trained model alone and steer it only at the moment a molecule comes out. The researchers tried two such methods on three diffusion baselines: DiffSBDD, DecompDiff and TargetDiff. One is classifier-free guidance, which pulls generation harder toward the desired condition. The other is Feynman-Kac correction, which floats several molecules at once and, partway through, discards the ones scoring badly while copying the ones scoring well. The second is closer to casting a wide net and hauling it in.

It was not that nothing happened. The paper reports that these techniques generally lead to modest improvements. But the sentence right after them stands in for this section's title: despite those gains, the fraction of molecules passing the strictest bar remains low, rarely exceeding 50%. The strict bar here is the line where the on-target score has to beat the off-target by one standard deviation to count as a pass. The spread of those 128 reference ligands in that pocket is what sets the length of one step. Falling short of half means the molecules that come out cannot be carried straight into the next stage.

There is a place where these techniques earn their keep. Break the large-scale task down by difficulty and the effect of guidance is clearer in the medium and hard regimes and fainter on easy, because easy pairs are already separable and leave little headroom to push. So this is not a verdict that the techniques are useless. It is that the selectivity rate decreases monotonically from easy to hard without ever recovering, and that on the safety panel the top-20% figures typically stay below 0.4. Reliable safety-aware generation remains an open problem, is what the paper writes at that point.

The limitations the researchers set down themselves belong in the same reading. The current tasks do not account for allosteric effects, where a molecule binds somewhere other than the active site and changes the protein's shape. Expanding the synthetic task to cover a larger portion of CrossDocked2020 would improve scale and diversity, the paper says, which touches on the fact that the hard band holds only 28 pairs. Adding more scoring functions, and going as far as checking whether the three-dimensional pose a molecule settles into makes physical sense, are left as future work.

Reading this result as the models' fault is reading half of it. There is a link further up the chain behind their failure to handle selectivity. A record that some molecule bound some protein becomes a paper and enters a database when the experiment succeeds. The fact that it did not bind usually gets written down nowhere. So data on off-target interactions runs short, and short data means no scorecard, and no scorecard means nothing to set as a training objective.

Data gathered without a use in mind will not score well on a use. Binding records were collected to predict binding, not to sort out safety. Even when they hold the same molecules and the same proteins, a difference in what they were gathered to ask changes the range of questions they can answer. That difference is invisible as long as data is counted only by volume.

The most valuable thing SELECTBENCH did is not the scoring but the scorecard. Once a task and a metric stand in a place that was empty, there is suddenly a reason to collect the data that fills it. The task also tells you which data to collect more of. That easy pairs already number 9,834 while what is really wanted is the 28 hard ones is visible only because the task was laid out by difficulty in the first place.

Getting data into a usable state does not end with filling in gaps and tidying labels. Writing down what question the data was gathered to answer comes before that. Drug design is simply a field where the gap opens unusually wide; the gap itself exists everywhere.

Thank you for reading this far. The facts here were checked against the paper published on arXiv and its evaluation setup, along with a public review of why drug candidates fail in the clinic. If you have run into a wall trying to put data gathered for one purpose to work on another question, we would be glad to hear about it.

Pebblous Data Communication Team
October 21, 2026

Academic Papers

• 1.Rajaonson, E.M., Lee, J., Chau, W., Aspuru-Guzik, A., Sanchez-Lengeling, B., Beaini, D., Neklyudov, K., Skreta, M. (2026). "On the Selectivity of Generative Models in Structure-Based Drug Design." arXiv:2610.24124.
• 2.Sun, D., Gao, W., Hu, H., Zhou, S. (2022). "Why 90% of clinical drug development fails and how to improve it?" Acta Pharmaceutica Sinica B, 12(7), 3049–3062.
• 3.Francoeur, P.G., Masuda, T., Sunseri, J., Jia, A., Iovanisci, R.B., Snyder, I., Koes, D.R. (2020). "Three-Dimensional Convolutional Neural Networks and a Cross-Docked Data Set for Structure-Based Drug Design." Journal of Chemical Information and Modeling, 60(9), 4200–4215.

Official Resources & Industry Coverage

• 4.lovetaro. "SelectBench." GitHub (benchmark code and data repository).
• 5.ASBMB Today (2022). "90% of drugs fail clinical trials." (summarizes Sun et al. 2022)