Executive Summary

Key Numbers

Put what changed next to what it changed, and the movement sits in placement, not scale.

Source: Talaei et al., Communications AI & Computing 1, 7 (2026)

An antibody is a Y-shaped protein. The variable domains of a heavy chain and a light chain pair up and together form the surface that meets the antigen. Inside each variable domain, four framework regions that hold the structure alternate with three complementarity-determining regions that touch the antigen directly. Two chains means six of those regions in all, and they are usually shortened to CDRs. The paper puts them at roughly 20% of all residues.

The other 80% looks much the same from one antibody to the next. Which pathogen an antibody recognizes, and how tightly it holds on, is settled in the six loops. John Misasi, a co-author and assistant professor of virology, immunology, and microbiology at BU's Chobanian & Avedisian School of Medicine, compared an antibody to a screwdriver in the university's announcement. Whether the shaft is long or short does not matter; the shape of the tip decides which screw it fits, and the CDRs are that tip. He added that because these regions vary so much from one antibody to another, they are also the hardest part for AI to model.

A general protein language model knows nothing of that asymmetry. It trains as though every residue were equally important. For most proteins that is a sound assumption. Biologically important information is spread across the molecule, so hiding residues anywhere and asking the model to restore them still surfaces the patterns of structure and function. Antibodies are the exception where the assumption breaks.

The break shows up in numbers. Ask the publicly released ESM2-3B model to recover masked residues and it gets 72% to 92% right in the framework regions, but only 35.69% in HCDR3, the third loop of the heavy chain, and 46.06% in LCDR3 on the light chain. The positions the model handles well and the positions that decide what an antibody is are precisely out of alignment. HCDR3 is hard for a reason: it is assembled by stitching three kinds of gene segment together at random, which makes it far more diverse than the other loops, which join only two.

There is a second misalignment. General models like ESM2 and ESM C are pretrained on single-chain sequences. How a heavy chain and a light chain interlock to build a binding site was never in the training signal to begin with.

What the researchers changed is not the grammar of the training objective but a single line of masking policy. The standard recipe draws 15% of the residues at random from the whole sequence, then replaces 80% of them with a mask token, swaps 10% for a different amino acid, and leaves 10% untouched. The new recipe narrows the draw to the six loops and masks 50% of what is inside them. The 80/10/10 treatment stays exactly as it was.

The reasoning behind that 50% is the heart of the study. The paper states that the rate was chosen empirically to match the number of masked residues in the 15% whole-chain scheme, so that the comparison would be fair. The six loops are about 20% of the sequence, so half of them is a little over 10% of the whole. The number of problems the model has to solve is therefore nearly identical across the two schemes, and the only thing that differs is where in the sequence those problems sit. Nothing extra was spent; the same resource was placed somewhere else.

The training data came from OAS, the Observed Antibody Space database of antibody sequences. The team kept only human sequences, dropped the repertoires of patients with autoimmune disease, then clustered what remained at 95% sequence identity and kept one representative per cluster. That left 1,617,948 paired heavy and light chains. The boundaries of the six loops came straight from the IMGT annotations OAS already carries. To say which positions get masked more often, those positions have to be named, and the names were already in the data.

Three masking regimes went head to head: 15% across the whole chain, 50% inside the loops, and a hybrid that gives 80% of each batch the loop treatment and 20% the whole-chain treatment. The hybrid was there because masking only the loops risks having the model forget the framework regions. That worry turned out to be real. Push loop-only training long enough and framework recovery starts to slip, which thins out the context the model can lean on when it predicts a loop.

So the team wrote a separate rule for choosing a checkpoint. Take the earliest epoch with the highest average loop accuracy, subject to average framework accuracy dropping no more than 0.1 percentage points from the previous stage. It stops at a balance point between the two regions rather than pushing either to its maximum, and because it never consults downstream performance, it does not lean on the evaluation data. The two checkpoints this rule selected are AbCDR-ESM2, built on the 3-billion-parameter ESM2, and AbCDR-ESMC, built on the 600-million-parameter ESM C.

Splitting the stages apart shows where the gain came from. Before any loop masking, Stage I alone, which is nothing more than whole-chain adaptation on paired data, already lifted average loop recovery by 35 percentage points to 85.65%. Even HCDR3, the hardest of them, went from 35.69% to 62.38%. Simply showing the model paired human antibody sequences recovered much of the ground, and loop masking is the layer that goes on top of that.

What that layer adds appears first in recovery accuracy. On a test that masks 50% inside the loops, the model trained this way reached 74.87% on the heavy-chain loops and 89.88% on the light-chain loops, the highest across every loop region. The authors note, though, that IgBERT and IgT5 were tested on splits different from their own, which makes the comparison approximate, and that the baseline values measured here should be read as an upper bound favorable to those models.

The hybrid strategy, the safeguard, ends up pointing the same way. It did protect framework accuracy while nudging the loops up a little, HCDR3 from 62.38% to 63.09%, but loop-only training lost just 0.10 percentage points of framework accuracy, so there was not much left to protect. Move to binding affinity and the gap sharpens: the loop-only model beat the hybrid on three of six datasets and matched it within standard error on the other three. For ESM C, no hybrid variant was built at all.

Recovering masked amino acids well is no guarantee of predicting binding well, so the researchers checked the connection separately. They turned antibody sequences into embeddings with the fine-tuned language model, averaged the vectors across every position, and put a single ridge regression on top to predict the logarithm of the dissociation constant. The language model stays frozen and only the linear regression is fitted, so what separates the runs is not the strength of the predictor but the quality of the embedding itself.

Evaluation came in two kinds. One is three single-mutant datasets, where a single amino acid position is changed: G6, which targets vascular endothelial growth factor, D44, which targets hen egg-white lysozyme, and trastuzumab, which targets HER2. The other is three combinatorial datasets, where several positions change at once, targeting the HR2 region of the SARS-CoV-2 spike protein, fluorescein, and H1 hemagglutinin. Taken together the six datasets hold more than 90,000 antibody variants across six antigens.

What these numbers weigh depends on how they were measured. The single-mutant sets reused the data splits from the IgBERT and IgT5 paper and were scored by ten-fold cross-validation; the combinatorial sets were scored by averaging 40 repetitions of a random 90/10 split. The regularization strength of the ridge regression was chosen separately inside each training fold by a nested five-fold cross-validation, which keeps the test data from leaking into that choice. The team also reproduced the published figures of AntiBERTy and AbLang through their own pipeline and confirmed that the values lined up.

On the single-mutant sets, adapting a base model to antibody data paid off consistently. For the ESM2 line, R² rose from 0.302 to 0.359 on D44, from 0.264 to 0.298 on G6, and from 0.335 to 0.350 on trastuzumab.

The smaller model is the more striking half of the story. The 600-million-parameter AbCDR-ESMC scored 0.313 on G6, the highest of everything compared, and 0.345 on D44, second best. Those figures put it 16% ahead of the 3-billion-parameter IgT5 on D44 and 25% ahead on G6. It did that with one fifth the parameters and with no unpaired pretraining, fine-tuned on paired data alone.

The combinatorial sets moved further. On anti-fluorescein, AbCDR-ESM2 lifted R² from 0.547 to 0.693, a 26.6% relative gain over the base model and 30.5% over AbLang's 0.531, the best published model on that set. This is where the abstract's figure of up to 27% comes from. The authors read it as a consequence of the mutations in this dataset spanning several loops, which lets the model apply the correlations between loops that it picked up while masking 50% of them. On the largest set, anti-HR2 SARS-CoV-2 with 71,830 variants, R² reached 0.396, 8.8% ahead of IgT5, and mean absolute error fell from 0.865 to 0.841.

Three qualifiers belong next to those numbers. First, the authors write that on this task an R² between 0.3 and 0.4 counts as strong performance, because binding measurements are noisy in themselves. Turned around, that means even the best model leaves more than half the variance unexplained. Second, it did not win everywhere. On H1 hemagglutinin, whole-chain masking beat every loop-masking strategy, and on trastuzumab the strategies were indistinguishable while AbLang2 alone came out ahead at 0.460. The authors point out that these two sets, at 1,038 and 422 measurements, are small enough to make their means hard to trust.

Third, the headline figure of up to 27% belongs to the 3-billion-parameter model on a combinatorial set. What the 600-million-parameter model achieved sits elsewhere, not in the size of the improvement but in matching or exceeding much larger antibody-specific models at one fifth the scale. Folding the two into a single claim, that the 600M model improved prediction by 27%, states something the paper does not.

OAS holds more than two billion single-chain sequences with no pairing information and around two million sequences where heavy and light chains are paired. Strip out the near-duplicates and 1,220,072,064 representatives remain on the unpaired side against 1,617,948 pairs on the other. With 750 times more data sitting on one side, the customary order in this field has been to pretrain there and finish on the paired set. IgBERT and IgT5 were built that way.

The researchers pulled that step out and compared. Pretraining ESM2-3B on unpaired sequences for a single epoch took more than 300 GPU-hours on 32 A100s, and they then fine-tuned two models on the same paired data, one starting from that checkpoint and one starting straight from the publicly released base.

Measured right after unpaired pretraining, the model was clearly better than the base across every region, which says single-chain data does carry information. But once paired fine-tuning finished, the two paths converged. The paired-only model matched or slightly exceeded the other, and no benefit from unpaired pretraining showed up in binding affinity prediction either. Three hundred GPU-hours and a whole pipeline stage failed to pay for themselves.

The authors explain the result as representation drift. Training on single chains tilts a model toward framework regularities and the frequency biases that come from germline genes. Those statistics describe a repertoire well, but they sit at an angle to the binding that arises when two chains interlock. Data at scale, in other words, can pull a model away from the answer. The authors add that in IgBERT and IgT5 too, the unpaired-to-paired route does not improve consistently from benchmark to benchmark.

In self-supervised learning, deciding what to mask is deciding what problem the model will be asked to solve. The answers already sit in the raw data, so no new labels have to be made, but which positions become questions is a human decision. Domain knowledge enters through that decision, which makes it closer to label design than it looks. This study changed one such decision and came out with a model five times smaller and better results.

Diane Joseph-McCarthy, the study's principal investigator and executive director of BU's Bioengineering Technology & Entrepreneurship Center, said in the announcement that rather than treating antibodies like generic proteins, the team designed an antibody-specific language model that learns the fundamental patterns in the regions responsible for antigen binding. Ioannis Paschalidis, a co-author and director of the Hariri Institute for Computing, put the starting point more directly: a great deal of AI research has gone into building larger models, and they asked a different question, which was how to teach the model the biology that matters most. Among the exciting findings, he said, was that they needed neither a larger model nor vastly more data, and he noted that the same thing has been seen in human-language AI, where smaller domain-specific models trained on high-quality data often outperform much larger and more general ones.

Even in work that has nothing to do with antibodies, this design carries over as three questions.

5.1Is the Place Where Signal Concentrates Already Named?

This design was possible because OAS had already annotated the boundaries of the six loops under the IMGT scheme. Without that annotation there would have been no way to specify which positions to mask more often. Knowing where the dense 20% of your own data lies and having that boundary written into the data as a field are two different things. The first lives in a domain expert's head; the second is what lets a training pipeline actually use the knowledge.

5.2Have You Rearranged the Budget Before Raising It?

The 50% rate was not set high to chase performance. It was set to match the number of residues the existing scheme masks. That is what makes the comparison a question of placement rather than of scale. Experiments that gather more data or add parameters give results whose cause is hard to isolate, while an experiment that fixes the total and changes only the distribution leaves the cause in plain view.

5.3Are You Discarding the Small Set That Holds the Structure?

Paired sequences number one for every 750 unpaired ones, yet the information about how the two chains interlock exists only there. And scale did not fill that information in. Every dataset has a part that is expensive to collect, and therefore small, but that holds the structure intact. Blend that part into the common data because it is small, and later scaling may never bring it back.

The limits the authors set down deserve equal attention. Only two models from the ESM family were evaluated, so whether the same effect appears in families with different positional encodings is unconfirmed. Only one property was measured, binding, which leaves it an open question whether loop-centered representations help with other therapeutic properties such as stability, expression, or immunogenicity. The trade that gives up framework accuracy to gain loop accuracy may work differently for those. Finding the optimal masking rate is also left as future work.

The materials for checking any of this are open. Both model checkpoints are on Hugging Face under the MIT license, the code is on GitHub, and the curated training and evaluation datasets are on Zenodo under CC-BY 4.0.

Academic Paper & Preprint

• 1.Talaei, M. et al. (2026). "Preferential CDR masking in paired antibody language models improves binding affinity prediction." Communications AI & Computing 1, 7.
• 2.Talaei, M. et al. (2025). "CDR-aware masked language models for paired antibodies enable state-of-the-art binding prediction." bioRxiv preprint.

Institutional Press & Science Media

• 3.Boston University Rafik Hariri Institute for Computing. (2026). "Teaching AI the Biology of Antibodies Speeds Drug Discovery."
• 4.Phys.org. (2026). "AI learns the biology of antibodies to speed up drug discovery."

Open Model

• 5.NOC-Lab. (2026). "NOC-Lab/AbCDR-ESMC." Hugging Face. MIT License.