Executive Summary
NVIDIA has turned chemistry lab records into an open dataset and released it alongside a model. Nemotron for Science, the science bundle unveiled on November 16 at SC26, the supercomputing conference in Chicago, contains a vision-language model, a post-training recipe and LitmusBench v0.2, a chemistry dataset of nearly 50,000 questions. The questions were built from molecular property data and records of real drug discovery work, and all of it is on Hugging Face.
The number quoted most often from the announcement is 61%. Lila Sciences, a partner, post-trained the model on laboratory feedback until its accuracy on quantum dot synthesis protocols reached that level, ahead of the 54% GPT-6 Astra scored on the same task. The party that designed and ran that comparison was neither NVIDIA nor an independent evaluator; it was Lila Sciences. The test bed behind the figure was not LitmusBench v0.2 either, but a task Lila Sciences picked for itself. No one outside has posted a score on the 50,000 open questions.
Sections 1 through 4 follow what NVIDIA announced and what the coverage reported. Section 5, which reads the release as a question about AI-Ready Data, is this article's own interpretation.
Key Numbers
Source: NVIDIA press release (November 16, 2026) and reporting by SiliconANGLE and CDO Magazine (November 17, 2026)
The first card gives the size of what was opened, and the second puts that 61% next to the score it beat. That comparison came out of a task the partner chose for itself, so it is not a score on the open questions. The third card counts the scores posted on those questions from outside, and the fourth is the cost gap NVIDIA stressed alongside the accuracy.
~50,000
Questions in LitmusBench v0.2
Drawn from molecular property data and drug discovery records
61% vs 54%
Quantum dot protocol accuracy
The model Lila Sciences post-trained, against GPT-6 Astra
0
Outside groups with a published score
The dataset is days old, so no third party has posted one
10-20x
Inference cost gap
Lower than the comparison model, by NVIDIA's account
What LitmusBench Actually Asks
Picture LitmusBench v0.2 as a chemistry quiz and you miss what it is. The raw material is experimental. How well a molecule dissolves in water, how the body absorbs and breaks it down, and other property measurements were turned into questions, together with records accumulated during real drug discovery work. The questions ask what a molecule will do rather than what is already recorded about it.
The size is nearly 50,000 questions, and they were built for two uses at once. They score models, and they were also designed to serve as post-training material for tuning a model toward scientific work. The exam paper and the textbook arrived in one package.
One choice in how it was published stands out. According to secondary coverage that went through the dataset itself, every record carries a question_license field, with CC-BY-SA-4.0 applied to questions pulled from ChEMBL and OpenMDW-1.1 to the rest. Set against the habit of stamping one license across a whole dataset and leaving it there, that is extra work. In return, anyone picking the data up can tell question by question which ones they may reuse and on what terms.
The Party That Chose the Task Also Measured the Result
Every scientific result in this announcement came from a partner. Put the three most quoted results side by side and the conditions turn out to differ. The tasks differ, the comparison targets differ, and the numbers do not mean the same thing.
| Partner | Task | Reported figure | Who measured it |
|---|---|---|---|
| Lila Sciences | Quantum dot synthesis protocol accuracy | 61%, against 54% for GPT-6 Astra on the same task. Nearly double the accuracy the same model had before the post-training | Measured it after post-training the model itself on laboratory feedback |
| Therna | Predicting mRNA protein-expression ranges | Nearly double the accuracy of frontier models, with scientists preferring its explanations in 96% of comparisons | Measured it after adapting the model itself |
| Terray Therapeutics | Extracting reactants, products and conditions from reaction diagrams | Recovered about 76% of annotated reactions on the RxnScribe test | Measured it after adapting the model itself |
▲ Partner results as given in NVIDIA's announcement and in reporting dated November 17, 2026. The three rows are different tasks and must not be summed or averaged. | Sources: SiliconANGLE, StorageNewsletter
All three converge in the rightmost column. The party that picked the task, the party that post-trained the model on its own data and the party that measured the result are the same party. OrcaRouter, which published an analysis of the release, states it plainly: "Not one of the headline numbers comes from the checkpoint as shipped." The same piece adds that "every science number in the SC26 release comes from a party with a stake in the result."
NVIDIA's press release frames Lila Sciences' 61% as the result of post-training that nearly doubled the accuracy. Those two figures sit in one sentence and measure different axes. The seven points between 61% and 54% separate two models, while the near doubling separates one model from its own state before post-training. The credit the second figure points at belongs not to the shipped model but to the feedback Lila Sciences gathered in its laboratory.
None of this is a reason to doubt the seven-point gap itself. If the protocol was built properly, the difference may well be real. What the number guarantees is narrow. It came out of a task Lila Sciences chose, through post-training Lila Sciences performed, against an opponent Lila Sciences named. How the same model handles other chemistry tasks, and what another team would get by running the same task again, nobody knows yet.
OrcaRouter treats the 10-20x gap in inference cost rather than the seven points as the more consequential figure. An accuracy gap moves when the conditions move, but paying a tenth of the price for the same work survives a change of task. The cost multiple is a figure NVIDIA supplied, exactly like the accuracy.
Verifiable Is Not the Same as Verified
Releasing the evaluation dataset alongside the model shifts this picture somewhat, and it is worth being exact about which part. The 50,000 open questions cannot be used to retrace the 61%. NVIDIA's press release never says which test bed produced that figure, and OrcaRouter describes the comparison as one run by the party that did the post-training, on a task it chose, against a competitor it named. The shift is in the scores that come next. The questions are on Hugging Face with terms of use attached to each one, so from here on, whenever someone puts a chemistry score forward, another team can run the same questions themselves. Before anyone argues about whether to believe a figure, the instrument for testing it is in hand.
That does not mean the verification is finished. What holds today is only the sentence about being verifiable. No outside group has posted a score on LitmusBench v0.2, which is unsurprising given that the dataset is only days old. Between saying something became verifiable and saying it was verified sits an entire piece of work nobody has done.
How easily a self-measured figure can inflate was demonstrated recently in this same field. On PMO, a molecular optimization benchmark, erasing the task name and the reference compound from the prompt dropped an LLM agent's summed AUC from 14.44 to 4.85, about a third of where it started. The same model was solving the same tasks, and what was printed on the exam paper accounted for two thirds of the score. That case is covered in detail in On the PMO Drug Molecule Test, Does the AI Discover or Look Up?
The bundle's model, Nemotron 3.5 Super VL, is a name Pebblous has written about once already. In October, when the model's training data was opened, we checked who had written the answers inside it (Who Wrote the Answers in Nemotron 3.5 Super VL's Open Data?), and in early November we asked where the video used to train the same company's robot model had come from (Whose Video Is NVIDIA's GR00T 2 Learning From?). In October it was training data; in November it is test data. The question is the same. Where did this data come from, and who checks it?
NVIDIA Puts a Dataset Next to the Model
This time the list of things released grew. Instead of model weights alone, an evaluation dataset and a post-training recipe came with them. Behind that sits a judgment that one model is not enough to make a difference in science. The wall a pharmaceutical or materials company hits after collecting a general-purpose model is not the size of the model but the problem of telling whether it is any use on the tasks in front of it. Hand over the exam paper and the method for tuning against it, and the wall gets lower.
Nemotron 3.5 Super VL, the model in the bundle, uses 12 billion of its 120 billion parameters at inference time by NVIDIA's account, with a context length of 1 million tokens. NVIDIA says it placed first on 7 of 11 science benchmarks and gave ChemBench 75.9 and ether0 39 alongside that. On distribution, it was optimized to run on a single DGX Spark with 128GB of memory and posted to Hugging Face at BF16 precision. The proposal amounts to running a science agent on one machine that fits on a desk.
The list of collaborators reaches past companies as well. Argonne National Laboratory put the model into a multi-agent system that designs protein-based therapeutic candidates, and KTH Royal Institute of Technology reported recovering 81% of broken molecular dynamics setups overnight on a single DGX Spark. RIKEN in Japan attached Nemotron to the inference gateway of its own supercomputing platform, and the Jülich Supercomputing Centre in Germany is on the list too.
NVIDIA opened resources to the sciences a month ago as well. In October it joined the European Union's RAISE pledge, agreeing to hand European scientists compute, models and datasets (EU RAISE Asks Companies to Stock Europe's Science With AI). The same strategy repeats at a month's interval. Being a company that sells the machines these models run on, rather than one that earns its money selling the models, explains the choice.
Why Pebblous Is Watching This Release
LitmusBench v0.2 turned records piled up in laboratories into a form that can be graded. That change matters more than the figure of 50,000. A solubility measurement for a single molecule is, on its own, one line in a log. The moment it becomes a pair consisting of the question "how well does this molecule dissolve in water" and the answer, it can teach a model and it can evaluate one. This is the work the phrase AI-Ready Data points at.
This case also shows that the work carries one further condition. Turning an accumulation into questions does not by itself produce trust. Only when outsiders can establish who built the questions and under what conditions a score was produced does a number become evidence. NVIDIA has done the first half and left the second half for later.
The same three questions can be put to the accumulation in your own field.
- How much of what has piled up has an answer attached to it? If the measurement survives but what was asked to produce it does not, there is no question to be made from it.
- Can a score from those questions be produced again outside the organization? Unless the data and the evaluation procedure are released together, a score stays a claim.
- Does each question carry its source and its terms of use? That is why LitmusBench writes a license onto every record. Data with no terms written on it has to be cleared for use before anything else.
All three questions ask about data rather than models. Data is also what NVIDIA took the trouble to open this time.
Editor's Note
Opening something does not stand in for verifying it. Without the opening, though, verification cannot even begin. LitmusBench v0.2 sits on that starting line. This article did not open the Hugging Face dataset card to inspect the questions directly, and for the licensing structure it relies on reporting that did examine the records. The next thing to watch for is when the first score on these questions arrives from someone who is neither NVIDIA nor a partner.
References
NVIDIA Official Source
- 1.NVIDIA. (2026). "NVIDIA Introduces Nemotron for Science to Accelerate Discoveries With Agents." NVIDIA Newsroom, November 16, 2026.
Press Coverage and Analysis
- 2.SiliconANGLE. (2026). "NVIDIA at SC26: 'Agents are the new scientists,' and they need a lab." November 17, 2026.
- 3.CDO Magazine. (2026). "NVIDIA launches open science AI models and 50,000-question chemistry dataset for research agents." November 17, 2026.
- 4.StorageNewsletter. (2026). "SC26: NVIDIA Introduces Nemotron for Science to Accelerate Discoveries with Agents." November 17, 2026.
- 5.Orca Router. (2026). "Nemotron for Science at SC26: What Actually Shipped." orcarouter.ai.