Executive Summary

Key Numbers

The four numbers below have to be read together. The first two say that there is no discrimination and that the cause is not a bad choice of features. The last two point to where something usable still survives in a model like this.

Source: Adetunji & Oyewusi, arXiv:2608.17712 (2026-08-18, accepted for EMBC 2026)

Preterm birth means delivery before 37 completed weeks of gestation. It affects roughly 10.6% of live births worldwide, about 14.84 million babies a year.

Behind that percentage sit concrete outcomes. Respiratory distress syndrome, intraventricular hemorrhage, necrotizing enterocolitis, sepsis and cerebral palsy account for a substantial share of neonatal and under-five mortality, and survivors carry elevated neurocognitive and cardiometabolic risk for years. More than 80% of the absolute burden is concentrated in South Asia and sub-Saharan Africa.

It is still a problem that can be acted on, provided it is caught in advance. That is why the search for a cheap and noninvasive screening tool has been going on for so long, and why the need is greatest exactly where the burden is concentrated. Electrohysterography is one of the candidates. Electrodes on the mother's abdomen record the electrical activity of the uterine muscle for about half an hour.

Studies using this signal to predict preterm birth have accumulated since 2008, and the reported AUROCs climbed past 0.80 and reached 0.97. AUROC is the probability that, given one mother who will deliver preterm and one who will go to term, the model assigns the higher risk score to the first. A value of 0.5 is a coin flip, and 0.97 means the model is almost never wrong. On the numbers alone, this looks like a field ready to discuss clinical deployment.

The paper that Sunday A. Adetunji of the Department of Biostatistics and Epidemiology at Oregon State University's College of Health and Rhoda O. Oyewusi of Alifort Hospital in Lagos posted to arXiv on 18 August reports 0.493 on the same public dataset. The 95% confidence interval runs from 0.467 to 0.520. The area under the precision-recall curve came to 0.122, essentially identical to 0.127, the rate at which preterm birth actually occurs in the cohort. In other words, the model carried no information at all. The paper has been accepted for EMBC 2026.

The data was not changed. The model was not sabotaged. The one thing that moved was where the line between training and validation was drawn.

The dataset this field shares is TPEHGDB, collected at the University Medical Centre in Ljubljana, Slovenia. It holds records from 300 mothers, 38 of whom delivered preterm, a rate of 12.7%. Thirty-eight cases out of 300. By machine learning standards this is a very small sample.

So the standard preprocessing step cuts each 30-minute recording into many windows. What was 300 samples now looks like several thousand. The trouble starts next. If training and validation are divided window by window, windows drawn from the same mother scatter across both sides. The model has already seen the signal of the mother it is about to be tested on.

What the model learns in that setup is not the uterine physiology that leads to preterm delivery but what this particular mother's signal looks like. The validation score comes out high. In front of a mother it has never seen, that score does not reproduce. The phrase the authors put in their title, leakage-proof, means they closed this channel.

This is not a matter of taste. PROBAST, the standard tool for assessing risk of bias in prediction-model studies, counts failure to preserve subject independence as a high-risk analysis feature. Tougui and colleagues showed in 2021 that allowing correlated observations to occur across validation partitions can materially inflate diagnostic performance estimates, and Varma and Simon showed in 2006 that model selection performed outside the validation hierarchy biases estimates of prediction error. This paper carries those principles into obstetric electrophysiology.

The principle the authors draw from the case fits in one sentence. The unit of resampling must correspond to the unit at which predictive performance is intended to generalize. A model meant for new mothers has to be split by mother, and the fact that signal segments are independent of one another does not mean the patients are. The paper is explicit that this distinction is inferential rather than procedural. Cutting the recording into finer pieces does not add samples. It makes the sample look larger than it is.

Sealing off leakage does not end with cutting the data into two piles. The authors made the record identifier the only grouping variable, then split the training records inside each outer fold into three further non-overlapping subsets: one to fit the model, one to calibrate the probabilities, one to set the threshold for the prediction sets. Median imputation and scaling parameters were estimated only within that fold's training partition. A record under evaluation was exposed at no stage of the process.

The model itself is unglamorous: elastic-net logistic regression on 92 engineered features. The features are amplitude, median frequency, peak frequency and sample entropy drawn from three band-pass filters across three channels, plus the differences between filters. That combination produced an AUROC of 0.493 and a Brier score of 0.115.

Looking inside the feature space told the same story. In a two-dimensional principal component projection of the 92 dimensions, used for visualization only and not part of the predictive pipeline, term and preterm records overlapped heavily, and the distributions of the model's risk scores overlapped as well. The coefficients, though, held steady. The peak-frequency minimum in the 0.3 to 4.0 Hz band survived as a nonzero coefficient in 98% of folds, and the cross-filter amplitude difference in 100%. A feature being selected again and again is a different claim from that feature separating mothers the model has never seen.

Whether the missing performance came from a bad choice of features was checked separately. Using one filter view at a time gave 0.483 to 0.500, and randomly dropping 10% and 30% of the 92 features gave 0.478 to 0.504. No combination manufactured any discrimination. Splitting by gestational age gave 0.514 for recordings taken at or before 26 weeks and 0.469 after, a visible direction, but the authors drew a line under it: no formal test of distribution shift was performed, so the difference should be read as hypothesis-generating evidence and nothing more.

That direction touches a design question on its own. Uterine electrical activity changes as pregnancy progresses. If so, gestational age is not one more baseline covariate to feed the model but an axis that determines when the signal was acquired in the first place. Gestational time, the authors write, constitutes an explicit dimension of model transportability.

3.1How Not to Overstate This Result

One point deserves care here. The figures of 0.80 to 0.97 are values reported in the literature, not values this paper reproduced through its own pipeline. The paper defines segment-level cross-validation as its first regime but states that its primary inferential claims rest on the patient-level regime and the gestational-age-stratified regime. And among prior studies that did group by record, a deep bidirectional LSTM model reported roughly 0.74 to 0.78.

The authors write that such differences may come from signal representation, model class, preprocessing or partitioning, and that only a comparison under matched conditions can settle it. Splitting properly, in other words, does not automatically collapse everything to 0.5. They state their own limits too. This result covers 300 records from a single center and one set of precomputed features, and it cannot be extended to models that work on raw waveforms or combine other obstetric measures.

They also spell out how they handled uncertainty in the numbers. The confidence intervals are percentiles from 1,000 record-level bootstrap resamples, and with only 38 preterm events the sampling variability remains substantial. The authors name that as a limitation.

Had the story ended at the absence of discrimination, this paper would be a refutation and little else. The authors added one more piece. Instead of forcing a binary answer, they let the model decline to answer.

Conformal prediction returns a set of labels rather than a single label. It can say preterm, it can say term, or it can return a set holding both and pass the case to clinical judgment. Set the miscoverage level to 0.10 and the threshold is chosen so that the true outcome falls inside the prediction set at least 90% of the time. In this study the realized coverage was 0.897, close to the target.

On 72.7% of cases the model returned a set containing both labels. It said, in effect, that it did not know. On the 27.3% where it issued a single label, accuracy was 0.624. That is 4.9 times the 0.127 rate at which preterm birth actually occurs. A model with no discrimination across the population was still leaving usable signal inside the narrow band where it was confident.

The authors insist these numbers cannot be read apart. Coverage only says whether the true outcome fell inside the set. It says nothing about how vague that set was. Rule nothing out and coverage is perfect every time. Coverage, abstention rate and singleton accuracy therefore have to be reported together. The premise of the whole mechanism is that in clinical work uncertainty is information to be displayed rather than a flaw to be hidden.

The open question is on the ground. If three cases out of four go to clinical judgment, who absorbs that judgment and at what volume becomes an immediate problem. The authors stop here as well. Whether calibration holds over time, whether referral frequency and clinical workflow can carry the load, and whether maternal and neonatal outcomes actually improve are all left to prospective multicenter studies.

Dividing observations from the same subject at random is not a mistake confined to obstetric signals. Sayash Kapoor and Arvind Narayanan of Princeton surveyed fields that have adopted machine learning and found reproducibility failures caused by data leakage in 329 papers across 17 disciplines. They sorted leakage into eight types, ranging from textbook errors to open research problems. When they reproduced the civil war onset literature directly, not one of the papers claiming that complex machine learning beats decades-old logistic regression held up.

Industrial equipment shows the same shape. A group at the Federal University of Santa Catarina in Brazil reevaluated models that diagnose bearing faults from vibration signals. They showed that the common segment-wise and condition-wise splits create spurious correlations that inflate performance, and proposed a bearing-wise split that keeps the same physical component out of both training and test. They confirmed this on four standard datasets: CWRU, Paderborn, Ottawa and Hanoi. They also looked at how data diversity affects generalization, and the number of unique training bearings turned out to be the decisive factor in how robust the performance was.

Cell imaging is no different. C-NMC 2019, a benchmark widely used for leukemia diagnosis, has a long record of near-perfect scores, yet splitting it randomly by image puts cells from the same patient in both training and test. Under a subject-disjoint reevaluation the best model reached an AUROC of 0.913, and in the conservative frozen-feature setting, random splitting inflated AUROC by about 0.04. The size of the inflation varies by domain. The direction never does.

The nature of this defect is worth dwelling on. The values are not wrong. There may be no missing entries, no outliers, and perfectly accurate labels. Stare at the files as long as you like and nothing looks broken. What is wrong is where those values were cut apart. A quality regime that defines data quality as the integrity of values is structurally unable to see this. So there are three questions to ask when designing a benchmark.

• • Does this data contain repeated measurements. If several rows come from the same patient, the same machine, the same operator or the same document, those rows are not independent of one another.
• • What is the unit at which the performance should generalize. A model for unseen patients and a model for an unseen factory draw the line in different places. The split has to happen at that unit.
• • Are preprocessing, calibration and thresholds all confined to the training side. Estimating a single scaling parameter on the full dataset already lets information from the validation data seep in.

The paper is available at arXiv:2608.17712, and the authors have released their reproduction materials on GitHub.

Primary Sources

• 1.Adetunji, S. A. & Oyewusi, R. O. (2026). "A Leakage-Proof Benchmark and Conformal Selective Triage for Electrohysterogram-Based Preterm Birth Prediction." arXiv preprint (accepted for EMBC 2026).
• 2.Fele-Žorž, G., Kavšek, G., Novak-Antolič, Ž. & Jager, F. (2008). "A Comparison of Various Linear and Non-Linear Signal Processing Techniques to Separate Uterine EMG Records of Term and Pre-Term Delivery Groups." Medical & Biological Engineering & Computing, 46(9), 911-922. — Original TPEHGDB dataset, PhysioNet.

Methodological Foundations

• 3.Wolff, R. F., Moons, K. G. M., Riley, R. D. et al. (2019). "PROBAST: A Tool to Assess the Risk of Bias and Applicability of Prediction Model Studies." Annals of Internal Medicine, 170, 51-58.
• 4.Tougui, I., Jilbab, A. & El Mhamdi, J. (2021). "Impact of the Choice of Cross-Validation Techniques on the Results of Machine Learning-Based Diagnostic Applications." Healthcare Informatics Research, 27(3), 189-199.
• 5.Varma, S. & Simon, R. (2006). "Bias in Error Estimation When Using Cross-Validation for Model Selection." BMC Bioinformatics, 7, 91.

Cross-Domain Evidence

• 6.Kapoor, S. & Narayanan, A. (2023). "Leakage and the Reproducibility Crisis in Machine-Learning-Based Science." Patterns, 4(9), 100804.
• 7.Vieira, J. P., Bauler, V. A., Rosa, R. K. & Silva, D. (2025). "Towards a More Realistic Evaluation of Machine Learning Models for Bearing Fault Diagnosis." arXiv preprint.
• 8.Albzour, N. (2026). "A Leakage-Aware Comparative Benchmark of Machine Learning, Deep Learning, and Transformer Models for Reliable Leukemia Detection." arXiv preprint.