Executive Summary

Key Figures

Source: Zhang et al., arXiv:2608.25846 (2026-08-26)

Three public TB cough datasets supply the raw material. CODA was assembled for a TB diagnosis challenge and holds Android smartphone recordings from 2,143 outpatients across India, Madagascar, the Philippines, South Africa, Tanzania, Uganda, and Vietnam, with the phone model differing from country to country. TBscreen was collected in a controlled recording room in Nairobi, Kenya, capturing 195 adults simultaneously on a smartphone, a boundary microphone, and a condenser microphone, and it keeps coughs that happened on their own separate from coughs produced on request. The Zambia CIDRZ data is the same data behind Google's HeAR benchmark, recording the same coughs from 599 adults on one professional audio recorder and three smartphones at once. In all three datasets the TB label rests on a microbiological reference standard such as sputum Xpert testing and culture.

The released data was not used whole. From CODA the researchers kept only Vietnam, Madagascar, and Tanzania, the countries where a single phone model was used consistently, and within those they kept only solicited coughs so that device effects could be read cleanly. In TBscreen, passive coughs went to training and forced coughs to validation only, because the original study found the two types behaved differently and models trained on passive coughs did not transfer well to forced ones. In Zambia one site with a single TB-positive participant was dropped, and the audio recorder subset, being a small portion of the data, was used for validation only. CODA itself was released as roughly half of the full collection, so the volume that actually entered the analysis is smaller than the participant counts quoted above.

The appendix table that counts only the analyzed subsets shows how far apart the TB-positive rates already are. The Chawama site in Zambia runs 9 to 10 percent depending on the device, and Kanyama runs 21 to 22 percent. Inside CODA the split is Tanzania 16 percent, Vietnam 40 percent, Madagascar 48 percent. TBscreen, which recruited TB patients alongside controls with other respiratory illnesses, sits at 70 to 74 percent for passive coughs. What a model meets when it crosses from one dataset to another is not only a difference in sound.

The validation design closed off the usual ways of inflating a score. Within-dataset performance came from nested cross-validation with 5 outer folds and 4 inner folds, repeated twice for 10 estimates reported as a mean and standard deviation, and every split was cut at the subject level rather than the cough level. Several coughs from one person landing on both sides of a split would push the score up. Model selection also happened only inside the training dataset, and the chosen configuration was carried to the other datasets and applied once.

The classical machine learning pipeline reached 0.700 on Zambia, 0.711 on TBscreen passive cough, and 0.631 on CODA within each dataset. In the deep learning pipeline, a VGGish backbone fed with mel spectrograms worked best, and the Zambia-trained model produced this study's high mark at 0.755. The problem starts there. Applying that same model to the Zambia audio recorder subset gave 0.717 and to TBscreen forced coughs 0.741, but TBscreen passive coughs returned 0.632 and CODA returned 0.581. Models trained on TBscreen or CODA scored below 0.6 on every external target.

A bad pipeline choice is not the culprit either. Performance moved far more with which dataset a model was trained on and where it was applied than with which classifier or which features it used, the best configuration differed by dataset, and no classifier and no feature family led consistently. The training data was balanced on more than the label as well. For each dataset, wherever the factors were recorded, subgroups were defined by the combination of recording location, device, and TB label, and the sample counts were balanced at that level before training. Even so, the score fell below 0.6 outside.

A similar collapse had already been observed once. A study that validated the top CODA challenge models on an independent Peruvian cohort found internal scores of 0.689 to 0.743 falling to 0.480 to 0.615 externally. The Peru data had been collected under a closely matched acquisition protocol. That study attributed the drop mainly to differences in population. This paper takes the same phenomenon and asks again what caused it.

To find the cause, the researchers looked past the model's report card and into the space where the inputs sit. They measured distances between subgroups in CLAP audio embeddings using maximum mean discrepancy, then laid that distance structure out on a plane with multidimensional scaling. Subgroups were defined three ways: by recording device, by collection site, and by TB label.

The result points one way. Subgroups from different dataset sources barely overlapped. Subgroups split by device and by location spanned a wide area even within the same dataset. The TB-positive and TB-negative groups from a single dataset, meanwhile, sat close together. Ideally it would be the reverse. Groups split by device and site would overlap across datasets, and groups split by TB label would separate. Projecting individual samples with t-SNE produced the same picture. The broadest clusters formed by dataset source, additional structure appeared by device within each dataset, and separation by TB label was comparatively weak.

Shortcuts like this are not new to medical AI. The paper's introduction lines up earlier work on chest X-ray models that judged COVID-19 by picking up traces of the imaging conditions instead of the lesion, and reads the same thing happening in sound. That picture explains the collapse in the previous section two ways. One is that the variation introduced by acquisition conditions overwhelms the variation associated with TB in the feature space. A boundary drawn on top of it works inside a dataset where the conditions are approximately fixed, and falls apart when the conditions change. The remaining one is that acquisition conditions correlate with TB status, and the model learns the conditions themselves as a shortcut for prediction.

In CODA the phone model differs by country and so does the TB rate. The researchers measured the relationship between the mean predicted probability per country and the actual TB-positive rate per country under two settings. The first closely replicated the best-performing CODA challenge pipeline, feeding a ResNet-34 with spectrograms that reach into the high frequencies and resampling on the TB label alone. The second was a conservative setting designed to reduce shortcuts, using a VGGish-based model, an upper frequency limit lowered to 8,000 Hz, cough segmentation before feature extraction, and resampling balanced jointly by TB label and country.

Under the challenge-replication setting the model output moved with the country more than with the disease state. Regressing the mean predicted probability per country on the country TB-positive rate gives a slope of 1.20 with R² of 0.95 for the TB-negative group and a slope of 1.30 with R² of 0.89 for the TB-positive group. The rate on the horizontal axis here is not a community prevalence statistic but the TB-positive proportion measured per country inside the CODA cohort. Country-level prevalence alone accounts for nearly all the variance in mean output per group. In practice, fix a single decision threshold and almost everyone from a high-prevalence country is classified positive regardless of how their cough sounds, while almost everyone from a low-prevalence country is classified negative.

Under the conservative setting the link weakens. Both slopes drop to 0.33 and the R² values become 0.75 and 0.84. It does not disappear entirely. The researchers were explicit about what kind of experiment this was. It was not an attempt to evaluate a deployable TB classifier but a diagnostic experiment testing whether resampling and pre-processing choices could reduce the shortcuts. They also do not treat prevalence information as a defect in itself. Prior disease rates carry information that can aid a judgment in deployment. The problem is when that information reaches the model through traces of the acquisition conditions rather than through clinical evidence. A prior absorbed that way becomes a calibration error the moment the model moves somewhere with a different prevalence.

In CODA, country, device, recruitment setting, clinical population, and prevalence come as one bundle, which makes the device's share hard to isolate. So the researchers move to the Zambia data. Because four devices recorded the same coughs from the same participants at once, the device can be changed while the people and the place stay fixed. The devices are one professional audio recorder and three smartphones: the entry-level Pixel 3a, the mid-range Galaxy A12, and the higher-end Galaxy A22. Unlike the cross-dataset analysis earlier, this section's experiment also allows the audio recorder as a candidate training device.

In the experiment that trained on one device and applied it to all four, moving to a device other than the one used in training generally cost performance. The size of the drop varied by device, and the strongest and most consistent training device was the Galaxy A22. It was not the professional audio recorder with its flat frequency response. That cuts against the assumption that better equipment yields a better training signal. The researchers write that a device's suitability depends not on nominal recording quality but on which structures of the cough it captures and how closely its recordings resemble those of other devices.

Training on three devices together raises performance on a device absent from training. With the audio recorder held out, training on the Pixel 3a alone gave 0.675, the Galaxy A12 0.655, and the Galaxy A22 0.725, while training on all three together gave 0.732. The researchers use the numbers to rule out extra data as the cause. By the values stated in the paper's body, the three devices cover 632, 628, and 642 subjects respectively, and all three combined cover 650. What grew was the range of devices, not the number of people.

These headcounts are also one point where the paper does not agree with itself. The appendix table that summarizes the analyzed subsets puts the two Zambia sites at 316 to 323 subjects per device, and the original dataset publication reports 599 participants. The 632 to 650 in the body appears to be counted on a different basis. Whichever number you take, the three devices recorded the same coughs from the same participants at once, so combining them barely adds people, and the argument separating device diversity from data volume stands as it is.

The researchers do attach a condition to this prescription. Adding devices pays off only when it is introduced inside a balanced study design. If device type arrives tied to site, prevalence, or recruitment route, the dataset grows and the model gains one more shortcut to hold. CODA in section 3 is a dataset built exactly that way.

There is an objection to raise here. When the dataset changes, the people change too. Different countries, different recruitment routes, different prevalence. If performance fell simply because the population differed, then no amount of work on recording hardware would help. The researchers close that off with a control.

They picked only the questionnaire variables available in comparable form across all three datasets: age, sex, smoking status, HIV status, previous TB history, cough duration, fever, weight loss, and night sweats. Without a single fragment of sound, they trained a logistic regression and put it through the same external validation framework. The model trained on Zambia reached 0.767 inside its own dataset, 0.655 on TBscreen, and 0.673 on CODA. These are not high numbers. But in the same settings where the acoustic model sank to 0.581, this one stayed above chance and swung across a far narrower band.

The questionnaire variables face the same differences in population. They still transferred more stably than the sound. That makes it reasonable to read much of the acoustic model's collapse as coming from acquisition variability layered onto the audio rather than from the population. The researchers phrase it as not being explained by population shift alone, which is not the same as saying population plays no part. In the limitations the paper notes that differences in airway and vocal-tract anatomy by ethnicity and region, body size, respiratory comorbidity patterns, and culturally shaped coughing behaviour could all produce acoustic differences. The questionnaire variables were not entirely free from dataset shift either. Some, such as weight loss, held steady across datasets, while others contributed differently depending on the dataset.

The point of bringing out the questionnaire is not to swap sound for paperwork. What the researchers propose as a next direction is a design that divides the work. Clinical variables supply a stable baseline risk, and cough acoustics carry the information added on top. If the acoustic model does not have to draw the whole decision boundary by itself, it has less reason to lean on shortcuts locked inside a dataset. They also note the use of previous TB history, HIV status, smoking, and symptom combinations to first define subgroups that are more acoustically homogeneous, and the possibility that looking at cough frequency, bout structure, and change over time transfers better than examining each cough on its own.

The algorithmic route has already been tried. The researchers attached established domain generalization techniques such as MixStyle, Mixup, and domain-adversarial neural networks, and got no consistent improvement in external performance. The reason lies in what those techniques assume. Learning a representation that is not swayed by the domain requires source domains that are large enough and balanced. Today's TB cough datasets have country, device, site, prevalence, and participant characteristics entangled with one another, and within that the acoustic signal of TB is weak relative to the effects of device and protocol. Forcing domain invariance onto data like this can shave off useful information along with the rest.

So both routes the paper offers sit on the collection side. One is to collect at far larger scale across many devices, environments, countries, and participant groups, putting the variation of real deployment into training. The other is to collect on a small set of devices with known acoustic properties under a fixed protocol, reducing acquisition variability itself. Either way, some things are required in common: a balanced design that decouples device and site from disease status, detailed metadata on device and protocol, and prospective external validation under conditions close to deployment.

In the Zambia data, some recordings carried the wrong subject number. In a design that has to cut data at the subject level, an error like that turns straight into information leakage. The researchers used the structure of several devices recording the same person at once, computing cross-correlations between the signals from different devices to find the mislabeled recordings. The simultaneous multi-device design was a resource for measuring device effects and a resource for auditing labels at the same time.

This piece sits alongside two other medical AI validation cases we covered in August, and the point of breakage differs in each. The preterm birth prediction benchmark was a case of information leakage, records from the same mother spanning training and validation, and the thing to fix was the splitting method. The glucose prediction benchmark was a case where models generalized but error widened for particular subgroups, and the thing to fix was the unit of reporting. This TB cough model is a case where performance itself collapses on crossing to another dataset, and the thing to fix is the design of the collection. What the three say together is that one performance figure hides the facts in different ways depending on the validation design.

The paper does not conclude that cough sounds carry no information about TB. That external validation performance sits above chance is itself a signal that information is in the sound. What it says is that current dataset designs and validation practices make it impossible to tell whether a model learned the disease or the recording environment, and that a good number inside a dataset is not the same thing as clinical readiness. The researchers note that the validation and analysis procedure used here is not limited to TB cough and can be extended to any ML-based acoustic classifier. The paper is at arXiv:2608.25846, and the analysis code is public on GitHub.

Academic Papers

• 1.Zhang, W., Teijeiro, T., Thevenot, J., Atienza, D. (2026). "Why ML-based cough models do not generalize: a systematic cross-dataset evaluation for tuberculosis screening." arXiv:2608.25846. The primary source for this article.
• 2.Zimmer, A. J. et al. (2026). "External validation of cough-based algorithms for pulmonary tuberculosis screening from the CODA TB DREAM challenge using cough data from Peru." Scientific Reports.
• 3.DeGrave, A. J., Janizek, J. D., Lee, S. (2021). "AI for radiographic COVID-19 detection selects shortcuts over signal." Nature Machine Intelligence, 3(7), 610–619.
• 4.Pahar, M., Klopper, M., Reeve, B., Warren, R., Theron, G., Niesler, T. (2021). "Automatic Cough Classification for Tuberculosis Screening in a Real-World Environment." Physiological Measurement, 42(10).
• 5.Botha, G. H. R. et al. (2018). "Detection of tuberculosis by automatic cough sound analysis." Physiological Measurement, 39(4), 045005.
• 6.Frost, G., Theron, G., Niesler, T. (2022). "TB or not TB? Acoustic cough analysis for tuberculosis classification." arXiv:2209.00934.

Official Statistics

• 7.World Health Organization (2025). "Global Tuberculosis Report 2025."