Executive Summary

Key Numbers

Four figures below. The first two are what the trial produced; the last two are the design that made those results readable.

Source: American Heart Association Newsroom, "AI-interpreted echocardiograms comparable to human reports" (November 9, 2026), and ClinicalTrials.gov registration NCT07229300.

Kaiser Permanente Northern California presented the AI ECHO INSIGHT results on November 9, 2026, at the American Heart Association's Scientific Sessions in Chicago. The subject was 1,200 adult transthoracic echocardiograms, and the trial set a workflow in which the AI writes the preliminary read and a cardiologist finalizes it against the workflows hospitals run today. The studies were acquired over two weeks in March 2025 at 21 Northern California medical centers, across inpatient wards and outpatient clinics, and the patients averaged 69 years of age, 47% of them women. The lead investigator is Keane K. Lee of Kaiser Permanente Santa Clara Homestead Medical Center.

Substantial change came to 50.6% for the AI drafts and 49.7% for the pooled drafts written by sonographers and cardiologists. The difference is 0.9 percentage points, and the 95% confidence interval runs from −2.1 to 3.9 points. The line drawn before the trial opened was 5 points, so the whole interval came in inside it. This is what the investigators mean when they write that noninferiority held. Not that AI is better than people, but that it is not meaningfully worse.

Time went down. Reviewing an AI-written preliminary read and finalizing it took an average of 4.6 minutes per study, against 8.5 minutes to produce a report without AI assistance. A 46% cut. The investigators also estimated that sonographers would save about 10 minutes per study if the AI took over the preliminary read, but that number points at a different place than the cardiologist's 4.6 minutes. The first is the time a sonographer spends writing a draft; the second is the time a cardiologist spends finalizing one. The reason the investigators put weight on a few minutes is volume. Lee noted that cardiologists read so many echocardiograms that even a few minutes saved per interpretation adds up to a significant amount of time over the course of a day. Alongside the announcement, the American Heart Association carried a Health Resources and Services Administration projection of a shortage of roughly 7,000 cardiologists in the United States by 2036.

The cardiologists doing the finalizing did not know where the draft in front of them had come from. Every preliminary read was entered in the same standardized format, and the AI output was deliberately styled to look as though a person had written it, so that formatting alone would not give the source away. Lee's statement that the over-reading cardiologists could not distinguish AI drafts from human drafts rests on that design rather than on a measurement. The result itself Lee put in one sentence: "AI preliminary interpretation of adult transthoracic echocardiograms was noninferior to pooled sonographer and cardiologist preliminary interpretation."

Where this trial parts company with the usual medical AI study is in how it built its comparison. Rather than splitting patients into two groups and giving AI to one of them, it put a single study through all three arms. The AI writes the preliminary read in one, a sonographer writes it in another, a cardiologist writes it in the third. Each of the three drafts went to a different cardiologist to be finalized, and the one finalizing did not know the source. The AI in use is EchoPrime, a deep learning model that reads standard echocardiographic views with computer vision and produces a complete report.

That the three arms share one study is the heart of the design. The images are the same, the patient is the same, the credentials of the person finalizing are the same, so whatever difference remains in the result comes only from who wrote the draft.

The 1,200 studies were assembled from 1,000 consecutive ones plus 200 with moderate or greater valvular disease. Gather only the common studies and the hard cases, the ones where judgment splits, fall out of the sample. What counts as a substantially changed read was nailed down in advance as well. If left ventricular ejection fraction moves by 10 points or more, if the grade shifts by even one step on right ventricular function, chamber size, or the severity of valvular regurgitation or stenosis, or if the estimated right ventricular systolic pressure crosses the 30 mmHg line, that study is counted as a substantial change. One item is enough to qualify.

The protocol paper gives the grounds for setting the noninferiority margin at 5 points. When a first-year cardiology fellow acquires and interprets an echocardiogram that an attending reviews afterward, prior reports put major discordance, the kind that requires an immediate change in management, at 4.1%, and minor discordance at 17.4%. Five points falls between the two, picked to show that an AI draft adds no more variability than a human trainee does.

Put 50.6% on its own and there is only one way to read it. A doctor had to fix half the heart ultrasound reports the AI wrote. Turning the same figure into "you cannot trust five in ten" takes no great effort either. That is the story when 49.7% is not standing beside it.

Both rates run high because the net is wide. What this trial counted is not whether a read matched the truth but whether any one of a set of prespecified items moved between the preliminary read and the final one. Ten points of ejection fraction, one step of a grade, one crossing of the 30 mmHg line, and the study enters the substantial change column. Criteria this wide catch a great deal, and they catch it no matter who wrote the draft.

Weaving the net wide was deliberate. Lee said the team "designed this study to capture even small differences between the preliminary reports and the cardiologists' final interpretations." Hold up a ruler that counts small differences and both arms come back near half. 50.6% and 49.7% are marks on that ruler as much as they are facts about the drafts.

There is a gap between what the metric is called and what it counts, too. The trial registration names the primary endpoint an accuracy rate. Open the definition and it reads: the proportion of studies for which the final cardiologist interpretation is not significantly different from the preliminary interpretation. Not a rate of matching the truth, then, but a measure of how closely two reads agreed with each other. The name says accuracy; the count is agreement. Which is why this one number cannot say whether the AI read the heart well, and the investigators did not claim that it does.

The control arm is what settles the meaning of the number. Because the human drafts were caught by the same criteria and came back at 49.7%, the 50.6% turns into a piece of a different fact. Finalizing is a job that rewrites about half of whatever it gets. What this trial newly exposed may be less the AI's score than the baseline of echocardiogram reading. The step from preliminary read to final report was already this labor-intensive before AI arrived.

Drawn on one line, here is where the 0.9-point difference and its interval fall against the margin.

One more secondary result came out of the trial. Matched against each patient's own earlier clinical reads, the finalized reports from the AI-assisted arm agreed with the historical record more often. Lee explained that variability between reads is large overall and that AI assistance narrows it. That is a useful property for long-term follow-up. When the same patient is scanned year after year and the report wobbles with whoever is reading, there is no telling whether what changed was the disease or the person.

But matching the past record more closely does not by itself separate two things. One is AI applying the same criteria without wavering. The other is AI following the wording of the earlier report. If it is the second, a newly appeared finding can be pressed down by the old record, and that is not consistency but delay. Telling the two apart takes a standard independent of the historical record. The registration already lists one. A secondary endpoint compares the preliminary reads against a consensus cardiologist interpretation, and another asks the finalizing cardiologist to guess where the draft came from, measuring whether the blind held. Neither value appears in the materials released. On the published numbers alone, agreement cannot be pulled apart from accuracy.

The investigators named two limitations. Long-term patient outcomes were not measured, and what was measured is the change rate and the time. And the participating cardiologists knew they were in a study, which may have moved them off their ordinary reading habits. Two more remain. This presentation is a late-breaking abstract that has not been peer reviewed, and the full paper is not out yet. The result came from a single integrated health system, and whether other institutions or other models produce the same values has not been checked.

On where responsibility lies, the investigators and an outside commentator give the same answer. Hani Najm of the Cleveland Clinic, vice president of the American College of Cardiology, added this: "At the end of the day, we are dealing with human lives, and human life is precious. We are the final arbitrators at this point." Lee said much the same. "AI cannot, however, replace the human team. The cardiologist remains ultimately responsible for interpreting the echocardiogram and certifying the accuracy of the final report, even when AI assists with the process."

Where AI evaluation wobbles is a subject this blog has come back to more than once. One piece asked whether an AI that beat doctors on text alone would do the same in front of a real patient, and another looked at why a model that aces the exam cannot do the job. Both point at the same spot. Between scoring high and working well stands measurement design.

The device AI ECHO INSIGHT used is a simple one. It scored people on the same scale as the AI. That was not cheap. The same 1,200 studies had to be read three times over, and 9 sonographers and 20 cardiologists went into it. Paying that cost is what made it possible to say what 50.6% means. Had the design left human drafts unmeasured, this trial would have ended in one line: doctors had to fix half of what the AI wrote.

The same question works on your own team's model evaluation. Does the score we report come with one a person produced? If it does not, what are we leaning on when we say the model's number is good or bad? A metric with no human baseline usually reads only as up from the last version, and that comparison breaks the moment the task changes. Whether the name of the metric matches what it actually counts is worth checking in the same pass. If a column labeled accuracy is in fact counting agreement, the easiest way to raise that column is to copy what the last person wrote.

Just as saying the data is ready has to carry quality and provenance inside it, saying the evaluation is ready has to carry a baseline. Without knowing what people score on the same task, a model's score proves nothing on its own.

Thank you for reading this far. The presentation is at the American Heart Association Newsroom, and the trial design can be read directly in the ClinicalTrials.gov registration and the protocol paper. On the scorecards your team runs, is there a human control arm?

Primary Coverage

• 1.American Heart Association Newsroom. (2026). "AI-interpreted echocardiograms comparable to human reports, cardiologists' time cut by about 50%."
• 2.Buzby, S. (2026). "AI-interpreted echocardiograms comparable to human reports, may halve reading time." Cardiology Today (Healio).

Registry & Academic Source

• 3.ClinicalTrials.gov NCT07229300. "Artificial Intelligence Echocardiogram Interpretation for Systematic Integration and Guided Healthcare Transformation (The AI ECHO INSIGHT Study)."
• 4.Lee, K.K. et al. (2026). "AI Echo INSIGHT Study: A Prospective Blinded Randomized Trial of Artificial Intelligence Echocardiogram Interpretation." Contemporary Clinical Trials, 169, 108431. DOI: 10.1016/j.cct.2026.108431