The software being tested comes from Vara, a startup based in Germany that also led the study. The company’s AI is already used in more than a quarter of Germany’s breast cancer screening centers and was introduced earlier this year in a hospital in Mexico and another in Greece.
Vara’s team, with the help of radiologists from Essen University Hospital in Germany and the Memorial Sloan Kettering Cancer Center in New York, tested two approaches. In the first, AI works alone to analyze mammograms. On the other hand, AI automatically distinguishes between scans that it thinks seem normal and those that raise a concern. He refers the latter to a radiologist, who will review them before seeing the AI assessment. The AI would then issue a warning if it detected cancer when the doctor did not.
“In the proposed process driven by artificial intelligence, nearly three-quarters of screening studies did not have to be reviewed by a radiologist, while improving overall accuracy.”
Charles Langlotz
To train the neural network, Vara fed AI data from more than 367,000 mammograms, including radiologists ’notes, original assessments, and information on whether the patient eventually had cancer, to learn how to place these scans in one of three buckets: “normal safe.”, “not confident” (in which no prediction is made) and “safe cancer”. The findings of both approaches were then compared with the decisions that actual radiologists originally made on 82,851 mammograms from detection centers that did not provide scans used to train AI.
The second approach, the doctor and AI working together, was 3.6% better at detecting breast cancer than a doctor working alone and generating fewer false alarms. He achieved this while automatically setting aside scans that were safely classified as normal, which accounted for 63% of all mammograms. This intense rationalization could reduce the workload of radiologists.
After breast cancer testing, patients with a normal scan are sent on their way, while an abnormal or unclear scan triggers follow-up tests. But radiologists who examine mammograms lose 1 in 8 cancers. Fatigue, overwork, and even the time of day affect the way radiologists can identify tumors while seeing thousands of scans. Visually subtle signs are also less likely to trigger alarms and dense breast tissue, which is found mainly in younger patients, makes the signs of cancer more difficult to see.
Radiologists who use AI in the real world are required by German law to look at every mammogram, at least looking at those that AI considers good. The AI still gives them a hand by previously filling out reports on scans labeled as normal, although the radiologist can always reject the AI call.
Thilo Töllner, a radiologist who runs a German breast cancer screening center, has used the program for two years. He sometimes disagrees when the AI classified the scans as safe normal and manually filled out reports to reflect a different conclusion, but says “normal is almost always normal.” Mostly, “just press Enter.”
Mammograms that the AI has labeled as ambiguous or “safe cancer” are referred to a radiologist, but only after the doctor has offered an independent initial evaluation.