A new research report from Yale School of Medicine offers an up-close look at how bias in medical AI can affect clinical outcomes. The study focuses specifically on the different stages of AI model development, and shows how data integrity issues can impact health equity and care quality.

Why Bias in Medical AI Matters

Published earlier this month in PLOS Digital Health, the research on bias in medical AI gives both real-world and hypothetical illustrations of how AI bias impacts adversely affects healthcare delivery – not just at the point of care, but at every stage of medical AI development: training data, model development, publication and implementation.

"Bias in; bias out," said the study's senior author, John Onofrey, assistant professor of radiology & biomedical imaging and of urology at Yale School of Medicine, in a press statement.

"Having worked in the machine learning/AI field for many years now, the idea that bias exists in algorithms is not surprising," he said. "However, listing all the potential ways bias can enter the AI learning process is incredible. This makes bias mitigation seem like a daunting task."

As the study notes, bias in medical AI can crop up almost anywhere in the algorithm-development pipeline.

Bias in medical AI can occur in "data features and labels, model development and evaluation, deployment, and publication," researchers say. "Insufficient sample sizes for certain patient groups can result in suboptimal performance, algorithm underestimation, and clinically unmeaningful predictions. Missing patient findings can also produce biased model behavior, including capturable but nonrandomly missing data, such as diagnosis codes, and data that is not usually or not easily captured, such as social determinants of health."

Meanwhile, "expertly annotated labels used to train supervised learning models may reflect implicit cognitive biases or substandard care practices. Overreliance on performance metrics during model development may obscure bias and diminish a model’s clinical utility. When applied to data outside the training cohort, model performance can deteriorate from previous validation and can do so differentially across subgroups."

Read the full article about bias in medical AI by Mike Miliard at Healthcare IT News.