3 Keys to Addressing Bias in Health Data and Algorithms and Why it Matters

Underserved and marginalized populations in the United States and across the world have long suffered from unequal access to the basic human need of healthcare based on factors that should not be relevant: race, gender, ethnicity, geographic location and income, says Maryland Smith’s Ritu Agarwal in a recent American Medical Association “Making the Rounds” podcast.

The podcast explored a wide range of issues, Agarwal says, that medical students, researchers, and practitioners should be thinking about as the digitization of healthcare accelerates. “When you juxtapose this presence of health inequities with the digital transformation of healthcare -- with an increasing reliance on data and algorithms – ‘boom,’ we have the potential for a very serious problem that we need to be very, very vigilant to and fix,” she adds. “Increasing the diversity of datasets researchers use and algorithms are based upon is critical.”

One dimension of this issue is illustrated in a 2021 study of racial disparity in COVID-19 vaccine distribution co-authored by Agarwal, Distinguished University Professor, Robert H. Smith Dean’s Chair of Information Systems and co-director of the Center for Health Information and Decision Systems (CHIDS) at the University of Maryland’s Robert H. Smith School of Business.

In the podcast, Agarwal says when vaccines started coming out, Blacks were being vaccinated at much lower rates than whites and this – through the media -- was being widely attributed to mistrust in the healthcare system. “I did not agree with that narrative, and I felt we needed to debunk it or else find the data supporting it. I had an intuition that it was structural barriers and other institutional determinants of health more than anything else that was causing the disparity in vaccination rates among different races… We collected a unique dataset, and we addressed that question. In a nutshell, we found that politics and privilege, socio-economic status and education do matter.”

Agarwal’s work – published in the Proceedings of the National Academy of Sciences of the United States -- “laid out in explicit terms the racial and socioeconomic disparities in [COVID] vaccine distribution and uptake,” noted her AMA interviewer and University of Toledo medical student Kristofer Jackson. “That scholarship made a significant difference in how we rolled out the vaccine to underserved people,” he added. “It was a tool to actually drive change.”

How can such change be reinforced? Agarwal described three paths for students and physician-scientists toward health equity:

Ask the right questions for any automated algorithm used for diagnostic and frequent decisions. “What data were they trained on? What’s the quality of the data? How representative is [the data] of the patient population? In the genome-wide association studies, 78 percent of the data come from people of European descent and only 2 percent from Blacks and 1 percent from Hispanics… You check the quality of your instruments to make sure they’re not malfunctioning, but you need to be literate and informed about algorithms and data as well.”

In clinical work, capture all the social determinants of health in your unstructured notes. “This is going to be the goldmine of data that will drive future algorithms and inference. Our research at the center (CHIDS) has shown that EHR data capture these factors about marginalized populations at significantly lower rates than the majority. This will be your responsibility to make sure you understand the full spectrum of healthcare influences within which your patients’ clinical conditions are presented.”

Engage vigorously with the NIH’s AIM-AHEAD (Artificial Intelligence/Machine Learning Consortium to Advance Health Equity and Researcher Diversity). And engage in its mission to shine a light on bias in data and algorithms to make sure the scientists and individuals who are contributing to healthcare transformation through AI and machine learning come from all walks of life, all backgrounds, all genders, all races, all ethnicities.”

“Health IT, AI and ML have a very important role in addressing inequities,” Agarwal says. “Even though ‘bias in AI and ML’ is often referred to, it’s also critical to keep AI and ML in mind as instruments of positive change, as well, to help us overcome our biases.”

Listen to the podcast: “Ritu Agarwal, PhD, and CHIDS founding director on shaping health IT research, education and outreach,” via the American Medical Association’s Making the Rounds.

Read the study: “Socioeconomic privilege and political ideology are associated with racial disparity in COVID-19 vaccination,” in PNAS, by Ritu Agarwal, Michelle Dugas, Jui Ramaprasad and Gordon Gao.