AI Bias in Medicine 🩺

How intersectionality can help combat bias in medical algorithms.

Photo by National Cancer Institute on Unsplash

By: Cindy Wen

TL;DR:

Data bias in AI and its discriminatory power creates a pressing danger in the field of medicine. High stakes and potentially life-changing outcomes make it essential for AI algorithms to be constructed and trained with intersectional and diverse sets of data that are representative of the general population.

The Breakdown

• AI is often believed to be a calculating technology that makes purely data-driven decisions, however, it has been shown that these algorithms do have the ability to discriminate, providing results that are prejudiced due to unrepresentative training data or assumptions made during development
• For instance, in 2019, a prominent U.S health care prediction algorithm used on over 200 million citizens was found to be demonstrating racial bias against black patients
• Designed to identify high-risk patients for preemptive care-management programs, issues arose due to the use of previous health care spending as a proxy for medical needs/risk
• With a strong correlation between race and income, black patients generally have less to spend on healthcare, and when they do spend as much as their white counterparts, it is often for a far more serious/active intervention
• Ultimately, this meant many high-risk black patients were not tagged by the medical system even when showing a similar level of risk to white patients, because they hadn’t spent the same amount on healthcare services previously
• In another example, an AI model constructed for the detection of skin cancer was trained with a dataset that consisted primarily of Caucasian males. Due to this lack of diversity in the training data, once applied to the real world, the model was found to be extraordinarily inaccurate when used on the general population

The Solution

• According to CTO of Royal Philips, Henk van Houten, there are three types of diversity that will need to be built into every aspect of AI development in the future: Diversity in people, diversity in data, and diversity in validation
• In the U.S, there is currently a movement in support of the Algorithmic Accountability Act, aimed at enforcing and assessing AI/ML algorithms for discriminatory output. Other areas in the world are developing similar guidelines surrounding the reliability of regulation of these systems
• To this extent, the realm of Artificial Intelligence has become increasingly multi-disciplinary, drawing on fields such as philosophy to answer and quantify questions like “what is fairness” in order to build out reliable and non-bias models
• Actionable solutions to questions like these will ultimately require the help of social scientists and ethicists, just as much as Machine Learning Engineers and Data Scientists

The Significance

• Intersectionality is a crucial concept to understand and incorporate in the development of quality AI systems, which are by and large only as good as the quality of the data inputted into them. It is, therefore, both desirable and necessary to incorporate intersectional data in the development of future medical AI/ML systems and algorithms

Additional Info

Removing Bias in AI isn’t enough, it must take intersectionality into account.

7 Types of Data Bias in Machine Learning

How does bias affect healthcare AI, and what can be done about it?

Artificial Intelligence Bias in Healthcare

Health Care AI Systems Are Biased

Krita Sharma’s Ted Talk: How to Keep Human Bias Out of AI

Full Source List