What is it about?
Artificial intelligence (AI) has great potential to help doctors predict disease risk and personalise treatments for patients. However, a major challenge is that the data used to train these AI models can be biased. Often, the patient data collected for a research study does not perfectly represent the diverse range of patients in the real world. This problem, known as sample selection bias, can occur if, for example, certain types of patients are lost to follow-up during a study. When an AI model is trained on such biased data, it may become unreliable when used in a real hospital. It could make inaccurate and potentially harmful predictions for the patient groups it has not learned enough about. Our research first highlights how serious this problem is. We then propose a new, safer approach. Instead of trying to force the AI to make predictions for everyone, our method teaches the AI to first identify which patients are similar to those in the original study data. If it confidently recognises a patient, it provides a risk prediction. If not, it flags the case for a human clinician to review, ensuring the AI only operates within its area of expertise.
Featured Image
Photo by Steve Johnson on Unsplash
Why is it important?
The successful integration of artificial intelligence into everyday medical practice is currently hindered by the critical issue of data bias. An AI model that performs exceptionally well during development can fail unexpectedly and even dangerously when deployed in the real world if its training data was not representative of the general patient population. This failure can lead to inaccurate risk scores and unequal care, particularly for underrepresented patient groups, which ultimately erodes trust in medical technology. This research addresses this challenge directly. First, we provide clear evidence of the significant performance drop and potential harm that sample selection bias can cause. More importantly, we introduce a practical and novel solution to mitigate this risk. Our "identify first, then predict" approach acts as a crucial safety layer. It empowers the AI model to understand its own limitations and refrain from making predictions when it is likely to be wrong. This work paves the way for developing more robust, reliable, and ethically responsible AI tools that clinicians can trust, ensuring that new technology safely improves care for patients.
Perspectives
As researchers creating AI for healthcare, our goal is often focused on achieving the highest possible accuracy. However, this project highlighted that the true value of an AI tool in a clinical setting is not just about how often it is right, but also about its ability to recognise when it might be wrong. We realised that forcing a model to generalise to patient groups it has never seen before is not only technically challenging but also potentially unsafe. This led us to shift our perspective from simply trying to "correct" data bias to instead engineering AI systems that are aware of their own boundaries. We believe a more responsible paradigm is one that fosters a partnership between the AI and the clinician. Our work promotes this by designing an AI that can essentially say, "I am confident in my analysis for this patient, and here is my prediction," or alternatively, "This patient's profile is outside my training, so a human expert should take the lead." This concept of selective prediction is a vital step toward building trustworthy AI, ensuring these powerful tools can safely and effectively support doctors to enhance patient outcomes.
Dr Vinod Kumar Chauhan
University of Strathclyde
Read the Original
This page is a summary of: Sample Selection Bias in Machine Learning for Healthcare, ACM Transactions on Computing for Healthcare, August 2025, ACM (Association for Computing Machinery),
DOI: 10.1145/3761822.
You can read the full text:
Resources
Contributors
The following have contributed to this page
