Following up from Qserve’s blog by Paul Hoseit, I am drawn to consider health equity across the Waldo landscape which struck a chord with me from the recent seminar on transparency in AI held by FDA. [https://www.fda.gov/medical-devices/workshops-conferences-medical-devices/virtual-public-workshop-transparency-artificial-intelligencemachine-learning-enabled-medical-devices]
Transparency with regulators, physicians and patients as to how algorithms (AI/ML devices) work is essential for trust from all the stake holders. It is true as Paul discussed, that regulators are tasked with assuring safety and efficacy; and patients trust that the algorithm has come up with the correct result (sensitivity and specificity). But in addition, physicians want to know “Does this (device) work for MY patients in my practice”?
This latter question goes to the patient population for which the device was developed and then later clinically validated from the frozen design. This is a key factor with FDA and Notified Bodies as they consider the intended use of the device or diagnostic that includes the intended use population. Regulators will ask, “Is the population tested representative of the population where the device or diagnostic will be used”? Prevalence of disease/clinical condition is a factor in the clinical study acceptance criteria so one must ask, does the population in the study exhibit the same prevalence for which the acceptance criteria were established?
Using the Where’s Waldo illustration again: is this the intended use population?
Taking this one step further, we need to consider health equity where everyone has a fair and equal opportunity to be healthy. The idea is to eliminate disparities in health and offer the same quality of care and access to medical devices and diagnostics across all populations. As developers and manufacturers, we play a part in this by designing and testing products in diverse patient groups. Bias in training data can lead to systems that unintentionally reproduce or normalize racial, gender, age, economics and/or comorbidities biases. Clinical validation conducted across such intended applicable population demographics are essential in furthering health equity.
One example given during the FDA conference was about an algorithm that was used to look at chronic diseases and determine whether they were going to get worse in the years to come and direct resources to them now to prevent hospitalizations in the future or worsening of their disease. The way the algorithm learned was looking at data based on resource use as a proxy for health needs. Due to inequities in access to healthcare, the tool learned those biases. It was found that if only the race was changed in a data set, the tool directed more resources towards white patients than to the black patients.
Can you describe how you know who Waldo is? What about Wilma? 
– What if there are a few Wilma’s in the crowd with various hairdos? Or perhaps Waldo is in an environment that sort of masks him? 
We need to direct equity questions early on and ask questions: Are we including diverse patients? Are we deploying tools to diverse communities? Are we able to instill trust and confidence in the applicability of the AI/ML tool to MY patient?
Qserve is here to help with verification, analytical and clinical study designs. We can help tackle the difficult questions of training sets, population representation, intended use. Our Clinical Research Organization (CRO) is equipped and ready to assist with all or parts of the design, conduct, recording and reporting of your clinical studies in full compliance with good clinical practices and regulatory requirements. Together we can further health equity across the regions and help deliver safe and effective medical devices and diagnostics.