TY - JOUR
T1 - Exploring the risks of automation bias in healthcare artificial intelligence applications
T2 - A Bowtie analysis
AU - Abdelwanis, Moustafa
AU - Alarafati, Hamdan Khalaf
AU - Tammam, Maram Muhanad Saleh
AU - Simsekler, Mecit Can Emre
N1 - Publisher Copyright:
© 2024 China Science Publishing & Media Ltd.
PY - 2024/12
Y1 - 2024/12
N2 - This study conducts an in-depth review and Bowtie analysis of automation bias in AI-driven Clinical Decision Support Systems (CDSSs) within healthcare settings. Automation bias, the tendency of human operators to over-rely on automated systems, poses a critical challenge in implementing AI-driven technologies. To address this challenge, Bowtie analysis is employed to examine the causes and consequences of automation bias affected by over-reliance on AI-driven systems in healthcare. Furthermore, this study proposes preventive measures to address automation bias during the design phase of AI model development for CDSSs, along with effective mitigation strategies post-deployment. The findings highlight the imperative role of a systems approach, integrating technological advancements, regulatory frameworks, and collaborative endeavors between AI developers and healthcare practitioners to diminish automation bias in AI-driven CDSSs. We further identify future research directions, proposing quantitative evaluations of the mitigation and preventative measures.
AB - This study conducts an in-depth review and Bowtie analysis of automation bias in AI-driven Clinical Decision Support Systems (CDSSs) within healthcare settings. Automation bias, the tendency of human operators to over-rely on automated systems, poses a critical challenge in implementing AI-driven technologies. To address this challenge, Bowtie analysis is employed to examine the causes and consequences of automation bias affected by over-reliance on AI-driven systems in healthcare. Furthermore, this study proposes preventive measures to address automation bias during the design phase of AI model development for CDSSs, along with effective mitigation strategies post-deployment. The findings highlight the imperative role of a systems approach, integrating technological advancements, regulatory frameworks, and collaborative endeavors between AI developers and healthcare practitioners to diminish automation bias in AI-driven CDSSs. We further identify future research directions, proposing quantitative evaluations of the mitigation and preventative measures.
KW - Artificial intelligence
KW - Automation bias
KW - Bowtie analysis
KW - Decision support systems
KW - Medical errors
KW - Patient safety
UR - http://www.scopus.com/inward/record.url?scp=85206286839&partnerID=8YFLogxK
U2 - 10.1016/j.jnlssr.2024.06.001
DO - 10.1016/j.jnlssr.2024.06.001
M3 - Review article
AN - SCOPUS:85206286839
SN - 2096-7527
VL - 5
SP - 460
EP - 469
JO - Journal of Safety Science and Resilience
JF - Journal of Safety Science and Resilience
IS - 4
ER -