Exploring the risks of automation bias in healthcare artificial intelligence applications: A Bowtie analysis

Moustafa Abdelwanis, Hamdan Khalaf Alarafati, Maram Muhanad Saleh Tammam, Mecit Can Emre Simsekler

Research output: Contribution to journalReview articlepeer-review

16 Scopus citations

Abstract

This study conducts an in-depth review and Bowtie analysis of automation bias in AI-driven Clinical Decision Support Systems (CDSSs) within healthcare settings. Automation bias, the tendency of human operators to over-rely on automated systems, poses a critical challenge in implementing AI-driven technologies. To address this challenge, Bowtie analysis is employed to examine the causes and consequences of automation bias affected by over-reliance on AI-driven systems in healthcare. Furthermore, this study proposes preventive measures to address automation bias during the design phase of AI model development for CDSSs, along with effective mitigation strategies post-deployment. The findings highlight the imperative role of a systems approach, integrating technological advancements, regulatory frameworks, and collaborative endeavors between AI developers and healthcare practitioners to diminish automation bias in AI-driven CDSSs. We further identify future research directions, proposing quantitative evaluations of the mitigation and preventative measures.

Original languageBritish English
Pages (from-to)460-469
Number of pages10
JournalJournal of Safety Science and Resilience
Volume5
Issue number4
DOIs
StatePublished - Dec 2024

Keywords

  • Artificial intelligence
  • Automation bias
  • Bowtie analysis
  • Decision support systems
  • Medical errors
  • Patient safety

Fingerprint

Dive into the research topics of 'Exploring the risks of automation bias in healthcare artificial intelligence applications: A Bowtie analysis'. Together they form a unique fingerprint.

Cite this