Comprehensive review of reinforcement learning for medical ultrasound imaging

Medical Ultrasound (US) imaging has seen increasing demands over the past years, becoming one of the most preferred imaging modalities in clinical practice due to its affordability, portability, and real-time capabilities. However, it faces several challenges that limit its applicability, such as operator dependency, variability in interpretation, and limited resolution, which are amplified by the low availability of trained experts. This calls for the need of autonomous systems that are capable of reducing the dependency on humans for increased efficiency and throughput. Reinforcement Learning (RL) comes as a rapidly advancing field under Artificial Intelligence (AI) that allows the development of autonomous and intelligent agents through rewarded interactions with their environments. Several existing surveys on advancements in US imaging predominantly focus on partially autonomous AI solutions. However, none of these surveys explore the intersection between the stages of the US process and the recent advancements in RL solutions. To bridge this gap, this survey proposes a comprehensive taxonomy that integrates the stages of the US process with the RL development pipeline -including data preparation, problem formulation, simulation environment, RL training, validation and finetuning- and reviews current research efforts under this taxonomy. This work aims to highlight the potential of RL in building autonomous US solutions while identifying limitations and opportunities for further advancements in this field.