An appraisal of the performance of AI tools for chronic stroke lesion segmentation

Automated demarcation of stoke lesions from monospectral magnetic resonance imaging scans is extremely useful for diverse research and clinical applications, including lesion-symptom mapping to explain deficits and predict recovery. There is a significant surge of interest in the development of supervised artificial intelligence (AI) methods for that purpose, including deep learning, with a performance comparable to trained experts. Such AI-based methods, however, require copious amounts of data. Thanks to the availability of large datasets, the development of AI-based methods for lesion segmentation has immensely accelerated in the last decade. One of these datasets is the Anatomical Tracings of Lesions After Stroke (ATLAS) dataset which includes T1-weighted images from hundreds of chronic stroke survivors with their manually traced lesions. This systematic review offers an appraisal of the impact of the ATLAS dataset in promoting the development of AI-based segmentation of stroke lesions. An examination of all published studies, that used the ATLAS dataset to both train and test their methods, highlighted an overall moderate performance (median Dice index = 59.40%) and a huge variability across studies in terms of data preprocessing, data augmentation, AI architecture, and the mode of operation (two-dimensional versus three-dimensional methods). Perhaps most importantly, almost all AI tools were borrowed from existing AI architectures in computer vision, as 90% of all selected studies relied on conventional convolutional neural network-based architectures. Overall, current research has not led to the development of robust AI architectures than can handle spatially heterogenous lesion patterns. This review also highlights the difficulty of gauging the performance of AI tools in the presence of uncertainties in the definition of the ground truth.