Predicting Disaster Type from Social Media Imagery via Deep Neural Networks Directed by Visual Attention

Social media has become the primary source for the public for seeking news and updates in crisis such as disasters. However, the information sought from social media in disasters is usually in the form posts (images or texts) with unorganized content that often contains duplicate, feeds, inappropriate and irrelevant posts. Processing these posts and generating meaningful information out of them is a challenge. This research proposed deep neural network-based design driven by visual-attention mechanism for classifying disaster types from social media imagery. Deep neural networks were applied to raw datasets consisting of 71K images obtained from actual disasters and were split into training validation and test sets. Three approaches were applied including ‘Base Model’, ‘Bottleneck Attention Module’ and ‘Focus Attention Module’. The Base Model showed the highest accuracy, but the Focus Attention Module learnt faster than models and enabled to cut down the training time. The research enhanced disaster management capabilities of government, first responders, non-governmental organizations and other relevant aid agencies.