Advanced Visual Learning Frameworks for Detecting Security Threats in X-ray Baggage Scans

Abstract

Since the early 1970s, X-ray-based imaging systems have played a crucial role in mitigating security risks by enabling the identification of potential threats within baggage and cargo. However, manually scrutinizing highly cluttered baggage scans for prohibited items requires expertise and vigilance. The overlapping contours of occluded objects and the infrequent occurrence of illegal items in daily monitoring further complicate this task. Manual threat detection during rush hours can be particularly challenging for even well-trained staff due to exhausting work schedules, stress, or monotony. Studies indicate that human operators achieve only 80 to 90 percent success, a critical shortfall considering the potential consequences

Advancements in computer vision, particularly deep learning models, have led to the development of several computer-aided screening systems for threat identification. However, unlike conventional object detectors for RGB images, X-ray detectors face limited performance due to inherent challenges such as severe class imbalance, limited datasets, heavy occlusion, and lack of prior knowledge. Our comprehensive survey of the literature reveals that few approaches address the class imbalance in X-ray baggage security screening. Furthermore, most proposed approaches tackle only specific problems, such as image enhancement or occlusion, and are often evaluated on a single dataset or require extensive para metric configurations based on different scanner specifications.

To address these limitations, this thesis explores the potential of visual transformers for security threat recognition in highly imbalanced and cluttered datasets, presenting the first attempt to apply transformers to the heavily imbalanced abnormality classification settings in X-ray baggage threat recognition. Recognizing the heavy reliance of most approaches on densely annotated training data—which are laborious and expensive to procure—this thesis also introduces a novel context-aware transformer architecture for weakly supervised baggage threat localization. This architecture leverages transformers’ ability to learn long-range semantic relations, capturing the object-level context of concealed baggage threats. Both proposed methods were evaluated on public X-ray security benchmarks and compared with state-of-the-art methods using various performance criteria to validate their effectiveness.

Additionally, this thesis proposes a contour-driven broad learning system for the detection and segmentation of occluded prohibited items in cluttered X-ray baggage imagery, addressing severe class imbalance and occlusion using a single training routine with limited scans. This is the first exploration of broad learning systems for detection and segmentation using complex, deep hierarchical representations. The framework is trained with minimal supervision using resource-efficient image-level labels rather than labor-intensive pixel-level annotations.

The contributions of this thesis are fourfold: (1) a comprehensive survey of the existing literature on X-ray baggage security screening methods; (2) explore the potential of visual transformers for threat recognition in highly imbalanced and cluttered datasets; (3) a context-aware transformer for weakly supervised threat localization, reducing the dependency on extensive annotations; and (4) a semi-supervised broad learning system for effective threat detection and segmentation in X-ray scans. Finally, the thesis provides a comprehensive evaluation and comparative analysis with state-of-the-art methods, demonstrating significant performance improvements.

Date of Award	16 Dec 2024
Original language	American English
Supervisor	Naoufel Werghi (Supervisor)