DLSeF: A dynamic key-length-based efficient real-time security verification model for big data stream

Applications in risk-critical domains such as emergency management and industrial control systems need near-real-time stream data processing in large-scale sensing networks. The key problem is how to ensure online end-to-end security (e.g., confidentiality, integrity, and authenticity) of data streams for such applications. We refer to this as an online security verification problem. Existing data security solutions cannot be applied in such applications as they cannot deal with data streams with high-volume and high-velocity data in real time. They introduce a significant buffering delay during security verification, resulting in a requirement for a large buffer size for the stream processing server. To address this problem, we propose a Dynamic Key-Length-Based Security Framework (DLSeF) based on a shared key derived from synchronized prime numbers; the key is dynamically updated at short intervals to thwart potential attacks to ensure end-to-end security. Theoretical analyses and experimental results of the DLSeF framework show that it can significantly improve the efficiency of processing stream data by reducing the security verification time and buffer usage without compromising security.