Abstract:Abnormal behavior detection in crowded scenes has drawn extensive interests in computer vision community due to its various applications, e.g., video surveillance, video analytics, and action recognition. However, it is still a challenging task due to the large-scale variation, perspective distortion, and labeling biases. In this paper, an abnormal crowd behavior detection method based on streak flow and convolutional neural network (SFCNN–ABD) was proposed to address this problem. The SFCNN–ABD was composed of two stream CNNs, in which the backbone network was composed of two deep residual networks including spatial network and temporal network respectively. The spatial network utilized the raw video frames to extract the appearance features of crowded scenes, whereas the temporal network used the streak flow to extract the kinematic features. The streak flow can accurately recognize the spatial and temporal changes in the scene, and thus can further boost the proposed the performance. For abnormal behavior detection, salient spatial features were learned via a convolutional neural network (CNN) and temporal features with the aid of streak flow and CNN. Finally, the outputs of these two networks were averaged to detect the abnormal crowd behavior. Experimental results on UMN and VIF benchmark datasets indicate that the proposed method outperforms several state-of-the-art methods in terms of the accuracy and thus demonstrates the effectiveness of the proposed method.