Categorizing Crowd Emotions based on Cross Division Expressions and Anomalies
Abstract
The crowd emotion sensing is a critical element in surveillance and management of the crowd in different environments. With exploding populations, and developing nations, the crowd in urban cities mandate state of art surveillance methodologies involving continuous monitoring and reporting of criminal activities. The research article presents a novel technique to compute the spatial and temporal features obtained from the crowd environments and combine the novelty of neural networks for detecting the emotions of crowds with better accuracy and swiftness. The features are obtained from the continuous feed of surveillance videos typically categorized into the common features of human beings namely anger, sadness, disgust, surprise, fear, happiness and obviously neutrality. Such features are extracted after careful background separation which are typically difficult in crowded environments, using techniques namely SIFT, and FAST termed to be the visual descriptors. Once the features are extracted, spatial and temporal features are classified into individual and combined features as defined in the cross-division environment in order to portray the crowd dynamics and characteristics. Cross division environment computes the necessary features for identifying the anomalies in the crowded situations in a neural network, after a series of operations such as dimensionality reduction, and principal component analysis. From the semantic information, crowd behaviours are detected based on interactive features in a dynamic environment and the proposed technique has demonstrated effective results in terms of 98.9% accuracy in detecting especially violence in crowd datasets collected from UMN.
Downloads
References
Manoj Kumar .K, L. Sujihelen. (2022). Recognising Actions with Segmentation and Prediction Techniques in ROI based Deep Learning Framework. Mathematical Statistician and Engineering Applications, 71(4), 4072–4090. https://doi.org/10.17762/msea.v71i4.977
M. K and L. Sujihelen, "Behavioural Analysis For Prospects In Crowd Emotion Sensing: A Survey," 2021 Third International Conference on Inventive Research in Computing Applications (ICIRCA), Coimbatore, India, 2021, pp. 735-743, doi: 10.1109/ICIRCA51532.2021.9544607.
W. Halboob, H. Altaheri, A. Derhab and J. Almuhtadi, "Crowd Management Intelligence Framework: Umrah Use Case," in IEEE Access, vol. 12, pp. 6752-6767, 2024, doi: 10.1109/ACCESS.2024.3350188.
Liu, D.; Liu, W.; Yuan, X.; Jiang, Y. Conscious and Unconscious Processing of Ensemble Statistics Oppositely Modulate Perceptual Decision-Making. Am. Psychol. 2023, 78, 346–357.
List of Human Stampedes and Crushes, Aug. 2022, [online] Available: https://en.wikipedia.org/wiki/List_of_human_stampedes_and_crushes.
Aljuaid, H.; Akhter, I.; Alsufyani, N.; Shorfuzzaman, M.; Alarfaj, M.; Alnowaiser, K.; Jalal, A.; Park, J. Postures anomaly tracking and prediction learning model over crowd data analytics. PeerJ Comput. Sci. 2023, 9, e1355.
Y. Wang, X. Luo and Z. Zhou, "Contrasting Estimation of Pattern Prototypes for Anomaly Detection in Urban Crowd Flow," in IEEE Transactions on Intelligent Transportation Systems, vol. 25, no. 8, pp. 10231-10245, Aug. 2024, doi: 10.1109/TITS.2024.3355143.
A. M. Al-Shaery et al., "Open Dataset for Predicting Pilgrim Activities for Crowd Management During Hajj Using Wearable Sensors," in IEEE Access, vol. 12, pp. 72828-72846, 2024, doi: 10.1109/ACCESS.2024.3402230.
L. Luo, Y. Li, H. Yin, S. Xie, R. Hu and W. Cai, "Crowd-level abnormal behavior detection via multi-scale motion consistency learning", Proc. AAAI Conf. Artif. Intell., pp. 8984-8992, 2023.
L. Luo, S. Xie, H. Yin, C. Peng and Y. -S. Ong, "Detecting and Quantifying Crowd-Level Abnormal Behaviors in Crowd Events," in IEEE Transactions on Information Forensics and Security, vol. 19, pp. 6810-6823, 2024, doi: 10.1109/TIFS.2024.3423388.
X.-C. Liao, W.-N. Chen, X.-Q. Guo, J. Zhong and X.-M. Hu, "Crowd management through optimal layout of fences: An ant colony approach based on crowd simulation", IEEE Trans. Intell. Transp. Syst., vol. 24, no. 9, pp. 9137-9149, Sep. 2023.
F. Abdullah, M. Abdelhaq, R. Alsaqour, M. H. Alatiyyah, K. Alnowaiser, S. S. Alotaibi, et al.,"Context aware crowd tracking and anomaly detection via deep learning and social force model", IEEE Access, vol. 11, pp. 75884-75898, 2023.
X. Ding, F. He, Z. Lin, Y. Wang, H. Guo and Y. Huang, "Crowd density estimation using fusion of multi-layer features", IEEE Trans. Intell. Transp. Syst., vol. 22, no. 8, pp. 4776-4787, Aug. 2021.
T. H. Noor, "Behavior analysis-based IoT services for crowd management", Comput. J., vol. 66, no. 9, pp. 2208-2219, Sep. 2023.
S. Alsubai et al., "Design of Artificial Intelligence Driven Crowd Density Analysis for Sustainable Smart Cities," in IEEE Access, vol. 12, pp. 121983-121993, 2024, doi: 10.1109/ACCESS.2024.3390049.
L. Luo, Y. Li, H. Yin, S. Xie, R. Hu and W. Cai, "Crowd-level abnormal behavior detection via multi-scale motion consistency learning", Proc. AAAI Conf. Artif. Intell., pp. 8984-8992, 2023.
R. Zhao et al., "Dynamic crowd accident-risk assessment based on internal energy and information entropy for large-scale crowd flow considering COVID-19 epidemic", IEEE Trans. Intell. Transp. Syst., vol. 23, no. 10, pp. 17466-17478, Oct. 2022.
T. Yang, C. Wang, T. Zhou, Z. Cai, K. Wu and B. Hou, "Identification of anomalous behavioral patterns in crowd scenes", Comput. Mater. Continua, vol. 71, no. 1, pp. 925-939, 2022.
S. Yadav, P. Gulia, N. S. Gill and J. M. Chatterjee, "A real-time crowd monitoring and management system for social distance classification and healthcare using deep learning", J. Healthcare Eng., vol. 2022, pp. 1-11, Apr. 2022.
J. Long, W. Liang, K.-C. Li, Y. Wei and M. D. Marino, "A regularized cross-layer ladder network for intrusion detection in industrial Internet of Things", IEEE Trans. Ind. Informat., vol. 19, no. 2, pp. 1747-1755, Feb. 2023.
Y. Li, Z. Xie, B. Li and M. Mohiuddin, "The impacts of in situ urbanization on housing mobility and employment of local residents in China", Sustainability, vol. 14, no. 15, pp. 9058, Jul. 2022.
G. Yu, S. Wang, Z. Cai, X. Liu, E. Zhu and J. Yin, "Video anomaly detection via visual cloze tests", IEEE Trans. Inf. Forensics Security, vol. 18, pp. 4955-4969, 2023.
M. Zhang, T. Li, Y. Yu, Y. Li, P. Hui and Y. Zheng, "Urban anomaly analytics: Description detection and prediction", IEEE Trans. Big Data, vol. 8, no. 3, pp. 809-826, Jun. 2022.
R. Lalit and R. K. Purwar, "Crowd abnormality detection using optical flow and GLCM-based texture features", J. Inf. Technol. Res., vol. 15, no. 1, pp. 1-15, Jun. 2022.
T. N. Nguyen and S. Zeadally, "Mobile crowd-sensing applications: Data redundancies challenges and solutions", ACM Trans. Internet Technol., vol. 22, no. 2, pp. 1-15, May 2022.
S. Zhang, Y. Yang, W. Liang, V. K. A. Sandor, G. Xie and K. R. Choo, "MKSS: An effective multi-authority keyword search scheme for edge-cloud collaboration", J. Syst. Archit., vol. 144, 2023.
P. Martí, L. Serrano-Estrada, A. Nolasco-Cirugeda and J. L. Baeza, "Revisiting the spatial definition of neighborhood boundaries: Functional clusters versus administrative neighborhoods", J. Urban Technol., vol. 29, no. 3, pp. 73-94, Jul. 2022.
C. Chen et al., "Comprehensive regularization in a bi-directional predictive network for video anomaly detection", Proc. AAAI Conf. Artif. Intell., vol. 36, no. 1, pp. 230-238, Jun. 2022.
L. Deng, D. Lian, Z. Huang and E. Chen, "Graph convolutional adversarial networks for spatiotemporal anomaly detection", IEEE Trans. Neural Netw. Learn. Syst., vol. 33, no. 6, pp. 2416-2428, Jun. 2022.
T. Yang, C. Wang, T. Zhou, Z. Cai, K. Wu and B. Hou, "Identification of anomalous behavioral patterns in crowd scenes", Comput. Mater. Continua, vol. 71, no. 1, pp. 925-939, 2022.
Y. Liu, X. Liu, X. Li, M. Li and Y. Li, "Participants recruitment for coverage maximization by mobility predicting in mobile crowd sensing", China Commun., vol. 20, no. 8, pp. 163-176, Aug. 2023.
Y. Chen and A. Deng, "Using POI data and Baidu migration big data to modify nighttime light data to identify urban and rural area", IEEE Access, vol. 10, pp. 93513-93524, 2022.
C. Cao, Y. Lu and Y. Zhang, "Context recovery and knowledge retrieval: A novel two-stream framework for video anomaly detection", IEEE Trans. Image Process., vol. 33, pp. 1810-1825, 2024.
C. H. Liu et al., "Modeling citywide crowd flows using attentive convolutional LSTM", Proc. IEEE 37th Int. Conf. Data Eng. (ICDE), pp. 217-228, Apr. 2021.
J. Zhang and X. Zhang, "Multi-task allocation in mobile crowd sensing with mobility prediction", IEEE Trans. Mobile Comput., vol. 22, no. 2, pp. 1081-1094, Feb. 2023.
H. Wang, S. Zeng, Y. Li and D. Jin, "Predictability and prediction of human mobility based on application-collected location data", IEEE Trans. Mobile Comput., vol. 20, no. 7, pp. 2457-2472, Jul. 2021.
T. Liu, C. Zhang, K.-M. Lam and J. Kong, "Decouple and resolve: Transformer-based models for online anomaly detection from weakly labeled videos", IEEE Trans. Inf. Forensics Security, vol. 18, pp. 15-28, 2023.
G. Woo, C. Liu, D. Sahoo, A. Kumar and S. Hoi, "CoST: Contrastive learning of disentangled seasonal-trend representations for time series forecasting", Proc. Int. Conf. Learn. Represent., pp. 1-18, 2022.
X. Liu and J. Liu, "A truthful double auction mechanism for multi-resource allocation in crowd sensing systems", IEEE Trans. Serv. Comput., vol. 15, no. 5, pp. 2579-2590, Sep./Oct. 2022.
P. Wu et al., "VadCLIP: Adapting vision-language models for weakly supervised video anomaly detection", Proc. AAAI Conf. Artif. Intell., pp. 6074-6082, 2024.
Y. Yang, C. Zhang, T. Zhou, Q. Wen and L. Sun, "DCdetector: Dual attention contrastive representation learning for time series anomaly detection", Proc. 29th ACM SIGKDD Int. Conf. Knowl. Discovery Data Mining (KDD), pp. 3033-3045, 2023.
S. Zhang, J. He, W. Liang and K. Li, "MMDS: A secure and verifiable multimedia data search scheme for cloud-assisted edge computing", Future Gener. Comput. Syst., vol. 151, pp. 32-44, 2024.
Copyright (c) 2025 Manojkumar Kothandapani, Suji Helen L
This work is licensed under a Creative Commons Attribution-ShareAlike 4.0 International License.
Authors who publish with this journal agree to the following terms:
- Authors retain copyright and grant the journal right of first publication with the work simultaneously licensed under a Creative Commons Attribution-ShareAlikel 4.0 International (CC BY-SA 4.0) that allows others to share the work with an acknowledgement of the work's authorship and initial publication in this journal.
- Authors are able to enter into separate, additional contractual arrangements for the non-exclusive distribution of the journal's published version of the work (e.g., post it to an institutional repository or publish it in a book), with an acknowledgement of its initial publication in this journal.
- Authors are permitted and encouraged to post their work online (e.g., in institutional repositories or on their website) prior to and during the submission process, as it can lead to productive exchanges, as well as earlier and greater citation of published work (See The Effect of Open Access).
.png)
.png)
.png)
.png)
.png)
