Development of Human Activity Recognition (HAR) for Health Rehabilitation Using MMWAVE Radar with 3D Point Cloud Data

Raden Rofiq Yudha Setyawan; Fiky Y. Suratman; Khilda Afifah

doi:10.35882/jeeemi.v7i2.642

Raden Rofiq Yudha Setyawan Telkom University Center of Excellence Intelligence Sensing-IoT, Telkom University, Bandung, Indonesia https://orcid.org/0009-0002-6940-492X
Fiky Y. Suratman Telkom University Center of Excellence Intelligence Sensing-IoT, Telkom University, Bandung, Indonesia; School of Electrical Engineering, Telkom University, Bandung, Indonesia https://orcid.org/0000-0003-4212-5242
Khilda Afifah Telkom University Center of Excellence Intelligence Sensing-IoT, Telkom University, Bandung, Indonesia; School of Electrical Engineering, Telkom University, Bandung, Indonesia https://orcid.org/0009-0009-1048-2166

DOI: https://doi.org/10.35882/jeeemi.v7i2.642

Abstract

Postoperative recovery is a crucial phase in ensuring successful rehabilitation. However, many healthcare facilities face challenges due to the limited availability of medical personnel, making routine patient monitoring difficult. This limitation can delay the early detection of complications and reduce overall recovery effectiveness. To address this issue, this study proposes a non-invasive, radar-based system for remote postoperative patient monitoring. The proposed system utilizes the IWR6843AOP radar to generate 3D point cloud data, spatially representing patient movements. This approach enables continuous monitoring without compromising patient privacy, allowing healthcare providers to offer more efficient care. The collected data undergoes preprocessing, including normalization, labeling, and dataset splitting, before being classified using deep learning models such as 3D CNN, 3D CNN+LSTM, 3D CNN+Bi-LSTM, PointNet, PointNet++, and RNN. The dataset consists of six activity categories: empty space, sitting, standing, walking, running, and squatting, recorded at a frame frequency of 18.18 Hz. Experimental results show that the 3D CNN combined with Bi-LSTM achieves the highest accuracy of 90%, surpassing models like PointNet and RNN. These findings indicate that a radar-based and deep learning-driven approach offers an accurate, efficient, and non-intrusive solution for postoperative monitoring, reducing the need for direct medical supervision. This technology has significant potential for broader healthcare applications, contributing to more advanced, accessible, and technology-driven patient monitoring systems. By integrating artificial intelligence and radar sensing, this research paves the way for innovative solutions in modern healthcare, ensuring better postoperative outcomes while optimizing medical resources.

Downloads

Download data is not yet available.

References

F. Brodersen, J. Wagner, F. G. Uzunoglu, and C. Petersen-Ewert, “Impact of Preoperative Patient Education on Postoperative Recovery in Abdominal Surgery: A Systematic Review,” World J. Surg., vol. 47, no. 4, pp. 937–947, 2023, doi: 10.1007/s00268-022-06884-4.

T. C. De Klerk, D. M. Dounavi, D. F. Hamilton, N. D. Clement, and K. T. Kaliarntas, “Effects of home-based prehabilitation on pre- and postoperative outcomes following total hip and knee arthroplasty: A systematic review and meta-analysis,” Bone Jt. Open, vol. 4, no. 5, pp. 315–328, 2023, doi: 10.1302/2633-1462.45.BJO-2023-0021.

K. N. Dainty, M. Bianca Seaton, and P. Richard Verbeek, “Moving from physical survival to psychologic recovery: a qualitative study of survivor perspectives on long-term outcome after sudden cardiac arrest,” Resusc. Plus, vol. 5, no. September, p. 100055, 2021, doi: 10.1016/j.resplu.2020.100055.

W. H. O. (WHO), Global lob ppatient safety report 2024, vol. 15, no. 1. 2024.

S. Montaha, S. Azam, A. K. M. R. H. Rafid, M. Z. Hasan, A. Karim, and A. Islam, “TimeDistributed-CNN-LSTM: A Hybrid Approach Combining CNN and LSTM to Classify Brain Tumor on 3D MRI Scans Performing Ablation Study,” IEEE Access, vol. 10, pp. 60039–60059, 2022, doi: 10.1109/ACCESS.2022.3179577.

A. K. Alhazmi et al., “Intelligent Millimeter-Wave System for Human Activity Monitoring for Telemedicine,” Sensors, vol. 24, no. 1, pp. 1–23, 2024, doi: 10.3390/s24010268.

F. Yan, N. Li, A. M. Iliyasu, A. S. Salama, and K. Hirota, “Insights into security and privacy issues in smart healthcare systems based on medical images,” J. Inf. Secur. Appl., vol. 78, no. October, p. 103621, 2023, doi: 10.1016/j.jisa.2023.103621.

A. P. N.Lyons, A. Santra, “Improved Deep Representation Learning for Human Activity Recognition using IMU Sensors,” 2021 20th IEEE Int. Conf. Mach. Learn. Appl. (ICMLA)., vol. 10.1109/IC, pp. 326–332, 2021, [Online]. Available: https://ieeexplore.ieee.org/document/9680027

A. D. Singh, S. S. Sandha, L. Garcia, and M. Srivastava, “Radhar: Human activity recognition from point clouds generated through a millimeter-wave radar,” Proc. Annu. Int. Conf. Mob. Comput. Networking, MOBICOM, pp. 51–56, 2019, doi: 10.1145/3349624.3356768.

K. Boikanyo, A. M. Zungeru, B. Sigweni, A. Yahya, and C. Lebekwe, “Remote patient monitoring systems: Applications, architecture, and challenges,” Sci. African, vol. 20, 2023, doi: 10.1016/j.sciaf.2023.e01638.

W. Gu, Zhanzhong, He, Xiangjian, Fang, Gengfa, Xu, Chengpei, Xia, Feng. Jia, “MMWAVE radar based HAR for health care monitoring robot.pdf,” 2024, Archive Prefix: arXiv.

Texas Instruments, “Texas Instruments Official Website,” 2024, [Online]. Available: https://www.ti.com/

S. An and U. Y. Ogras, “MARS: mmWave-based Assistive Rehabilitation System for Smart Healthcare,” ACM Trans. Embed. Comput. Syst., vol. 20, no. 5s, pp. 1–22, 2021, doi: 10.1145/3477003.

K. Mirzaei, M. Arashpour, E. Asadi, H. Masoumi, Y. Bai, and A. Behnood, “3D point cloud data processing with machine learning for construction and infrastructure applications: A comprehensive review,” Adv. Eng. Informatics, vol. 51, p. 101501, 2022.

A. Gorji, H. U. R. Khalid, A. Bourdoux, and H. Sahli, “On the Generalization and Reliability of Single Radar-Based Human Activity Recognition,” IEEE Access, vol. 9, pp. 85334–85349, 2021, doi: 10.1109/ACCESS.2021.3088452.

V. C. Chen, F. Li, S. S. Ho, and H. Wechsler, “Micro-doppler effect in radar: Phenomenon, model, and simulation study,” IEEE Trans. Aerosp. Electron. Syst., vol. 42, no. 1, pp. 2–21, 2006, doi: 10.1109/TAES.2006.1603402.

C. Zhou, “Review: mmWave Radar Point Cloud Processing Technology for Human Activity and Posture Recognition,” 2023.

A. Prabhakara et al., “High Resolution Point Clouds from mmWave Radar,” Proc. - IEEE Int. Conf. Robot. Autom., vol. 2023-May, pp. 4135–4142, 2023, doi: 10.1109/ICRA48891.2023.10161429.

I. G. Van De Zande, “3D Point Cloud Object Detection for Millimeter Wave Radar: a Synthesis Study,” no. September, 2023.

C. R. Qi, H. Su, K. Mo, and L. J. Guibas, “PointNet: Deep learning on point sets for 3D classification and segmentation,” Proc. - 30th IEEE Conf. Comput. Vis. Pattern Recognition, CVPR 2017, vol. 2017-Janua, pp. 77–85, 2017, doi: 10.1109/CVPR.2017.16.

J. Zhu, H. Chen, and W. Ye, “A Hybrid CNN-LSTM Network for the Classification of Human Activities Based on Micro-Doppler Radar,” IEEE Access, vol. 8, pp. 24713–24720, 2020, doi: 10.1109/ACCESS.2020.2971064.

L. Cao, S. Liang, Z. Zhao, D. Wang, C. Fu, and K. Du, “Human Activity Recognition Method Based on FMCW Radar Sensor with Multi-Domain Feature Attention Fusion Network,” Sensors, vol. 23, no. 11, 2023, doi: 10.3390/s23115100.

Y. Wang, H. Liu, K. Cui, A. Zhou, W. Li, and H. Ma, “m-Activity : ACCURATE AND REAL-TIME HUMAN ACTIVITY RECOGNITION VIA MILLIMETER WAVE RADAR Beijing University of Posts and Telecommunications , School of Computer Science,” no. Dl, pp. 8298–8302, 2021.

B. J. Gluckman, T. I. Netoff, E. J. Neel, W. L. Spano, M. L. Spano, and S. J. Schiff, “Stochastic Resonance in a Neuronal Network from Mammalian Brain,” Phys. Rev. Lett., vol. 77, no. 19, pp. 4098–4101, 1996, doi: 10.1103/PhysRevLett.77.4098.

Y. Lin, H. Li, and D. Faccio, “Human Multi-Activities Classification Using mmWave Radar: Feature Fusion in Time-Domain and PCANet,” in Sensors, 2024, pp. 1–24. doi: 10.3390/s24165450.

H. Zhou, Y. Zhao, Y. Liu, S. Lu, X. An, and Q. Liu, “Multi-Sensor Data Fusion and CNN-LSTM Model for Human Activity Recognition System,” Sensors, vol. 23, no. 10, 2023, doi: 10.3390/s23104750.

H. Khalid et al., “Arsitektur Multi-View CNN-LSTM untuk Pengenalan Aktivitas Manusia Berbasis Radar,” pp. 24509–24519, 2022.

A. H. Victoria, S. V Manikanthan, M. A. Wildan, and K. H. Kishore, “International Journal of Communication Networks and Information Security Radar Based Activity Recognition using CNN-LSTM Network Architecture Article History,” vol. 14, no. 3, pp. 303–312, 2022, [Online]. Available: https://ijcnis.org

D. Tran, L. Bourdev, R. Fergus, L. Torresani, and M. Paluri, “Learning spatiotemporal features with 3D convolutional networks,” Proc. IEEE Int. Conf. Comput. Vis., vol. 2015 Inter, pp. 4489–4497, 2015, doi: 10.1109/ICCV.2015.510.

Z. Hameed and B. Garcia-Zapirain, “Sentiment Classification Using a Single-Layered BiLSTM Model,” IEEE Access, vol. 8, pp. 73992–74001, 2020, doi: 10.1109/ACCESS.2020.2988550.

X. Glorot and Y. Bengio, “Understanding the difficulty of training deep feedforward neural networks,” J. Mach. Learn. Res., vol. 9, pp. 249–256, 2010.

D. P. Kingma and J. L. Ba, “Adam: A method for stochastic optimization,” 3rd Int. Conf. Learn. Represent. ICLR 2015 - Conf. Track Proc., pp. 1–15, 2015.

A. Mao, M. Mohri, and Y. Zhong, “Cross-Entropy Loss Functions: Theoretical Analysis and Applications,” Proc. Mach. Learn. Res., vol. 202, pp. 23803–23828, 2023.

R. O. Sinnott, F. Wu, and W. Chen, “A Mobile Application for Dog Breed Detection and Recognition Based on Deep Learning,” Proc. - 5th IEEE/ACM Int. Conf. Big Data Comput. Appl. Technol. BDCAT 2018, pp. 87–96, 2018, doi: 10.1109/BDCAT.2018.00019.

M. Trăscău, M. Nan, and A. M. Florea, “Spatio-temporal features in action recognition using 3D skeletal joints,” Sensors (Switzerland), vol. 19, no. 2, 2019, doi: 10.3390/s19020423.