Optimized PCA Infused Liquid Neural Network Deployment for FPGA-Based Embedded Systems

Bhupesh Deka; Sayanti Chatterjee; S Rao Chintalapudi; Jajala Nikitha; Lakshminarayana Kodavali; Kancharagunta Kishan Babu

doi:10.35882/jeeemi.v7i2.681

Bhupesh Deka Department of Computer Science and Engineering (AIML & IoT), Vallurupalli Nageswara Rao Vignana Jyothi Institute of Engineering &Technology, Hyderabad, India https://orcid.org/0000-0001-9460-5796
Sayanti Chatterjee Department of Computer Science and Engineering (AIML & IoT), Vallurupalli Nageswara Rao Vignana Jyothi Institute of Engineering &Technology, Hyderabad, India
S Rao Chintalapudi Department of Computer Science and Engineering (AI&ML), CMR Technical Campus, Hyderabad, India https://orcid.org/0000-0002-5023-6711
Jajala Nikitha Department of Emerging Technologies (DS,CyS,IoT), Mallareddy College of Engineering and Technology (MRCET), Hyderabad, India https://orcid.org/0009-0009-6178-1967
Lakshminarayana Kodavali Department of Computer Science and Engineering, Koneru Lakshmaiah Education Foundation (KLEF), Guntur, AP, India https://orcid.org/0000-0003-2403-8532
Kancharagunta Kishan Babu Department of Computer Science and Engineering (AIML & IoT), Vallurupalli Nageswara Rao Vignana Jyothi Institute of Engineering &Technology, Hyderabad, India https://orcid.org/0000-0001-5613-804X

DOI: https://doi.org/10.35882/jeeemi.v7i2.681

Abstract

The integration of neural networks into FPGA-based systems has revolutionized embedded computing by offering high performance, energy efficiency, and reconfigurability This paper introduces a novel optimization framework integrating Principal Component Analysis (PCA) to reduce the complexity of input data while preserving essential features for accurate neural network processing. By applying PCA for dimensionality reduction, the computational burden on the FPGA is minimized, enabling more efficient utilization of hardware resources. Combined with hardware-aware optimizations, such as quantization and parallel processing, the proposed approach achieves superior performance in terms of energy efficiency, latency, and resource utilization. Simulation results demonstrate that the PCA-enhanced Liquid Neural Network (LNN) deployment significantly outperforms traditional methods, making it ideal for edge intelligence and other resource-constrained environments. This work emphasizes the synergy of PCA and FPGA optimizations for scalable, high-performance embedded systems. A comparison study using simulation results between cascaded feed forward neural network (CFFNN), deep neural network (DNN) and liquid neural network (LNN) has been encountered here for the embedded system to show the efficacy of PCA based LNN. It has been shown from case studies that the average F1score is 98% in case of proposed methodology and accuracy is also 98.3% for high clock value.

Downloads

Download data is not yet available.

References

P. Patel, “Embedded systems design using FPGA,” in 19th International Conference on VLSI Design held jointly with 5th International Conference on Embedded Systems Design (VLSID’06), IEEE, 2006, p. 1 pp. doi: 10.1109/VLSID.2006.83.

Fuming Sun, Xiaoying Li, Qin Wang, and Chunlin Tang, “FPGA-based embedded system design,” in APCCAS 2008 - 2008 IEEE Asia Pacific Conference on Circuits and Systems, IEEE, Nov. 2008, pp. 733–736. doi: 10.1109/APCCAS.2008.4746128.

Sedat Sonko, Ayodeji Matthew Monebi, Emmanuel Augustine Etukudoh, Femi Osasona, Akoh Atadoga, and Cosmas Dominic Daudu, “REVIEWING THE IMPACT OF EMBEDDED SYSTEMS IN MEDICAL DEVICES IN THE USA,” International Medical Science Research Journal, vol. 4, no. 2, pp. 158–169, Feb. 2024, doi: 10.51594/imsrj.v4i2.767.

A. Kushwah et al., “A novel embedded system for tractor implement performance mapping,” Cogent Eng, vol. 11, no. 1, Dec. 2024, doi: 10.1080/23311916.2024.2311093.

K. P. Seng, P. J. Lee, and L. M. Ang, “Embedded Intelligence on FPGA: Survey, Applications and Challenges,” Electronics (Basel), vol. 10, no. 8, p. 895, Apr. 2021, doi: 10.3390/electronics10080895.

M. Javaid, A. Haleem, R. P. Singh, and R. Suman, “Artificial Intelligence Applications for Industry 4.0: A Literature-Based Study,” Journal of Industrial Integration and Management, vol. 07, no. 01, pp. 83–111, Mar. 2022, doi: 10.1142/S2424862221300040.

R. Vazquez, A. Gordon-Ross, and G. Stitt, “Machine Learning-based Prediction for Dynamic, Runtime Architectural Optimizations of Embedded Systems,” in 2019 IEEE Nordic Circuits and Systems Conference (NORCAS): NORCHIP and International Symposium of System-on-Chip (SoC), IEEE, Oct. 2019, pp. 1–7. doi: 10.1109/NORCHIP.2019.8906901.

T. Gong, T. Fan, J. Guo, and Z. Cai, “GPU-based parallel optimization of immune convolutional neural network and embedded system,” Eng Appl Artif Intell, vol. 62, pp. 384–395, Jun. 2017, doi: 10.1016/j.engappai.2016.08.019.

S. N. Shahrouzi and D. G. Perera, “Dynamic partial reconfigurable hardware architecture for principal component analysis on mobile and embedded devices,” EURASIP Journal on Embedded Systems, vol. 2017, no. 1, p. 25, Dec. 2017, doi: 10.1186/s13639-017-0074-x.

T. S. Ajani, A. L. Imoize, and A. A. Atayero, “An Overview of Machine Learning within Embedded and Mobile Devices–Optimizations and Applications,” Sensors, vol. 21, no. 13, p. 4412, Jun. 2021, doi: 10.3390/s21134412.

A. K. Jain, K. D. Pham, J. Cui, S. A. Fahmy, and D. L. Maskell, “Virtualized Execution and Management of Hardware Tasks on a Hybrid ARM-FPGA Platform,” J Signal Process Syst, vol. 77, no. 1–2, pp. 61–76, Oct. 2014, doi: 10.1007/s11265-014-0884-1.

C. Plessl and M. Platzner, “Zippy - A coarse-grained reconfigurable array with support for hardware virtualization,” in 2005 IEEE International Conference on Application-Specific Systems, Architecture Processors (ASAP’05), IEEE, pp. 213–218. doi: 10.1109/ASAP.2005.69.

P. Bachanna, B. Gadgay, and S. Chatterjee, “Probabilistic Principle Component Analysis based Feature Extraction of Embedded System Applications with Deep Neural Network based Implementation in FPGA,” International Journal on Recent and Innovation Trends in Computing and Communication, vol. 11, no. 6, pp. 45–51, Jul. 2023, doi: 10.17762/ijritcc.v11i6.6771.

S. Chatterjee, “Fault Detection for a Nonlinear Switched Continuous Time Delayed System Using Machine Learning and Self-Switched UKF,” Journal Européen des Systèmes Automatisés, vol. 55, no. 2, pp. 245–251, Apr. 2022, doi: 10.18280/jesa.550212.

D. G. Perera and K. F. Li, “Embedded hardware solution for principal component analysis,” in Proceedings of 2011 IEEE Pacific Rim Conference on Communications, Computers and Signal Processing, IEEE, Aug. 2011, pp. 730–735. doi: 10.1109/PACRIM.2011.6032984.

O. Ituabhor, J. Isabona, J. T. zhimwang, and I. Risi, “Cascade Forward Neural Networks-based Adaptive Model for Real-time Adaptive Learning of Stochastic Signal Power Datasets,” International Journal of Computer Network and Information Security, vol. 14, no. 3, pp. 63–74, Jun. 2022, doi: 10.5815/ijcnis.2022.03.05.

J. Qiu et al., “Going Deeper with Embedded FPGA Platform for Convolutional Neural Network,” in Proceedings of the 2016 ACM/SIGDA International Symposium on Field-Programmable Gate Arrays, New York, NY, USA: ACM, Feb. 2016, pp. 26–35. doi: 10.1145/2847263.2847265.

K. Pradeep, K. Kamalavasan, R. Natheesan, and A. Pasqual, “EdgeNet: SqueezeNet like Convolution Neural Network on Embedded FPGA,” in 2018 25th IEEE International Conference on Electronics, Circuits and Systems (ICECS), IEEE, Dec. 2018, pp. 81–84. doi: 10.1109/ICECS.2018.8617876.

M. Zhao, C. Hu, F. Wei, K. Wang, C. Wang, and Y. Jiang, “Real-Time Underwater Image Recognition with FPGA Embedded System for Convolutional Neural Network,” Sensors, vol. 19, no. 2, p. 350, Jan. 2019, doi: 10.3390/s19020350.

R. T. Syed, Y. Zhao, J. Chen, M. Andjelkovic, M. Ulbricht, and M. Krstic, “FPGA Implementation of a Fault-Tolerant Fused and Branched CNN Accelerator With Reconfigurable Capabilities,” IEEE Access, vol. 12, pp. 57847–57862, 2024, doi: 10.1109/ACCESS.2024.3392240.

S. N. Shahrouzi and D. G. Perera, “Optimized hardware accelerators for data mining applications on embedded platforms: Case study principal component analysis,” Microprocess Microsyst, vol. 65, pp. 79–96, Mar. 2019, doi: 10.1016/j.micpro.2019.01.001.

T. S. Ajani, A. L. Imoize, and A. A. Atayero, “An Overview of Machine Learning within Embedded and Mobile Devices–Optimizations and Applications,” Sensors, vol. 21, no. 13, p. 4412, Jun. 2021, doi: 10.3390/s21134412.

L. N. S. Andreasen Struijk et al., “Development and functional demonstration of a wireless intraoral inductive tongue computer interface for severely disabled persons,” Disabil Rehabil Assist Technol, vol. 12, no. 6, pp. 631–640, Aug. 2017, doi: 10.1080/17483107.2016.1217084.

M. Aqeel Iqbal, F. Azam, U. Saeed Awan, and S. Hammad, “Performance Enhancement Techniques for Modern Reconfigurable Computing Systems,” Int J Comput Appl, vol. 27, no. 9, pp. 33–38, Aug. 2011, doi: 10.5120/3327-4577.

I. Bravo, M. Mazo, J. L. Lázaro, A. Gardel, P. Jiménez, and D. Pizarro, “An Intelligent Architecture Based on Field Programmable Gate Arrays Designed to Detect Moving Objects by Using Principal Component Analysis,” Sensors, vol. 10, no. 10, pp. 9232–9251, Oct. 2010, doi: 10.3390/s101009232.

M. Elnawawy, A. Sagahyroon, and T. Shanableh, “FPGA-Based Network Traffic Classification Using Machine Learning,” IEEE Access, vol. 8, pp. 175637–175650, 2020, doi: 10.1109/ACCESS.2020.3026831.

A. Biglari and W. Tang, “A Review of Embedded Machine Learning Based on Hardware, Application, and Sensing Scheme,” Sensors, vol. 23, no. 4, p. 2131, Feb. 2023, doi: 10.3390/s23042131.

K. Kumar, A. Verma, N. Gupta, and A. Yadav, “Liquid Neural Networks: A Novel Approach to Dynamic Information Processing,” in 2023 International Conference on Advances in Computation, Communication and Information Technology (ICAICCIT), IEEE, Nov. 2023, pp. 725–730. doi: 10.1109/ICAICCIT60255.2023.10466162.

M. Chahine et al., “Robust flight navigation out of distribution with liquid neural networks,” Sci Robot, vol. 8, no. 77, Apr. 2023, doi: 10.1126/scirobotics.adc8892.

P. K. Karn, I. Ardekani, and W. H. Abdulla, “Generalized Framework for Liquid Neural Network upon Sequential and Non-Sequential Tasks,” Mathematics, vol. 12, no. 16, p. 2525, Aug. 2024, doi: 10.3390/math12162525.

R. Hasani, M. Lechner, A. Amini, D. Rus, and R. Grosu, “Liquid Time-constant Networks,” Proceedings of the AAAI Conference on Artificial Intelligence, vol. 35, no. 9, pp. 7657–7666, May 2021, doi: 10.1609/aaai.v35i9.16936.

S. Il Yu, C. Rhee, K. H. Cho, and S. G. Shin, “Comparison of different machine learning algorithms to estimate liquid level for bioreactor management,” Environmental Engineering Research, vol. 28, no. 2, pp. 220037–0, Apr. 2022, doi: 10.4491/eer.2022.037.

K. V. Santhosh, B. Joy, and S. Rao, “Design of an Instrument for Liquid Level Measurement and Concentration Analysis Using Multisensor Data Fusion,” J Sens, vol. 2020, pp. 1–13, Jan. 2020, doi: 10.1155/2020/4259509.

B. Wu, X. Wu, P. Li, Y. Gao, J. Si, and N. Al-Dhahir, “Efficient FPGA Implementation of Convolutional Neural Networks and Long Short-Term Memory for Radar Emitter Signal Recognition,” Sensors, vol. 24, no. 3, p. 889, Jan. 2024, doi: 10.3390/s24030889.

Y. Gao, S. Miyata, and Y. Akashi, “How to improve the application potential of deep learning model in HVAC fault diagnosis: Based on pruning and interpretable deep learning method,” Appl Energy, vol. 348, p. 121591, Oct. 2023, doi: 10.1016/j.apenergy.2023.121591.