Robust Fault Detection Of A Hybrid Control System Using Derivative Free Estimator And Reinforcement Learning Method

Sayanti Chatterjee; Bhupesh Deka; Dipanwita Debnath; Sasmita Tripathy; Bindu Agarwalla

doi:10.35882/jeeemi.v7i1.644

Sayanti Chatterjee Department of Computer Science and Engineering (AIML &IoT), Vallurupalli Nageswara Rao Vignana Jyothi Institute of Engineering & Technology, Hyderabad, India https://orcid.org/0000-0002-3675-2672
Bhupesh Deka Department of Computer Science and Engineering (AIML &IoT), Vallurupalli Nageswara Rao Vignana Jyothi Institute of Engineering & Technology, Hyderabad, India https://orcid.org/0000-0001-9460-5796
Dipanwita Debnath Department of Computer Science and Engineering, Koneru Lakshmaiah Education Foundation, Hyderabad-500043, Telangana, India https://orcid.org/0000-0002-7579-3979
Sasmita Tripathy Centurion University of Technology and Management, Bhubaneswar, Odissa, India https://orcid.org/0009-0005-7156-0448
Bindu Agarwalla School of Computer Engineering, Kalinga Institute of Industrial Technology, University, Bhubaneswar, Odisha ,India https://orcid.org/0000-0002-0133-4913

DOI: https://doi.org/10.35882/jeeemi.v7i1.644

Keywords: CSTR, Fault Detection, Hybrid System, Markov Decision Process, Q-Adaptive, Robustness Study

Abstract

Fault detection in hybrid control systems (HCS) poses significant challenges due to dynamic variations in system dynamics caused by event-based inputs and the existence of unknown large process noise. A novel scheme for optimized robust fault detection of HCS has been proposed and projected here that can effectively handle dynamic system changes and process noise along with the fault detection while achieving high accuracy and reliability. The challenge with the HCS is the presence of a large process noises due to changing of state equations drastically with dynamical input making the fault detection a complex task. The derivative-free estimator minimizes process noise and provides reliable state estimation, while the Markov Decision Process (MDP) framework is employed to optimize fault detection. MDP has been chosen here due to its mathematical introspection for dynamic system's decision-making process when the results are random or under the control of a decision maker. The data generated by the derivative-free estimator is used to train this deep learning model. Simulation studies were conducted to evaluate the scheme’s performance, and additional tests for convergence, optimization, and robustness were performed using MDP infused with adaptive estimators. The efficacy of the proposed estimators has been confirmed on a benchmark problems, namely the liquid level control system for an chemical stirred tank reactor (CSTR) model. Simulation studies has been employed to prove the efficacy of the proposed method. The proposed method achieved 98.6% fault detection accuracy and a 12% mean error reduction compared to existing techniques. It demonstrated robustness under varying noise levels, dynamic conditions and presence of external disturbances . The results confirm the method's effectiveness for robust and optimized fault detection in HCS, offering a scalable, accurate, and noise-resilient solution for real-world industrial systems.

Downloads

Download data is not yet available.

References

J. Lygeros, ‘An Overview of Hybrid Systems Control’, in Handbook of Networked and Embedded Control Systems, Boston, MA: Birkhäuser Boston, 2005, pp. 519–537. doi: 10.1007/0-8176-4404-0_22.

R. Goebel, R. G. Sanfelice, and A. R. Teel, ‘Hybrid dynamical systems’, IEEE Control Syst, vol. 29, no. 2, pp. 28–93, Apr. 2009, doi: 10.1109/MCS.2008.931718.

N. N. Nandola and S. Bhartiya, ‘A multiple model approach for predictive control of nonlinear hybrid systems’, J Process Control, vol. 18, no. 2, pp. 131–148, Feb. 2008, doi: 10.1016/j.jprocont.2007.07.003.

K. Ito and K. Xiong, ‘Gaussian filters for nonlinear filtering problems’, IEEE Trans Automat Contr, vol. 45, no. 5, pp. 910–927, May 2000, doi: 10.1109/9.855552.

M. Das, A. Dey, S. Sadhu, and T. K. Ghoshal, ‘Adaptive central difference filter for non‐linear state estimation’, IET Science, Measurement & Technology, vol. 9, no. 6, pp. 728–733, Sep. 2015, doi: 10.1049/iet-smt.2014.0299.

A. Dey, M. Das, S. Sadhu, and T. K. Ghoshal, ‘Adaptive divided difference filter for parameter and state estimation of non‐linear systems’, IET Signal Processing, vol. 9, no. 4, pp. 369–376, Jun. 2015, doi: 10.1049/iet-spr.2013.0395.

D.-J. Lee and K. Alfriend, ‘Adaptive Sigma Point Filtering for State and Parameter Estimation’, in AIAA/AAS Astrodynamics Specialist Conference and Exhibit, Reston, Virigina: American Institute of Aeronautics and Astronautics, Aug. 2004. doi: 10.2514/6.2004-5101.

M. Ahmadi, A. Khayatian, and P. Karimaghaee, ‘Attitude estimation by divided difference filter in quaternion space’, Acta Astronaut, vol. 75, pp. 95–107, Jun. 2012, doi: 10.1016/j.actaastro.2011.12.022.

S. Chatterjee, ‘Improved fault detection and identification for nonlinear hybrid systems using self switched CDKF’, in 2015 Annual IEEE India Conference (INDICON), IEEE, Dec. 2015, pp. 1–6. doi: 10.1109/INDICON.2015.7443175.

S. Vedvik and C. D. Karlgaard, ‘Iterated Sigma-Point Kalman Filtering for Trajectory Reconstruction’, 2024. [Online]. Available: http://www.sti.nasa.gov

A. Almagbile, J. Wang, and W. Ding, ‘Evaluating the Performances of Adaptive Kalman Filter Methods in GPS/INS Integration’, Journal of Global Positioning Systems, vol. 9, no. 1, pp. 33–40, Jun. 2010, doi: 10.5081/jgps.9.1.33.

F. Deng, H.-L. Yang, and L.-J. Wang, ‘Adaptive Unscented Kalman Filter Based Estimation and Filtering for Dynamic Positioning with Model Uncertainties’, Int J Control Autom Syst, vol. 17, no. 3, pp. 667–678, Mar. 2019, doi: 10.1007/s12555-018-9503-4.

M. Liu, J. Yu, and J. J. Rodríguez-Andina, ‘Adaptive Event-Triggered Asynchronous Fault Detection for Nonlinear Markov Jump Systems With Its Application: A Zonotopic Residual Evaluation Approach’, IEEE Trans Netw Sci Eng, vol. 10, no. 4, pp. 1792–1808, Jul. 2023, doi: 10.1109/TNSE.2023.3235008.

S. Chatterjee, ‘Estimation of Nonlinear Hybrid Systems Using Second-Order Q-Adaptive Self-switched Derivative-Free Estimators’, 2020, pp. 693–703. doi: 10.1007/978-981-15-2256-7_63.

X. Chen, X. Lü, W. Zhang, C. Xue, X. Zhu, and W. Bao, ‘Miniaturized buried low-frequency acoustically actuated magnetoelectric antenna for soil moisture adaptive underground communication’, iScience, vol. 27, no. 11, Nov. 2024, doi: 10.1016/j.isci.2024.111151.

J. J. Q. Yu, Y. Hou, A. Y. S. Lam, and V. O. K. Li, ‘Intelligent Fault Detection Scheme for Microgrids With Wavelet-Based Deep Neural Networks’, IEEE Trans Smart Grid, vol. 10, no. 2, pp. 1694–1703, Mar. 2019, doi: 10.1109/TSG.2017.2776310.

R. İnan, B. Aksoy, and O. K. M. Salman, ‘Estimation performance of the novel hybrid estimator based on machine learning and extended Kalman filter proposed for speed-sensorless direct torque control of brushless direct current motor’, Eng Appl Artif Intell, vol. 126, p. 107083, Nov. 2023, doi: 10.1016/j.engappai.2023.107083.

D. Zhang, Y. Fu, Z. Lin, and Z. Gao, ‘A reinforcement learning based fault diagnosis for autoregressive-moving-average model’, in IECON 2017 - 43rd Annual Conference of the IEEE Industrial Electronics Society, IEEE, Oct. 2017, pp. 7067–7072. doi: 10.1109/IECON.2017.8217236.

Z. Wang and J. Xuan, ‘Intelligent fault recognition framework by using deep reinforcement learning with one dimension convolution and improved actor-critic algorithm’, Advanced Engineering Informatics, vol. 49, p. 101315, Aug. 2021, doi: 10.1016/j.aei.2021.101315.

K. Salahshoor, M. Kordestani, and M. S. Khoshro, ‘Fault detection and diagnosis of an industrial steam turbine using fusion of SVM (support vector machine) and ANFIS (adaptive neuro-fuzzy inference system) classifiers’, Energy, vol. 35, no. 12, pp. 5472–5482, Dec. 2010, doi: 10.1016/j.energy.2010.06.001.

A. Carron, M. Todescato, R. Carli, L. Schenato, and G. Pillonetto, ‘Machine learning meets Kalman Filtering’, in 2016 IEEE 55th Conference on Decision and Control (CDC), IEEE, Dec. 2016, pp. 4594–4599. doi: 10.1109/CDC.2016.7798968.

S. Chatterjee, ‘Fault Detection for a Nonlinear Switched Continuous Time Delayed System Using Machine Learning and Self-Switched UKF’, Journal Européen des Systèmes Automatisés, vol. 55, no. 2, pp. 245–251, Apr. 2022, doi: 10.18280/jesa.550212.

X. Chen, R. Sun, M. Liu, and D. Song, ‘Two-stage exogenous Kalman filter for time-varying fault estimation of satellite attitude control system’, J Franklin Inst, vol. 357, no. 4, pp. 2354–2370, Mar. 2020, doi: 10.1016/j.jfranklin.2019.11.078.

Q. Zhang, P. Wang, and Z. Chen, ‘An improved particle filter for mobile robot localization based on particle swarm optimization’, Expert Syst Appl, vol. 135, pp. 181–193, Nov. 2019, doi: 10.1016/j.eswa.2019.06.006.

X. Zhu, M. Zhong, T. Xue, and J. Fang, ‘An Extended H i /H ∞ Optimization Approach to Dynamic Event-Triggered Fault Detection for a Class of Nonlinear Systems’, IEEE Trans Instrum Meas, vol. 73, pp. 1–10, 2024, doi: 10.1109/TIM.2024.3472851.

M. Weiss et al., ‘Applications of the Kalman Filter to Chemical Sensors for Downstream Machine Learning’, IEEE Sens J, vol. 18, no. 13, pp. 5455–5463, Jul. 2018, doi: 10.1109/JSEN.2018.2836183.

N. Laouti, S. Othman, M. Alamir, and N. Sheibat-Othman, ‘Combination of Model-based Observer and Support Vector Machines for Fault Detection of Wind Turbines’, International Journal of Automation and Computing, vol. 11, no. 3, pp. 274–287, Jun. 2014, doi: 10.1007/s11633-014-0790-9.

L. Hong, Z. Chen, Y. Wang, M. Shahidehpour, and M. Wu, ‘A novel SVM-based decision framework considering feature distribution for Power Transformer Fault Diagnosis’, Energy Reports, vol. 8, pp. 9392–9401, Nov. 2022, doi: 10.1016/j.egyr.2022.07.062.

Y. Gao, S. Miyata, and Y. Akashi, ‘How to improve the application potential of deep learning model in HVAC fault diagnosis: Based on pruning and interpretable deep learning method’, Appl Energy, vol. 348, p. 121591, Oct. 2023, doi: 10.1016/j.apenergy.2023.121591.

C. Andrieu, A. Doucet, and E. Punskaya, ‘Sequential Monte Carlo Methods for Optimal Filtering’, in Sequential Monte Carlo Methods in Practice, New York, NY: Springer New York, 2001, pp. 79–95. doi: 10.1007/978-1-4757-3437-9_4.

F. Cadini, E. Zio, and G. Peloni, ‘Particle Filtering for the Detection of Fault Onset Time in Hybrid Dynamic Systems With Autonomous Transitions’, IEEE Trans Reliab, vol. 61, no. 1, pp. 130–139, Mar. 2012, doi: 10.1109/TR.2011.2182224.

K. V. Santhosh, B. Joy, and S. Rao, ‘Design of an Instrument for Liquid Level Measurement and Concentration Analysis Using Multisensor Data Fusion’, J Sens, vol. 2020, pp. 1–13, Jan. 2020, doi: 10.1155/2020/4259509.

S. Il Yu, C. Rhee, K. H. Cho, and S. G. Shin, ‘Comparison of different machine learning algorithms to estimate liquid level for bioreactor management’, Environmental Engineering Research, vol. 28, no. 2, pp. 220037–0, Apr. 2022, doi: 10.4491/eer.2022.037.

M. Zadkarami, M. Shahbazian, and K. Salahshoor, ‘Pipeline leakage detection and isolation: An integrated approach of statistical and wavelet feature extraction with multi-layer perceptron neural network (MLPNN)’, J Loss Prev Process Ind, vol. 43, pp. 479–487, Sep. 2016, doi: 10.1016/j.jlp.2016.06.018.