Hostname: page-component-745bb68f8f-b95js Total loading time: 0 Render date: 2025-01-24T22:06:36.976Z Has data issue: false hasContentIssue false
  • English
  • Français

Development of deep reinforcement learning-based fault diagnosis method for actuator faults in unmanned aerial vehicles

Published online by Cambridge University Press:  24 January 2025

M. Saied Open the ORCID record for M. Saied [Opens in a new window] ,
N. Tahan Open the ORCID record for N. Tahan [Opens in a new window] ,
K. Chreif ,
C. Francis  and
Z. Noun
M. Saied*
Affiliation:
Scientific Research Center in Engineering (CRSI), Faculty of Engineering, Lebanese University, Hadath, Lebanon Department of Electrical and Electronics Engineering, School of Engineering, Lebanese International University, Bekaa, Lebanon
N. Tahan
Affiliation:
Scientific Research Center in Engineering (CRSI), Faculty of Engineering, Lebanese University, Hadath, Lebanon
K. Chreif
Affiliation:
Department of Electrical and Electronics Engineering, School of Engineering, Lebanese International University, Bekaa, Lebanon
C. Francis
Affiliation:
Arts et Metiers ParisTech, Campus Châlons en Champagne, Châlons en Champagne, France
Z. Noun
Affiliation:
Department of Electrical and Electronics Engineering, School of Engineering, Lebanese International University, Bekaa, Lebanon
*
Corresponding author: M. Saied; Email: majd.elsaied@ul.edu.lb
Get access Rights & Permissions [Opens in a new window]

Abstract

Actuator faults in unmanned aerial vehicles (UAVs) can have significant and potentially adverse effects on their safety and performance, highlighting the critical importance of fault diagnosis in UAV design. Ensuring the reliability of these systems in various applications often requires the use of advanced diagnostic algorithms. Artificial intelligence methods, such as deep learning and machine learning techniques, enable fault diagnosis through sample-based learning without the need for prior knowledge of fault mechanisms or physics-based models. However, UAV fault datasets are typically small due to stringent safety standards, which presents challenges for achieving high-performance fault diagnosis. To address this, deep reinforcement learning (DRL) algorithms offer a unique advantage by combining deep learning’s automatic feature extraction with reinforcement learning’s interactive learning approach, improving both learning capabilities and robustness. In this study, we propose and evaluate two DRL-based fault diagnosis models, which demonstrate remarkable accuracy in fault diagnosis, consistently exceeding $99{\rm{\% }}$. Notably, under small sample scenarios, the proposed models significantly outperform traditional classifiers such as decision trees, support vector machines, and multilayer perceptron neural networks. These findings suggest that the integration of DRL enhances fault diagnosis performance, particularly in data-limited environments.

Keywords

Deep reinforcement learningfault diagnosisunmanned aerial vehicles
Type
Research Article
Information
The Aeronautical Journal , First View , pp. 1 - 17
Check for updates
Copyright
© The Author(s), 2025. Published by Cambridge University Press on behalf of Royal Aeronautical Society

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

Bui, S.T., Luu, Q.K., Nguyen, D.Q., Loianno, G. and Ho, V.A. Tombo propeller: bioinspired deformable structure toward collision-accommodated control for drones, IEEE Trans. Rob., 2023, 39, pp 521–538.CrossRefGoogle Scholar
Jinquan, X., Xiaohong, W., Dawei, L. and Lizhi, W. Research on the safety assessment of the brushless DC motor based on the gray model, Adv. Mech. Eng., 2017, 9, pp 1–15.Google Scholar
Duccard, G. Fault-tolerant Flight Control and Guidance Systems: Practical Methods for Small Unmanned Aerial Vehicles, Springer, Springer-Verlag, London, 2009.CrossRefGoogle Scholar
Ding, S.X. Model-Based Fault Diagnosis Techniques: Design Schemes, Algorithms and Tools, Springer, Springer-Verlag, London, 2013.CrossRefGoogle Scholar
Saied, M., Attieh, H., Mazeh, H., Shraim, H. and Francis, C. Supervised learning classification applications in fault detection and diagnosis: an overview of implementations in unmanned aerial systems, SAE Int. J. Aerospace, 2022, 16, pp 57–73.CrossRefGoogle Scholar
Nor, N.M., Che Hassan, C.R. and Hussain, M.A. A review of data-driven fault detection and diagnosis methods: applications in chemical process systems, Rev. Chem. Eng., 2020, 36, pp 513–553.CrossRefGoogle Scholar
Saied, M., Shraim, H. and Francis, C. A review on recent development of multirotor UAV fault-tolerant control systems, IEEE Aerospace Electron. Syst. Mag., 2023. doi: 10.1109/MAES.2023.3327697 Google Scholar
Lei, Y., Yang, B., Jiang, X., Jia, F., Li, N. and Nandi, A.K. Applications of machine learning to machine fault diagnosis: a review and roadmap, Mech. Syst. Signal Process., 2020, 138, 106587.CrossRefGoogle Scholar
Park, J., Jung, Y. and Kim, J.H. Multiclass classification fault diagnosis of multirotor UAVs utilizing a deep neural network, Int. J. Control Autom. Syst., 2022, 20, pp 1316–1326.CrossRefGoogle Scholar
Alos, A. and Dahrouj, Z. Using MLSTM and multioutput convolutional LSTM algorithms for detecting anomalous patterns in streamed data of unmanned aerial vehicles, IEEE Aerospace Electron. Syst. Mag., 2022, 37, pp 6–15.CrossRefGoogle Scholar
Saied, M., Lussier, B., Fantoni, I., Shraim, H. and Francis, C. Fault diagnosis and fault-tolerant control of an octorotor UAV using motors speeds measurements, IFAC-PapersOnLine, 2017, 50, pp 5263–5268.CrossRefGoogle Scholar
Sadhu, V., Anjum, K. and Pompili, D. On board deep-learning-based unmanned aerial vehicle fault cause detection and classification via FPGAs, IEEE Trans. Rob., 2023, 39, pp 3319–3331.CrossRefGoogle Scholar
Benkuan, W., Xiyuan, P., Min, J. and Datong, L. Real-time fault detection for UAV based on model acceleration engine, IEEE Trans. Instrum. Meas., 2020, 69, pp 9505–9516.Google Scholar
Qian, G. and Liu, J. Development of deep reinforcement learning-based fault diagnosis method for rotating machinery in nuclear power plants, Prog. Nuclear Energy, 2022, 152, 104401.CrossRefGoogle Scholar
Mumuni, A. and Mumuni, F. Data augmentation: a comprehensive survey of modern approaches, Array, 2022, 16, 100258.CrossRefGoogle Scholar
Yao, S., Kang, Q., Zhou, M., Rawa, M.J. and Abusorrah, A. A survey of transfer learning for machinery diagnostics and prognostics, Artif. Intell. Rev., 2023, 56, pp 2871–2922.CrossRefGoogle Scholar
Nian, R., Liu, J. and Huang, B. A review on reinforcement learning: Introduction and applications in industrial process control, Comput. Chem. Eng., 2020, 139, 106886.CrossRefGoogle Scholar
He, L., Aouf, N. and Song, B. Explainable deep reinforcement learning for UAV autonomous path planning, Aerospace Sci. Technol., 2021, 118, 107052.CrossRefGoogle Scholar
Ouahouah, S., Bagaa, M., Prados-Garzon, J. and Taleb, T. Deep-reinforcement-learning-based collision avoidance in UAV environment, IEEE Internet Things J., 2022, 9, pp 4015–4030.CrossRefGoogle Scholar
Bayerlein, H., Theile, M., Caccamo, M. and Gesbert, D. Multi-UAV path planning for wireless data harvesting with deep reinforcement learning, IEEE Open J. Commun. Soc., 2021, 2, pp 1171–1187.CrossRefGoogle Scholar
Wang, R. and Xu, L. Application of deep reinforcement learning in UAVs: a review, Proc. 34th Chinese Control and Decision Conference, Heifei, China, 2022, pp 4096–4103. doi: 10.1109/CCDC55256.2022.10034357 CrossRefGoogle Scholar
Wang, H., Xu, J., Sun, C., Yan, R. and Chen, X. Intelligent fault diagnosis for planetary gearbox using time-frequency representation and deep reinforcement learning, IEEE/ASME Trans. Mechatron., 2022, 27, pp 985–998.CrossRefGoogle Scholar
Li, G., Wu, J., Deng, C., Vu, X. and Shao, X. Deep reinforcement learning-based online domain adaptation method for fault diagnosis of rotating machinery, IEEE/ASME Trans. Mechatron., 2021, 27, pp 2796–2805.CrossRefGoogle Scholar
Fan, S., Zhang, X. and Song, Z. Imbalanced sample selection with deep reinforcement learning for fault diagnosis, IEEE Trans. Ind. Inf., 2022, 18, pp 2518–2527.CrossRefGoogle Scholar
Rose, C., McMurray, R. and Hadi, M.U. A reinforcement learning control and fault detection method for the MADNI drone, Proc. 2024 35th Irish Signals and Systems Conference (ISSC), Belfast, United Kingdom, 2024. doi: 10.1109/ISSC61953.2024.10602956 CrossRefGoogle Scholar
Wang, S., Jian Feng, T., Qin Cheng, J., Wei, C. and Cheng Liang, L. Manipulator joint fault localization for intelligent flexible manufacturing based on reinforcement learning and robot dynamics, Rob. Comput.-Integr. Manuf., 2024, 86, 102684.CrossRefGoogle Scholar
Wang, R., Zhuang, Z., Tao, H., Paszke, W. and Stojanovic, V. Q-learning based fault estimation and fault tolerant iterative learning control for MIMO systems, ISA Trans., 2023, 142, pp. 123–135.CrossRefGoogle ScholarPubMed
Mnih, V. et al., Human-level control through deep reinforcement learning, Nature, 2015, 518, pp 529–533.CrossRefGoogle ScholarPubMed
Zhang, J., Zhang, Z., Han, S. and Lu, S. Proximal policy optimization via enhanced exploration efficiency, Inf. Sci., 2022, 609, pp 750–765.CrossRefGoogle Scholar
Meng, W., Zheng, Q., Shi, Y. and Pan, G. An off-policy trust region policy optimization method with monotonic improvement guarantee for deep reinforcement learning, IEEE Trans. Neural Networks Learn. Syst., 2022, 33, pp 2223–2235.CrossRefGoogle ScholarPubMed