Hostname: page-component-76c49bb84f-c7tcl Total loading time: 0 Render date: 2025-07-12T13:03:03.308Z Has data issue: false hasContentIssue false
  • English
  • Français

Trajectory planning of free-floating space robot for non-cooperative tumbling target capture based on deep reinforcement learning

Published online by Cambridge University Press:  11 July 2025

Yaqiang Wei Open the ORCID record for Yaqiang Wei [Opens in a new window] ,
Xinlin Bai  and
Han Lu
Yaqiang Wei*
Affiliation:
State Key Laboratory of Mechanics and Control for Aerospace Structures, Nanjing University of Aeronautics and Astronautics, Nanjing, China
Xinlin Bai
Affiliation:
Shenyang Institute of Automation, Chinese Academy of Sciences, Shenyang, China
Han Lu
Affiliation:
Shenyang Institute of Automation, Chinese Academy of Sciences, Shenyang, China University of Chinese Academy of Sciences, Beijing, China
*
Corresponding author: Yaqiang Wei; Email: weiyaqiang@nuaa.edu.cn
Get access Rights & Permissions [Opens in a new window]

Abstract

Capturing the non-cooperative tumbling target by the free-floating space robot stands as a crucial task within on-orbit servicing. However, the strong dynamic coupling of the base-spacecraft and the manipulator seriously disturbs the base-spacecraft, which reduces the power generation efficiency of solar panels and the communication quality with the earth station. In this paper, the trajectory planning method of the free-floating space robot for non-cooperative tumbling target capture based on deep reinforcement learning is proposed, which can reduce the disturbance of the base-spacecraft during target capture. First, the generalized Jacobian matrix of the space robot is derived, from which the dynamics model is obtained. The kinematics model of the space non-cooperative tumbling target is established. And the contact collision dynamics between the space robot and the tumbling target are analysed. Second, the twin delayed deep deterministic policy gradient algorithm is introduced to plan the trajectory for capturing the non-cooperative tumbling target, where apart from the motion parameters of the manipulator and the generalized manipulability of the space robot, the pose disturbance of the base-spacecraft is initially added to the reward function. Finally, the simulation for target capture is carried out. The results show that compared with the existing method, the proposed method converges faster with a larger reward, and the pose disturbance of the base-spacecraft is reduced. Moreover, the method performs well for capturing the non-cooperative tumbling target with different initial rotational angular velocities.

Keywords

free-floating space robotnon-cooperative tumbling target capturetrajectory planningdeep reinforcement learningpose disturbance of the base-spacecraft

Information

Type
Research Article
Information
Robotica , First View , pp. 1 - 19
Copyright
© The Author(s), 2025. Published by Cambridge University Press

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Article purchase

Temporarily unavailable

References

Kalaycioglu, P. H. S. and de Ruiter, A., “Dual arm coordination of redundant space manipulators mounted on a spacecraft,” Robotica 41(8), 2489–2518 (2023). doi: 10.1017/S0263574723000504.CrossRefGoogle Scholar
Rybus, T., Prokopczuk, J., Wojtunik, M., Aleksiejuk, K. and Musiał, J., “Application of bidirectional rapidly exploring random trees (BiRRT) algorithm for collision-free trajectory planning of free-floating space manipulator,” Robotica 40(12), 4326–4357 (2022). doi: 10.1017/S0263574722000935.CrossRefGoogle Scholar
Flores-Abad, A., Ma, O., Pham, K. and Ulrich, S., “A review of space robotics technologies for on-orbit servicing,” Prog. Aeosp. Sci. 68, 1–26 (2014). doi: 10.1016/j.paerosci.2014.03.002.CrossRefGoogle Scholar
Yan, W., Liu, Y., Lan, Q., Zhang, T. and Tu, H., “Trajectory planning and low-chattering fixed-time nonsingular terminal sliding mode control for a dual-arm free-floating space robot,” Robotica 40(3), 625–645 (2022). doi: 10.1017/S0263574721000734.CrossRefGoogle Scholar
Yan, L., Yuan, H., Xu, W., Hu, Z. and Liang, B., “Generalized relative jacobian matrix of space robot for dual-arm coordinated capture,” J. Guid. Control Dyn. 41b(5), 1202–1208 (2018). doi: 10.2514/1.G003237.CrossRefGoogle Scholar
Peng, J., Xu, W., Hu, Z., Liang, B. and Wu, A., “Modeling and analysis of the multiple dynamic coupling effects of a dual-arm space robotic system,” Robotica 38(11), 2060–2079 (2020). doi: 10.1017/S0263574719001826.CrossRefGoogle Scholar
Tortopidis, I. and Papadopoulos, E., “On point-to-point motion planning for underactuated space manipulator systems,” Robot. Auton. Syst. 55(2), 122–131 (2007). doi: 10.1016/j.robot.2006.07.003.CrossRefGoogle Scholar
Zhou, C., Jin, M. H., Liu, Y. C., Zhang, Z., Liu, Y. and Liu, H., “Singularity robust path planning for real time base attitude adjustment of free-floating space robot,” Int. J. Autom. Comput. 14(02), 169–178 (2017). doi: 10.1007/s11633-017-1055-1.CrossRefGoogle Scholar
Wang, M., Luo, J., Fang, J. and Yuan, J., “Optimal trajectory planning of free-floating space manipulator using differential evolution algorithm,” Adv. Space Res. 61(6), 1525–1536 (2018). doi: 10.1016/j.asr.2018.01.011.CrossRefGoogle Scholar
Shan, M., Guo, J. and Gill, E., “Review and comparison of active space debris capturing and removal methods,” Prog. Aeosp. Sci. 80, 18–32 (2016). doi: 10.1016/j.paerosci.2015.11.001.CrossRefGoogle Scholar
Xu, S., Wang, H., Zhang, D. and Yang, B., “Adaptive reactionless motion control for free-floating space manipulators with uncertain kinematics and dynamics,” IFAC Proc. 46(20), 646–653 (2013). doi: 10.3182/20130902-3-CN-3020.00145.CrossRefGoogle Scholar
Luo, J., Zong, L., Wang, M. and Yuan, J., “Optimal capture occasion determination and trajectory generation for space robots grasping tumbling objects,” Acta Astronaut. 136, 380–386 (2017). doi: 10.1016/j.actaastro.2017.03.026.CrossRefGoogle Scholar
Xu, W., Liang, B., Li, C. and Xu, Y., “Autonomous rendezvous and robotic capturing of non-cooperative target in space,” Robotica 28(5), 705–718 (2010). doi: 10.1017/S0263574709990397.CrossRefGoogle Scholar
Nguyen-Huynh, T.-C. and Sharf, I.. Adaptive Reactionless Motion for Space manipulator When Capturing an Unknown Tumbling Target. In: 2011 IEEE International Conference on Robotics and Automation, Shanghai, China (2011) pp. 4202–4207, doi: 10.1109/ICRA.2011.5980398 CrossRefGoogle Scholar
Xie, Z., Sun, T., Kwan, T. H., Mu, Z. and Wu, X., “A new reinforcement learning based adaptive sliding mode control scheme for free-floating space robotic manipulator,” IEEE Access 8, 127048–127064 (2020). doi: 10.1109/ACCESS.2020.3008399.CrossRefGoogle Scholar
Wang, S., Cao, Y., Zheng, X. and Zhang, T.. An End-to-End Trajectory Planning Strategy for Free-Floating Space Robots. In: 40th Chinese Control Conference (CCC), Shanghai, China (2021) pp. 4236–4241, 2021, doi: 10.23919/CCC52363.2021.9550509 CrossRefGoogle Scholar
Yan, C., Zhang, Q., Liu, Z., Wang, X. and Liang, B.. Control of Free-Floating Space Robots to Capture Targets using Soft q-Learning. In: 2018 IEEE International Conference on Robotics and Biomimetics (ROBIO), Kuala Lumpur, Malaysia (2018) pp. 654–660, doi: 10.1109/ROBIO.2018.8665049 CrossRefGoogle Scholar
Wang, S., Zheng, X., Cao, Y. and Zhang, T.. A Multi-Target Trajectory Planning of a 6-Dof Free-Floating Space Robot via Reinforcement Learning. In: 2021 IEEE/RSJ International Conference on Intelligent Robots and Systems (IROS), Prague, Czech Republic (2021) pp. 3724–3730, doi: 10.1109/IROS51168.2021.9636681 CrossRefGoogle Scholar
Du, D., Zhou, Q., Qi, N., Wang, X. and Liu, Y.. Learning to Control a Free-Floating Space Robot using Deep Reinforcement Learning. In: 2019 IEEE International Conference on Unmanned Systems (ICUS), Beijing, China (2019) pp. 519–523, doi: 10.1109/ICUS48101.2019.8995991 CrossRefGoogle Scholar
Hu, X., Huang, X., Hu, T., Shi, Z. and Hui, J.. Mrddpg Algorithms for Path Planning of Free-Floating Space Robot. In: 2018 IEEE 9th International Conference on Software Engineering and Service Science (ICSESS), Beijing, China (2018) pp. 1079–1082, doi: 10.1109/ICSESS.2018.8663748 CrossRefGoogle Scholar
Li, Y., Hao, X., She, Y., Li, S. and Yu, M., “Constrained motion planning of free-float dual-arm space manipulator via deep reinforcement learning,” Aerosp. Sci. Technol. 109, 106446 (2021). doi: 10.1016/j.ast.2020.106446.CrossRefGoogle Scholar
Wu, Y.-H., Yu, Z.-C., Li, C.-Y., He, M.-J., Hua, B. and Chen, Z.-M., “Reinforcement learning in dual-arm trajectory planning for a free-floating space robot,” Aerosp. Sci. Technol. 98, 105657 (2020). doi: 10.1016/j.ast.2019.105657.CrossRefGoogle Scholar
Fujimoto, S., van Hoof, H. and Meger, D.. Addressing Function Approximation Error in Actor-Critic Methods. In: 35th International Conference on Machine Learning, Stockholm, Sweden (2018) pp. 1587–1596, doi: 10.48550/arXiv.1802.09477 CrossRefGoogle Scholar
Bao, J., Zhang, G., Peng, Y., Shao, Z. and Song, A., “Learn multi-step object sorting tasks through deep reinforcement learning,” Robotica 40(11), 3878–3894 (2022). doi: 10.1017/S0263574722000650.CrossRefGoogle Scholar
Eyiguler, E. K., Pandey, K., Howarth, A., Holley, W., Danskin, D., Hussey, G., Gillies, R. and Yau, A., “Effect of spacecraft attitude on radio wave polarization measurements for the radio receiver instrument on swarm-e,” Adv. Space Res. 72(11), 4836–4855 (2023). doi: 10.1016/j.asr.2023.09.001.CrossRefGoogle Scholar
Zhou, Y., Luo, J. and Wang, M., “Dynamic manipulability analysis of multi-arm space robot,” Robotica 39(1), 23–41 (2021). doi: 10.1017/S0263574720000077.CrossRefGoogle Scholar
Gong, S., Gong, H. and Shi, P., “Shape-based approach to attitude motion planning of reconfigurable spacecraft,” Adv. Space Res. 70(5), 1285–1296 (2022). doi: 10.1016/j.asr.2022.06.004.CrossRefGoogle Scholar
Tsiotras, P., King-Smith, M. and Ticozzi, L., “Spacecraft-mounted robotics,” Annu. Rev. Control Robot. Auton. Syst. 6(1), 335–362 (2023). doi: 10.1146/annurev-control-062122-082114.CrossRefGoogle Scholar
Yang, Y., “Spacecraft attitude determination and control: Quaternion based method,” Annu. Rev. Control 36(2), 198–219 (2012). doi: 10.1016/j.arcontrol.2012.09.003.CrossRefGoogle Scholar
Hamdan, M. O. and Abu-Nabah, B. A., “Modeling meniscus rise in capillary tubes using fluid in rigid-body motion approach,” Commun. Nonlinear Sci. Numer. Simul. 57, 449–460 (2018). doi: 10.1016/j.cnsns.2017.11.004.CrossRefGoogle Scholar
Yang, Y., Hu, W. and Liu, Z., “Configuration design and collision dynamics analysis of flexible nets for space debris removal,” Acta Astronaut. 211, 249–256 (2023). doi: 10.1016/j.actaastro.2023.06.024.CrossRefGoogle Scholar
Han, D., Huang, P., Liu, X. and Yang, Y., “Combined spacecraft stabilization control after multiple impacts during the capture of a tumbling target by a space robot,” Acta Astronaut. 176, 24–32 (2020). doi: 10.1016/j.actaastro.2020.05.035.CrossRefGoogle Scholar
Lachner, J., Schettino, V., Allmendinger, F., Fiore, M. D., Ficuciello, F., Siciliano, B. and Stramigioli, S., “The influence of coordinates in robotic manipulability analysis,” Mech. Mach. Theory 146, 103722 (2020). doi: 10.1016/j.mechmachtheory.2019.103722.CrossRefGoogle Scholar
Wei, Y., Yang, X., Xu, Z. and Bai, X., “Novel ground microgravity experiment system for a spacecraft-manipulator system based on suspension and air-bearing,” Aerosp. Sci. Technol. 141, 108587 (2023). doi: 10.1016/j.ast.2023.108587.CrossRefGoogle Scholar