Hostname: page-component-7dd5485656-zlgnt Total loading time: 0 Render date: 2025-10-30T21:12:06.375Z Has data issue: false hasContentIssue false
  • English
  • Français

Gait generation via unified learning optimal control of Hamiltonian systems

Published online by Cambridge University Press:  23 January 2013

Satoshi Satoh ,
Kenji Fujimoto  and
Sang-Ho Hyon
Satoshi Satoh*
Affiliation:
Division of Mechanical Systems and Applied Mechanics, Faculty of Engineering, Hiroshima University, 1-4-1, Kagamiyama, Higashi-Hiroshima 739-8527, Japan
Kenji Fujimoto
Affiliation:
Department of Aerospace Engineering, Graduate School of Engineering, Kyoto University, Yoshida-Honmachi, Sakyo-ku, Kyoto 606-8501, Japan
Sang-Ho Hyon
Affiliation:
Department of Robotics, Ritsumeikan University, Noji Higashi 1-1-1, Kusatsu, Shiga 525-8577, Japan
*
*Corresponding author. E-mail: s.satoh@ieee.org
Get access Rights & Permissions [Opens in a new window]

Summary

This paper proposes a repetitive control type optimal gait generation framework by executing learning control and parameter tuning. We propose a learning optimal control method of Hamiltonian systems unifying iterative learning control (ILC) and iterative feedback tuning (IFT). It allows one to simultaneously obtain an optimal feedforward input and tuning parameter for a plant system, which minimizes a given cost function. In the proposed method, a virtual constraint by a potential energy prevents a biped robot from falling. The strength of the constraint is automatically mitigated by the IFT part of the proposed method, according to the progress of trajectory learning by the ILC part.

Keywords

Gait generationBiped robotsRepetitive controlIterative learning controlHamiltonian systems

Information

Type
Articles
Information
Robotica , Volume 31 , Issue 5 , August 2013 , pp. 717 - 732
Copyright
Copyright © Cambridge University Press 2013 

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

Article purchase

Temporarily unavailable

References

1.Vukobratović, M. and Stepanenko, J., “On the stability of anthropomorphic systems,” Math. Biosci. 15, 1–37 (1972).CrossRefGoogle Scholar
2.Takanishi, A., Ishida, M., Yamazaki, Y. and Kato, I., “The Realization of Dynamic Walking by the Biped Walking Robot WL-10RD,” In: Proceedings of the International Conference on Advanced Robotics (1985) pp. 459–466.Google Scholar
3.Hirai, K., Hirose, M., Haikawa, Y. and Takenaka, T., “The Development of Honda Humanoid Robot,” In: Proceedings of the IEEE International Conference on Robotics and Automation (1998) pp. 1321–1326.Google Scholar
4.Kajita, S., Kanehiro, F., Kaneko, K., Fujiwara, K., Harada, K., Yokoi, K. and Hirukawa, H., “Biped Walking Pattern Generation by Using Preview Control of Zero-Moment Point,” In: Proceedings of the IEEE International Conference on Robotics and Automation (2003) pp. 1620–1626.Google Scholar
5.McGeer, T., “Passive dynamic walking,” Int. J. Robot. Res. 9 (2), 62–82 (1990).CrossRefGoogle Scholar
6.Garcia, M., Chatterjee, A., Ruina, A. and Coleman, M., “The simplest walking model: Stability, complexity, and scaling,” ASME J. Biomech. Eng. 120, 281–288 (1998).CrossRefGoogle ScholarPubMed
7.Osuka, K. and Kirihara, K., “Motion Analysis and Experiments of Passive Walking Robot QUARTET II,” In: Proceedings of the IEEE International Conference on Robotics and Automation (2000) pp. 3052–3056.Google Scholar
8.Sano, A., Ikemata, Y. and Fujimoto, H., “Analysis of Dynamics of Passive Walking from Storage Energy and Supply Rate,” In: Proceedings of the IEEE International Conference on Robotics and Automation (2003) pp. 2478–2483.Google Scholar
9.Goswami, A., Espiau, B. and Keramane, A., “Limit cycles in a passive compass gait biped and passivity-mimicking control laws,” Auton. Robots 4 (3), 273–286 (1997).CrossRefGoogle Scholar
10.Spong, M. W., “Passivity Based Control of the Compass Gait Biped,” In: Proceedings of the IFAC World Congress (1999) pp. 19–23.Google Scholar
11.Asano, F., Yamakita, M., Kamamichi, N. and Luo, Z. W., “A novel gait generation for biped walking robots based on mechanical energy constraint,” IEEE Trans. Robot. Autom. 20 (3), 565–573 (2004).CrossRefGoogle Scholar
12.Grizzle, J. W., Abba, G. and Plestan, F., “Asymptotically stable walking for biped robots: Analysis via systems with impulse effects,” IEEE Trans. Autom. Control 46 (1), 51–64 (2001).CrossRefGoogle Scholar
13.Hyon, S. and Emura, T., “Symmetric Walking Control: Invariance and Global Stability,” In: Proceedings of the IEEE ICRA (2005) pp. 1455–1462.Google Scholar
14.Endo, G., Morimoto, J., Matsubara, T., Nakanishi, J. and Cheng, G., “Learning CPG-based biped locomotion with a policy gradient method: Application to a humanoid robot,” Int. J. Robot. Res. 27 (2), 213–228 (2008).CrossRefGoogle Scholar
15.Morimoto, J. and Atkeson, C., “Robust Low Torque Biped Walking Using Differential Dynamic Programming with a Minimax Criterion,” In: Proceedings of the 5th International Conference on Climbing and Walking Robots (2002) pp. 453–459.Google Scholar
16.Tedrake, R., Zhang, T. W. and Seung, H. S., “Stochastic Policy Gradient Reinforcement Learning on a Simple 3D Biped,” In: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (2004) pp. 2849–2854.Google Scholar
17.Theodorou, E., Buchli, J. and Schaal, S., “A generalized path integral control approach to reinforcement learning,” J. Mach. Learn. Res. 11, 3153–3197 (2010).Google Scholar
18.Kappen, H. J., “An Introduction to Stochastic Control Theory, Path Integrals and Reinforcement Learning,” In: Proceedings of the 9th Granada Seminar on Computational Physics: Cooperative Behavior in Neural Systems (2007) pp. 149–181.Google Scholar
19.Satoh, S., Fujimoto, K. and Hyon, S., “Biped Gait Generation via Iterative Learning Control Including Discrete State Transitions,” In: Proceedings of the 17th IFAC World Congress (2008), pp. 1729–1734.Google Scholar
20.Satoh, S., Fujimoto, K. and Hyon, S., “A gait generation framework via learning optimal control considering discontinuous state transitions,” [in Japanese] J. Robot. Soc. Japan 29 (2), 90–100 (2011).CrossRefGoogle Scholar
21.Satoh, S., “Control of Deterministic and Stochastic Hamiltonian Systems Application to Optimal Gait Generation for Walking Robots,” Ph.D. dissertation (Nagoya University, Aichi, Japan, 2010), available at: http://home.hiroshima-u.ac.jp/satoh/index.html.Google Scholar
22.Fujimoto, K. and Sugie, T., “Iterative learning control of Hamiltonian systems: I/O based optimal control approach,” IEEE Trans. Autom. Control 48 (10), 1756–1761 (2003).CrossRefGoogle Scholar
23.Crouch, P. E. and van der Schaft, A. J., Variational and Hamiltonian Control Systems. Lecture Notes on Control and Information Science, vol. 101 (Springer, Berlin, 1987).CrossRefGoogle Scholar
24.Maschke, B. and van der Schaft, A. J., “Port-Controlled Hamiltonian Systems: Modelling Origins and System Theoretic Properties,” In: Proceedings of the 2nd IFAC Symposium on Nonlinear Control Systems (1992) pp. 282–288.Google Scholar
25.Arimoto, S., Kawamura, S. and Miyazaki, F., “Bettering operation of robotics,” J. Robot. Syst. 1 (2), 123–140 (1984).CrossRefGoogle Scholar
26.Fujimoto, K. and Koyama, I., “Iterative Feedback Tuning for Hamiltonian Systems,” In: Proceedings of the 17th IFAC World Congress (2008) pp. 15,678–15,683.Google Scholar
27.Bruyne, F. D., Anderson, B. D. O., Gevers, M. and Linard, N., “Iterative Controller Optimization for Nonlinear Systems,” In: Proceedings of the 36th IEEE Conference on Decision and Control, vol. 4 (1997), pp. 3749–3754.CrossRefGoogle Scholar
28.Hjalmarsson, H., “Iterative feedback tuning – An overview,” Int. J. Adapt. Control. Signal Process. 16, 373–395 (2002).CrossRefGoogle Scholar
29.Hara, S., Yamamoto, Y., Omata, T. and Nakano, M., “Repetitive control system: A new type servo system for periodic exogenous signals,” IEEE Trans. Autom. Control 33 (7)659–668 (1988).CrossRefGoogle Scholar
30.Satoh, S., Fujimoto, K. and Hyon, S., “A Framework for Optimal Gait Generation via Learning Optimal Control Using Virtual Constraint,” In: Proceedings of the IEEE/RSJ International Conference on Intelligent Robots and Systems (2008) pp. 3426–3432.Google Scholar
31.Satoh, S., Fujimoto, K. and Hyon, S., “Periodic Gait Generation via Repetitive Optimal Control of Hamiltonian Systems,” In: Proceedings of the 18th IFAC World Congress (2011) pp. 6912–6917.Google Scholar
32.Fujimoto, K. and Sugie, T., “Canonical transformation and stabilization of generalized Hamiltonian systems,” Syst. Control Lett. 42 (3), 217–227 (2001).CrossRefGoogle Scholar
33.Gregorio, P., Ahmadi, M. and Buehler, M., “Design, control, and energetics of an electrically actuated legged robot,” IEEE Trans. Syst. Man Cybern. 27 (4), 626–634 (1997).CrossRefGoogle ScholarPubMed
34.Asano, F., Luo, Z. W. and Yamakita, M., “Biped gait generation and control based on a unified property of passive dynamic walking,” IEEE Trans. Robot. 21 (4), 754–762 (2005).CrossRefGoogle Scholar
35.Collins, S., Ruina, A., Tedrake, R. and Wisse, M., “Efficient bipedal robots based on passive-dynamic walkers,” Science, 307 (5712), 1082–1085 (2005).CrossRefGoogle ScholarPubMed
36.Satoh, S. and Fujimoto, K., “On Passivity Based Control of Stochastic Port-Hamiltonian Systems,” In: Proceedings of the 47th IEEE Conference on Decision and Control (2008) pp. 4951–4956.Google Scholar
37.Satoh, S. and Fujimoto, K., “A Symmetric Structure of Variational and Adjoint Systems of Stochastic Hamiltonian Systems,” In: Proceedings of the 49th IEEE Conference on Decision and Control (2010) pp. 1423–1428.Google Scholar
38.Satoh, S. and Fujimoto, K., “Observer Based Stochastic Trajectory Tracking Control of Mechanical Systems,” In: Proceedings of the ICROS–SICE International Joint Conference (2009) pp. 1244–1248.Google Scholar