Hostname: page-component-cd9895bd7-dzt6s Total loading time: 0 Render date: 2024-12-29T07:38:19.395Z Has data issue: false hasContentIssue false
  • English
  • Français

Neuro-fuzzy-based skill learning for robots

Published online by Cambridge University Press:  08 December 2011

Hsien-I. Lin  and
C. S. George Lee
Hsien-I. Lin*
Affiliation:
Graduate Institute of Automation Technology, National Taipei University of Technology, Taipei, Taiwan
C. S. George Lee
Affiliation:
School of Electrical and Computer Engineering, Purdue University, West Lafayette, Indiana, USA
*
*Corresponding author. E-mail: sofin@ntut.edu.tw
Get access Rights & Permissions [Opens in a new window]

Summary

Endowing robots with the ability of skill learning enables them to be versatile and skillful in performing various tasks. This paper proposes a neuro-fuzzy-based, self-organizing skill-learning framework, which differs from previous work in its capability of decomposing a skill by self-categorizing it into significant stimulus-response units (SRU, a fundamental unit of our skill representation), and self-organizing learned skills into a new skill. The proposed neuro-fuzzy-based, self-organizing skill-learning framework can be realized by skill decomposition and skill synthesis. Skill decomposition aims at representing a skill and acquiring it by SRUs, and is implemented by stages with a five-layer neuro-fuzzy network with supervised learning, resolution control, and reinforcement learning to enable robots to identify a sufficient number of significant SRUs for accomplishing a given task without extraneous actions. Skill synthesis aims at organizing a new skill by sequentially planning learned skills composed of SRUs, and is realized by stages, which establish common SRUs between two similar skills and self-organize a new skill from these common SRUs and additional new SRUs by reinforcement learning. Computer simulations and experiments with a Pioneer 3-DX mobile robot were conducted to validate the self-organizing capability of the proposed skill-learning framework in identifying significant SRUs from task examples and in common SRUs between similar skills and learning new skills from learned skills.

Keywords

Self-organizing skill learningSkill decompositionSkill synthesisNeuro-fuzzy networkReinforcement learning
Type
Articles
Information
Robotica , Volume 30 , Issue 6 , October 2012 , pp. 1013 - 1027
Copyright
Copyright © Cambridge University Press 2011

Access options

Get access to the full version of this content by using one of the access options below. (Log in options will check for institutional or personal access. Content may require purchase if you do not have access.)

References

1.Nenchev, D. N. and Nishio, A., “Ankle and hip strategies for balance recovery of a biped subjected to an impact,” Robotica 26, 643–653 (2008).CrossRefGoogle Scholar
2.Kanda, T., Ishiguro, H., Imai, M., Ono, T. and Mase, K., “A Constructive Approach for Developing Interactive Humanoid Robots,” In: Proceedings of the IEEE/RSJ International Conference on Intelligent and Robotic Systems, Lausanne, Switzerland (2002) pp. 1265–1270.Google Scholar
3.Tomporowski, P., The Psychology of Skill: A Life-Span Approach (Praeger, Westport, CT, 2003).Google Scholar
4.Mosemann, H. and Wahl, F., “Automatic decomposition of planned assembly sequences into skill primitives,” IEEE Trans. Robot. Autom. 17, 709–718 (2001).CrossRefGoogle Scholar
5.Shibata, T., Abe, T., Tanie, K. and Nose, M., “Motion Planning of a Redundant Manipulator Based on Criteria of Skilled Operators,” In: Proceedings of the IEEE International Conference Systems, Man and Cybernetics, Vancouver, BC, Canada (1995) pp. 3730–3735.Google Scholar
6.Bonea, G. M. and Elbestawi, M. A., “Robotic force control for deburring using an active end effector,” Robotica 7, 303–308 (1989).CrossRefGoogle Scholar
7.Albus, J., “A new approach to manipulator control: The cerebellar model articulation controller (CMAC),” Trans. ASME J. Dynamic Syst. Meas. Contr. 63, 220–227 (1975).CrossRefGoogle Scholar
8.Poggio, T. and Girosi, F., “Networks for approximation and learning,” Proc. IEEE, 78 (9):1481–1497 (1990).CrossRefGoogle Scholar
9.Buhmann, M. D., Radial Basis Functions: Theory and Implementations (Cambridge University Press, Cambridge, UK, 2003).CrossRefGoogle Scholar
10.Baroglio, C., Attilio, G., Kaiser, M., M. Nuttin and Piola, R., “Learning controllers for industrial robots,” Mach. Learn. 23, 221–249 (1996).CrossRefGoogle Scholar
11.Nechyba, M. and Xu, Y., “Human Skill Transfer: Neural Networks as Learners and Teachers,” In: Proceedings of the IEEE/RSJ International Conference Intelligent Robots and Systems, Vancouver, BC, Canada (1995) pp. 314–319.Google Scholar
12.Wasik, Z. and Safiotti, A., “A Fuzzy Behavior-Based Control System for Manipulation,” In: Proceedings of the IEEE/RSJ International Conference Intelligent and Robotics Systems, Lausanne, Switzerland (2002) pp. 1596–1601.Google Scholar
13.Yang, J., Xu, Y. and Chen, C. S., “Human action learning via hidden Markov model,” IEEE Trans. Syst. Man Cybern. A 27, 34–44 (1997).CrossRefGoogle Scholar
14.Hovland, G., Sikka, P. and McCarragher, B., “Skill Acquisition from Human Demonstration Using a Hidden Markov Model,” In: Proceedings of the IEEE International Conference on Robotics and Automation, Minneapolis, MN (1996) pp. 2706–2711.CrossRefGoogle Scholar
15.Speeter, T., “Primitive-Based Control of the Utah/MIT Dextrous Hand,” In: Proceedings of the IEEE International Conference on Robotics Automation, Sacramento, CA (1991) pp. 866–877.Google Scholar
16.Milighetti, G., Kuntze, H. B., Frey, C. W., Diestel-Feddersen, B. and Balzer, J., “On a Primitive Skill-Based Supervisory Robot Control Architecture,” In: Proceedings of International Conference on Advanced Robotics, Seattle, WA (2005) pp. 141–147.Google Scholar
17.Matarić, M., “Behavior-based control: Examples from navigation, learning, and group behavior,” J. Exp. Theor. Artif. Intell. 9, 323–336 (1997).CrossRefGoogle Scholar
18.Arkin, R., “Motor Schema Based Navigation for a Mobile Robot: An Approach to Programming by Behavior,” In: Proceedings of the IEEE International Conference on Robotics and Automation, Raleigh, NC (1987) pp. 264–271.Google Scholar
19.Arbib, M., “Perceptual structures and distributed motor control,” In: Handbook of Physiology – The Nervous System II: Motor Control (Brooks, V. B., ed.) (American Physiological Society, Bethesda MD, 1981)pp. 1449–1480.Google Scholar
20.Brooks, R., “A robust layered control system for a mobile robot,” IEEE J. Robot. Autom. 2, 14–23 (1986).CrossRefGoogle Scholar
21.Purnamadjaja, A. H. and Russell, R. A., “Pheromone communication in a robot swarm: Necrophoric bee behaviour and its replication,” Robotica 23, 731–742 (2005).CrossRefGoogle Scholar
22.Matarić, M., Zordan, V. and Mason, Z., “Movement Control Methods for Complex, Dynamically Simulated Agents: Adonis Dances the Macarena,” In: Proceedings of the 2nd International Conference Autonomous Agents, Stuttgart, Germany (1998) pp. 317–324.CrossRefGoogle Scholar
23.Connell, J. and Viola, P., “Cooperative Control of a Semi-Autonomous Mobile Robot,” In: Proceedings of the IEEE International Conference on Robotics and Automation, Cincinnati, OH (1990) pp. 1118–1121.CrossRefGoogle Scholar
24.Martineza, A., Tunstela, E. and Jamshidi, M., “Fuzzy logic based collision avoidance for a mobile robot,” Robotica 12, 521–527 (1994).CrossRefGoogle Scholar
25.Slack, M., Situationally Driven Local Navigation for Mobile Robots, JPL Publication 90-17 (California Institute of Technology, Jet Propulsion Lab., Pasadena, CA, 1990).Google Scholar
26.Connolly, C., “Applications of Harmonic Functions to Robotics,” In: Proceedings of the IEEE International Symposium on Intelligent Control, Vancouver, British Columbia (1992) pp. 498–502.Google Scholar
27.Singh, S., “Transfer of learning by composing solutions of elemental sequential tasks,” Mach. Learn. 8, 323–339 (1992).CrossRefGoogle Scholar
28.Singh, S., “Reinforcement Learning with a Hierarchy of Abstract Models,” In: Proceedings of the 10th Nat. Conference Artificial Intelligence (AAAI Press, San Jose, CA, 1992) pp. 202–207.Google Scholar
29.Sutton, R., Precup, D. and Singh, S., “Between MDPs and semi-MDPs: A framework for temporal abstraction in reinforcement learning,” Artif. Intell. 112, 181–211 (1999).CrossRefGoogle Scholar
30.Parr, R. and Russell, S., “Reinforcement learning with hierarchies of machines,” In: Advances in Neural Information Processing Systems 10 (MIT Press, Cambridge, MA, 1998).Google Scholar
31.Dietterich, T., “Hierarchical reinforcement learning with the MAXQ value function decomposition,” J. Artif. Intell. Research 13, 227–303 (2000).CrossRefGoogle Scholar
32.Fitts, P. M. and Posner, M. I., Human Performance (Brooks/Cole, Belmont, CA, 1967).Google Scholar
33.Proctor, R. and Dutta, A., Skill Acquisition and Human Performance (Sage, London, 1995).Google Scholar
34.Schaal, S., “Is imitation learning the route to humanoid robots?” Trends Cogn. Sci. 3, 233–242 (1999).CrossRefGoogle ScholarPubMed
35.Dordevic, G. S., Rasic, M., Kostic, D. and Potkonjak, V., “Representation of robot motion control skill,” IEEE Trans. Syst. Man Cybern. C 30, 219–238 (2000).CrossRefGoogle Scholar
36.Nicolescu, M. N. and Matarić, M. J., “Natural Methods for Robot Task Learning: Instructive Demonstrations, Generalization and Practice,” In: Proceedings of the Second International Joint Conference on Autonomous Agents and Multi-Agent Systems, New York, NY, USA (2003) pp. 241–248.CrossRefGoogle Scholar
37.Lin, C. T. and Lee, C. S. G., “Neural-network-based fuzzy logic control and decision system,” IEEE Trans. Comput. 40, 1320–1336 (1991).CrossRefGoogle Scholar
38.Lin, C. T. and Lee, C. S. G., Neural Fuzzy Systems: A Neuro-Fuzzy Synergism to Intelligent Systems (Prentice-Hall, Upper Saddle River, NJ, 1996).Google Scholar
39.Wang, J. S. and Lee, C. S. G., “Self-adaptive neuro-fuzzy inference systems for classification applications,” IEEE Trans. Fuzzy Syst. 10, 790–802 (2002).CrossRefGoogle Scholar
40.Setnes, M., Babuška, R., Kaymak, U. and van Nauta Lemke, H., “Similarity measures in fuzzy rule base simplification,” IEEE Trans. Syst. Man Cybern. B 28, 376–386 (1998).CrossRefGoogle Scholar
41.Barto, A., Sutton, R. and Anderson, C., “Neuron-like adaptive elements that can solve difficult learning control problems,” IEEE Trans. Syst. Man Cybern. 13, 834–846 (1983).CrossRefGoogle Scholar
42.Lee, C. S. G. and Lin, C. T., “Supervised and Unsupervised Learning with Fuzzy Similarity for Neural-Network-Based Fuzzy Logic Control Systems,” In: Proceedings of the IEEE International Conference Systems, Man and Cybernetics (1992) pp. 688–693.Google Scholar
43.Jin, Y., Von Seelen, W. and Sendhoff, B., “On generating FC3 fuzzy rule systems from data using evolution strategies,” IEEE Trans. Syst. Man Cybern. B 29, 829–845 (1999).Google ScholarPubMed
44.Papoulis, A. and Pillai, S. U., Probability, Random Variables and Stochastic Processes (McGraw-Hill, Columbus, OH, 2001).Google Scholar
45.Janglova, D., “Neural networks in mobile robot motion,” Int. J. Adv. Robot. Syst. 1, 15–23 (2004).CrossRefGoogle Scholar
46.Anmin, Z. and Yang, S. X., “Neurofuzzy-based approach to mobile robot navigation in unknown environments,” IEEE Trans. Syst. Man Cybern. C, 37 610–621 (2007).Google Scholar