22 - Learning for Movement Actions

Summary

Models for TAMP problems are complex and challenging to develop. The high-dimensional sensory-motor space and the required integration of metric and symbolic state variables augment the challenges. Machine learning addresses these challenges at both the acting level and the planning level. But ML in robotics faces specific problems: lack of massive data; experiments needed for RL are scarce, very expensive, and difficult to reproduce; realistic sensory-motor simulators remain computationally costly; and expert human input for RL, e.g., for specifying or shaping reward functions or giving advices, is scarce and costly. The functions learned tend to be narrow: transfer of learned behaviors and models across environments and tasks is challenging. This chapter presents approaches for learning reactive sensory-motor skills using deep RL algorithms and methods for learning heuristics to guide a TAMP planner avoiding computation on unlikely feasible movements.