16 - Learning Hierarchical Refinement Models

Summary

The hierarchical refinement approach in the previous two chapters requires a priori domain knowledge of the methods, action models, and heuristics used by RAE and UPOM. The topic of this chapter is to use machine learning techniques to synthesize planning heuristics and domain knowledge. It illustrates the "planning to learn" paradigm for learning domain-dependent heuristics to guide RAE and UPOM. Given methods and a sample function, UPOM generates near-optimal choices that are taken as targets by a deep Q-learning procedure. The chapter shows how to synthesize methods for tasks using hierarchical reinforcement techniques.