The hierarchically structure is improved in learning process, it forms different hierarchical reinforcement learning (RL) method by adopting different RL algorithm.

• 递阶结构中嵌入不同的再励学习方法可以形成不同的递阶学习。