Journal Presentation - Nature Machine-Intelligence

The journal paper Hybrid Hierarchical Learning for Solving Complex Sequential Tasks Using the RObotic MAnipulation Network - ROMAN can be accessed in Nature Machine Intelligence, with a video presentation of the work also available on YouTube.

Paper Abstract

Solving long sequential tasks remains a nontrivial challenge in the field of embodied artificial intelligence. Enabling a robotic system to perform diverse sequential tasks with a broad range of manipulation skills is a significant open problem and continues to be an active area of research. In this work, we present a Hybrid Hierarchical Learning framework, the Robotic Manipulation Network (ROMAN), to address the challenge of solving multiple complex tasks over long time horizons in robotic manipulation. By integrating behavioural cloning, imitation learning, and reinforcement learning, ROMAN achieves task versatility and robust failure recovery. ROMAN consists of a central manipulation network that coordinates an ensemble of various neural networks, each specialising in different re-combinable sub-tasks to generate their correct in-sequence actions, in order to solve complex long horizon manipulation tasks. Our experiments show that by orchestrating and activating these specialised manipulation experts, ROMAN generates correct sequential activations accomplishing long sequences of sophisticated manipulation tasks and achieving adaptive behaviours beyond demonstrations, while exhibiting robustness to various sensory noises. These results highlight the significance and versatility of ROMAN’s dynamic adaptability featuring autonomous failure recovery capabilities, and underline its potential for various autonomous manipulation tasks that require adaptive motor skills.