Offline Deep Reinforcement Learning for Robotic Arm Control in the ManiSkill Environment

Abstract

Offline reinforcement learning (Offline RL) has been widely used in robot control tasks, while online reinforcement learning is often abandoned due to its high interaction cost with environment. In order to face this situation, Behavior Cloning (BC), an approach of Imitation learning (IL), is often considered a suitable choice for solving this prob-lem, in which the agent could learn from a offline dataset. In this study, we propose a intuitive way, in which we add a Proximal Policy Optimization (PPO) loss as a correction term to the BC loss. The models are trained on a static dataset with four different robotic arm control tasks given by the ManiSkill environment. We demonstrate a comparison of the proposed approach with other existing Offline RL algorithms.