DOMAIN KNOWLEDGE INTEGRATION BY GRADIENT MATCHING FOR SAMPLE-EFFICIENT REINFORCEMENT LEARNING

• Chadha, Parth*
• Accepted abstract
• [PDF] [Slides] [Join poster session]
  Poster session from 15:00 to 16:00 EAT and from 20:45 to 21:45 EAT
  Obtain the zoom password from ICLR

Abstract

Model-free deep reinforcement learning (RL) agents can learn an effective policy directly from repeated interactions with a black-box environment. However, in practice the algorithms often require large amounts of training experience to learn and generalize well. In addition, classic model-free learning ignores the domain information contained in the state transition tuples. Model-based RL, on the other hand, attempts to learn a model of the environment from experience and is substantially more sample efficient, but suffers from significantly large asymptotic bias owing to the imperfect dynamics model. In this paper, we propose a gradient matching algorithm to improve sample efficiency by utilizing target slope information from the dynamics predictor to aid the model-free learner. We demonstrate this by presenting a technique for matching the gradient information from the model-based learner with the model-free component in an abstract low-dimensional space and validate the proposed technique through experimental results that demonstrate the efficacy of this approach.