The workshop hosts virtual poster sessions, invited talks, contributed talks, and a panel.
The livestream will be held from 17:00 EAT to 00:00 EAT (UTC+3).

Our program has three major components:


If videos are not appearing, disable ad-block!

You need an ICLR registration to ask questions live using zoom, to enter the chat system, and to participate in the poster sessions. Passwords can be found on the ICLR page. You have received your rocket chat user account in an email from ICLR.


The schedule can be added to your calendar and viewed in your timezone using this Google calendar.
All talks include live Q & A.

Times below are in EAT (UTC+3).

15:00 to 16:00 Poster session
17:00 to 17:15 Livestream start: Opening remarks
17:15 to 17:55 Invited talk: “Understanding Inductive Biases for Betrrl Agents” by Martha White
17:55 to 18:15 A Boolean Task Algebra for Reinforcement Learning” by Geraud Nangue Tasse
18:15 to 18:30 Break
18:30 to 19:10 Invited talk: “Rethinking Supervision in Meta-Reinforcement Learning” by Abhishek Gupta
19:10 to 19:30 Fast Adaptation to New Environments via Policy-Dynamics Value Functions” by Roberta Raileanu
19:30 to 20:30 Panel discussion
20:30 to 20:45 Break
20:45 to 21:45 Poster session
21:45 to 22:25 Invited talk: “Meta-learning causal reasoning” by Ishita Dasgupta
22:25 to 22:45 Think, Learn, and Act on An Episodic Memory Graph” by Ge Yang
22:45 to 23:00 Break
23:00 to 23:40 Invited talk: “Learning to Continually Learn” by Jeff Clune
23:40 to 00:00 Multi-Task Reinforcement Learning with Soft Modularization” by Ruihan Yang


Invited Speakers

Title Speaker
Understanding Inductive Biases for Betrrl Agents White, Martha
Rethinking Supervision in Meta-Reinforcement Learning Gupta, Abhishek
Meta-learning causal reasoning Dasgupta, Ishita
Learning to Continually Learn Clune, Jeff

Abhishek Gupta (UC Berkeley), “Rethinking Supervision in Meta-Reinforcement Learning”

Abhishek Gupta

Abhishek Gupta is a 5th year PhD student at UC Berkeley working with Pieter Abbeel and Sergey Levine, where he is interested in algorithms that can leverage reinforcement learning algorithms to solve real world robotics tasks. Currently he has been pursuing the directions of effective reward supervision in reinforcement learning, learning from demonstrations, meta-reinforcement learning and multi-task reinforcement learning. He has also spent time at Google Brain. He is also the recipient of the NDSEG and NSF graduate research fellowships, and several of his works have been presented as spotlight presentations at top-tier machine learning and robotics conference. His work has been covered by multiple popular news outlets such as the New York Times and VentureBeat.

Ishita Dasgupta (Harvard University), “Meta-learning Causal Reasoning”

Ishita Dasgupta

Ishita Dasgupta is a 5th year graduate student in the physics department at Harvard University, advised by Prof. Sam Gershman. Her research is at the intersection of machine learning and computational cognitive science, both on a) leveraging methods from machine learning to shed light on process-level accounts of how humans make inferences, and b) using frameworks from cognitive science to build a better understanding of black-box machine learning algorithms. Her thesis is on trying to understand how humans infer probabilities in the real world. Specifically, how people might be trading-off the precision/accuracy and the computational costs of various algorithms for statistical inference.

Martha White (University of Alberta), “Understanding Inductive Biases for Betrrl Agents”

Martha White

Martha White is an Assistant Professor in the Department of Computing Sciences at the University of Alberta, Faculty of Science. Martha is a PI of AMII, the Alberta Machine Intelligence Institute, and a director of RLA, the Reinforcement Learning and Artificial Intelligence Lab at the University of Alberta. Her primary research goal is to develop techniques for adaptive autonomous agents learning on streams of data. Her research focus to achieve this goal is on reinforcement learning and representation learning. In particular, efficient, practical algorithms that enable learning from large amounts of data. She has also been working on off-policy reinforcement learning.

Jeff Clune (OpenAI), “Learning to Continually Learn”

Jeff Clune

Jeff Clune is a Research Manager at OpenAI and the former Loy and Edith Harris Associate Professor in Computer Science at the University of Wyoming. Previously he was the Senior Research Manager and founding member of Uber AI Labs, which was formed after Uber acquired the startup Geometric Intelligence. Jeff focuses on robotics and training neural networks via deep learning and deep reinforcement learning. He has also researched open questions in evolutionary biology using computational models of evolution, including studying the evolutionary origins of modularity, hierarchy, and evolvability. Prior to becoming a professor, he was a Research Scientist at Cornell University, received a PhD in computer science and an MA in philosophy from Michigan State University, and received a BA in philosophy from the University of Michigan.  

Invited Panelists

In addition to invited speakers, we are happy to have the following invited panelists:

Jürgen Schmidhuber (The Swiss AI Lab IDSIA / NNAISENSE)

Jürgen Schmidhuber

Jürgen Schmidhuber is the Scientific Director of IDSIA and Professor of Artificial Intelligence. His research group has revolutionised machine learning and AI and also established the fields of metalearning, mathematically rigorous universal AI and recursive self-improvement in universal problem solvers that learn to learn. He also generalized algorithmic information theory and the many-worlds theory of physics, and introduced the concept of Low-Complexity Art, the information age’s extreme form of minimal art. He is the recipient of numerous awards, author of over 350 peer-reviewed papers, and Chief Scientist of the company NNAISENSE. He is also advising various governments on AI strategies.

Panel questions

Contributed Talks / Spotlights

The zoom meetings, videos, and chats can be found by clicking on the link for each paper.

Title Authors
Multi-Task Reinforcement Learning with Soft Modularization Yang, Ruihan*; Xu, Huazhe; Wu, Yi; Wang, Xiaolong
Fast Adaptation to New Environments via Policy-Dynamics Value Functions Raileanu, Roberta*; Goldstein, Max; Szlam, Arthur; Fergus, Rob
A Boolean Task Algebra for Reinforcement Learning Nangue Tasse, Geraud*; James, Steven D; Rosman, Benjamin
Think, Learn, and Act on An Episodic Memory Graph Yang, Ge*; Zhang, Amy; Morcos, Ari S; Pineau, Joelle; Abbeel, Pieter; Calandra, Roberto

Poster Sessions

Our virtual poster sessions will take place from 15:00 to 16:00 EAT and from 20:45 to 21:45 EAT. Attendees will be able to join zoom meetings to discuss the research in more detail and switch from one meeting to the other similar to a physical poster session. Click each talk / abstract to obtain the zoom links. We recommend watching the video before joining the session.

Contributed Abstracts

The zoom meetings, videos, and chats can be found by clicking on the link for each paper.

Title Authors
Goal-conditioned Batch Reinforcement Learning for Rotation Invariant Locomotion Mavalankar, Aditi*
Adaptive Exploration via Modulated Behaviour Schaul, Tom*; Borsa, Diana; Ding, David; Szepesvari, David; Ostrovski, Georg; Dabney, Will; Osindero, Simon
BabyAI++: Towards Grounded-Language Learning beyond Memorization Cao, Tianshi; Wang, Jingkang; Zhang, Annie; Manivasagam, Sivabalan*
Robust Visual Domain Randomization for Reinforcement Learning Clements, William R*; Slaoui, Reda; Foerster, Jakob; Toth, Sébastien
Goal-Aware Prediction: Learning to Model What Matters Nair, Suraj*; Savarese, Silvio; Finn, Chelsea
MULTIPOLAR: Multi-Source Policy Aggregation for Transfer Reinforcement Learning between Diverse Environmental Dynamics Barekatain, Mohammadamin; Yonetani, Ryo*; Hamaya, Masashi
Latent State Models for Meta-Reinforcement Learning Nagabandi, Anusha; Rakelly, Kate*; Zhao, Zihao; Finn, Chelsea; Levine, Sergey
Accelerating Meta-Learning by Sharing Gradients Chang, Oscar*; Flokas, Lampros; Lipson, Hod
Generating Automatic Curricula via Self-Supervised Active Domain Randomization Raparthy, Sharath Chandra*; Mehta, Bhairav J; Golemo, Florian; Paull, Liam
Invariant Causal Prediction for Block MDPs Zhang, Amy*; Lyle, Clare; Sodhani, Shagun; Filos, Angelos; Kwiatkowska, Marta; Pineau, Joelle; Gal, Yarin; Precup, Doina
Estimating Q(s,s') with Deep Deterministic Dynamics Gradients Edwards, Ashley D.*; Sahni, Himanshu; Liu, Rosanne; Hung, Jane; Jain, Ankit; Wang, Rui; Ecoffet, Adrien; Miconi, Thomas; Isbell, Charles; Yosinski, Jason
Curriculum for Gradient-Based Meta-Learners Mehta, Bhairav J*; Deleu, Tristan; Raparthy, Sharath Chandra; Pal, Chris J; Paull, Liam
Multi-Task Reinforcement Learning with Soft Modularization Yang, Ruihan*; Xu, Huazhe; Wu, Yi; Wang, Xiaolong
Deep Modular Reinforcement Learning for Physically Embedded Abstract Reasoning Karkus, Peter*; Mirza, Mehdi; Guez, Arthur; Jaegle, Andrew; Lillicrap, Timothy; Buesing, Lars; Heess, Nicolas; Weber, Theophane
Time Adaptive Reinforcement Learning Reinke, Chris*
ITER: Iterated Relearning for Improved Generalization in Reinforcement Learning Igl, Maximilian*; Boehmer, Wendelin; Whiteson, Shimon
Deep Sets for Generalization in Reinforcement Learning Karch, Tristan; Colas, Cédric*; Teodorescu, Laetitia; Moulin-Frier, Clément; Oudeyer, Pierre-Yves
Fast Adaptation to New Environments via Policy-Dynamics Value Functions Raileanu, Roberta*; Goldstein, Max; Szlam, Arthur; Fergus, Rob
The NetHack Learning Environment Küttler, Heinrich; Nardelli, Nantas; Raileanu, Roberta*; Selvatici, Marco; Grefenstette, Edward ; Rocktäschel, Tim
A Boolean Task Algebra for Reinforcement Learning Nangue Tasse, Geraud*; James, Steven D; Rosman, Benjamin
Exploration in Approximate Hyper-State Space Zintgraf, Luisa M*; Feng, Leo; Igl, Maximilian; Hartikainen, Kristian; Hofmann, Katja; Whiteson, Shimon
Visual Control with Variational Contrastive Dynamics Luo, Calvin*; Hafner, Danijar
Deconstructing Model-Based Visual Reinforcement Learning Babaeizadeh, Mohammad*; Saffar, Mohammad; Hafner, Danijar; Erhan, Dumitru; Kannan, Harini; Finn, Chelsea; Levine, Sergey
Weakly-Supervised Trajectory Segmentation for Learning Reusable Skills Mahmoudieh, Parsa*; Darrell, Trevor; Pathak, Deepak
Attention Driven Dynamic Memory Maps Sahni, Himanshu*; Bansal, Shray; Isbell, Charles
Safely Transferring to Unsafe Environments with Constrained Reinforcement Learning Knight, Ethan*; Achiam, Joshua
Planning to Explore via Latent Disagreement Sekar, Ramanan; Rybkin, Oleh*; Daniilidis, Kostas; Abbeel, Pieter; Hafner, Danijar; Pathak, Deepak
Offline Meta-Reinforcement Learning with Advantage Weighting Mitchell, Eric A*; Rafailov, Rafael; Peng, Xue Bin; Levine, Sergey; Finn, Chelsea
Improving Policy Gradient via Parameterised Reward Liu, Hao*; Abbeel, Pieter
Dyna-AIL : Adversarial Imitation Learning by Planning Saxena, Vaibhav*; Sivanandan, Srinivasan; Mathur, Pulkit
Trying again instead of trying longer: Prior learning for Automatic Curriculum Learning Portelas, Rémy*; Hofmann, Katja; Oudeyer, Pierre-Yves
Privileged Information Dropout in Reinforcement Learning Kamienny, Pierre-Alexandre; Behbahani, Feryal; Arulkumaran, Kai*; Boehmer, Wendelin; Whiteson, Shimon
Bayesian Online Meta-Learning with Structured Laplace Approximation Yap, Pau Ching*; Ritter, Hippolyt; Barber, David
Fast adaptation with importance weighted priors Galashov, Alexandre*; Sygnowski, Jakub; Desjardins, Guillaume; Humplik‎, Jan; Hasenclever, Leonard; Heess, Nicolas; Teh, Yee Whye
Think, Learn, and Act on An Episodic Memory Graph Yang, Ge*; Zhang, Amy; Morcos, Ari S; Pineau, Joelle; Abbeel, Pieter; Calandra, Roberto
MGHRL: Meta Goal-generation for Hierarchical Reinforcement Learning Fu, Haotian*; Tang, Hongyao; Hao, Jianye; Liu, Wulong; Chen, Chen
Transfer Learning via Diverse Policies in Value-Relevant Features Luketina, Jelena*; Smith, Matthew; Igl, Maximilian; Whiteson, Shimon
Generative Adversarial Simulator Raiman, Jonathan*

Program committee

Main organizers:

Louis Kirsch
Ignasi Clavera
Kate Rakelly
Jane Wang
Chelsea Finn
Jeff Clune

Thanks to all of our reviewers:

Alexandre Galashov
Andrei Rusu
Ashvin Nair
Aviral Kumar
Bradly Stadie
Brandon Schoenfeld
Charles Blundell
Devendra Singh Chaplot
Dumitru Erhan
Dushyant Rao
Haoran Tang
Hugo Jair Escalante
Jakub Sygnowski
Jan Humplik
Jessica Hamrick
Karol Hausman
Kelvin Xu
Krsto Proroković
Kyle Hsu
Luisa Zintgraf
Marc Pickett
Marcin Andrychowicz
Marta Garnelo
Maximilian Igl
Misha Denil
Parminder Bhatia
Piotr Mirowski
Sayna Ebrahimi
Tin Ho
Tom Schaul
Vitchyr Pong