Events

Reinforcement Learning (RL) is known for its ability to discover new knowledge, while modern generative models excel at distilling and summarizing internet data—data that represents existing human knowledge. Integrating RL and generative models promises a path towards creating AI systems that can surpass human capabilities. In this talk, Sun focuses on two critical aspects of the synergy between RL and generative models: (1) designing advanced RL algorithms to enable more effective generative models, such as language models that outperform humans on specific tasks and image generation models that produce novel, yet realistic images, and (2) leveraging generative models to enable more powerful RL. In the first part, Sun explores two new RLHF (Reinforcement Learning with Human Feedback) algorithms that are theoretically sound, scalable to models with billions of parameters, versatile across multimodal generation, and exhibit impressive practical performance on benchmarks. The second part of the talk delves into the framework of inverse RL, demonstrating how a powerful diffusion model can facilitate more effective learning from human demonstrations, leading to improved performance in tasks such as whole-body humanoid control. Sun will highlight the critical theoretical benefits of employing diffusion models in inverse RL, which are reflected in empirical advancements. Sun will conclude by discussing new approaches for integrating RL with generative models and how this represents a promising path toward developing more capable AI systems.