The most exciting perspective of RL for post-training is the ability to craft tailor-made rewards to achieve specific goals; more to follow 🚀. 📷 Blogpost: � 🙏 Thanks to @colinraffel & @aahmadian_ for the cooperation on this project!

Our latest blog From Zero to PPO: Understanding the Path to Helpful AI Models builds an intuitive understanding of PPO and its differentiation from other tuning techniques:

📣 At #VDS2024, Adaptive ML CTO @BaptistePannier joined Ahmed Menshawy of @Mastercard, @margaridagsl of @poolsideai, and @NeemaBal of NEEMA AI for a discussion on the challenges and rewards of getting GenAI into production. 📣 🎉 Thanks to @VDS_event for hosting! 🎉

A more effective training process would be for the LLM to suggest completions and learn through the evaluation of these completions instead. See how in our blog From Zero to PPO: Understanding the Path to Helpful AI Models -

We connect the dots between rejection sampling, REINFORCE, and Advantage Actor Critic, drawing a deeper understanding of how to tune LLMs to deliver helpful, harmless, and honest answers. Read our blog:

Taken at face value, PPO is puzzling. It involves four different versions of the model interacting together (policy, value, reward, and reference), and is driven by an intricate loss function. In our blog, we build-up to PPO, starting from supervised fine-tuning (SFT).

Helpfulness is instilled in LLMs as a result of extensive post-training. One approach in particular has been exceptionally successful: Reinforcement Learning from Human Feedback (RLHF). One of the engines of RLHF is Proximal Policy Optimization (PPO).

@lae_teo Welcome to the team!