@jramapuram In the case of language modeling, KL is computed only over the next token's distribution, but completions' prefixes are different. So, dataset expansion is the simplest way to increase diversity of possible contexts (prompt + prefix) for next-token-KL computations.

6/ Our paper is out: This work was the result of my internship at @GoogleDeepMind—huge thanks to the team: Daniele Calandriello, @johanferret, @sarah_perrin_, @nino_vieillard, @ramealexandre, @mblondel_ml!

🔍 Check out our paper and code and see you at @aistats_conf in Valencia! Of course, a lot of thanks to my colleagues Nikita Morozov, Alexey Naumov and Dmitry Vetrov!

RT @misovalko: Let’s get closer to understanding the ➡️ scaling laws ⬅️ for human feedback in RHLF! 🚀 Our rising star student @dtiapkin ha…