Here is the recording of the presentation:

@fhuszar @andrewgwils I also found: and:

@DHolzmueller I wonder if schedule-free AdamW could help here.

@glouppe Maybe this is just a copy and paste error.

@jn2clark How many epochs? Maybe ADOPT works best (as an optimizer) but makes it easier to overfit as a result? Strong regularization might be needed (e.g. dropout or SAM-style updates?)

@nworbmot Indeed, I read some of the details in the links of the thread after asking :) I suppose this kind of energy storage can help justify the increase of demand side flexibility from 5% to 10%.

@PreetumNakkiran Or "creativity".

I am looking forward to giving a keynote presentation at @PyDataParis 2024, September 25-26, Cité des Sciences. This will be an opportunity to reflect on the meaning of "noise" and predictive uncertainty in machine learning applications.

@ML_BenWalker @srush_nlp But one could argue this is just a UI problem: the LLM could be prompted to output an inner voice "train of thoughts" first followed by a final answer and the UI hides the transient inner voice output to the user. I am sure that some 'chat' platforms already implement this.

@karpathy s/state/step/

@tomgoldsteincs Do you think this idea could be adapted to generative image / video models (stable diffusion and the like)?

RT @PyDataParis: The schedule of PyData Paris 2024 is LIVE! 🚀 Have you secured your tickets yet? 🎟️ Join us for two inspiring days of ope…