George E. Dahl
@GeorgeEDahl
Followers
4K
Following
145
Statuses
130
Machine learning researcher @Google research. My opinions do not necessarily represent my employer. Prefer email over DMs. https://t.co/FI9gvTzCbO…
Joined March 2021
Congratulations to the winners and everyone who submitted! Now that we finally have a rigorous definition of the state of the art training algorithm, we will see rapid innovation in training algorithms!
@MLCommons #AlgoPerf results are in! 🏁 $50K prize competition yielded 28% faster neural net training with non-diagonal preconditioning beating Nesterov Adam. New SOTA for hyperparameter-free algorithms too! Full details in our blog. #AIOptimization #AI
1
5
26
RT @JeffDean: So, it turns out you don't get much time to throw together a Nobel Prize party, but we got people together to celebrate! 🎉 H…
0
137
0
RT @annadgoldie: We are also releasing a model checkpoint pre-trained on 20 TPU blocks, so our open-source method now has open-weights as w…
0
15
0
RT @annadgoldie: In 2020, we introduced an AI method capable of generating superhuman chip layouts. Today, we describe its impact on the fi…
0
25
0
RT @aaron_defazio: Schedule-Free Wins AlgoPerf Self-Tuning Track 🎉 I'm pleased to announce that Schedule-Free AdamW set a new SOTA for self…
0
28
0
RT @zacharynado: see this thread for an in-depth description of the benchmark setup, if you're interested in submitting your ideas you shou…
0
1
0
RT @GoogleAI: Congratulations to everyone who submitted to the @MLCommons AlgoPerf training algorithms competition! We were delighted to pr…
0
25
0
RT @zacharynado: "Non-diagonal preconditioning has dethroned Nesterov Adam" 🧴👑 shampoo wins, finally the community can know what we have f…
0
23
0
RT @frankstefansch1: The inaugural AlgoPerf results are in, highlighting a new generation of neural net training algorithms! Get 28% faster…
0
9
0
RT @deepcohen: to focus less on inventing new algorithms, and to focus more on understanding the ones we already have.
0
2
0
RT @deepcohen: Algorithm design is traditionally considered to be the most important type of work. Yet the story we keep seeing in modern…
0
7
0
@Euclaise_ @zacharynado Please do! And join the working group and let us know if you have any issues (or create github issues/discussions)
0
0
2
@omeadsthename @zacharynado Harder to aggregate across workloads if per-workload results have different units. If someone can figure out a nice way to deal with that, I'd love to see the "dual" benchmark where time is fixed and loss is minimized instead of fixing loss/eval targets and minimizing runtime
1
0
2
@tOSUFever @zacharynado MLCommons is providing the prize money, Google is providing computational support for evaluating the top submissions. Since I'm one of the co-chairs organizing, Googlers can't win prizes, but we will still submit stuff. But no one has to beat us to win money so don't worry 😀
0
0
2
@GaelVaroquaux Partly. There are also issues with invalid comparisons even on a single workload due to improper accounting of hparam tuning, differences in tuning goals, or even incoherent (or at best non-quantitative) definitions of training speed.
1
0
1
RT @MLCommons: Improved training algorithms can save time, computational resources, and lead to better, more accurate, models. Thank you @A…
0
1
0
@GaelVaroquaux IMO confidence intervals (or lack thereof) are not even a top 3 issue with comparisons of training algs (see for the largest issues) . However, AlgoPerf will score based on the median of 5 repetitions of the entire hparam tuning and training procedure.
1
0
0
RT @ekindogus: Thrilled to share this work on materials discovery! We found that OOD generalization of GNNs improves predictably, with in…
0
33
0
RT @JeffDean: Exciting! "The AlgoPerf: Training algorithms benchmark is a competitive, time-to-result benchmark that runs on a fixed syste…
0
31
0
RT @MLCommons: A big thank you to @GoogleAI for providing the compute resources for the @mlcommons Algorithm benchmark efficiency competiti…
0
5
0