Jon Deaton
@deaton_jon
Followers
82
Following
182
Statuses
26
Researcher at @EvoscaleAI
San Francisco, CA
Joined November 2012
This is an incredibly cool plot - it shows how protein language models form internal representations of physical structure when trained only on amino acid sequences selected by evolution. The representation fidelity scales with compute.
Information about protein structure in ESM C representations improves predictably with increasing training compute, demonstrating linear scaling across multiple orders of magnitude. (We overtrained the 300M and 600M models past the predicted point of compute optimality).
0
2
15
I'm excited to share the release of ESM3 and the launch of EvolutionaryScale! I've had the privilege of joining this talented team of researchers to build foundation models for biology. So proud of the work that we've done and to make ESM3 openly available!
We have trained ESM3 and we're excited to introduce EvolutionaryScale. ESM3 is a generative language model for programming biology. In experiments, we found ESM3 can simulate 500M years of evolution to generate new fluorescent proteins. Read more:
1
0
13
@MaximZiatdinov Very cool! I see that the inferred noise uncertainty extends to negative near X=2.0. Wondering if the noise GP should operate in log space to handle that variance must be positive?
1
0
0
RT @irhumshafkat: New post's up! This one takes a look at the interaction of LoRA with weight decay (WD). In brief: LoRA + default WD produ…
0
2
0
@PatrickKidger Actually, I should say: "Are your datasets well shuffled?" So as not to assume you're using beam, and also that it could alternatively be caused by poorly shuffled training dataset too.
0
0
0
RT @GoogleHealth: 2 billion people globally suffer from skin conditions. Our CE-marked AI-powered dermatology tool will help people researc…
0
165
0
RT @vivnat: @ckbjimmy will be presenting this work tmrw afternoon at the #NeurIPS2020 #ML4H workshop. If you are working on datasets with…
0
2
0
Proud to have contributed to the ML in Healthcare aspects here!
NEW from a big collaboration at Google: Underspecification Presents Challenges for Credibility in Modern Machine Learning Explores a common failure mode when applying ML to real-world problems. 🧵 1/14
0
0
3
New paper!
Our new work "Addressing the real world class imbalance problem in dermatology"- has been accepted as a full paper at #NeurIPS2020 ML4H workshop! Work led by brilliant @ckbjimmy with @GoogleHealth /Brain team mates Jon Deaton, @gamaleldinfe and Yuan Liu
0
0
1
RT @BarackObama: "No one is born hating another person because of the color of his skin or his background or his religion..." https://t.co/…
0
1M
0
Look I did a science PhaMers identifies novel bacteriophage sequences from thermophilic hot springs
0
0
0