Chief Scientist, Google DeepMind & Google Research. Opinions stated here are my own, not those of my company. TensorFlow, MapReduce, Bigtable, Spanner, Gemini..
In April, '17,
@jsomers
of
@NewYorker
reached out & said he wanted to do a small profile of me & my longtime colleague Sanjay Ghemawat, watch us work for a few hours, maybe dinner, etc.
It came out today. I think it captures our working style really well.
I’m very excited to share our work on Gemini today! Gemini is a family of multimodal models that demonstrate really strong capabilities across the image, audio, video, and text domains. Our most-capable model, Gemini Ultra, advances the state of the art in 30 of 32 benchmarks,
Google Dataset Search is now officially out of beta.
"Dataset Search has indexed almost 25 million of these datasets, giving you a single place to search for datasets & find links to where the data is."
Nice work, Natasha Noy and everyone else involved!
Gemini 1.5 Pro - A highly capable multimodal model with a 10M token context length
Today we are releasing the first demonstrations of the capabilities of the Gemini 1.5 series, with the Gemini 1.5 Pro model. One of the key differentiators of this model is its incredibly long
Introducing Gemma - a family of lightweight, state-of-the-art open models for their class, built from the same research & technology used to create the Gemini models.
Blog post:
Tech report:
This thread explores some of the
25 years ago today I officially started at a small search engine company, wedged into a tiny office space above what's now a T-Mobile store in downtown Palo Alto. Since then, I have had the incredible pleasure of working with awesome colleagues on software used by billions of
I am very hopeful that we move the U.S. back to welcoming the best & brightest students from all around the world to our colleges and universities. For so many decades, this was an incredible strength of the U.S. I hope we get back to welcoming amazing students with open arms!
A reminder that some people in our field are alienating our female colleagues by flirting in settings that are meant to be professional and by trying to turn what should be topic- & setting-appropriate conversations into dates.
Please don't do this.
Last week, many of my colleagues got together to throw a surprise party to celebrate my 20th anniversary of working
@Google
. I am delighted & incredibly fortunate to have worked with so many amazing people over the years, on such a wide variety of projects. Thank you all!
Happy 25th Birthday Google! 🎉
I have gotten incredible enjoyement from being along for the ride for 24+ of these years. When I joined, we were a handful of people wedged into a small office area in downtown Palo Alto above what is now a T-Mobile store.
1/
Congratulations to my colleague
@geoffreyhinton
, along w/
@ylecun
, & Yoshua Bengio for receiving this year's
@TheOfficialACM
Turing Award (the highest honor in computer science) for their pioneering work in deep learning!
So, it turns out you don't get much time to throw together a Nobel Prize party, but we got people together to celebrate! 🎉
Here's me & Geoff before the party started, & a nice group photo of all of Geoff's former PhD students that were able to attend on short notice.
This is a small research community. To all my friends, former colleagues and those that I don't know at OpenAI, the last few days have been a rollercoaster for those of us watching from the outside: I can't imagine the ebb & flow of emotions you're all experiencing. ♥️
I'm very happy to see
@Google
&
@Apple
partnering on technology to help w/ COVID-19 contact tracing. Technology, coupled w/ strong consideration of privacy, will enable public health authorities to create apps w/ appropriate user opt-in.
@TSA
Ahh, yes. This was why you confiscated my then 13-year old vegetarian daughter's unopened peanut butter jar that was going to be a significant source of protein for her on an overseas trip to a less-than-friendly-to-vegetarian-diets region. 🙏
Check out NotebookLM! Create a notebook, upload one or more sources (e.g. PDFs of research papers, your favorite PhD thesis, a newspaper article, etc) then click on 'Generate' to create a podcast of two voices talking about the content you've uploaded.
We're starting to roll out API support for Gemini 1.5 Pro for developers. We're excited to see what you build with the 1M token context window!
We'll be onboarding people to the API slowly at first, and then we'll ramp it up. In the meantime, developers can try out Gemini 1.5
Gemini 1.5 Pro - A highly capable multimodal model with a 10M token context length
Today we are releasing the first demonstrations of the capabilities of the Gemini 1.5 series, with the Gemini 1.5 Pro model. One of the key differentiators of this model is its incredibly long
On behalf of the Google Brain team, I wrote a summary of some of the things we did in 2017. This is part 1 of 2. I'm very proud of the whole team and what we accomplished in 2017!
I understand the concern over Timnit’s resignation from Google. She’s done a great deal to move the field forward with her research. I wanted to share the email I sent to Google Research and some thoughts on our research process.
On behalf of the whole Google Research &
@GoogleAI
community, I was excited to put together a post describing some of the work that we collectively did in 2018. I hope you enjoy it!
Thanks to everyone who helped make this work possible!
We're continuing to expand our
@GoogleAI
teams around the world. We’ll be opening our first research center in Africa in
#Ghana
later this year!
If you're a machine learning researcher interested in working in Accra, Ghana, apply:
AI is full of promise, with the potential to revolutionize so many different areas of modern society.
In order to realize its true potential, our field needs to be welcoming to all people. As it stands today, it is definitely not.
Our field has a problem with inclusiveness.
Welcome, AlphaChip!
Today, we are sharing some exciting updates on our work published in
@Nature
in 2021 on using reinforcement learning for ASIC chip floorplanning and layout. We’re also naming this work AlphaChip.
Since we first published this work, our use of this approach
Our AI for chip design method AlphaChip has transformed the way we design microchips. ⚡
From helping to design state-of-the-art TPUs for building AI models to CPUs in data centers - its widespread impact can be seen across Alphabet and beyond.
Find out more →
AI System Achieves Silver Medal-level score in IMO
The International Mathematical Olympiad (IMO) is the oldest, largest & most prestigious competition for young mathematicians. Every year, countries send their top young mathematicians to take a 6 problem test spanning two days.
At Google, we've been getting a better understanding of issues of bias & fairness in machine learning models as we've used ML throughout more of our products. We've also created training for Google engineers on these topics, and we've now made this material available externally.
Introducing the MLCC Fairness in
#MachineLearning
module (), which looks at different types of human biases that can manifest in training data, and provides strategies to identify them and evaluate their effects. Learn more at:
Congratulations to my good friend & former Google colleague Geoff Hinton for winning this year's Nobel Prize in Physics (along w/John Hopfield)! 🎉
Geoff's impact on so many scientific fields continues to grow as neural networks are applied in more & more domains.
Celebratory
BREAKING NEWS
The Royal Swedish Academy of Sciences has decided to award the 2024
#NobelPrize
in Physics to John J. Hopfield and Geoffrey E. Hinton “for foundational discoveries and inventions that enable machine learning with artificial neural networks.”
Excited to share the first of a series of
@GoogleAI
blog posts summarizing our research work from 2022. This covers language & multimodal models, computer vision, and generative models. We'll have ~7 posts covering other areas over next few weeks!
AI can unlock joint human/computer creativity! Imagen is one direction we are pursuing:
"A high contrast portrait of a very happy fuzzy panda dressed as a chef in a high end kitchen making dough. There is a painting of flowers on the wall behind him."
My
@Google
colleague and longtime
@UCBerkeley
faculty member David Patterson has a great essay out in this month's Communications of the ACM (
@TheOfficialACM
):🎉
"Life Lessons from the First Half-Century of My Career
Sharing 16 life lessons, and nine magic words."
I saw an
Human language LLMs are so passe. Read about this work on a whale bioacoustics model that can identify eight distinct species, including multiple calls for some of these species (e.g. the "biotwang" of the Bryde's whale).
I'm going to say this until I'm blue in the face, but
Introducing our new whale bioacoustics model, which can identify eight distinct species, including multiple calls for two of those species. The model even includes the “Biotwang” sounds recently attributed to the Bryde’s whale. Learn more at:
Gemini is Latin for "twins".
The Gemini effort came about because we had different teams working on language modeling, and we knew we wanted to start to work together. The twins are the folks in the legacy Brain team (many from the PaLM/PaLM-2 effort) and the legacy DeepMind
Bard (the product) is now Gemini. Gemini Advanced is powered by the Gemini Ultra 1.0 model, our most advanced multimodal model. Congrats to everyone who worked on this effort, including the Bard and Gemini teams! Lots more updates in the blog post!
When I was finishing grad school, I was applying for both academic faculty positions and industrial research positions. Only got one academic interview (& offer), and not from a top- or even mid-tier place, so I went into industrial research. It's turned out okay..
Being denied tenure is a life-twisting thing, and there's no one best strategy for dealing with it. It's okay to recommit to your academic goals, or to try something completely different. Take the opportunity to have your mid-life crisis a little bit early.
The denial of visas for people to attend scientific conferences inhibits the free flow of ideas that is essential for scientific progress. I wish visas weren't such a hindrance, especially for those from underrepresented countries in the field of ML. We need everyone's voice!
This never ends. This year, so far, 15 out of 44 people to attend
@black_in_ai
workshop at
@NeurIPSConf
(which is still in Canada) have been denied visas. That's 33%. We had all this press last year, they were supposed to help us this year.
.
@geoffreyhinton
was in Mountain View this week, so we decided to hold a Turing award celebration for him at
@GoogleAI
. Deanna Chen obtained secret info from Mohammad Norouzi about Geoff's favorite dessert, and ordered a version to feed 100. Behold our Turing-misu celebration!
Congrats to my colleagues
@demishassabis
& John Jumper for winning the Nobel Prize in chemistry for their awesome AlphaFold work. AlphaFold has had a dramatic impact across many fields including medicine, drug development & more.
And yes, this is my second Nobel-congratulating
This year’s chemistry laureates Demis Hassabis and John Jumper have developed an AI model, AlphaFold2, to solve a 50-year-old problem: predicting proteins’ complex structures.
Check out two examples of protein structures determined using AlphaFold2. First up, a bacterial enzyme
Currently, the OPT program enables talented graduate students to work in the US for a period after they graduate. Eliminating this program will cause graduate students, with newly minted PhDs, to be forced to leave the US, hurting US competitiveness.
The launch of
@TensorFlow
~5 years ago this week was a pretty exciting milestone for a lot of us. We all gathered in a conference room early one morning to watch the original blog post announcing it go live & people start downloading it:
Five years ago, we open sourced
@TensorFlow
, our machine learning framework that's now the most popular machine learning library in the world. 🌎 To celebrate, we’re sharing few interactive demos and tutorials you can try, no experience required →
As in past years, I've spent part of the holiday break summarizing much of the work we've done in
@GoogleResearch
over the last year. On behalf of
@Google
's research community, I'm delighted to share this writeup (this year grouped into five themes).
Some new work from the part of Google Brain that works on ml for healthcare: we have encouraging early signs that non-invasive retinal images contain subtle indicators of cardiovascular health that ml models can pick up on, that weren't previously even known to human doctors.
On behalf of the entire Google Research &
@GoogleAI
communities, I'm excited to share an overview of some of our research in 2020.
Thanks to everyone who helped make this work possible!
This ICE action against students on F-1 visas is incredibly damaging in many ways. It hurts U.S. competitiveness, damages our world-class universities, and, if enacted, will disrupt the scholarly study of millions of the brightest students from around the world.
This is bad. ICE just told students here on student visas that if their school is going online-only this fall, the students must depart the United States and cannot remain through the fall semester.
An update:
@geoffreyhinton
has been in contact with Canadian immigration officials. They told him that anyone who has been denied a visa to attend
@NeurIPS
can request their case to be reconsidered via this form:
No guarantees, but please pass along!
The denial of visas for people to attend scientific conferences inhibits the free flow of ideas that is essential for scientific progress. I wish visas weren't such a hindrance, especially for those from underrepresented countries in the field of ML. We need everyone's voice!
I'm very excited to see the launch of Dataset Search, which can help researchers, scientists, and others around the world find open datasets!
To get a feel for it, try it with things like
… or
… or
…
Announcing the launch of Dataset Search, a new way for researchers to find the datasets they need, wherever they’re hosted, whether it’s a publisher's site, a digital library, or an author's personal web page. Learn more at
We have an experimental updated version of Gemini 1.5 Pro that is
#1
on the
@lmsysorg
Chatbot Arena. This model is a significant improvement over earlier versions of Gemini 1.5 Pro (it cracks into 1300+ elo score territory).
I'm really proud of the whole team of people that
Exciting News from Chatbot Arena!
@GoogleDeepMind
's new Gemini 1.5 Pro (Experimental 0801) has been tested in Arena for the past week, gathering over 12K community votes.
For the first time, Google Gemini has claimed the
#1
spot, surpassing GPT-4o/Claude-3.5 with an impressive
This is crazy. Study shoes three people in a conference room over 2 hours can result in a Co2 level that can impair cognitive functioning. Ie. If you’re making decisions at the end of the meeting, you’re mentally less qualified to do so.
On behalf of our co-authors Tomáš Mikolov,
@ilyasut
and Kai Chen,
@greg_corrado
and I were delighted to accept the
#NeurIPS2023
Test of Time Award for the "word2vec" paper (). Thanks to the
@NeurIPSConf
test of time committee for honoring us with this
Congratulations to Jeff Dean, Greg Corrado, & co-authors of the paper “Distributed Representations of Words and Phrases and their Compositionality”, for winning the
#NeurIPS2023
Test of Time Award! This prize recognizes a highly impactful paper published at NeurIPS 10 years ago.
The multimodal and reasoning capabilities of Gemini are quite strong. The benchmark results, which I’ll discuss in a moment are nice, but I’m most excited by demonstrations of what it can do.
Consider the image below. A teacher has drawn a physics problem of a skier going down
Bard, powered by the Gemini Pro-scale model, debuts at the
#2
position on the independent lmsys leaderboard. 🔥
Give it a try at . Bard is much better & has many more capabilities since its debut in March, thanks to everyone on the Bard/Gemini teams!
🔥Breaking News from Arena
Google's Bard has just made a stunning leap, surpassing GPT-4 to the SECOND SPOT on the leaderboard! Big congrats to
@Google
for the remarkable achievement!
The race is heating up like never before! Super excited to see what's next for Bard + Gemini
@ilyasut
@sama
@gdb
@miramurati
Ilya, best of luck in all that comes next for you. I was pleased to call you a colleague for many years, and I've admired everything you've accomplished at OpenAI. I always enjoy when our paths cross (even if just for coffee).
School "buses" like this would be awesome! Health benefits, a bit of excess energy burned off before arriving at school, chance to all work together on getting to school, great for the environment, ... Probably way cheaper than a fuel-burning school bus, too.
These Dutch “bicycle buses” are transporting 50+ kids getting the kind of exercise that leads to better learning, who DON’T need to be driven & dropped off by parents adding lots of local car traffic. Nijmegen, NL video via
@MartijnLi
HT
@urbanthoughts11
What a time to be alive: collecting whale mucus via UAV! No doubt the first stage in a complex pipeline for data analysis or machine learning. Anyone else have equally exotic data collection techniques in work they're doing?
What a great photo capturing that exhilirating moment when your hard work all comes together for the first time. Congrats to Katie Bouman and the whole team behind this work!
Today, the part of Google Brain that had been working on healthcare problems open sourced a set of protocol buffer definitions for the FHIR standard, which makes it much easier to manipulate data in FHIR format. We've been using this for our recent work on ML for medical records.
Wonderful to see my good friend & colleague
@geoffreyhinton
today at the retirement party we organized for him. He retired in May, but this was the first chance to have a proper party for him in the Bay Area.
To your amazing career & a happy retirement,
@geoffreyhinton
! 🥂
I am deeply saddened by the passing of my dear colleague and friend
@SusanWojcicki
. She has had a profound influence on everyone at Google and impacted the lives of so many. My heart goes out to her entire family and to all who knew her. 😥 💙
Unbelievably saddened by the loss of my dear friend
@SusanWojcicki
after two years of living with cancer. She is as core to the history of Google as anyone, and it’s hard to imagine the world without her. She was an incredible person, leader and friend who had a tremendous
I don't think that people appreciate how different the voice to text experience on a Pixel is from an iPhone. So here is a little head to head example. The Pixel is so responsive it feels like it is reading my mind!
🎉🇨🇦 Congratulations to the godfather of Deep Learning, AI pioneer and
@Google
Engineering Fellow Geoffrey Hinton on being named a Companion of the
#OrderofCanada
.
@GoogleAI
@UofT
"A photo of the back of a wombat wearing a backpack and holding a walking stick. It is next to a waterfall and is staring at a distant mountain."
#parti
Not that many systems handle 6B QPS. 😀
"Bigtable has been in continuous production use at Google for more than 15 years now, processing over 6 billion requests per second at peak and with over 10 exabytes of data under management. "
Very excited to see the MLPerf 0.7 results released today, where Google TPUs set records in six of the eight benchmarks!
We need bigger benchmarks, because we can now train the ResNet-50, BERT, Transformer, & SSD benchmarks each in under 30 seconds.
It's nice to have good names for things. I'm proud to have named or been involved in naming a bunch of things at Google over the years, including:
MapReduce
Bigtable
Spanner
TensorFlow
Tensor Processing Units (TPUs)
Pathways
Protocol Buffers
PaLM
Gemini
"I felt the twins aspect of the name ‘Gemini’ was a great fit. The twins here are the folks in the legacy Brain team and the legacy DeepMind team, who started to work together on this ambitious multimodal model project.” -
@JeffDean
Current U.S. policies that make it hard or impossible for some of the most talented people in the world to enter this country, or the proposed tax policies that will tax graduate students that make $28k per year as if they make $71k will definitely hurt the U.S.
A good friend, AI scientist, recently returned from China in awe: “they are toe to toe with us, but moving faster.” In Singapore I was told, “BTW the best thing to happen to China AI is your immigration policy...”
@karpathy
Congrats! AI is going to have a huge impact on education & I know it's something you're passionate about (seeing how you organized cs231n years ago when I gave a guest lecture it was clear you put lots of thought into the best way to teach people different kinds of material)
東京オフィスでAI研究に取組む仲間を募集します!Happy to see our
#GoogleAI
efforts expanding w/ Google Brain now having a research presence in Tokyo. We’re hiring machine learning researchers there, if you’re interested in helping advance AI, apply here —>
Today, Google Cloud is announcing the first of many AutoML products, AutoML Vision. It's the result of close collaboration between the Google Brain and Cloud AI teams. Automatically solve vision problems to high accuracy with no ML expertise required!
Incredibly cool work by my friend and compatriot
@PetarV_93
! TacticAI uses GNNs & Geometric deep learning to make tactical suggestions for the corner kicks and it turns out those suggestions (compared to real corners from previous games) are favored by football experts 90% of
Needle in a Haystack Tests Out to 10M Tokens
First, let’s take a quick glance at a needle-in-a-haystack test across many different modalities to exercise Gemini 1.5 Pro’s ability to retrieve information from its very long context. In these tests, green is good, and red is not
What I did over my winter break!
It gives me great pleasure to share this summary of some of our work
in 2019, on behalf of all my colleagues at
@GoogleAI
&
@GoogleHealth
.
Great article.
"Countries that are prepared will see a fatality rate of ~0.5% (South Korea) to 0.9% (rest of China).
Countries that are overwhelmed will have a fatality rate between ~3%-5%"
Taking decisive action early is important: it can reduce the fatality rate 10X.
Gemini and I also got a chance to watch the
@OpenAI
live announcement of gpt4o, using Project Astra! Congrats to the OpenAI team, super impressive work!
Gemini 1.5 Model Family: Technical Report updates now published
In the report we present the latest models of the Gemini family – Gemini 1.5 Pro and Gemini 1.5 Flash, two highly compute-efficient multimodal models capable of recalling and reasoning over fine-grained information
"I feel like Salieri. I understand the greatness. I don't understand how it's done."
Well said, Craig Silverstein! We all feel that way when looking at Sanjay Ghemawat's code.
(Aside: if you haven't seen the movie "Amadeus", you should!)
I strongly believe that the U.S. is at its best when it brings in talented scientists and engineers from all over the world to study, to be hired into companies to push forward our most advanced science and engineering efforts, and to start new U.S.-based companies. We should
If this were a science paper, you would expect a country that picks its science workforce at random as a “weak baseline” and a leading nation like the US to actively experiment towards state-of-the-art, or at least beat the baseline.
Not providing a guaranteed path for
Lots of excitement about the Gemini announcement, but
@GoogleCloud
also announced availability of the newest TPU system today, TPU v5p. These systems are quite a bit higher performance and much cost effective than earlier generations.
Compared to TPU v4, TPU v5p (see table
ML assisted coding is helping thousands of Google software engineers.
"We see a 6% reduction in coding iteration time (time between builds and tests) [vs. control]" &
"Currently, 3% of new code (measured in characters) is now generated from accepting ML completion suggestions."
Check out a new hybrid approach that leverages both ML and rule-based semantic engines to build a Transformer-based semantic code completion model, which we show can improve developer efficiency. Read more ↓
Our
@GoogleAI
flood forecasting system is now live in all of India & Bangladesh, and we are working to expand to countries in South Asia and South America. And eventually, we want it to be available everywhere.
A good article about the tragic death of my dear friend & colleague, Luiz André Barroso.
Luiz was incredible in so many ways. We collaborated on many things, made 20,000+ cappuccinos together & he was an amazing engineer, leader & friend. 😥
Happy
#InternationalWomensDay
!
I want to recognize all the female computer scientists & other researchers who are advancing our knowledge of the world. Thank you for everything you do!
To any young girls out there, please consider STEM fields in your future careers!
We have a poster child for excellent parenting and research!
Seriously, this is great to see, and I wish it was more common in academic conferences and professional settings.
When we released
@TensorFlow
as an open source project in Nov. 2015, we hoped external machine learning researchers & practicioners would find it as useful as we had internally at
@GoogleAI
. Very proud to see us hit 100M downloads!
2015 blog post: