David Luan
@jluan
Followers
9,873
Following
1,035
Media
22
Statuses
533
led Google’s large models effort, director @googleai . former vp engineering @openai . interested in ML + society. all about type II fun.
irl
Joined March 2009
Don't wanna be here?
Send us removal request.
Explore trending content on Musk Viewer
The U
• 631794 Tweets
#TGS2024
• 181335 Tweets
Miami
• 97891 Tweets
Washington
• 87428 Tweets
Knicks
• 79998 Tweets
Towns
• 71153 Tweets
Minnesota
• 68952 Tweets
León
• 64720 Tweets
Randle
• 60155 Tweets
Donte
• 38449 Tweets
Wolves
• 37973 Tweets
भगत सिंह
• 36891 Tweets
The ACC
• 29197 Tweets
#DAY6INBKK2024
• 28648 Tweets
Asharamji Bapu Case
• 26409 Tweets
Equality Exploited
• 24217 Tweets
Hart
• 22829 Tweets
日本破壊クソメガネ
• 19629 Tweets
Virginia Tech
• 18445 Tweets
Cam Ward
• 18291 Tweets
Canes
• 14294 Tweets
Rutgers
• 12511 Tweets
Celtics
• 11867 Tweets
A bunch of top ML folks from Google, DeepMind, OpenAI, etc have come together to build Adept! It’s a pleasure to be working with this kind and extremely talent-dense crew, incl. the folks who invented Transformer.
We’re doing something a bit different… (thread)
45
124
2K
This is tech’s “let them eat cake” moment: I only see talk of machine learning papers, WFH setups, and the escapist fantasy of space.
Wake up, everybody. AI doesn’t matter if we can’t even treat everyone in this country like a real human being.
20
306
2K
Excited to share the news that we’ve raised $350M to build a natural language interface to your computer!
Having a strong coalition of strategic partners for Adept (Atlassian, Microsoft, NVIDIA, and Workday) is going to be awesome!
We’re also excited to partner with Addition, Greylock, Atlassian Ventures, Microsoft, NVIDIA, Workday Ventures, Caterina Fake, Frontiers Capital, PSP Growth, SV Angel and A_Capital, and others, who supported the round.
More from
@Forbes
below:
(2/4)
2
9
71
21
32
379
People have invested a ton of time and expertise to create software tools that help them get their work done. Rather than replacing these tools, we want to build a natural language interface to all of them — an NL frontend to your computer.
5
13
210
This is also the reverse of most AGI work out there. Rather than automating economically valuable tasks, we want to keep humans in the driver’s seat, by building AI tools that people can work with to do things together.
4
8
200
It’s just been three months since we got started and we’ve already built a ton. If the idea of building a foundational general AI product – and using it to solve general intelligence – excites you, please reach out :)
If you’re interested in what we’re up to, please visit our jobs page at or email us at hello
@adept
.ai
11
9
115
8
7
184
I had a wonderful 1.25 years at Google Research! It was a real privilege to get to lead the large models effort there and work with folks like
@RandomlyWalking
,
@achowdhery
,
@elicollins
and
@JeffDean
on PaLM etc. After OpenAI and Google, I’ll be doing something totally different!
9
3
179
Finally out!
AI is as much about engineering as it is about research. PaLM required solving hard problems across all levels of the stack—networking, XLA, distributed training infra, optimizers, model architecture, data.
Our group’s model scaling effort did whatever it took.
1
22
175
Solving Rubik's Cube with a humanoid hand shows my favorite part of
@OpenAI
's research philosophy: choose a hard task that we don't think is doable with today's techniques, then use or invent whatever technique to solve it. This is the transpose of how research is often done.
We’re all used to robots that fail when their environment changes unpredictably. Our robotic system is adaptable enough to handle unexpected situations not seen during training, such as being prodded by a stuffed giraffe:
46
387
2K
4
15
141
In the future, we’ll be able to ask our computers to do increasingly abstract and complex things in natural language—and it’ll be the default way people use their machines.
Excited to share our first step in this direction! Some thoughts on why I think this is really cool:
1/7 We built a new model! It’s called Action Transformer (ACT-1) and we taught it to use a bunch of software tools. In this first video, the user simply types a high-level request and ACT-1 does the rest. Read on to see more examples ⬇️
136
919
5K
4
7
125
Give me an artist, genre, and lyrics (or not), and this neural network will generate you a song! It can even rap.
This is one of the coolest results from OpenAI’s Algorithms and Language team. Loved being involved in it as a bureaucrat.
Next up, 24/7 lo-fi vaporwave.
Introducing Jukebox, a neural net that generates music, including rudimentary singing, as raw audio in a variety of genres and artist styles. We're releasing a tool for everyone to explore the generated samples, as well as the model and code:
219
2K
8K
7
26
100
From now on, all of my mistakes will be rebranded as unintentional ablation studies. h/t
@machinaut
2
12
76
Making AI agents reliable has been a big challenge for the community, in part because they can’t see. GPT-V/Gemini are not yet generally available.
Adept’s releasing an open 8B multimodal model! Fast, good at regular photos, but also handles unstructured knowledge worker data.
It actually has an extremely simple architecture. Fuyu-8B doesn’t have an image encoder. This allows easy interleaving of text and images and handling arbitrary image resolutions! And it’s super fast for copilot use cases where latency really matters.
4
3
43
2
9
77
Congrats to the OpenAI, Microsoft, and GitHub teams on this pretty sweet set of results! Program synthesis is going to get really lit in the next few years, thanks in large part to scale.
Today,
@GitHub
,
@OpenAI
and
@Microsoft
launched a technical preview of GitHub Copilot. It’s a great example of how advancements in
#AI
are producing powerful new tools to help developers write better code - and spur more creativity and innovation.
8
225
716
0
8
61
Adept's Fuyu architecture scales really well! Fuyu-Heavy is the best model in its weight class and outperforms Gemini Pro :)
In particular, it's super good at being an AI agent on your computer...
Introducing Fuyu-Heavy, our new multimodal model. Fuyu-Heavy is the world’s third-most-capable multimodal model, behind only GPT4-V and Gemini Ultra, which are 10-20 times larger. In particular, it outperforms Gemini Pro at both MMLU and MMMU...
19
131
617
4
2
48
We’ve been training giant neural networks that do stuff for you on your computer!
In the first three months at Adept, we taught it to query databases, make visualizations, and fetch data from the web, but we want to teach it how to use every software tool in the world.
We made a fun video of some of the earliest things our system can do! If you want to help us build useful general intelligence, please reach out -- we are hiring.
30
105
712
4
4
49
The improvement in the quality of images sampled from generative models over the last few years has been astounding.
See in particular 5:35. This GAN has learned 3d structure of cars and bedrooms through only 2d images, and interpolates smoothly!
1
13
38
My lovely motorcycle (and main commuting tool) was stolen in front of my house at 6:30AM today. It’s a 2018 KTM 500 EXC with crazy graphics that the previous owner put on. SF friends, keep an eye out for me! Living in this city has not been a good time. Nest shots attached.
6
27
39
Excited that Overcooked-like environments are being used to study cooperative multiagent RL!
I’m happy that the growth of indie gaming has led to new creative gameplay mechanics, which makes our jobs as researchers easier in that useful environments sometimes come to us :)
Excited to share our work: collaboration requires understanding! In Overcooked, self-play doesn't gel with humans: it expects them to play like itself. (1/4)
Demo:
Blog:
Paper:
Code:
4
119
367
1
2
33
I know this headline "OpenAI Wants to Move Slow and Not Break Anything" is written tongue-in-cheek, but I could not agree more.
ML research has real downstream consequences. The Silicon Valley cowboy mentality is irresponsible for ML.
0
6
31
Turns out training high-capacity language models on chunks of the internet produces a flexible, general tool for language understanding tasks! I was surprised by how many tasks it had learned to do with no task-specific data, setting SOTAs on some and solid performance on others.
We've trained an unsupervised language model that can generate coherent paragraphs and perform rudimentary reading comprehension, machine translation, question answering, and summarization — all without task-specific training:
173
2K
6K
2
4
27
Returning from a long hiatus to say -- this was an exceptionally fun podcast thanks to
@HarryStebbings
!
Give it a watch for a collection of hot takes on path to AGI, limitations, hardware-model vertical integration, and the crucial role that interaction design now plays...
3. Why Every Cloud Provider Must Have a Model Play
As models become smarter, they’ll become the base computing primitive.
The logic of software will be handled by LLMs in the future.
Whoever controls the model layer controls all of the underlying compute.
1
1
6
4
0
29
Belated commentary:
AI research is extremely expensive, and this partnership provides a level of stability where OpenAI can comfortably take an even longer-term view with the research problems we pursue, and the diff ways our policy work engages with society.
Microsoft is investing $1 billion in OpenAI, the research lab overseen by startup guru Sam Altman that says (with all seriousness) that it wants to build "artificial general intelligence, or AGI, a machine that can do anything the human brain can do:
98
462
1K
1
4
26
Congrats to
@GreylockVC
on their new early-stage fund! I've been grateful for their support since day 0 for Adept... they've done a bunch for us, including helping us hire some of our strongest folks on the team.
1/ With a long history of partnering w/ founders from idea to IPO, we
@GreylockVC
are excited to announce 2 new updates to advance this mission:
1/ Fund 17, our new $1B early-stage fund
2/ Edge, a bespoke program to help founders initiate new companies
39
36
208
0
6
24
Finally, it’s still the early days, but we think the future of these interfaces will be less like an assistant (you tell the model to do stuff), and more like a collaborator (you and the model work together to solve a problem).
A true bicycle for the mind!
2
1
22
Fuyu mafia! Glad to see our models make high res image understanding easier :)
OtterHD: A High-Resolution Multi-modality Model
Presents OtterHD-8B, an innovative multimodal model evolved from Fuyu-8B, specifically engineered to interpret high-resolution visual inputs with granular precision
2
44
281
0
0
15
Help us flex on giant language models by creating tasks that show their limitations!
CALL FOR TASKS CAPTURING LIMITATIONS OF LARGE LANGUAGE MODELS
We are soliciting contributions of tasks to a *collaborative* benchmark designed to measure and extrapolate the capabilities and limitations of large language models. Submit tasks at
#BIGbench
14
73
279
0
1
20
Pretty amazing improvements to making longer sequence models more efficient to train—have loved collaborating with
@tri_dao
!
I’ve been working with
@AdeptAILabs
and we’ve made FlashAttention even faster for long sequences! For seqlen 8K, FlashAttention is now up to 2.7x faster than a standard PyTorch implementation even at small batch, making it easier to train better LMs with longer context 1/7
7
86
605
0
1
18
ML is increasingly as much about new ideas as it is about big engineering investments. Standardizing frameworks lets us share a lot more across teams. Excited to be leading this transition with our head of infra, Chris Berner! Keep an eye out for blocksparse wrappers for PyTorch.
We're standardizing OpenAI's deep learning framework on PyTorch to increase our research productivity at scale on GPUs (and have just released a PyTorch version of Spinning Up in Deep RL):
34
541
2K
0
2
19
bleep bloop!
We’re hiring our first designer at Adept. We’re building a teammate anyone can work with to get stuff done in front of a computer.
If you’re interested in defining how people and increasingly capable AI systems interact, we’d love to chat!
We’re hiring Adept’s first designer! If you want to shape the new era of human/machine interaction, apply at . Bonus points if you have experience with interactive and/or multimodal ML products.
3
6
37
1
3
19
Unfortunately I won't be directly working on AI policy at the moment but am still looking forward to staying engaged.
2
0
18
high school essay writing will never be the same!
0
2
16
It’s irresponsible for Amazon to push their facial recognition product onto law enforcement, especially with how much higher the error rates are for POC and women. The price of mistakes is paid not by Amazon, but by minorities and the over-policed.
New
@MIT
study shows gender and racial bias in
@amazon
Rekognition AI product -100% accuracy on pale males vs 69% accuracy on women of color
- Study link
@ACLU
@Data4BlackLives
@black_in_ai
@medialab
@AIESConf
@AINowInstitute
@AOC
10
322
417
0
3
17
Interesting analysis on some of the upcoming research challenges we’ll have to solve at Adept together with the research community! There’s so much still to do.
This is the dream: having a system whose action space is universal (at least in the world of bits). And with foundation models, it is actually possible now to produce sane predictions in that huge action space. Some interesting challenges:
2
16
147
0
2
16
@Miles_Brundage
Hmm. Why is the eagle meddling with the connections?
Maybe, to be charitable, it's been assigned to do dropout?
3
0
15
Every enterprise service I use regularly now has a logo indistinguishable from a Google Cloud product. It is so confusing.
Now, in a fancy SoHo store, tshirts that could pass as swag for a Google Cloud product. Have high fashion and high tech converged at last?!
1
0
15
Welcome, Ken! Open-endedness and a diversity of behaviors are critical and so far still understudied in ML.
I'm thrilled to announce that I will be joining the superb team at
@OpenAI
in June, where I will be starting a group (and indeed hiring) focused on achieving open-endedness in machine learning. Looking forward to exploring a novel path!
43
36
746
1
0
13
Jeff's vision for the emergent complexity that comes from generating environments is incredible, and in my opinion, a necessary piece towards advancing intelligence. I'm so happy to welcome him to the team!
I am extremely excited to announce (1) I've joined OpenAI to lead a large-scale effort into AI-generating Algorithms research, & (2) I'll be an Associate CS Professor at U. British Columbia in 2021, where I will continue to lead the OpenAI project. Both are dreams come true! 1/2
78
110
2K
0
1
14
@hunterwalk
@levie
It’s what we’re up to at Adept:
1/7 We built a new model! It’s called Action Transformer (ACT-1) and we taught it to use a bunch of software tools. In this first video, the user simply types a high-level request and ACT-1 does the rest. Read on to see more examples ⬇️
136
919
5K
0
0
14
Just give me MuseNet-generated vaporwave when I type, and my life is complete.
I just need you all to know how much my VS Code theme slaps. I *finally* got the glow working 😍😍😍
Theme: Synthwave x Fluoromachine
Font: Fira Code
308
1K
8K
0
2
13
@adamdangelo
@sama
Anecdotally seems to skew more toward folks earlier on in their careers—is that what you two are seeing as well?
3
0
12
Neat survey from
@CadeMetz
of the next frontier of progress toward useful AGI—agents that can do more than just talk, but actually use software to do stuff on your computer.
Featuring some of Adept’s early work and some great directions from my former colleague
@jeffclune
!
Today our research is on the homepage of the New York Times, covering VPT and implications for AI like it (eg
@jluan
's
@AdeptAILabs
), & great work by
@DrJimFan
&
@AnimaAnandkumar
).As a fun bonus, they also included my children's doodles! Thanks
@CadeMetz
!
0
3
28
0
2
12
I'm very hopeful that we can add Activation Atlases to the arsenal of tools people use to assess for bias and spurious correlations in machine learning.
For the last few years, one of the staples of my research has been visualizing individual neurons in vision models. But that's only a partial picture -- neurons work together.
Activation Atlases are a way to explore the space neurons jointly represent.
4
172
505
0
3
13
OK, typography friends: in the right image, does Google Imagen seem to prefer drawing Helvetica or Arial?
19
0
12
As I was working on making the partnership happen, the thing that most struck me was the degree to which MSFT leadership cared about OpenAI's independence and the importance of our nonprofit board.
0
1
12
;)
The most important user interface of the next decade
52
168
2K
2
0
11
Untitled Goose Game as a benchmark for exploration in RL, to replace Montezuma ;)
2
0
11
Side note, the pace of progress in AI is astounding. First text generation, then image generation, now computer use :)
1
0
10
Can we please make this the next great meme format?
2
0
11
Go team!
Really surprised and happy to receive an Honorable Mention for Outstanding Paper on "Generative Pretraining from Pixels" ()! Thanks so much to the ICML awards committee!
4
22
219
0
0
10
Danielle’s stuff is so brilliant that I am often left speechless. Like today.
1
0
9
Folks at OpenAI gathered around to try basic facts, like "Q: what's the tallest mountain on Earth?" which it answered correctly. And it would produce surprisingly strong samples, like an essay on recycling.
And then we tried English to French translation on a whim. 😮
1
0
9
ACT-1 is also surprisingly sample efficient when it comes to human feedback. A single demonstration or piece of documentation can be all that’s needed to do something new.
We think human feedback is by far the best path to improve capabilities and alignment.
6/7 ACT-1 doesn’t know how to do everything, but it’s highly coachable. With 1 piece of human feedback, it can correct mistakes, becoming more useful with each interaction.
5
19
240
1
0
9
When I was a kid, it was boring days of program induction on times tables. This looks more fun. cc
@kevinakwok
@eugenewei
My friend has been making
#MachineLearning
books for *5 year olds* and they are freaking adorable - - - and actually educational. Asian parents would go crazy with these books
#rocketbabyclub
5
26
71
0
0
9
@dennybritz
We do at OpenAI--we have toy tasks that are meant to encourage advances in planning, etc. At the same time, we also work on solving complex environments that require the composition of problems to make sure we're not overfitting on the toy ones.
1
0
9
ACT-1 maps natural language to actions. All of us already know that natural language input is extremely flexible—we’ve seen it with language models. The surprise is that our model, ACT-1, is similarly flexible with its outputs, aka what software tools it can use.
3/7 Working in-depth in tools like spreadsheets, ACT-1 demonstrates real-world knowledge, infers what we mean from context, and can help us do things we may not even know how to do.
6
32
397
2
0
8
Amazing progress using clever ideas that are also simple to explain.
Want to improve accuracy and robustness of your model? Use unlabeled data!
Our new work uses self-training on unlabeled data to achieve 87.4% top-1 on ImageNet, 1% better than SOTA. Huge gains are seen on harder benchmarks (ImageNet-A, C and P).
Link:
20
426
1K
0
1
8
I’m pretty sure turnip prices in Animal Crossing are pegged to the S&P 500.
0
0
8
@HarryStebbings
Shoutout to
@HarryStebbings
who unbeknownst to me logged onto this podcast filming having just had his wisdom teeth removed. Legend!
1
0
8
nathan is amazing—thankful to be able to work together!
👋 I'm excited to unveil
@airstreet
’s second fund of $121,212,121 as we accelerate our mission to back ambitious AI-first companies in North America and Europe!
🧵 My reflections on the journey, opportunity and what this means for our founders and community:
90
55
702
2
0
4
@mer__edith
at yale, there were nice dorms and financial aid dorms. financial aid dorms used to not have dining halls b/c their students worked as servants in the halls of the rich kid dorms. most full ride kids incl me were drafted to the financial aid dorms bc we don’t benefit from legacy.
2
0
7
On top of the great research progress, these incredible photorealistic samples point to the need for society to quickly adapt to fake imagery and videos.
0
0
7
SDCs are insidious and they’ve plagued almost every giant model training run I’ve been familiar with… here’s an expose!
If your loss curves look sus, join the club! Giant LLM training runs are full of pitfalls. We learned the hard way. We wrote a deep dive for the community on silent data corruptions (SDCs).
Problem and mitigations here:
1
26
108
1
0
4
For a long time, things didn't work. But the team pushed through, and along the way we came up with ADR and made sim2real work zero-shot!
0
0
6
^^ if the video above excites you, please work with us!
or write back if you have things you think our model should do in the future :)
1
0
6
It was the first time I'd seen a model resemble a competent generalist. As initial results rolled in, we started trying things we thought there would be no way it would have learned zero-shot: article summarization, question answering. The model gave reasonable answers!
1
0
6
@anniefryman
it was
@kane
, the only person i know who has both a gas mask and an inexplicable love of fernet
1
0
5
@nathanbenaich
@ClementDelangue
@AdeptAILabs
@airstreet
@NEARProtocol
@CohereAI
@GoogleAI
@OpenAI
@ashVaswani
@huggingface
@karpathy
@nikiparmar09
@lukaszkaiser
@stateofaireport
the best researchers are calculated risk-takers in all parts of life :)
1
0
5
Fun fact: GPT-2 was almost named after a muppet.
OpenAI doubled down on its no-muppet strategy with a follow up in February, GPT-2. It generates impressively fluid text. (See: ). China's Tsinghua University chose to sustain the trend, with Enhanced Language Representation with Informative Entities: ERNIE.
1
0
7
1
1
5
@mhdempsey
@boop
@dipalua_
@ghosttyped
@kevinakwok
michael! thank you for volunteering to be LA Kevin! there will be an amazon box waiting for you that you can use as furniture in this new LA house
1
0
4
