Ruiqi Gao
@RuiqiGao
Followers
5,995
Following
580
Media
22
Statuses
251
Research scientist @GoogleDeepmind . Generative modeling, representation learning.
San Francisco
Joined June 2019
Don't wanna be here?
Send us removal request.
Explore trending content on Musk Viewer
Beyoncé
• 279770 Tweets
Flamengo
• 275495 Tweets
sabrina
• 182605 Tweets
Camper Right Nina Pop Up Event
• 170924 Tweets
#CamperXBecky
• 169522 Tweets
京都国際
• 160566 Tweets
Cruzeiro
• 126695 Tweets
7DREAM ENGLISH SINGLE OUT NOW
• 91265 Tweets
Sian
• 82440 Tweets
#ShortNSweet
• 81205 Tweets
renjun
• 79354 Tweets
Rossi
• 72357 Tweets
Yachty
• 58510 Tweets
甲子園決勝
• 56813 Tweets
علي النبي
• 52039 Tweets
関東第一
• 51432 Tweets
Sabine
• 44037 Tweets
ラストマイル
• 41066 Tweets
PIDOM
• 37829 Tweets
タイブレーク
• 35231 Tweets
ハングル校歌
• 32785 Tweets
Hayırlı Cumalar
• 29560 Tweets
おじさんの詰め合わせ
• 23190 Tweets
チー付与
• 21781 Tweets
#يوم_Iلجمعه
• 19052 Tweets
Bed Chem
• 14157 Tweets
#NationalSpaceDay
• 13448 Tweets
Good Graces
• 13436 Tweets
T'Kor
• 11195 Tweets
セカンドライフ
• 10370 Tweets
राष्ट्रीय अंतरिक्ष
• 10271 Tweets
Pinned Tweet
🌟 Create anything in 3D! 🌟 Introducing CAT3D: a new method that generates high-fidelity 3D scenes from any number of real or generated images in one minute, powered by multi-view diffusion models.
w/ lovely coauthors
@holynski_
,
@poolio
and an amazing team!
24
64
407
🥳Thrilled to share Imagen Video: our new text-to-video diffusion model generating 1280x768 24fps HD videos!
#ImagenVideo
Website:
Excited to announce Imagen Video, our new text-conditioned video diffusion model that generates 1280x768 24fps HD videos!
#ImagenVideo
Work w/
@wchan212
@Chitwan_Saharia
@jaywhang_
@RuiqiGao
@agritsenko
@dpkingma
@poolio
@mo_norouzi
@fleet_dj
@TimSalimans
58
734
3K
10
66
402
Impressed by the amazing progress of diffusion models?
📢 Check our
#CVPR2022
tutorial on
#diffusion_models
next Sunday morning! w/
@karsten_kreis
&
@ArashVahdat
Agenda:
An
#Imagen
(awesome diffusion models) preview of what to expect 😉:
6
66
374
🔥Latent Diffusion Models Tutorial🔥
#NeurIPS2023
@karsten_kreis
,
@ArashVahdat
and I will present the tutorial on latent diffusion models tomorrow! Save the location & time, and excited to see you all there!
Tomorrow, Monday, 9:45am-12:15pm, Hall E2.
3
57
344
Looking for diffusion model advancements at
#NeurIPS2023
? Come to check our oral work "Understanding Diffusion Objectives as the ELBO with Simple Data Augmentation" w/
@dpkingma
.
New theoretical understanding, SOTA empirical results, and more!
Arxiv:
6
30
311
The preliminary slides of our
#CVPR2022
tutorial on
#diffusion_models
are available on our website:
Looking forward to seeing you, w/
@karsten_kreis
&
@ArashVahdat
, at 8:30 CDT on Sunday in New Orleans or virtually!
3
60
265
Pleased to share our new work on learning energy-based models:
By maximizing recovery likelihoods on increasingly noisy data, the MCMC becomes more tractable. We achieve (1)high quality samples (2)stable long-run chains (3)estimated likelihoods. (1/n)
2
43
230
🥳Check the re-recording of our tutorial on denoising diffusion models on YouTube:
The tutorial was originally presented at
#CVPR2022
with
@karsten_kreis
@ArashVahdat
. Slides and additional info:
Hope you enjoy the learning!!😃
1
35
220
📢 Booking your NeurIPS trip? Wondering the advancement of latent diffusion models, and how they have been applied to various domains?
@karsten_kreis
,
@ArashVahdat
and I will present the
#NeurIPS2023
tutorial on "Latent Diffusion Models" on Mon, Dec 11.
4
27
197
Excited to see people coming for our
#NeurIPS23
tutorial on “latent diffusion models” at Hall E2. This happening. Hope you’d enjoy it! ☺️
4
10
169
📢 Check our new work (w/
@dpkingma
) on theoretical understanding of weighted diffusion objectives!
See details in this thread 👇
New theoretical work on diffusion objectives:
We e.g. show that under a simple condition (monotonic weighting, satisfied by e.g. the v-prediction loss), diffusion objectives equal the ELBO with data augmentation, namely additive noise.
1/2
3
105
636
3
18
151
(1/2) Interested in understanding towards grid cells? Check our preprint "A Representational Model of Grid Cells Based on Matrix Lie Algebras" (with
@jianwen_xie
, SC Zhu, YN Wu), with learned regular hexagon grid patterns!
6
32
147
Check out our work "Flow Contrastive Estimation of Energy-Based Models" at
#CVPR2020
(with
@erik_nijkamp
,
@dpkingma
, Z Xu,
@iamandrewdai
, YN Wu): , a scalable method for learning EBM, w/o MCMC.
Oral session: 2.3B, 2-4pm (PDT) Wed.
Glad to get in touch!
1
17
125
Excited to get this tutorial proposal accepted to
#CVPR2022
. Stay tuned! 😄
🎉 Our tutorial proposal on 𝗗𝗲𝗻𝗼𝗶𝘀𝗶𝗻𝗴 𝗗𝗶𝗳𝗳𝘂𝘀𝗶𝗼𝗻-𝗯𝗮𝘀𝗲𝗱 𝗚𝗲𝗻𝗲𝗿𝗮𝘁𝗶𝘃𝗲 𝗠𝗼𝗱𝗲𝗹𝗶𝗻𝗴 has been accepted to
#CVPR2022
. Stay tuned for some cool lectures from
@karsten_kreis
,
@RuiqiGao
and me at
@CVPR
2022.
3
73
510
2
12
109
Spending 1/4 of my birthday flying towards Honolulu. Looking forward to catching up with people! 😊
12
2
106
🐈 Here is my favorite 3D asset from CAT3D. Her name is Mochi. You can find her by searching for her name in our gallery 😊.
🌟 Create anything in 3D! 🌟 Introducing CAT3D: a new method that generates high-fidelity 3D scenes from any number of real or generated images in one minute, powered by multi-view diffusion models.
w/ lovely coauthors
@holynski_
,
@poolio
and an amazing team!
24
64
407
4
8
97
More interesting 3D scenes generated by CAT3D 🐈
Single image input, generated in 1 min, rendered in real time.
Check out gallery for more samples 😃:
🌟 Create anything in 3D! 🌟 Introducing CAT3D: a new method that generates high-fidelity 3D scenes from any number of real or generated images in one minute, powered by multi-view diffusion models.
w/ lovely coauthors
@holynski_
,
@poolio
and an amazing team!
24
64
407
2
6
93
Finally this work has been accepted to
#ICLR2022
. MCMC could mix, as long as it is in the latent space :).
Check our work "Learning Energy-based Model with Flow-based Backbone by Neural Transport MCMC"(w/
@erik_nijkamp
, etc.): .
We learn EBM by exp. tilting of flow, with mixing MCMC chains for 2000+ steps! Very excited that long run MCMC finally works for EBM.
0
3
30
1
3
76
Super excited to be at
#NeurIPS2022
from Wed to Sat this week! Looking forward to catching up with people. DM me if you're around and I'd love to chat! 😃
1
0
50
Come to check our award candidate paper at
#CVPR2023
about distillation of guided diffusion models in the afternoon poster session tomorrow (poster
#186
)! Excited to chat about how to make sampling of diffusion models faster 🥳
1
6
51
It has been a while but if you haven't checked it out, could be a fun reading if you're excited about flow matching by SD3 and want to understand more about the connection between that and diffusion models 😃
Happy to see that our work on diffusion model objectives () is starting to get noticed, with e.g. Stable Diffusion 3 () building on our result that the flow matching objective can be simply understood as a special case of diffusion.
6
43
371
0
1
47
If you missed Durk’s talk at SPIGM
@ICML2023
, check the recording, slides, and the latest version of our paper under this thread! 😃👇
Thanks to the SPIGM organizers and those who attended!
If you're an ICML registree but missed my talk, you can find it at 2:21:41 under this link:
Our updated paper (including new results) will appear on arXiv on Aug 1. I'll also post the slides then.
1
12
104
0
7
47
Our work "Flow Contrastive Estimation of Energy-Based Models" with
@erik_nijkamp
,
@dpkingma
, Z Xu,
@iamandrewdai
, YN Wu at
#NeurIPS2019
: . Glad to get in touch!
Our work "Flow Contrastive Estimation of Energy-Based Models" (
@RuiqiGao
,
@erik_nijkamp
,
@dpkingma
, Z. Xu
@iamandrewdai
, YN Wu) at
#NeurIPS2019
:
We show (1) joint learning of EBM & Glow, (2) correction of over-dispersion, (3) competitive semi-super cls.
1
17
80
2
4
40
⏰Check our poster later today at
#ICLR2021
! A tractable MCMC based learning method of energy-based models.
time: 5-7pm PDT
link:
paper:
Pleased to share our new work on learning energy-based models:
By maximizing recovery likelihoods on increasingly noisy data, the MCMC becomes more tractable. We achieve (1)high quality samples (2)stable long-run chains (3)estimated likelihoods. (1/n)
2
43
230
2
6
36
Thanks for featuring our work!
EM Distillation for One-step Diffusion Models
While diffusion models can learn complex distributions, sampling requires a computationally expensive iterative process. Existing distillation methods enable efficient sampling, but have notable limitations, such as
2
23
105
0
0
34
It's still quite challenging to maintain the exact content for few step distillation of diffusion models. Very neat work to make this happen, and more (advanced deterministic sampling techniques etc.). Congrats to the team! 🥳
Fast sampling with 'Multistep Consistency Models': We get 1.6 FID on Imagenet64 in 4 steps and scale text-to-image models, generating 256x256 images with 16 steps.
Guess which row is distilled?
With
@emiel_hoogeboom
@TimSalimans
Arxiv:
2
19
86
0
3
29
Looking forward to seeing you there and discuss diffusion models!
Join us on Friday at Hall B1 for the
#NeurIPS2023
workshop on diffusion models!
We are honored to host a panel of experts
@ArashVahdat
@RuiqiGao
@robrombach
@sedielem
Have questions for the panel? Use this link to submit:
0
4
35
1
2
31
Check our work "Learning Energy-based Model with Flow-based Backbone by Neural Transport MCMC"(w/
@erik_nijkamp
, etc.): .
We learn EBM by exp. tilting of flow, with mixing MCMC chains for 2000+ steps! Very excited that long run MCMC finally works for EBM.
(1/2) "Learning Energy-based Model with Flow-based Backbone by Neural Transport MCMC" (
@erik_nijkamp
*,
@RuiqiGao
*, P. Sountsov, S. Vasudevan,
@bo_pang0
, S.-C. Zhu, Y. N. Wu):
We show (1) exponential tilting of Glow, (2) mixing MCMC, (3) improved synthesis.
6
12
82
0
3
30
Website:
Paper:
Also, check out our gallery for hundreds of diverse generated 3D scenes!
1
7
23
In our new work led by brilliant
@chenlin_meng
, guided diffusion models can be distilled to very few sampling steps with no loss on perceptual quality! It is also adopted by
#ImagenVideo
and works pretty well. Check the details in Chenlin’s thread 👇.
Excited to share our work "On distillation of guided diffusion models"! Our distillation approach allows classifier-free guided diffusion models to generate high-quality samples using as few as 1-4 sampling steps😮
10
89
582
0
4
20
Come to check Aleks’ talk about CAT3D tmrw at 2:30pm at CVPR!
I'll be presenting CAT3D tomorrow at CVPR. Come say hi!
Monday 2:30pm at AI for 3D Generation (Summit Flex A)
3
12
115
0
0
17
📢 Check our latest work ReconFusion, 3D reconstruction of realistic scenes with 2D diffusion priors!
Excited to share ReconFusion! 3D reconstruction of real-world scenes from only a few photos, powered by diffusion priors:
w/ amazing team
@ChrisWu6080
@BenMildenhall
@philipphenzler
@KeunhongP
@RuiqiGao
@watson_nn
@_pratul_
@dorverbin
@jon_barron
@poolio
9
60
355
0
1
15
Come to check Chenlin’s presentation on our paper “On distillation of Guided Diffusion Models” at
#NeurIPS22
workshop on score-based models this afternoon! 👇
Interested in making diffusion models fast? I’ll present our paper On Distillation of Guided Diffusion Models and other interesting works at
#NeurIPS
workshop on score-based methods this afternoon.
📍/🕜: room 293-294 / 2:30pm - 2:40pm and 3:00pm-4:30pm
2
3
51
0
1
15
CAT3D uses a multi-view diffusion model to synthesize a large set of consistent novel views from the input image(s). These views are then used to reconstruct a detailed 3D scene using robust reconstruction methods.
1
1
12
We are looking for reviewers for the 2nd ICML SPIGM workshop (). Welcome to register here! 👇
0
3
12
Exciting!
We just finished a joint code release for CamP () and Zip-NeRF (). As far as I know, this code is SOTA in terms of image quality (but not speed) among all the radiance field techniques out there. Have fun!
24
156
947
0
0
12
Big shout out to the incredible team:
@holynski_
@philipphenzler
@ArthurBrussee
@rmbrualla
@_pratul_
@jon_barron
@poolio
1
0
11
🥳🥳
We’re excited to announce 𝗚𝗲𝗺𝗶𝗻𝗶:
@Google
’s largest and most capable AI model.
Built to be natively multimodal, it can understand and operate across text, code, audio, image and video - and achieves state-of-the-art performance across many tasks. 🧵
171
2K
6K
0
0
11
@poolio
’s talk is happening in 12mins at Room 214!
Drop by the workshop today at
#NeurIPS2023
to learn about the intersection of deep learning and inverse problems! I'll be presenting our work on diffusion priors for 3D generation and reconstruction at 11:30.
2
4
58
1
0
11
Deepest gratitude to my amazing advisors Song-Chun Zhu and Ying Nian Wu! This would not have been possible without your guidance and support. Thanks to all my colleagues and collaborators!
0
0
10
CAT3D can create entire 3D scenes in as little as one minute! ⚡ Plus, it achieves state-of-the-art performance for few-view 3D reconstruction.
1
0
10
A joint work with my amazing collaborators
@YSongStanford
@poolio
Ying Nian Wu and
@dpkingma
! We will release the code soon. (8/n)
0
0
10
Key to our method is that we maximize the recovery likelihood: conditional probability of data at a noise level given their versions at a higher noise level. Compared to marginal likelihood, the MCMC is more tractable for the relatively concentrated conditional distribution.(2/n)
1
0
9
Our
#NeurIPS2021
paper "On Path Integration of Grid Cells: Group Representation and Isotropic Scaling"
☑️Theoretical analysis of general RNN models
☑️Algebraic & geometric structure of linear models
☑️Clear learned hexagon patterns
Session today 4:30PT:
1
0
8
For practitioners, based on this understanding, we propose new weighting functions, that lead to SOTA FID scores on high-res image generation. Ready for use on your own diffusion models with a minimal change of code and get performance boost!
0
2
9
We are also sincerely thankful to
@shlomifruchter
,
@sirbayes
,
@babaeizadeh
,
@Han_Zhang_
and Amir Hertz for training the base text-to-image model that supports our work!
0
1
8
Very interesting and solid work!
New theory paper: . We show rigorously that EBMs of the type E(x|y) = f(x)•g(y) are, under fairly mild conditions, (1) identifiable in functions f and g, and (2) universal conditional density approximators, generalizing previous forms of nonlinear ICA.
4
32
254
0
1
8
Just to clarify, the student researcher position is targeted at current PhD students and it’s short-term (very similar to research intern), not for full-time job after graduation.
0
0
8
Highly recommend to check this thread for more insightful thoughts about ReconFusion!
ReconFusion = standard single-scene optimized 3D reconstruction, additionally guided by a multi-view diffusion prior to allow for decent outputs from significantly fewer input views.
some thoughts below for view synthesis fanatics...
1/
6
49
419
0
0
7
This looks impressive! 🔥
Today, we’re excited to release the first step in our mission to build real time multimodal intelligence for every device: Sonic, a blazing fast (🚀 135ms model latency), lifelike generative voice model and API.
Read and try Sonic
43
164
819
0
0
7
For theory enthusiasts, we show that existing diffusion training objectives can be understood as the ELBO with data augmentation, under mild assumptions. It allows for apple-to-apple comparison of DMs with other likelihood-based models, e.g., autoregressive models.
1
2
7
Although learning from conditional distributions, this estimation method is theoretically consistent in terms of estimating marginal distributions at each noise level, given sufficient data. This is also verified on 2D examples. (4/n)
1
0
7
We learn a sequence of EBMs trained on increasingly noisy versions of a dataset. After training, samples are generated by progressively sampling from the conditional distributions at decreasingly lower noise levels. (3/n)
1
0
6
We make connections of our method to denoising score matching (Song & Ermon, 2019, 2020) and denoising diffusion probabilistic models (Ho et al. 2020). They correspond to the normal approximation of the recovery likelihood. (7/n)
1
0
6
@BAristimunha
@Warvito
@karsten_kreis
@ArashVahdat
Yes it will be recorded! It will also be livestreamed via zoom that is available to all cvpr attendees.
2
0
5
Moreover, the long-run MCMC samples do not diverge and still represent realistic images, allowing us to accurately estimate the normalized density of data. Our model obtains 3.18 bits/dim, the first competitive estimated log-likelihood using EBMs. (6/n)
1
0
5
@zhoubolei
I tested negative for the first three days after coming back and with symptoms, but now positive 🤒️. Hope it’s not the case for you 🙏
0
0
4
1
0
4
@erik_nijkamp
@dpkingma
@iamandrewdai
Welcome to our poster & spotlight sessions at BDL workshop on Friday at
#NeurIPS2019
:
0
0
3
On CIFAR-10, our methods achieve FID 9.60, outperforming existing methods of learning pure energy-based models by a large margin. (5/n)
1
0
4
0
0
3
Amazing work by colleagues at Brain!
Diffusion models are becoming incredibly powerful.
We are thrilled to announce Imagen, a text-to-image model with unprecedented photorealism and deep language understanding. Explore and Imagen!
A large rusted ship stuck in a frozen lake. Snowy mountains and beautiful sunset in the background.
#imagen
60
312
2K
0
0
3
It also includes a panel discussion with an awesome group of panelists:
@dpkingma
,
@talidekel
,
@robrombach
,
@chenlin_meng
, and Ying Nian Wu.
Looking forward to seeing you there in New Orleans! 🥳🎷
1
0
3
Interesting work that combines VAE with neuroscience and improves identifiability.
NeurIPS time: Check our new paper on exacting identifiable neural manifolds by leveraging both neural and behavioral data, “Learning identifiable and interpretable latent models of high-dimensional neural activity using pi-VAE”, with
@DingZhou2
.
3
3
16
0
0
3
(2/2) We show (1) a representational model of grid cells with two coupled matrix Lie algebras, (2) connection of path integral and dim reduction models via group rep. theory , (3) learned hexagon patterns & faithful path integral.
1
0
2
@XPe79361163
@DocXavi
@ArashVahdat
@karsten_kreis
Hi Xiong, thanks for your interest in our tutorial! We will post the video on our website soon. Stay tuned!
0
0
2
2
0
1
@arjunreddy2613
@schangpi
@GoogleAI
Congrats Arjun! Very well done. Haven’t received that hat this year...
0
0
2
Don't miss this great opportunity to work with brilliant
@wgrathwohl
!
Very excited to be hosting a student researcher on my team at Google Deepmind next year.
I'm looking to work with a PhD student who has a background in generative models and sampling.
If you think that describes you, please apply here!
1
7
78
0
0
2
✨ EMD achieves this through a novel MLE-based approach inspired by Expectation-Maximization.
✨ Bonus content: a reparametrized sampling scheme + a noise cancellation technique, stabilizing distillation.
✨ New understanding on existing approaches like DMD & Diff-instruct
1
0
2
0
0
2
☑️accurate path integration
paper:
code:
with wonderful collaborators
@jianwen_xie
@weixx2
S.C. Zhu, Y.N. Wu
0
0
0
0
0
1
1
0
1
Sorry didn’t open DM previously 😅, now it’s open! Feel free to shoot me a message ;).
0
0
1