Patrick Esser
@pess_r
Followers
5,055
Following
314
Media
27
Statuses
1,119
Walking on the generative side of computer vision @bfl_ml . he/him
Heidelberg
Joined September 2016
Don't wanna be here?
Send us removal request.
Explore trending content on Musk Viewer
Brazil
• 1180576 Tweets
Elon Musk
• 721961 Tweets
Meu Twitter
• 483724 Tweets
Alexandre de Moraes
• 471351 Tweets
Bluesky
• 214134 Tweets
Adeus Twitter
• 183091 Tweets
Politico
• 160259 Tweets
Xandão
• 158295 Tweets
#BUS1stFANCON_KnockKnockKnock
• 142808 Tweets
Wisconsin
• 122381 Tweets
Tchau
• 95672 Tweets
Caitlin Clark
• 67630 Tweets
Temple
• 59620 Tweets
ブラジル
• 41051 Tweets
Angel Reese
• 34390 Tweets
#プレイバックガチャ
• 33552 Tweets
Michigan State
• 23195 Tweets
#INZM_30Mviews
• 21988 Tweets
Janta Kee Awaaz
• 20278 Tweets
野菜の日
• 18524 Tweets
#GiftOfEducation
• 17048 Tweets
Djokovic
• 13593 Tweets
Stanford
• 12119 Tweets
カネゴン
• 11431 Tweets
#TheLastDriveIn
• 10770 Tweets
台風一過
• 10188 Tweets
Pinned Tweet
#stablediffusion
text-to-image checkpoints are now available for research purposes upon request at
Working on a more permissive release & inpainting checkpoints.
Soon™ coming to
@runwayml
for text-to-video-editing
102
1K
5K
Pre-tamed transformers now at
Just added a Colab demo to start sampling right away:
5
96
490
We present Latent Diffusion Models in tomorrow's Image & Video Synthesis and Generation session
@CVPR
in Hall B1 (Thur. 08:30-10:18).
Join us for a chat at Poster
#7
between 10:00-12:30 in Hall B2-C and sample some jazz!
With
@robrombach
@andi_blatt
D. Lorenz
@runwayml
B. Ommer
5
36
379
demo to run our GeoGPT models on images in the wild.
Code at
Detes at
8
69
350
Geometry-Free View Synthesis: We don't need no 3D priors. Leave them transformers unbiased!
Without coding 3D transformations into the model, they learn to synthesize novel views from a single input image.
1
30
148
Excited about ML for creative video editing? Come work with an amazing team
@runwayml
! We have open positions for Research Scientists and other roles. The team is diverse and distributed all over the world with a great remote work culture.
Thrilled to announce Runway’s $35M Series B led by Coatue.
Content creation and video editing are being massively transformed by machine learning and the web. Excited to double down on our mission to keep reimagining how we tell stories.
More here:
41
30
313
4
18
92
Denoising Diffusion Probabilistic Models converted to PyTorch with Streamlit Demo
Run demo:
pip install -e git+
pytorch_diffusion_demo
1
19
70
❤️ let's gooo 🚀
Party time! The SD3 paper made it to arxiv:
Key takeaways:
- flow matching is very nice.
- back to work with
@pess_r
and a fantastic team ♥️
The paper is full of details on improved flow matching, scaling and engineering. Enjoy!
10
38
266
3
3
55
So many good news! 🥳
I joined
@runwayml
as a Research Scientist!
I'll present Geometry-Free View Synthesis () at
#ICCV2021
today 6 pm and Friday 11 am EDT 👋
ImageBART () was accepted at
#NeurIPS2021
w/
@robrombach
@andi_blatt
& BO
6
6
55
sooooo good! 🤩 amazing work, congrats to the whole
@runwayml
team!!
Introducing Gen-3 Alpha: Runway’s new base model for video generation.
Gen-3 Alpha can create highly detailed videos with complex scene changes, a wide range of cinematic choices, and detailed art directions.
(1/10)
254
930
4K
2
1
54
Packing things up.
Our goal is to make creative experimentation possible for everyone. Turning prototypes into milestones on our way towards this goal requires a shared vision and true team efforts.
Fast-forwarded prototype vs real-time app:
1
6
43
Demo of our
#ECCV2020
work "Making Sense of CNNs" with
@robrombach
project:
run
@streamlit
demo in 5 lines:
git clone
cd invariances
conda env create -f environment.yaml
conda activate invariances
streamlit run invariances/demo.py
2
16
37
Training state-of-the-art models is getting too expensive. We present Net2Net at
#NeurIPS2020
to efficiently recombine existing models for new tasks. Try it and translate between low- and high-resolution models to obtain a new super-resolution model!
1
5
24
Generating videos.
Videos introduce additional challenges: Searching for frames where the background is visible is expensive and exposure changes between frames require a seamless blending strategy that produces temporally consistent results without flickering.
1
1
22
Moving forward.
Anticipating every possible use case is impossible. That's why we release an early open beta to learn more about how inpainting will be used. If you are excited about boosting video creativity with new tools, join our team:
1
0
21
Filling unknown regions of images.
Filling in a masked region of an image is an underdetermined problem: There are many possible options and we need to come up with one of them.
More about our research on generative models for inpainting and other tasks:
1
1
17
🤗
Stable Diffusion Inpainting is now available in the latest version of the
@huggingface
diffusers library (v0.6.0) 🎉
Get started here:
0
20
114
0
4
16
What to remove?
To avoid excessive manual masking, we follow a similar approach as in GreenScreen, except that we use a simpler & faster model, because we don't need very accurate masks, but we do need some extra time for the actual inpainting.
More at
1
1
15
Using temporal information.
For video inpainting, we don't necessarily need to come up with plausible content if we can see what's behind a masked region in other parts of the video.
This becomes more clear if we explicitly align a video to its first frame:
1
0
15
Great article "Understanding the Role of Individual Units in a Deep Network" on network dissection for classifiers and generators by
@davidbau
et al.
New code with pretrained segmentation model analyzes 1825 semantic concepts (1.5x)
0
1
7
"Maya the bee as an astronaut"
I finally managed to find hosting for my CLIP conditioned transformer (it's on GitHub as a release right now), so here's the Colab:
11
34
264
0
1
5
Great talks and papers at the AI for Content Creation workshop at
#CVPR2020
We'll present our work "Network Fusion for Content Creation with Conditional INNs". Live Q&A @ 4pm PDT. Joint work with
@robrombach
& B. Ommer
0
3
3
We have also added code and a
@streamlit
demo for unpaired image translation tasks from our work "A Note on Data Biases" which will be presented at the
#neurips4creativity
workshop.
0
0
2
@MonaJalal_
No GAN, it's an autoregressive model scaled to large images via a multiscale approach
0
1
1
"The important thing in science is not so much to obtain new facts as to discover new ways of thinking about them. ~Sir William Bragg"
0
0
1
Towards Principled Methods for Training Generative Adversarial Networks
0
0
1
0
0
1