Yuge Shi (Jimmy)
@YugeTen
Followers
4,049
Following
486
Media
58
Statuses
605
石宇歌 · Research Scientist @DeepMind · Past: PhD at Oxford, intern at Google Brain, FAIR, CSIRO · she/her
London, England
Joined December 2017
Don't wanna be here?
Send us removal request.
Explore trending content on Musk Viewer
#KawalPutusanMK
• 2275705 Tweets
#TolakPolitikDinasti
• 1668132 Tweets
#TolakPilkadaAkal2an
• 1666372 Tweets
Nobara
• 237328 Tweets
FB JOIN OPENING BTV HOUSE
• 177648 Tweets
#BeyondTheVinesXFreenBecky
• 172415 Tweets
Mulyono
• 152122 Tweets
BATALKAN BUKAN TUNDA
• 147632 Tweets
Nigerians
• 104834 Tweets
Bolt
• 60412 Tweets
BOTO PARA SA SB19
• 53303 Tweets
鯉登少尉
• 36565 Tweets
チャイルドシート
• 33251 Tweets
ポケ森サ終
• 30400 Tweets
ポケットキャンプ
• 28194 Tweets
PilihDAMAI BarengPRABOWO
• 27911 Tweets
BBFA LAST HURRAH
• 24659 Tweets
LebihSEJUK LebihNYAMAN
• 23039 Tweets
首都高バトル
• 22923 Tweets
Widjiatno Notomihardjo
• 21605 Tweets
買い切り
• 19362 Tweets
中川大志
• 17270 Tweets
本人確認
• 13684 Tweets
GCSE
• 12896 Tweets
BTV X FORCEBOOK
• 11052 Tweets
月見バーガー
• 10617 Tweets
Dear conference organisers: I promise I will choose a better birth place next time, but in the meantime can we please 1) host conferences at more visa-friendly locations or 2) leave more time between paper decision time and conference so we have enough time to apply for visas?
18
357
3K
Successfully defended my thesis with Max Welling (
@wellingmax
) and Andrew Zisserman today! No professors were physically harmed in the process.
A huge thank you to both for the insightful discussions!
25
6
517
Not everyone can get into the US by taking 20 minutes to fill out an online form, some has to dig up their entire employment/travelling history and refresh the visa appointment website desperately for days and days
3
19
483
After much procrastination, finally uploaded my first blog post "Gaussian Processes, not quite for dummies"! Have a lookie here:
Share if like :)
9
119
471
Random pixel mask is NOT random word mask in image space: we propose ADIOS, which learns semantically meaningful masks for masked image modelling in a self-supervised manner.
Joint work with Adam Kosiorek (
@arkosiorek
), now accepted to
#ICML2022
.
📜:
7
47
308
Pretty pissed off about not getting to ICML for the third time because of visa, but super proud to have 2 orals, 1 spotlight and 1 workshop at the conference!
I'm not going to fix open borders with this tweet but I do hope it nudges you to go talk to my amazing collaborators --
4
10
310
The school of informatics at the University of Edinburgh and DeepMind are offering an ML PhD scholarship for students who identify as gender/racial/ethnic minorities in 2022/23. See thread for details. (1/n)
4
77
236
New blog post: How I learned to stop worrying and write ELBO in a billion ways.
I talk about a list of interesting approaches including IWAE and DReG that are important to the evolution of ELBO over the past few years.
Have fun reading, RT if like!
9
48
228
Dare I say...computer vision with human feedback is on its way in? Super happy to have contributed to this important piece of work during my internship at Google Brain!
Vision meets RL! We reveal that policy gradient can be used for tuning vision models to optimize complex metrics, such as mAP, PQ or “color diversity”, observing large performance boosts on tasks like object detection, panoptic segmentation, etc.
4
131
641
0
19
200
How robust are unsupervised representation learning methods (e.g. SSL) to distirbution shift compared to supervised learning?
𝐒𝐡𝐨𝐫𝐭 𝐚𝐧𝐬𝐰𝐞𝐫: Quite!
𝐋𝐨𝐧𝐠 𝐚𝐧𝐬𝐰𝐞𝐫: Our
#ICLR2023
paper
Joint work with Imant Daunhawer &
@AmartyaSanyal
.
4
37
200
We built an *action-controlled video generative model* with video data only 🤯 GenAI now officially also means generative environments!
So incredibly proud of this work that I spent my first year at
@GoogleDeepMind
on 💙 more work to be done, watch this space!
I am really excited to reveal what
@GoogleDeepMind
's Open Endedness Team has been up to 🚀. We introduce Genie 🧞, a foundation world model trained exclusively from Internet videos that can generate an endless variety of action-controllable 2D worlds given image prompts.
145
572
3K
11
23
170
My supervisor Sid has 2 PhD positions available at University of Edinburgh! A great learning opportunity for those interested in probabilistic generative models.
See more details on Sid's website here:
1
76
149
Super thrilled to have our work "MEME" on multimodal VAE accepted to ICLR as a spotlight paper!
@iclr_conf
2
22
139
New blog post: An incomplete and slightly outdated literature review on augmentation-based self-supervise learning.
Read about me trying my best to summarise 9 SSL methods (e.g. SimCLR, BYOL, SimSiam) here 👇
3
25
135
It feels weird that these days "overqualification" is considered as a merit for PhD applicants. It is as if the point of a PhD degree is not to receive education, but to compete.
2
4
131
Our paper was desk rejected by Neurips because I cut out the appendix by "printing to local files" and it added 1-2cm of margin each side.
Is there any mistake out there that I haven't made yet???
9
12
102
Finally getting visa for the US feels like a lucid dream -- will be attending neurips2023 this year 😊
6
0
101
Another year, another missed conference in the US due to visa issues :(
Do go check out our poster and chat with
@giffmana
!!
Will be presenting the poster this afternoon at ICML.
Sadly, none of my coauthors were able to attend, so I'll be all alone 😔
1
11
112
1
10
87
While you're at it, please also rank your children from "most promising" to "least promising" and enter the probability of them achieving success in life
0
4
84
We are hiring a research engineer on the Openendedness team at
@DeepMind
! Please reach out if you have any questions 😌
Interested in scaling open-ended learning systems? Check out the new RE posting in our team 🚀. Feel free to DM with any questions!
0
17
104
2
8
76
So much fun presenting Genie at
@UCL_DARK
yesterday with
@ashrewards
and
@jparkerholder
!! 🧞✨ And thanks Tim for the candid photo it's time to make my Chinese parents proud again by sending this to the family group chat
How can we learn a foundational world model directly from Internet-scale videos without any action annotations?
@YugeTen
,
@ashrewards
and
@jparkerholder
from
@GoogleDeepMind
's Open-Endedness Team are presenting "Genie: Generative Interactive Environments" at the
@UCL_DARK
Seminar
2
17
116
0
6
73
Three months ago I said to my advisor "I'd rather rub glass into my eyes than do a post-doc" and this past week I have been like "hmm I think I want to do a post-doc"
What's happening????
13
0
69
Yesterday I started my internship at Google Brain with André Pinto, Alexander Kolesnikov (
@__kolesnikov__
) & Neil Houlsby (
@neilhoulsby
)! Excited to work with new people and tackle new problems :)
I took necessary steps in home decoration to make it feel more like the real deal
1
1
69
Had an amazing time at the Mediterranean Summer School
@M2lSchool
giving a lecture on introduction to Computer Vision 🥸 big kudos to all the organisers and students for making this such a fun and fulfilling experience 💙
2
3
68
New work on data-efficient multimodal VAE "Relating by Contrasting" is now available on Arxiv!
Joint work with Brooks Paige, Philip Torr and N. Siddharth.
1
10
62
We currently have 1 PhD position for machine learning research open in Torr Vision Group (
@OxfordTVG
) led by Professor Philip Torr!
If you are interested, please send your CV + short paragraph of relevant experience to philip.torr
@eng
.ox.ac.uk.
Oxford Professors Philip Torr and
@wooldridgemike
have been appointed as the first Turing Artificial Intelligence World-Leading Researcher Fellows. 👏
These fellowships aim to strengthen the UK’s position as a global leader in the field.
Read more ⬇️
3
26
147
1
14
64
Update: this has been reverted! Thank you to everyone who brought attention to this matter and saved me and my coauthors lots of heartbreaks ♥️
Our paper was desk rejected by Neurips because I cut out the appendix by "printing to local files" and it added 1-2cm of margin each side.
Is there any mistake out there that I haven't made yet???
9
12
102
0
2
61
Having so much fun presenting Genie alongside
@jparkerholder
at
@cveu_workshop
, AI4CC and 3DMV workshop at
@CVPR
! We will be at the Google booth on Thursday at 1:30pm if anyone wants to come say hi 🧞♀️
2
5
56
Super excited about hosting this workshop on multimodal representation learning at
#ICLR2023
! Is embedding multimodal data a waste of modelling capacity or a way to learn more generalisable representation? Submit your work and join our discussion!
Is having multiple modalities a blessing or a curse? What is a good representation? Let's find out together!
We are proud to announce the 1st hybrid workshop on Multimodal Representation Learning at
#ICLR2023
(
@iclr_conf
) 🚀
More info:
1
10
38
0
9
48
What good does labels do for OOD generalisation? Our new work says: not much!
Check out our ICML SCIS & pretraining workshop publication:
Joint work with Imant Daunhawer (
@ImantDaunhawer
) and Amartya Sanyal (
@AmartyaSanyal
).
#ICML2022
@icmlconf
2
11
46
🤯📽️🏃Annoucning controllable video generation workshop at ICML2024! Super excited for speaker line up, and looking forward to seeing submissions exploring controllability in video generation.
Submission deadline is 31st May AOE (but why wait till then if you can submit now?!)
We are pleased to announce the first *controllable video generation* workshop at
@icmlconf
2024! 📽️📽️📽️
We welcome submissions that explore video generation via different modes of control (e.g. text, pose, action).
Deadline: 31st May AOE
Website:
1
9
47
2
9
47
Improvements from using longer context is unsurprising, but we found a way to do it ✨in style✨ with x10 memory savings with greater perf gain.
A super fun and rewarding experience working with Ivana Balazevic (
@ibalazevic
) and Olivier Henaff (
@olivierhenaff
) and many more!
Humans and animals reason about events spanning days, weeks, and years, yet current CV systems live largely in the present.
Introducing Memory-Consolidated ViT, whose context extends far into the past and sets a new SOTA in long-video understanding with a 10x smaller model
5
27
172
2
11
41
Will be presenting this work at
#ICLR2023
in the poster this morning! Come say hi 👋
How robust are unsupervised representation learning methods (e.g. SSL) to distirbution shift compared to supervised learning?
𝐒𝐡𝐨𝐫𝐭 𝐚𝐧𝐬𝐰𝐞𝐫: Quite!
𝐋𝐨𝐧𝐠 𝐚𝐧𝐬𝐰𝐞𝐫: Our
#ICLR2023
paper
Joint work with Imant Daunhawer &
@AmartyaSanyal
.
4
37
200
0
7
41
Come join our
@iclr_conf
poster session at 2:30AM BST (6:30PM PDT) and watch me struggle to stay awake
Link:
0
10
40
Thinking about the time when I was in Japan arguing with a deer
4
0
35
We have seen in Genie again and again the importance of dataset quality, really cool to see more excellent work on automatic data curation. Congrats to the authors!
Excited to share our new preprint on data curation for large scale multimodal learning ()! TLDR; 1) we show that picking good batches of data is more important than selecting data points independently, 2) online model approximation can be used to filter
7
37
191
1
4
34
Thank you for answering the question that has been haunting me in my dreams and can't wait to read the paper!
New paper! We ask: How important is VQ for neural discrete representation learning? 🤔
I always felt it should be possible to “absorb” the VQ on top of a deep net into the net and use a simple grid-based quantization instead, without sacrificing much expressivity.
Summary 🧵👇
7
59
352
1
1
33
Today is a sad day. My best friend in the lab left and now we have 50% less women.
0
1
34
Fellow senior PhD/post-docs evaluating PhD candidates for the
@ELLISforEurope
program: How's your ego? Mine is not great
1
1
31
loll "mum I'm on twitter!!"
It's that day of the year again (
#ALevelResultsDay
) when we get to congratulate all of our new, incoming students who will be
#GoingtoOxford
this year! 🥳
A very warm welcome to you if you're going to be joining us as an
#Oxford
student 👋
We can't wait to meet you ☺️
3
15
122
3
0
31
Come find Sander and many other rockstars in video generation speaking at our workshop at ICML2024!
@cvgworkshop
🥰
0
2
28
Spending four hours to review a paper to and then discovering that the paper has been withdrawn is the perfect comeback for anyone lecturing you to be an early bird
0
0
28
I have a joke about all these "I have a joke about" jokes, but it fucking sucks
1
0
27
Our paper on Variational Mixture-of-Experts Autoencoders for Multi-modal Deep Generative Models is online! This is a joint work with N. Siddharth, Brooks Paige and Philip Torr
@OxfordTVG
.
Paper:
Video:
1
6
26
👇👇👇
We have received 10 *signed* hard copies of Machine Learning and Pattern Recognition by Christopher Bishop, as a generous donation by
@MSFTResearch
! We will be gifting these to students who most need them from our upcoming summer series --- stay tuned for more updates!
2
10
100
0
2
26
Thank you
@maxjaderberg
— as kids these days like to say, we feel very seen 🫶
Your work on XLand inspired many of us to work on Genie — we are standing on giant’s shoulders for sure!
Very cool to see the
@GoogleDeepMind
Genie results: learning an action-conditional generative model purely unsupervised from video data. This is close to my heart in getting towards truly open-ended environments to train truly general agents with RL 1/
2
18
151
0
0
24
It is now midnight and I still can't get over the fact that this afternoon some old guy in my building told me he's going to drown me and my dog for playing ball downstairs, what a wild fucking world this is
3
0
23
There's still time to submit to our workshop! Would love to see your work on controllable video generation 🫵
We are pleased to announce the first *controllable video generation* workshop at
@icmlconf
2024! 📽️📽️📽️
We welcome submissions that explore video generation via different modes of control (e.g. text, pose, action).
Deadline: 31st May AOE
Website:
1
9
47
1
5
23
👇👇👇 So glad to finally be able to announce this! We are super grateful for the generous donation by Microsoft Research at Cambridge
@MSFTResearchCam
for our summer program, enabling us to continue on with our mission and bring more diversity to this space. Stay tuned!
AMAZING NEWS!! We are thrilled to announce that we're now officially sponsored by
@MSFTResearch
!! Having their support will enable us to continue to deliver our workshops, but also provides us with greater resources to expand. Watch this space, big things are coming.
2
16
149
0
2
22
Another cool blog post by Adam on extracting/utilising geometric information from diffusion models! 🟡🟪🔻⬜️
Text-to-image diffusion models seem to have a good idea of geometry. Can we extract that geometry? Or maybe we can nudge these models to create large 3D consistent environments? Here's a blog summarizing some ideas in this space :)
0
26
141
0
2
22
Today I was told that this lovely early autumn weather is also called "Indian summer", and I'm so delighted to finally learn an idiom that uses Asian countries in a positive way
1
0
21
Writing paper is overrated and I just wanna cuddle my dog in bed for a day
0
0
21
Super interesting work on controllable video generation with diffusion!
Excited to share our work on Neural Assets: a new method for enabling 3D asset-level control in image diffusion models – scalable & without any 3D inductive biases.
Neural Assets goes beyond text or pixel-based control & provides an interface inspired by 3D graphics tools.
🧵
3
46
246
1
3
19
Just want to show off a great birthday present from my niece ❤️🎈
0
0
19
Will be presenting Genie with
@jparkerholder
at CVPR🧞🧞♀️ see you all in Seattle!
🔥 Ready to redefine the future? Join us at
#CVPR2024
for the
@cveu_workshop
on "The Future of Generative Visual Art"! Experience the latest in AI artistry and visionary tech. Let’s create the unimaginable! 🎨🤖
🗓️ June 18 📍 Summit 343
Be part of the revolution! 🌌
2
3
15
1
4
19
Post-doc positions available at TVG! Great opportunity those interested in continual learning/estimation of uncertainty/robustness 👇👇👇
@OxWoCS
@OxWEST_
@QueerinAI
@AiDisability
@black_in_ai
@Khipu_AI
@DeepIndaba
@_LXAI
@women_in_ai
Two post-doc positions open in the group now!
Apply before 11th November!
0
4
6
2
7
18
Submit your worst idea now for our Bad Hypothesis
@iclr_conf
social!
Tired of rigorous scientific processes? We offer the fastest publication cycle ever with no review (!!) at our
@iclr_conf
social: Bad Hypothesis Contest. Submit your 5 min fake science contribution:
with
@YugeTen
and
@karen_ullrich
5
32
140
0
0
18
and that's exactly the only thing that will happen, you will be heard -- anything beyond that, good luck you're by yourself
Sexual harassment and violence is never acceptable at the University of Oxford.
If you have been affected by sexual harassment or violence in any form, our Support Service is here to help as a safe place for you to be heard:
0
0
4
0
0
17
I think my timeline needed this 🧡
Dear Students, please don't get discouraged... we need you. Quoting another Geoff: "The future depends on some graduate student who is deeply suspicious of everything I have said" (8/8)
25
212
2K
0
3
17
Hey they made a podcast out of my necklace :)
@GuiltFemPod
Had to give the live show a miss when they are last in town in 2019 from the stress of writing the first paper of my PhD --- it's now finally the turn that I give the stress a miss for the live show!
1
0
17
A month ago we mourned the death of doctor Wenliang Li, the first to warn the public of COVID19. The western media called him the whistle blower. Now China, Italy, Iran, South Korea has been blowing this whistle for 2 months, but turns out some countries are just FUCKING DEAF
#uk
0
1
16
So honored to have had the opportunity to chair the seminar with
@timnitGebru
at
@oxwocs
! Thank you for the amazing talk and insightful discussion ❤️
We are super thrilled to have Dr. Timnit Gebru
@timnitGebru
joining us in our next seminar to talk about the recently founded
@DAIRInstitute
!
Time: 18th Jan, 5-6pm GMT
Sign up: |
3
21
126
1
0
16
Amazing opportunity for those looking for a phd 👀 can personally verify that both Antonio and the School of Informatics in Edinburgh has excellent vibes
I have fully-funded PhD positions (3.5 yrs) for troublemakers in
#ML
#AI
who want to design the next gen of
#probabilistic
#models
and
#programs
that are provably
#reliable
and
#efficient
Join
@InfAtEd
@EdinburghUni
Email me! ✉️
Please share! 🔁
Apply 👉
5
81
205
0
2
16
Our paper on “Variational Mixture-of-Experts Autoencoders for Multi-Modal Deep Generative Models” got in Neurips! Here’s just a photo of me and some random strangers I’m sitting with when I got the news
3
1
14
Nothing is more real for starting the second half of your 20s than your parents forgetting about your birthday aaaawwwwuuuuuuuuu
1
0
15
This is the dream supervisor team that will 100% strike a perfect balance between setting ambitious research goals while being practical and realistic, and Open-Endedness is such an exciting field with lots of cool work popping up -- apply now!
We (
@_rockt
,
@egrefen
,
@robertarail
, and
@jparkerholder
) are looking for PhD students to join us in Fall 2024. If you are interested in Open-Endedness, RL & Foundation Models, then apply here: and also write us at ucl-dark-admissions
@googlegroups
.com
3
20
65
0
1
15
Apparently it is culturally inappropriate in England to take your puppy that weighs 2kg downstairs for a wee without a leash on, and I am rude and ignorant because I am "not from around here"
2
0
15
I remember the very first thing my supervisor in Oxford said to me was "I am so sorry about the weather, but trust me it is LOVELY here in summer" -- we have had around 10 days of summer in the past three years
The sun came out for 7 minutes. People stood in the street, staring at it like it was Halley’s comet. Dogs howled, children wept.
766
4K
67K
0
0
15
Internship opportunity with one of the bestest and coolest researcher & mentor I know 👇👇
Interested in the unique challenges at the intersection of Reinforcement Learning & Drug Discovery?
New internship in Amsterdam:
Do research w/
@MaziarzKris
,
@fiberleif
,
@jagarridotorres
, Ran Bi,
@marwinsegler
& me in AI4Science!
Questions? Reach out!
3
27
88
1
0
15
🤔Has this ever happened to you -- you want to learn a multimodal VAE, but only have limited amount of paired data? Or, you want to incorporate NCE with ELBO, but not sure how?
Join us at our
@iclr_conf
poster 9AM (BST) tomorrow to find some answers:
0
4
14
Today my dog broke the internet cable and I wasted half a day trying to fix it. As I was lying on the floor feeling frustrated and desperate, she came and put her lil head on my belly while remaining in a weird standing pose.
Perform this magic to get out of any trouble
0
1
14
100000 times less offensive than "hope this email finds you well"
0
1
14
For interested applicants: please email anyone at the school of informatics in the University of Edinburgh depending on research interests to get more information on this. See below for more details on the degrees available: (n/n)
0
0
14
Hey Neurips: come to our paper’s poster to learn all about multimodal-VAE! Our session is tomorrow (Wednesday) 5pm-7pm, No.150 at east exhibition hall.
#NeurIPS2019
Paper:
Code:
0
2
14
Just your friendly neighbourhood workshop looking for solid submissions 👀 spread the words!
So thrilled to see so many interesting work accepted to
#ICLR2023
, can't wait to see you all in Kigali!
That said -- if you work did not get a chance to shine at the main conference, consider submitting to our multimodal learning workshop! Submission open till 30 Jan ⌛️
0
9
21
0
8
13
🧞 Generative interactive environments (oral): Hall C 4-9
#614
, Tuesday morning
Go chat with
@ashrewards
@jparkerholder
@_rockt
@MichaelD1729
, Jacob Bruce about video generation as world model!
5
1
13
Super excited to have
@shiorisagawa
speak at OxWoCS this Thursday!! Sign up to join:
For our seminar speaker event this Thursday, Shiori Sagawa (
@shiorisagawa
) from Stanford will be talking about her work on distributionally robust optimization (DRO) as well as the WILDS benchmark 👏👏👏
Time: 5-6pm BST, Thursday, 16th June
Sign up:
1
6
18
0
3
13
In lieu of positive results from experiments, I did a pull up as my weekly progress when Skyping with my supervisor.
0
0
13
My 13 year old niece just asked me, "will I be single forever?"
1
0
12
Elise is one of the most amazing person I know, so glad to see her recognised at WiML for her incredible research!
1/4 Continuing our
#WiML
Profiles series, we are delighted to highlight Dr.
@ElisevanderPol
, who is a Sr. Researcher at
@MSFTResearch
AI4Science Amsterdam, working on deep learning for molecular modeling and reinforcement learning. See full profile here:
1
4
35
1
1
12
Huge thanks to the amazing
@gradientpub
team for helping with editing! 🤗🤗🤗
1
1
12
Probably the best non-fake video call background I'll ever have 😌
Come join us in gathertown for our poster session
@mrlworkshop
!
Gather Town Link:
Password: mrl_workshop_2023!
0
0
12
You did such an amazing job! ♥️ Congrats to the 80 students who joined our first session and big thanks to all the demonstrators and content creators of the class:
@ShuyuLin_n
@linde_hesse
@fhuszar
@kiourlappou
@xinchiqiu
, Maz, Zack Lee, Adel Bibi and Louis-Pascal Xhonneux
quick snap of my (13" monitor) setup at the 1st
@GirlsWhoML
90-minute workshop on linear regression which I designed and delivered to 80+ girls yesterday! 🌟 many thanks to the team & the demonstrators & everyone who joined, it was so much fun!
1
4
34
0
4
11
📢📢📢 Code and models for ADIOS is now available!!
Check it out here:
@arkosiorek
and I will both be joining ICML remotely, be sure to stop by and have a chat if you are also attending this year!
Random pixel mask is NOT random word mask in image space: we propose ADIOS, which learns semantically meaningful masks for masked image modelling in a self-supervised manner.
Joint work with Adam Kosiorek (
@arkosiorek
), now accepted to
#ICML2022
.
📜:
7
47
308
1
1
11
You ask if it is a data problem or a modeling problem -- why not use one to address the other? Super interesting work on *online* data curation for CV tasks!
So excited to announce what we've been working on for the past ~year or so:
Active Learning Accelerates Large-Scale Visual Understanding
We show that model-based data selection efficiently and effectively speeds up classification- and multimodal pretraining by up to 50%
9
76
383
0
2
11