Rishabh Anand 🧬
@rishabh16_
Followers
4,992
Following
1,365
Media
534
Statuses
7,806
multiplying matrices @NUSingapore • geometric DL + generative modelling for proteins, RNA, and drug discovery @Cambridge_CL 🛠
Singapore
Joined June 2017
Don't wanna be here?
Send us removal request.
Explore trending content on Musk Viewer
SEBI
• 416615 Tweets
#にじ甲2024
• 153148 Tweets
#Number_i_RIJF2024
• 96610 Tweets
South Africans
• 82736 Tweets
NUNEW DREAM CATCHER
• 79394 Tweets
#NuNew1stConcertDay2
• 67281 Tweets
ギラホス
• 32131 Tweets
小倉記念
• 31122 Tweets
報徳学園
• 22523 Tweets
マーリン
• 22124 Tweets
ヘルナンデス
• 18919 Tweets
大社高校
• 18410 Tweets
Ferhat Gedik
• 16307 Tweets
Sifan Hassan
• 15824 Tweets
和田アキ子
• 15341 Tweets
#iyikivarsınEren
• 13832 Tweets
契約解除
• 12223 Tweets
リフレーミング
• 11908 Tweets
鈴木さん
• 10480 Tweets
Pinned Tweet
🚨 My Graph Deep Learning extended tutorial has a new home!!!
It's finally up on my personal website:
Do RT, share, and tag your GDL buddies if you find it useful 🙌🏻🙌🏻
🚨NEW WEEKEND READING🚨
I'm publishing "Graph Neural Networks for Novice Math Fanatics" – a primer on the math behind GNNs using colourful drawings & diagrams
(I hope this becomes the definitive guide to GDL for those hoping to enter the field!!!)
10
232
1K
10
303
1K
🚨NEW WEEKEND READING🚨
I'm publishing "Graph Neural Networks for Novice Math Fanatics" – a primer on the math behind GNNs using colourful drawings & diagrams
(I hope this becomes the definitive guide to GDL for those hoping to enter the field!!!)
gm 🚨🚨
New primer on "Math behind Graph Neural Networks and its Variants" dropping soon 👀👀
It's filled with drawings and diagrams too!!!
Stay tuned!
5
48
461
10
232
1K
Hey 👋🏻
Lately, I've been preparing for ML/DS internship interviews (mainly in Computer Vision) and decided to create a non-exhaustive set of notes to study.
Thought I'd share it here for those in the same boat :D
Happy learning!
15
176
874
My two contenders:
1) Why LayerNorm is so helpful in Attention and what it’s actually doing under the hood ()
2) Explaining the oversquashing phenomenon in GNNs ()
Really cool papers with superrr approachable math/theory 🙌🏻
what paper (not your own, maybe not even in your own area) can you not stop telling people about?
84
43
448
6
79
592
Hey 👋🏻
So a lot of you reached out to learn how to train your own models on *ALL* 8 cores of a Colab TPU using
@PyTorch
XLA
Not many good resources out there, so I wrote a super easy 8-step tutorial on doing exactly this:
Share if useful :D
7
142
582
CS kids will do everything except study for a non-CS exam ...
23
23
547
OpenAI’s embeddings will usher in a new age of vector search engines in latent spaces. Info retrieval’s coming back hotter than ever 🤔 🤔
18
46
473
I’m teaching my first ‘Intro to ML’ class at NUS today!!!
So excited omg 😍🤩
25 students but fully online. Oh well let’s see how it goes 🤔
20
18
475
gm 🚨🚨
New primer on "Math behind Graph Neural Networks and its Variants" dropping soon 👀👀
It's filled with drawings and diagrams too!!!
Stay tuned!
5
48
461
There’s a lot of math involved in current generative models. This is my favourite in-depth blog series about it all by
@ludiXIVwinkler
:
5
46
386
Happy Lunar NY 🎉
Just read
@PetarV_93
's "Everything is Connected". I spent most of 2022 studying Transformers for graphs and this paper intuitively connects most of what I've learned!
🔗:
Here's an executive summary if you haven't caught it yet!
1/9
7
65
367
🚨🚨 Exciting Update:
I've giving a talk on the Expressiveness of Graph Neural Networks at the ✨Workshop on Advances in Deep Learning and Applications✨ this Thursday!! ()
It's hosted by the Dept of Sci & Tech, Govt. of India 😱
Stay tuned for slides!!!
4
35
347
Once you've trained on *ALL* 8 cores of a TPU using PyTorch XLA, there's no going back ...
Everything else seems like a very bad downgrade 🥴🥴🥴
8
26
324
🚨🚨NEW BLOGPOST
I write about the new family of Expressive Graph Neural Networks and what expressiveness entails in the broader context.
It's full of colourful diagrams and intuitive explanations to the math behind the concepts ✨⚡️
Check it out:
5
65
314
no matter what you do in research, you *have* to learn tmux
ultimate lifesaver
23
15
269
3-WL spectral GNN with laplacian encodings, virtual nodes, and graph rewiring // 2 transformer layers
3
17
232
Shampoo now has a good official Pytorch implementation by FAIR (MAIR??) 🎉
(cc
@_arohan_
and friends)
0
35
206
🚨🚨 New Graph DL library!!!
Introducing ✨grafog✨ – a graph data augmentation library built on top of torch_geometric. I built this for a class project and am super excited to finally release it!
Stars appreciated if you like it!!! 😬
2
28
186
Why isn’t a single neuroscientist or neuro-person commenting on this Lamda thing??? Looks to me like a bunch of AI nerds are freaking about sentience when that’s not even their primary field of study …
29
15
170
So I was wondering why Softmax is so convenient that it gives out a probability distribution for any input vector ... and it turns out the proof is actually *really* simple
Thought it was much more complex than that ... 🤔🤷🏻
7
19
166
Good news!
Finally got the approval from NUS to start my UROP (undergrad research programme) in Jan 2022++ 🤘🏻🤘🏻
Will be focusing on Graph Deep Learning, Reinforcement Learning, & Contrastive Learning
Leaning on twitter to keep serving me loads of cool papers (as it does)🤝
2
5
151
✨Why do ResNet architectures prefer TWO sets of 3x3 filters back-to-back compared to a SINGLE 5x5 filter?✨
I was chatting with someone about it and it's super interesting!!
A thread filled with colorful diagrams and simple explanations 👇🏻👇🏻
1/n
6
33
143
Kinda late to the party but for the ML course I was TA'ing this semester, I made a 1-hour mini lecture on ✨Backpropagation✨ from scratch to help the cohort of students learning it
Check it out:
3
14
128
🧬🤖 Introducing RNA-FrameFlow –– an unconditional generative model for 3D RNA backbone design!
📑:
🧰:
Our method generates ≥ 40% self-consistent *all-atom* RNA backbones that are globally and locally realistic 💪🏻
1/9
3
25
123
Hi ML Twitter 👋🏻
Anyone in my circles working on Graph DL and NLP? I'd like to have a chat about this intersection :D
Appreciate any leads on this!
11
6
106
Interesting paper from CVPR 2021 that shows the efficacy embedding faces in *spherical spaces* as opposed to general Euclidean. Their "Sphere Confidence Face" (SCF) embeddings beat existing embedding techniques for face recognition with a ResNet🤔
3
25
102
Finally finally finally teaching Neural Networks in my ML class next week omg have been looking forward to this for a long time 🥴🥴🎉
1
3
99
Geometric DL is super hot right now. Want a front-row seat to what’s going on?
Then check out this repo NOW to learn about the many mechanisms that power the latest and greatest geometric models!!!
❓New to Geometric GNNs, GDL, PyTorch Geometric, etc.? Want to understand how theory/equations connect to real code?
Try this practical notebook before diving into this exciting area!
**Geometric GNNs 101**
7
167
808
4
12
97
Hey everyone 👋🏻
Yet another interesting paper ⬇️😲
Here's my unofficial PyTorch implementation of the Involution operator introduced in the paper:
(Of course, it's a pip-able wrapper in true
@lucidrains
fashion!)
Note: TF wrapper coming veryyyy soon 🏎
Involution: Inverting the Inherence of Convolution for Visual Recognition
Improve the perf of ResNet-50 by up to 1.6% top-1 accuracy, 2.5% and 2.4% bounding box AP, and 4.7% mean IoU absolutely while compressing the computational cost to about 60%.
1
67
330
1
22
95
👉🏻👉🏻To the graph DL folks:
If shallow GNNs perform decently well on certain benchmarks without risking oversmoothing/squashing, why’s there significant literature on building deeper GNNs?
What even is the point? Is it simply in hopes of richer feature extraction from data?
9
7
97
🚨🚨
I’m on the intern job market for ML Engineer/Research roles from May-September!! I love Graph DL, Computer Vision, and ML in general.
Find out more abt me:
Mostly established labs/companies. Remote but idm traveling
RTs/intros appreciated 🙏🏻
3
15
97
A career in machine learning is importing tensorflow.keras, training on MNIST, and immediately putting “machine learning” on your resume
A career in machine learning is moving data around in sql and pushing a button on a web interface to train your models
5
14
188
2
6
87
Super lit article by
@hardmaru
and team from Brain. It involves using more focused (attentive) RL agents. Results include lighter and smarter models in pixel-based envs
Recommend the read :O
1
17
90
It's crazy how close we are to *great* people in universities – that Prof teaching you an undergrad class? Yeah, they're Twitter besties with a Turing Award winner
Or that Prof you walked past in the corridor? They built a fancy algo 10 years ago that's used everywhere 🤯🤯
5
3
86
OMG my teaching feedback for this semester's AI and Machine Learning course came out and I scored a ✨4.8/5.0✨ average rating with the course-wide TA avg being only 4.2/5.0
The students' comments are so wholesome🥺🥺
Beyond elated 🙌🏻🙌🏻 Can't wait to do it again next sem!!!
5
1
85
🚨NEW PROJECT🚨
I built a PyTorch wrapper (<10LOC) of "TokenLearner" by
@ryoo_michael
,
@m__dehghani
, & others from
@GoogleAI
.
You can plug-and-play with ViT or any Transformer model!!
Check it out:
✨Install with pip✨
pip install tokenlearner-pytorch
While Vision Transformer models consistently obtain state-of-the-art results, they often require too many tokens for larger images and video. Read about TokenLearner, which adaptively generates fewer tokens but enables models to perform better, faster →
6
134
610
3
26
83
Yo 👋🏻
Neural Additive Models (NAM) by
@agarwl_
et al. was one of my fav papers of 2020 and I've been meaning to implement it for a while!
My pipable NAM PyTorch wrapper:
Paper:
You can install it like so:
pip install nam-pytorch
0
12
80
Sure, we all know
but imagine a platform to present ML research papers in a more interactive and fun manner – like the World Models demo website ()
same vibe as Papers w/ Code but for interactive ML literature only
2
15
80
Yo!
I'm very close to releasing "Long Short-Term Memories", a weekly podcast series where the hosts are 2 conversational ML agents talking to each other about random stuff powered by
@huggingface
DialoGPT models 😱🙌🏻
Super excited and can't wait to show it to y'all 🎉
3
6
77
Teaching tomorrow and I completely forgot to make slides for the students coz of how hectic the week was …
It’s 2am rn … never again 🥵😔🤕
6
1
79
What’s up with people and taking shortcuts 🤔🤔🤔🤔
Don't waste 6 months just to learn mathematics to start machine learning.
57
21
353
10
2
73
🚨 ML Twitter, I need your help:
Can I know how y'all calculate the FLOPS of a Transformer/model?
Do I need some custom method inside my nn.Module to compute it in the forward pass or are there libraries out there I can check out? What's a fast, easy way to do this?
Thanks!
13
6
74
Tech twitter is either UCBerkeley or UWaterloo.
Nothing more, nothing less.
Balanced, as all things should be.
2
3
70
Decided to have a mini solo hackathon today coz it was my free day at school + was bored.
Read the paper and built a discount version of GPT-3:
(I know I'm definitely late to the party ... also pretty sure there are some errors here and there 🤷🏻)
1
14
69
I'm in the UK!! 🇬🇧
Excited to join
@pl219_Cambridge
's lab at Cambridge as a visiting research student for the next one year! I'll be working on geometric deep learning and protein dynamics alongside
@chaitjo
and others
If you're here / London, I'd love to chat over ☕️ :D
8
1
67
Super cool video on how convolution should actually be taught in schools/education material instead of using the "sliding window on a 2D sheet" visualisation
(can't find the creator's twitter, help appreciated)
1
8
68
The sad thing is, for most adults, I'm guessing the last time they had n ≥ 2 months free for themselves to unwind (ie, a long break) was when they were teens
2
4
66
Wew some good news 🥳
Super excited to tell y'all I'm now a writer for
@gradientpub
, a leading AI/ML research magazine from
@StanfordAILab
🌲
I'll be writing in-depth articles about interesting research from recent papers, mainly in the RL area
Can't wait to get started!
1
3
66
Tiny win this semester 🥳
Just got back my reviews for the ML class I TA'd at NUS for ~4 months and received a rating of 4.7/5 from my students while the department average was only 4.3/5 😌
I'll be teaching next sem too so stay tuned for more notes I drop (occasionally) here!
2
0
60
⬜️⬜️⬜️⬜️🟩
⬜️💀⬜️⬜️⬜️
⬜️⬜️⬜️⬜️⬜️
⬜️⬜️⬜️💀⬜️
🤖⬜️💀⬜️⬜️
Not wordle, just my RL agent getting stuck in the corner of the maze for 3 episodes straight
0
0
60
Taking an advanced linear algebra class this sem ... pretty psyched to be learning about Tensors from a math/physics lens rather than the "oh it's just a n-dim array" perspective
5
1
60
Ever since GeoLDM dropped (), I've started believing that training gen models on stuff (esp molecules) in latent space makes so much more sense than directly on its constituent features/quantities like coords, atom IDs, etc
Really neat results!! 👇🏻
Excited to share that our paper, A Latent Diffusion Model for Protein Structure Generation has been accepted by LoG 2023
@LogConference
!
Our LatentDiff achieves ~88 faster speed than FrameDiff and is ~247 faster than RFdiffusion, and the performance gap is small.
3
20
178
1
13
60
Read the paper in greater detail and I'm fascinated by how (relatively) simple the underlying GNN architecture is for a *generative model*.
The community seems to love e3nn-style GNNs for generative setups coz of the extra expressiveness.
Surprised this still works so well :O
1/n: We are excited to share that our paper on Chroma, a general purpose diffusion model for proteins, is out today in
@Nature
!
A couple of my favorite highlights in the 🧵below 👇
15
317
1K
1
5
59
Broke: using a fully connected layer 😢
Woke: using 1x1 convolution 😎
4
2
57
Just watched Karpathy's CVPR talk on the ML that happens under the hood at Tesla
"Tesla wins in self-driving coz it is in full control of every component in the AV manufacturing pipeline" is an understatement
Must-watch if you're interested in AVs:
2
6
54
Meet
@drfeifei
✅
Super inspiring talk about the progress in vision systems/benchmarks and how it drives better understanding of the real world. Thank you for visiting NUS!
2
0
55
It’s ya boi’s 21st today 🎉
Super grateful to each and every one of you here who has given me so much. I am inspired by you daily and am rooting for you in you whatever you pursue 🥺
Looking forward to the exciting things coming up ahead 🌊🤟🏻
10
0
55
Hey 👋🏻
Here's my PyTorch wrapper around the Attention Free Transformer by
@zhaisf
,
@nitishsr
, et al. that aims to linearly approximate the expensive dot product operation🏎
You can even pip install the "AFT-Full" Attention layer:
pip install aft-pytorch
An Attention Free Transformer
pdf:
abs:
an efficient variant of Transformers that eliminates the need for dot product self attention
3
141
671
4
18
54
Seeing all these image transformer papers say they “fully remove convolutions” but use patch-wise embeddings on their inputs
3
1
54
I’ve deployed 3 websites on
@Netlify
just this week for school projects, and there hasn’t been a single moment where I’ve felt mentally drained
They really out here solving the webapp deployment crisis ✨✨✨
Couldn’t be more grateful for it existing omg 😭
4
4
54
My "Intro to CS" class had this regular thing called "Mastery Checks"
It's a 1-on-1 informal chat where students had to CONVINCE their TA they knew the concepts taught until that point
It's NOT an exam or test
Had HIGH effectiveness + helped students keep up
100% recommend it
2
4
53
Spent the evening on an idea I thought was novel (coz it sounded unorthodox but interesting) and after a while of searching for the PyTorch code for a tiny part of said "idea", I land on a random project's repo that's ~80% similar ... from NeurIPS 2018 🥲🤕
Defn. scooped /s
12
0
49
Taking an in-depth class in probability this semester and omg so many ML concepts are making wayyyyyyyyyyyyyyyyy more sense after re-reading the chapters 😱😱😱😱😱
The excitement is toooo real. Might write something about it if time allows :O
5
1
47
My 2022 plans:
- master JAX (im v scared of this lol)
- create more model implementations
- write more technical blogs
- read+document more papers properly
- be kinder, hungrier, and curious
(optional sidequest: find a way/reason to visit the US)
3
1
45
Combining DDPMs with Transformers has been done in other areas too!! Really cool work by
@Kevin_E_Wu
@KevinKaichuang
@avapamini
et al. combines the two to generate plausible protein structures. It diffuses over dihedral & bond angles
Paper:
Transformers are all the rage today. But neither DALLE nor Stable Diffusion uses Transformer for image generation. Instead, they rely on a 7-year-old, Jurassic-era neural architecture. Why? 🤷🏾♀️⁉️ It’s finally time for Transformer and Diffusion to join forces! Quick 🧵👇:
23
245
1K
2
9
45
Super cool tutorial and hands-on activities on GNN expressiveness by
@ffabffrasca
@beabevi_
and
@HaggaiMaron
at
@LogConference
!!
1
8
45
In the near future, I wish papers aren't *still* the only way to communicate new ML findings. It'd be awesome to have interactive demos of the NNs doing their thing. The "World Models" paper was super cool at doing that. ()
Looking forward to more like it
2
6
44
The placebo effect of a high refresh rate monitor is amazing, everything feels better:)
226
455
13K
3
1
43
Wew GNNs have finally entered HF territory :D 🙌🏻
🔔 New modality on
@huggingface
's hub: Graphs! 🎇
If you want to experiment, we already have 25 datasets and a model... and we're looking forward to seeing what else the community will add!
Ping me if you need a hand to upload your artifacts 🤗
5
63
326
0
2
41
@deliprao
@jonasgeiping
@tomgoldsteincs
While Cramming was a good read, this one is an equally good paper with similar lessons (minus the scaling laws stuff):
Learned quite a bit when I was playing around with a few NLP projects early this year
2
6
42
1. I dive deep into graphs and GNNs with their equations + drawings on them
2. I go through familiar training procedures for GNNs
3. I use the general GNN framework to introduce SOTA GNNs from popular literature
Please LIKE and SHARE if you find it useful 🙏🏻
2
3
40
✨✨OMGOMGOMGOMG
@ykilcher
covered my graph augmentation library in his ML news episode!!!
🗃Library repo:
📹 Youtube video:
Means so much!! I can finally rest 😩🙌🏻🤝🤘🏻
3
5
41
"Embeddings as a Service" is now an official thing 🤔🤔
We're introducing embeddings, a new feature of our API that distills relationships between concepts, sentences, and even code in a simple numerical representation — for more powerful search, classification, and recommendations.
34
233
1K
4
2
38
Teaching CNNs this week and super grateful that exists ... literal life saver for diagrams 🤝🙌🏻
0
1
40
🚨🚨Some news:
@chaitjo
and I are pleased to announce our work, "Recent Advances in Deep Learning for Routing Problems", has been accepted to the
@iclr_conf
Blog Post Track '22 🥳
We thank the reviewers and readers for their invaluable feedback!
🚨 New blogpost alert, co-authored with
@rishabh16_
:
"Recent Advances in Deep Learning for Routing Problems"
Read on if you are interested in Graph Neural Networks, Combinatorial Optimization, and exciting applications in their intersection! 👇
2
26
155
5
4
38
How do y'all at Google AI decide who your co-authors / research partners are for a specific project?
Suppose someone has a random idea, do they just ask "hey who's interested in working on XYZ together?" on a huge groupchat with all Google AI people from around the world?
1
0
39
What neural networks go through during backprop:
1
4
38
Hmm ... NUS just sent out its "AI Tools Policy" to the entire student body today. It's pretty decent and not super anti-AI or full of weird rules and restrictions (which is great!!)
So the approach seems to be, "just ask your prof if they're cool with it!". Very progressive 👍🏻
2
5
38
Check out this really cool library by
@tyleryep1
that summarises your pytorch models in a really nice way 😲
0
8
36
Don't think I got the memo but I have a strong feeling
@DynamicWebPaige
was behind this and I'm alllll for it :DDD
1
3
37
@thegautamkamath
I made a 1-hour "backprop bootcamp" video about it for the class I was teaching this semester and it received some great reception from the cohort (mostly Year 1/2 undergrads)!
2
2
35
Also, people acting like the creation of a sentient AGI will change the status quo. Like bro your GPT3/DALLE API key hasn’t been granted, what makes you think you’ll have access to AGI if built successfully? It’s gonna be even more inequitable
4
0
36
🔥🎉 It’s an honour to be a recipient of the Gradient Prize 2022! It was super fun writing this extended piece on self-driving car policy string the world and its implications.
Appreciate the editorial team’s deliberation!
Check it out here 👇🏻👇🏻
@mmbronstein
Our first runner up is
@rishabh16_
on self-driving cars. Our judges commented that "engaging with Disengagement elaborates on many of the policy issues that make it difficult to evaluate these systems on real roads."
1
0
11
4
4
36
Experiment Thread:
What happens if you add random Skip Connections (blue) INSIDE a layer between neurons, summing up the connected ones? We then send the sums and unconnected neuron outputs to the next layer.
Turns out, this performs BETTER than its non-intra-skip counterpart
3
9
36
Yo wot literally everyone I know from Canada on Twitter is apparently working at
@CohereAI
now lol
We love to see it 🙌🏻🙌🏻👏🏻👏🏻
2
3
36
🚨🚨NEW PROJECT –– I built a Chrome Extension to render LaTeX on Substack articles ✍🏻🚀
It allows you type in regular LaTeX during editing (wrapped in $$) and once published, renders it all like a good ol' Jekyll blog
Like and share! I'll be releasing this for download soon 🤘🏻
2
3
34
How are papers allowed to have Buzzfeed titles now?
3
0
34