Shashank Rajput
@shashank_r12
Followers
684
Following
566
Media
7
Statuses
169
Explore trending content on Musk Viewer
Kamala
• 4296149 Tweets
The Loyal Pin Trailer
• 510735 Tweets
#HouseOfTheDragon
• 286214 Tweets
Rhaenyra
• 163811 Tweets
Manchin
• 148539 Tweets
Seasmoke
• 88030 Tweets
#GranHermanoCHV
• 78858 Tweets
東海道新幹線
• 78276 Tweets
Daemon
• 72918 Tweets
Serena
• 66436 Tweets
Leah
• 62315 Tweets
Jana
• 61028 Tweets
#LaCasaDeLosFamososMx
• 58375 Tweets
Punk
• 53938 Tweets
Mysaria
• 53852 Tweets
Kordell
• 49572 Tweets
Alicent
• 48928 Tweets
Addam
• 30473 Tweets
Kendall
• 28703 Tweets
ポポロ休刊
• 27566 Tweets
भगवान शिव
• 25221 Tweets
#LaAcademia
• 24654 Tweets
#ピザBENTO
• 21312 Tweets
Paola
• 17986 Tweets
Yuhui
• 14264 Tweets
北陸新幹線
• 13892 Tweets
電波人間
• 13753 Tweets
Adrián Marcelo
• 13591 Tweets
Harrenhal
• 13218 Tweets
奈良の鹿
• 11578 Tweets
To all my friends: Over the past 3 months, whenever I said "Sorry, can't make it tonight, gotta work" or "Sorry, I'm busy this weekend", but then couldn't really say what exactly we were working on, THIS was the monster we were building
@DbrxMosaicAI
.
#DBRX
Meet DBRX, a new sota open llm from
@databricks
. It's a 132B MoE with 36B active params trained from scratch on 12T tokens. It sets a new bar on all the standard benchmarks, and - as an MoE - inference is blazingly fast. Simply put, it's the model your data has been waiting for.
33
266
1K
13
12
168
Thank you
@DimitrisPapail
for being such an amazing advisor!!! Your help and guidance have been invaluable throughout my phd! I am also extremely lucky to have had the opportunity to collaborate with some really brilliant researchers from various universities and organizations!
PapaiLLM lab represent. Congrats to
@shashank_r12
for defending a 600 page tour de force of a thesis!!
1
4
59
2
3
65
Our paper was recently featured in the
@tldrnewsletter
: !!!
Feeling lucky to have collaborated on this project with
@madiator
,
@_nikhilmehta
,
@YiTayML
,
@vqctran
, and other amazing people at Google Brain!
Happy to share our recent work "Recommender Systems with Generative Retrieval"!
Joint work with
@shashank_r12
,
@_nikhilmehta
,
@YiTayML
,
@vqctran
and other awesome colleagues at Google Brain, Research, and YouTube.
Preprint:
#GenerativeAI
🧵 (1/n)
13
73
480
1
4
30
Thank you
@sopharicks
for inviting me for the talk! It was great talking to you and the other members of BuzzRobot!
It was great to have
@shashank_r12
share with the BuzzRobot community details about DBRX, the large language model created by
@databricks
. Shashank walked us through the architecture of the model, hyperparameter choices, the software and hardware issues the team experienced
0
2
14
2
3
24
So true!!! 🥲
1
0
24
A big advantage of two column paper format is that people can comfortably read your paper on their phones. Kind of embarrassing that I only realized this today, after years of reading papers on my phone 😅
2
1
19
@OfirPress
@BlancheMinerva
@xlr8harder
In our (preliminary) experiments we also see ALiBi and RoPE have matching training curves (in fact ALiBi converges a bit faster initially). Performance is also similar for eval on seq lens less than the max seq len seen during training. After that, ALiBi extrapolates better.
4
1
16
@DbrxMosaicAI
Feeling ecstatic that all the hard work by the team payed off! It was the greatest experience working with all the talented and hardworking folks
@DbrxMosaicAI
! Looking forward to building even bigger and better LLMs!
0
0
11
@HongyiWang10
is one of the best researchers that I've worked with. I was really lucky to have him as a senior phd student in the lab when I started my phd. He is one of the few people I know who has comprehensive expertise in both ML and Systems. Congratulations Hongyi!!!
[1/n] I'm thrilled to share that I will join the Rutgers CS Department
@RutgersCS
as a tenure-track Assistant Professor in the summer of 2025!
I'm excited about and looking forward to this new chapter of my career journey!
42
8
263
1
1
9
Even GPT2-chatbot isn't immune to the Sharma-ji-ka-beta syndrome
1
0
6
@DimitrisPapail
@OfirPress
@BlancheMinerva
@xlr8harder
Yes, but we observed that it is very sensitive to learning rates.
0
0
4
@srchvrs
@madiator
@tingchenai
@_nikhilmehta
@YiTayML
@vqctran
Yes, the amazon dataset is a particularly difficult dataset in terms of recall. Other product recommendation datasets like REES46 and YOOCHOOSE are 'easier' where a recall of ~0.5 can be reached.
0
0
5
1
0
5
@Gradient_AI_
@AIatMeta
@huggingface
@CrusoeEnergy
Wow! Amazing work! It seems that you used 2.8 Billion as the RoPE theta (), which is much, much bigger than any RoPE thetas seen in other models. How did you come up with that value?
1
0
5
@DimitrisPapail
@mraginsky
You should join us at
#COLT2023
in Bangalore if you want to try some more 😋
2
0
5
0
0
4
Do single author papers qualify to have a Discussion section?
1
0
3
@JeffDean
@DimitrisPapail
Thank you
@JeffDean
! And thank you
@GoogleAI
for the opportunity! Looking forward to a productive research collaboration with Google!
0
0
3
Can we bring the mullet back into fashion? Cutting hair on back of the head is difficult!
#Quarantine
0
0
3
If WeChat is banned then how will Chinese people working in the US communicate with their families back home? I've been told of alternatives like QQ and Skype, but what is the guarantee that the US or even China (in retaliation) wouldn't ban these in the future!
0
1
2
@zeroXmusashi
@madiator
@_nikhilmehta
@YiTayML
@vqctran
Yes, in order to use this to build a recommender system for a dataset, you only need two things: user session data, and some semantic data about each item. For tiktok, the latter could be stuff like video title, tags, caption or creator's name.
1
0
1
@unsorsodicorda
@aminkarbasi
@DimitrisPapail
@ten10_93
Yes, this paper by Park et al. - - provides results for memorization using ReLU networks, and uses a separation assumption similar to ours.
1
0
1
@DimitrisPapail
Thank you
@DimitrisPapail
, it has been great working you! Looking forward to a lot more of productive and fun research ahead!
0
0
1
Congratulations Dr. Hongyi Wang!!!!
@HongyiWang10
My first PhD student defended today, and it filled my heart with bittersweet joy.
Congratulations Dr. Hongyi Wang
@HongyiWang10
!
It has been an incredible honor to serve as your advisor. I can't wait to see the great things you will accomplish.
10
1
352
1
0
1
@mvpatel2000
Keeps selecting the same person for random search because nobody bothered randomizing the global seed.
0
0
1
@unsorsodicorda
@aminkarbasi
@DimitrisPapail
@ten10_93
Great question! We have a margin assumption on the points, and in fact, the "exponential improvement" is in the dependence on the margin.
Indeed, modern DNN can interpolate training datasets, and this was one of the motivations for our work.
1
0
1