ML Collective
@ml_collective
Followers
14K
Following
274
Media
169
Statuses
797
We are an independent nonprofit organization that believes collaboration opportunities and research training should be openly accessible and free.
Joined April 2020
It's been 4 years since the inception of ML Collective. Through 500+ events, we nurtured ML researchers around the world. We've been operating w/o formally raising any money, but people's awareness and kindness. Today, we finally set up a donation page to help us do better.
3
22
62
We have free GPUs to support your research! . Thanks to a generous donation from @JeffDean, we have 28 GPUs to give away. Here is the full inventory:. (Nvidia GeForce GTX).Titan: 10 .1080: 2.Titan X: 6.Titan Z: 1.Tesla: 9. You are welcome to apply for 1-4 for research use.
17
95
566
8 researchers from Nigeria got accepted into Deep Learning Indaba @DeepIndaba but couldn't afford the trip. Access to research, mentorship, and networking opportunities like this is vital for early-stage researchers. You can help make their trips possible!.
7
196
242
ICML deadline is less than 2 days. If you are in need of some compute to push forward your submission, send an email to compute@mlcollective.org or post on help-compute channel on our Discord, we can help out!.
2
58
202
What's next after open-source and open-access? We believe it's *open collaboration*. The field is fairer, more inviting, diverse and inclusive only when we make mentorship and collaborators accessible. But how? We are discussing it on Tue at #NeurIPS2020:
6
32
192
ICLR 2021 deadline is 10 days away! Working on a draft but short on labmates or peers to proofread? ML Collective can help: send us a draft (review@mlcollective.org) between now and Oct 2 and one of our volunteer readers will read+comment. Undersupported researchers. (1/2).
1
49
181
Dive into the realm of Deep Learning: Classics and Trends (DLCT) with us tomorrow at 10 AM PST! 📷 . Join our discussion on "Understanding & Mitigating Pre-training Noise on Downstream Tasks." . See you there! [ #DLCT #DeepLearning #AI #PreTrainingNoise.
1
20
130
Working on an ML research project but don’t have labmates to show? Show a plot or three in our Research Jam in two weeks!. Also open to folks that want to bounce an idea off others or who just want to hang out and talk shop. Details:.
1
30
94
Do you know ML Collective has an "Office Hours" service? You can book 1:1 chats with researchers who kindly open up their calendars to serve the community. Current ongoing sessions are proudly hosted by @bneyshabur @AndreaMadotto @natolambert
1
15
90
Hi MLC folks and Open Collab advocates, It's been a while since we last hung out! #ICML2021 is next week, and we are having an *Open Collaboration in ML research* social (again!) on Thursday. Our theme is connecting (as usual) and story sharing (new!). 1/.
1
24
76
🎉WE DID IT!! 🎉 All GPUs got delivered at NeurIPS. Remember how it started? 👇 It's been a crazy week finding and coordinating with volunteers. You rock, all!. There are still more sitting in SF waiting to be shipped to members in need. Stay tuned! . But now, let's celebrate 🥳.
5
3
78
The long-awaited DALL-E 2 talk series is finally coming to DLCT! #dalle. Starting this Friday, for 3 consecutive Fridays we are going over the science, engineering, and policy of DALL-E 2. Join the mailing list to stay tuned:.
1
13
68
Anyone in need of a workstation for research? A lab member has generously donated a workstation (specs below). “I bought this machine during my Ph.D. as I didn't have computing resources. I suppose there will be some students who might still be facing similar problems as mine.”.
3
13
67
We had an impromptu hangout at @DeepIndaba ! This is us: ~30 Nigerian students (who rode a 20h bus to Accra) with @savvyRL all shouting "M-L-C!!" (which turns out to work better than "cheese") #Indaba2023 .@abrahamowos @_busayo_awobade
1
16
70
Already 3 ICLR papers we know of that came from collaborations facilitated by ML Collective! 😲. Reach out if there are more you know of, we'd love to collect them and put them in a "feel good" folder for future gloomy days . Thread for those papers:.
2
6
64
Hebbian learning made easy for multi-layer CNNs! 👏👏
Fun little project:. Problem: implementing Hebbian learning in convolutional networks with PyTorch/TF is a pain. Solution: just use a special loss whose gradients turn out to produce exactly the desired Hebbian weight updates!.
0
5
64
[Help Session] It’s grad school application season! We are hosting an impromptu live help session this Saturday! Including:. - group SoP editing. - 1:1 or small group Q&A. - casual hangout. Applicants: . - bring you SoP. - bring your questions. Helpers:. - bring your expertise
2
22
63
Going to NeurIPS next week? Anxiously scraping around for party invites? Rest assured, MLC parties are always open to all (. and scrappy and low budget). We will have a walk and picnic at the park! .
1
7
64
Do you know that the best way to ensure personal research progress is to *set small milestones and deadlines*? . We understand doing research without institutional support is hard. That's why our most successful event series — "research jams" — is here to help set the cadence.
2
3
56
We understand life is busy — but if you booked an Office Hour slot with one of our hosts, please set reminders for yourself, show up, or cancel in advance! We've seen some number of no-shows, and it's a wasted time for the hosts and wasted chance for others who could've used it😢.
1
1
50
Appreciating what @joinClubhouse offers but don't want to get behind its exclusive vibe? . Join MLC’s first *OpenClubHouse* event on our community Discord (NOT on ClubHouse), Sunday March 14, 10am PT! . Join our Discord: and enter "OpenClubHouse" channel.
1
7
42
Read Behnam’s PhD application story here! (Be prepared for some jaw dropping).
In 2010 when I applied for PhD programs, I was a Master student in Iran with no published papers, undergrad GPA C and poor to mediocre English test scores. 2/.
0
2
41
We are discussing CLIP today (in case you didn't realize from the paper title)!.
0
7
42
Our research jam is in two days! Schedule, line-up, and ways to join:. *It's 100% open to the public* . Should be straightforward how to join. But if you are the type of person that needs a calendar invite to remember anything, DM me your preferred email.
0
12
37
Anyone can apply! You don't have to live in the US. We will (hopefully) find someone near your region at NeurIPS to help ship to you. On that note, reach out if you are coming to NeurIPS from a different continent and willing to help with the distribution!.
We have free GPUs to support your research! . Thanks to a generous donation from @JeffDean, we have 28 GPUs to give away. Here is the full inventory:. (Nvidia GeForce GTX).Titan: 10 .1080: 2.Titan X: 6.Titan Z: 1.Tesla: 9. You are welcome to apply for 1-4 for research use.
1
9
37
Independent researchers! Come share your work to receive: . - feedback. - compute assistance. - a sense of community!. Research jam #17 is next week! Submission is open.
2
3
39
Best moment today: ."It's named "Shampoo" because it is about *preconditioning* matrix.".🤣🤣.
Awesomeness with @ml_collective, “Scalable second order optimization for deep learning”. Credit to @_arohan_ & @GuptaVineetG 🤘 🚀.
0
3
35
Tomorrow at DLCT ( we will hear from @lilianweng on "asymmetric self-play for automatic goal discovery in robotic manipulation" as well as her insights on:. - unknown struggles in becoming a researcher. - underrated skills for an ML researcher.
3
3
33
Application form: Please apply before Nov 20!. We will be distributing them at NeurIPS 2022 in New Orleans. If you can't be there in person, we will ship to you.
0
2
38
WE DID IT!!!!!! Thank you all ✨.
Thanks to everyone that donated, we reached and even surpassed our target. We're very grateful to you all.🙏
3
5
35
New year, new jam! 🎆🎇. Research jam—our most famous event—is returning for 2022 this Wednesday. If you are working on an idea, bring it to our group meeting to receive feedback, possible collaborators and compute assistance.
1
8
33
Oh god what just happened to me. Did I have… wait for it…. A NEW LOGO?
1
2
35
[Call for Submissions] Research jam #2 is in two weeks! We invite two types of submissions: "I did a thing" and "I have an idea," with a focus on early-stage, prototype work or unimplemented, big-picture ideas.
1
6
32
Quick summary of our GPU Giveaway:. - We received 28 donated GPUs and 185 applications. - We picked 28 awardees (15%) based on 1) need 2) usability (computer slot & research plan). - 21 brought to NeurIPS. - 3 picked up in person. - 18 taken by volunteers to bring to destination.
1
2
33
It was great to see 60 (mostly) smiling researchers at the Town Hall today! 😃😃😬😃😃😐😃😃. Slides and discussion notes at
1
1
34
Tomorrow at DLCT we are learning about the intriguing "early phase of training," how it's unstable even when using a small learning rate, and how, using large learning rates can improve generalization due to implicit regularization.
1
2
32
Applied for PhD programs this past season? Now that all PhD admission decisions are made, we'd like to hear what your experiences have been. Whether it's a happy, sad, or dramatic story, we are here for you. Join OpenClubHouse #11 (in 10h) to share your story!.
1
2
32
Happy new year! (You are allowed to say HNY however late it is, as long as it is the first time you speak this year, right?). This year we are still committed to democratizing research training for all. To start, we have (and will keep having) an amazing lineup at DLCT.
0
3
31
I am personally (wait, am I a person?) very excited about all the upcoming talks in our reading group! . Thank you to all our past and future speakers.
0
4
30
Do you know ML Collective has a reading group where people from the community gather weekly to read through this awesome book, chapter by chapter? We've had just finished Chapter 10! All meetings are recorded here: Led by the greatest @charles_irl.
My editors just shared with me the feedback from early reviewers and I'm in tears 😭. With the help of so many people, I worked really hard on this book. I'm grateful that people gave it a chance. Read the book online: . Pre-order:
1
9
29
ICLR starts Monday! We have an event page 💕. Stop by our Social on Day 4 and share what you've seen and whom you've met at the conference! #ICLR2021.
0
3
28
Good news! We received a renewed @googlecloud award to support our members' research. If your projects have been shut down due to the unfortunate gap between grants, please let us know and we can help you set up again. Email: compute AT
0
4
29
In 5 hours we will be hosting research jam #7—the most radically open research meeting to get to know what MLC members are working on. It is our mission to make research *process*—not just results—transparent, accessible, inviting and welcoming. Research jams manifest that.
1
1
29
Research jam in ~10 hours! The schedule is online; come join the fun and support independent research!.
1
4
29
Join our ~very~ active NLP interest group to stay on top of papers, LLMs and beyond! .💜 Led by @Aiswarya_Sankar @infiniteakashe
1
5
29
@DeepIndaba This is a full statement from the 8 researchers:. @peculiarabolade @neo_fweranmi @__Oluwadamilola.@manny_uncharted @AdejumobiJoshua @hemhemoh @steveddev @abrahamowos
1
17
29
We silently dropped a new mission statement. Let me know what you think:
1
0
23
Thanks everyone for joining our 1st OpenClubHouse! . We chatted about:.1/ legal aspects on open collab; possibly/impossibly out-lawyer lawyers🤣.2/ skills grad schools don't really prepare you for.3/ recent papers.4/ underrated/overrated research directions.5/ robot overlords 🤪.
1
2
27
A few hundreds new followers have found us over the last week and pushed our followership over 10K — thank you! If you are new and wondering how MLC works, our Town Hall tomorrow is a great chance to catch up. What is MLC Town Hall:.
1
7
26
We are raising $15k to support these researchers to attend @DeepIndaba. This amount will help cover their airfare, visa fees, and housing while at the conference. This sheet lists a breakdown of how the raised fund is going to be allocated:.
1
16
27
To everyone receiving tough news from #NeurIPS2020 and considering resubmitting to #ICLR2021, the abstract is due tomorrow. Our draft proofread and review service can help! Volunteers from MLC and beyond are here with one goal: help improve your work. 1/3.
1
9
27
"Creating noise from data is easy; creating data from noise is generative modeling." . Guess which paper this line is from and you'd have guessed today's topic! #dlct @YSongStanford @poolio .
0
2
26
New Office Hour hosts alert! . So lucky to be doing service to a great community with you amazing people 🤝 @bneyshabur @AndreaMadotto @natolambert @yash_j_sharma @rishiyer @pcastr .
Are you an ML researcher looking for ways to give back to the community? Hosting an office hour with us! . It's really easy: decide on a recurring time with any length and frequency of your choice, create a calendly (or equivalent) link and we can set it up for you.
0
6
25
We are hearing about this paper today at DLCT! Happening in 3 hours.
🆕The Role of Permutation Invariance in Linear Mode Connectivity of Neural Networks. Our conjecture: Taking permutations into account, there is likely no barrier in the linear interpolation between SGD solutions. w @HanieSedghi @osaukh @bneyshabur.1/10
0
8
26
Sometimes our loss functions lead us to sunny, wide basins: @hoony, @zach_nussbaum, @jasonyo, and @Sosanyo followed their gradients to this impromptu MLC lunch in NYC ☀️
0
2
23
We have a YouTube channel! . More content is coming up, but for tonight, listen to @jasonyo explain "What is a RFP" and why it is suitable for distributed research in 80 seconds.
1
3
23
Happening in 3 hours!.
This Friday (since everyone is busy with ICML and I'm pathetically free) I am giving a talk at *my own* reading group -- extreme nepotism or what? 😬. Title: "Unconventional ways of training neural networks and what they teach us about model capacity".
0
1
23
Hey! What progress have you been making with your research? Or in what ways are you stuck? We'd like to hear about them so we know how to best support you!. Time to get together for a jam! Submit your "thing" or "idea" here:
0
5
23
Recording of our research jam #1 is out! . Video: .Event page (with slides): If you enjoy it, consider submitting to our next jam on June 2!
0
2
21
DLCT year finale is tomorrow! Last talk of the year! If you miss it……well there’s always next year. So do whatever you feel like doing. Speaker: @kchonyc . Title: oversmoothing of <eos> in neural autoregressive modeling. Abstract: (see pic). More:
0
3
21
Can you believe I'm only 2 years old? I cannot. I feel 80.
1
1
22
Over the years we’ve unlocked a secret weapon of online talks: having 2+ authors in charge of 1) delivering the talk live & 2) answering questions in the chat AT THE SAME TIME! We might have made a groundbreaking discovery that’s gonna change all future talks! 😬.
1
3
21
At MLC we have lots of ongoing events: regularly-occurring ones like DLCT, Research jam, Town hall, and interest group meetups. We also have random, one-off events because, well, we'd be reading something fun and go "I wonder how it works exactly. Let's have a hacking session!".
1
3
20
We reached our fundraising goal today. Here is a histogram of all 101 donations from 95 donors. Thank you all — you make the world an unbelievably wonderful place!
8 researchers from Nigeria got accepted into Deep Learning Indaba @DeepIndaba but couldn't afford the trip. Access to research, mentorship, and networking opportunities like this is vital for early-stage researchers. You can help make their trips possible!.
1
2
22
Great resource that supports PhD applicants! You are always welcome to contact MLC (hello@mlcollective.org) if you wish to connect with a research member to chat about applications, career paths, specific research areas, etc.
This year, CS PhD applications are different. There are many more 'graduate application support programs' for applicants to get informal feedback (on statements, etc) from current PhD students before formally applying. It's an awesome resource! I've link some below:.
0
2
23
Time to JAM! Submit to research jam on Wed for a chance to:. - tell people about your project. - find collaborators. - get feedback/mentorship. - get sponsored compute. But most importantly, get to feel like a real research member of the MLC community 💞.
0
7
21
**DLCT today** We'll be learning about differential privacy (DP) and how to do that in large language models (LLMs). A "match made in heaven" as said by @thegautamkamath 😻 .
Think privacy in ML ➡️ bad accuracy? 😭. Think again! We show that large language models (e.g., RoBERTa and GPT-2) can be privately fine-tuned with minimal loss in utility! 🥳. A super fun @UWCheritonCS @MSFTResearch collab!. 📝Paper: 🧵Thread ⬇️ 1/8
1
1
22
The future of community is hybrid, and we are committed to not just make it happen, but make it work. Make it work for not just some, but all who thrive with and make thrive a community.
1
1
19
Part of ML Collective and want to help steer it? Or curious where we're heading? Stop by our first Town Hall meeting on Oct 24th to share, to listen, to help create the future!.
0
3
22
Last week we had one of the best talks at DLCT! Learn about loss landscape, basins, permutation symmetry, and the unreasonable effectiveness of SGD:. Thanks for delivering such knowledge and fun to us @SamuelAinsworth.
📜🚨📜🚨.NN loss landscapes are full of permutation symmetries, ie. swap any 2 units in a hidden layer. What does this mean for SGD? Is this practically useful?. For the past 5 yrs these Qs have fascinated me. Today, I am ready to announce "Git Re-Basin"!.
0
3
20
Failing to attend #ICML2021? 😅. Don’t worry, the MLC social is only thing you need to take home with! Starting in 9h (8am PT/11am ET). Full schedule is online:.
Hi MLC folks and Open Collab advocates, It's been a while since we last hung out! #ICML2021 is next week, and we are having an *Open Collaboration in ML research* social (again!) on Thursday. Our theme is connecting (as usual) and story sharing (new!). 1/.
3
5
20
Heard from a #dlct speaker that "I had the best audience in years" "It was a talk that I will remember" 😭😭😭 Thanks all for being the best audience ever! We don't know how that happened -- we never curated the audience; it's a group anyone can join from anywhere in the world.
1
1
20
OpenClubHouse #9 is canceled today, sadly, due to certain key personnels missing. Last time seen: Friday 3:04pm PST. @Luke_Metz might have a clue of their whereabouts 🤔🧐. Hope y’all have a wonderful rest of your Sunday!
0
0
21
We are a ~warm~ “collective” and we just reached 900 followers! You guys are real, right?.
There's a lonely feeling doing sth entirely by yourself, unsupported, unseen, unheard. Whether it's founding a company, managing a team, organizing an event. The warmest prefix in the world is "co-". The warmest word "collective" (OK I AM BIASED). Best phrase, "partner in crime".
0
0
19
Update: in 5 days we received astounding 155 requests for our 28 GPUs. To accommodate everyone's need as much as possible, we are expanding the program to include a "Request for GPUs":. Do you have unused or retired GPUs to donate to those in need? 🙏.
We have free GPUs to support your research! . Thanks to a generous donation from @JeffDean, we have 28 GPUs to give away. Here is the full inventory:. (Nvidia GeForce GTX).Titan: 10 .1080: 2.Titan X: 6.Titan Z: 1.Tesla: 9. You are welcome to apply for 1-4 for research use.
3
6
19
This is our first ever crowdfunding! Would appreciate RTs :) .@forai_ml @MasakhaneNLP @arabicml2 @BigscienceW @WiMLworkshop @wimlds @black_in_ai @QueerinAI.@timnitGebru @shakir_za @sarahookr @avishkar58 @siminyu_kat @A__Diack @seb_ruder @alienelf @vukosi @nakadori ❤️.
0
8
20
Score-based generative modeling slides from @YSongStanford is up on our website! *Beware, Large File* . One of the best talks ever! ❤️.
"Creating noise from data is easy; creating data from noise is generative modeling." . Guess which paper this line is from and you'd have guessed today's topic! #dlct @YSongStanford @poolio .
1
1
18
We are hiring an Executive Assistant! Starting part-time (with a possibility to grow into full-time), can work from anywhere, and with competitive pay and growth opportunities! Reach out if it sounds like you or someone you know!.
3
6
18
Our crowd-sourced collection of resources on "how to grow as a researcher" is getting some traction!. What should we add to the list?
This is a great post on some things to keep in mind while conducting research! @ml_collective .
1
6
19
Today's #DLCT features this great work unifying hyperarameters across architectures and how to tune any large model by just tuning a small version of it! Brought to you by @TheGregYang in 2.5h!.
1/ You can't train GPT-3 on a single GPU, much less tune its hyperparameters (HPs). But what if I tell you…. …you *can* tune its HPs on a single GPU thanks to new theoretical advances?. paper code blog
2
3
19
Our NLP reading group is 1.5yo and growing strong!.
In honor of our 1.5 year anniversary, the NLP Reading group is in full swing for the rest of 2022 - join us for. - Aug - Sept: LLM limitations .- Oct: Open Dialogue systems.- Nov: Conditional information retrieval.- Dec: Fact verification.- Jan: Open Domain QA.@ml_collective.
0
1
19
We just heard that two accepted ICLR 2022 Blog Posts came out of our NLP interest group alone! So we updated the "celebration" slide used at the Town Hall. Keep doing cool stuff and let us know at the Town Hall!
1
1
18
MLC compute grant for 2023 is renewed! For those who already have a project set up with us, you should've received an email about how to un-pause. For anyone wanting to start a project using our compute, email us! Read more:
0
0
17
Anyone wants to lead a hackathon to play with these language models? . We can supply:. - GCP instances, with credits. - a room of helpers. You need to:. - read the README, follow the steps. - be happy to share your screen so others can learn and play with you!.
A big step! Congrats all!!. If anyone in @ml_collective wants to try out these models with GCP and Cloud TPU, we can easily set up a hackathon session, with hopefully some consultations from you guys if running into problems? @jamesleethorp @ada_rob @LiamFedus ?😊.
1
1
19
We have decided to award this workstation to @tomatosaucedo to support her psychology+ML research! Congrats, Jennifer!.
Anyone in need of a workstation for research? A lab member has generously donated a workstation (specs below). “I bought this machine during my Ph.D. as I didn't have computing resources. I suppose there will be some students who might still be facing similar problems as mine.”.
1
0
17
We read papers that reviewers may not want you to read! 🤫.
Reviewer 2 complains it has "too many ideas". @prfsanjeevarora says "gave me some new ideas". This Friday at Noon PT… come hear about the paper that reviewers don't want you to read! I'm speaking at @ml_collective's DLCT:
0
0
16
It is beyond satisfying to see that all presenters tomorrow are *returning* presenters. It's like running a local convenience store, and the true rewarding feeling comes when you've run it long enough to start seeing familiar faces every time 😊.
1
0
17
After the NeurIPS fun, time for some serious research updates! If you are working on a project and could use some feedback and compute, signing up for a short presentation at our [check notes] last of the year Research Jam is a good idea!.
0
5
18
ICLR social “Open Collaboration in ML Research” is a week away! Submit an RFP before May 1 (Sat) to earn a spot to speak at our event!.
0
7
16
We are officially one of the "weird misfit organizations at the intersection of research and technology" (c/o @_joelsimon) 🙌.
0
0
15
Hi 200 people who responded to our #NeurIPS2022 picnic lunch + river walk tomorrow (Thursday)!. We updated the exact meeting point and time (it's now 12:30-2 so you won't miss poster session). Can still RSVP! Or not—just show up and meet us there🚶♂️🚶🚶♀️
1
1
18
Happening in 1 hour!!! . All the links to join are available on website:
What's next after open-source and open-access? We believe it's *open collaboration*. The field is fairer, more inviting, diverse and inclusive only when we make mentorship and collaborators accessible. But how? We are discussing it on Tue at #NeurIPS2020:
0
3
18