Humans and animals learn visual knowledge through continuous streams of experiences. How do we perform unsupervised continual learning (UCL) in the wild?
Yipeng's latest paper reveals three essential components for UCL success in real-world scenarios: Plasticity, Stability, and
What should you care about when continually improving your model’s representations with self-supervised learning? Check out our paper titled *Integrating Present and Past in Unsupervised Continual Learning* to appear at
#CLVision2024
and
#CoLLAs2024
!
1/🧵
Wondering about how to train deep neural networks without backprop? Check out our ICLR 2023 paper:
Forward gradient computes gradient information from forward pass. But it is slow and noisy — it computes the directional gradient along a random weight
I am looking for incoming students at NYU Courant CS or Center for Data Sci starting fall 2022. If you have interest in working with me on topics like meta/continual/embodied/representation learning+vision, please reach out & mention me in your PhD application. Deadline is Dec 12
Dalle 3 generates watches and clocks pointing at 10:10 and fails to generate other times because almost all of the product images advertise timepieces using 10:10 since it is more visually appealing.
#dalle3
#GenAI
China, a country that produces one of the most papers in AI, will be underrepresented in today’s
#NeurIPS
in New Orleans, due to their zero-covid policy, the most draconian in the world. While I am boarding on a plane, many Chinese are still locked in their buildings for months.
ML in the past: Loss function, convergence, learning rate, generalization, complexity, gradient, variance, normalization, duality...
ML today: Calling a fine-tuning API and then a completion API. Learning how APIs work to keep myself up-to-date.
Although ConvNets are already translation-equivariant, we found that a general motion-equivariance objective actually guides the network to learn semantically meaningful representations. For more details, check out our latest
#ICCV2021
paper:
Curious about self-supervised learning from raw videos in the wild? Our
#ICCV2021
paper FlowE proposed a Flow-Equivariance objective that leverages moving objects, useful for segmentation/detection. Check out paper+video for detail:
@RaquelUrtasun
@mengyer
Our NeurIPS 2019 paper "Incremental Few-Shot Learning with Attention Attractor Network" studies how to learn few-shot new concepts on top of previously learned old classes by meta-learning a regularization function. Paper: GitHub:
🚨 New Research Alert!
People have found safety training of LLMs can be easily undone through finetuning. How can we ensure safety in customized LLM finetuning while making finetuning still useful? Check out our latest work led by Jiachen Zhao!
@jcz12856876
🔍 Our study reveals:
With 1000+ accepted paper in NIPS, it's virtually impossible to go through the whole list without questioning the meaning of life... We should have some cluster based visualization tool to help us navigate... Or, an auto generated news article highlights the latest breakthroughs.
Introducing LifelongMemory, an LLM-based personalized AI for egocentric video natural language query (NLQ). This amazing work is led by Ying Wang
@yingwww_
When AI is used by totalitarian regimes to censor, silence, and purge people everyday, as AI researchers, what can we do and what have we done? There is a fine line to be cut, and the future of humanity depends on it.
Why you should care about ongoing 🇨🇳 protests?For years, 🇨🇳 has been using AI to censor msgs &posts, and prosecute ppl w/ diff political views,or ppl who just wanna end zero-covid. You should care b/c the cool things you created—transformers, CNNs—are likely held in evil’s hands.
China, a country that produces one of the most papers in AI, will be underrepresented in today’s
#NeurIPS
in New Orleans, due to their zero-covid policy, the most draconian in the world. While I am boarding on a plane, many Chinese are still locked in their buildings for months.
China’s zero-covid policy is causing a humanitarian catastrophe. Ppl locked in homes for 100+ days. 10 died in a fire and firefighters couldn’t get thru blockades. 27 died in a bus to covid camps. Ppl starving, ppl dying. Now protest has begun. Let’s end zero-covid, end the CCP.
Can we ask self-driving cars to pay attention? Our recent ICRA'21 paper proposed a learned attention mechanism that is not only more efficient but also much safer! Joint work by Bob
@zengwenyuan1995
@binyangderek
Ming
@RaquelUrtasun
Humans learn new knowledge and concepts continually, incrementally, with only a few labeled examples, in a dynamically changing environment. Yet standard machine learning emphasizes "train-then-evaluate" which is inflexible to adapt once a model is trained.
1/10
Today 10:30AM ET: I will give a tea talk
@MILAMontreal
on "Towards continual & compositional few-shot learning". You can join livestream or watch recording here:
Check out our ICLR 2023 workshop on Scene Representations for Autonomous Driving. We have an amazing line of speakers below. See you in Kigali, Rwanda!
Excited to announce the ICLR 2022 workshop on Scene Representations for Autonomous Driving! Submission: (by 15.02.2023)
We're accepting contributions in the form of blogs, repos, or PDFs. We also have a track for commenting on others' work, check it out!
Code for our ICML 2018 paper "Learning to Reweight Examples for Robust Deep Learning" is shared here: Many thanks to co-authors and co-workers at Uber.
Congratulations to
@nicvadivelu
on receiving the Jessie W.H. Zou Memorial Award! I am honored to be his mentor at Uber and super excited to see his future trajectory as a rising star in the field!
Huge congratulations to Nicholas Vadivelu (
@nicvadivelu
), an exemplar
@UWCheritonCS
undergrad, who just won the Jessie W.H. Zou Memorial Award for Excellence in Undergraduate Research!! Time for a
#ResearchIntro
1/4
Can we push back
#neurips
deadline? In the midst of this global pandemic, the least we want is to increase stress on our already fragile mental state in an endless isolation. Also
#icml2020
cycle has been delayed. Show some coordination among big conference organizers?
After such rebuttal hustles, I realized that maybe it's to the best interest for both
#neurips
and myself to reject my papers, so I can save them for a non-virtual conference in the future.
Publishing a paper in the old days: Rounds of feedbacks/edits and eventually get accepted. Publishing a paper today: Open up the lottery ticket to see what are the scores, and withdraw and re-submit to the next conf deadline in a week.
Check out our ICML workshop on Localized Learning! We have an amazing line of speakers:
@geoffreyhinton
,
@irinarish
, Edouard Oyallon, Claudia Clopath,
@timos_m
, Qu Yang, and Stephen Gou! Our workshop paper deadline is May 24 (1 week after NeurIPS). See you in Hawaii!
This is an extraordinary, historic moment in China
Protests are breaking out across the country-from Beijing, to elite colleges, to other major cities, and even far flung places
Shocking to hear people chanting for Xi to step down
This is people past their breaking point
@cnn
Inspired by how humans use sketches to communicate novel visual concepts, our SketchEmbedNet learns to draw and can be used for few-shot learning. We also show interesting properties like linear embedding composition. arXiv: GitHub:
Aloha! Our ICML workshop on localized learning is happening this Saturday at room 310. See our official ICML workshop page for a detailed schedule! We have an amazing line of speakers and panelists -->
Check out our ICML workshop on Localized Learning! We have an amazing line of speakers:
@geoffreyhinton
,
@irinarish
, Edouard Oyallon, Claudia Clopath,
@timos_m
, Qu Yang, and Stephen Gou! Our workshop paper deadline is May 24 (1 week after NeurIPS). See you in Hawaii!
China has officially given up its notorious zero-covid policy—a huge political progress for the protesters. As China is opening up, let’s remember the people in Shanghai, Xinjiang, and many other cities have been locked in their homes for months, all for nothing.
Hello world! 🎉 We are the Z-Group a.k.a. Rich Zemel’s lab at
@UofT
and
@VectorInst
! Excited to join the party on Twitter. We’ll be posting about ML papers, as well as other fun things that come to mind 🧠
Check the next tweet to follow some students in our lab:
Before desk rejecting papers with format issues, please provide a format checker next time to the authors in the portal. This is a new process introduced this year and it’s unfair to authors since we don’t understand which rules are flagging and there is no warning.
@NeurIPSConf
“To start a PhD in ML, without insider referral, you need to do work equivalent to half of a PhD.
Hence, in Apr 2019, I decided to dedicate all my time until Jan 2020 to publish in either NeurIPS or ICLR.
If I fail, I would become a JavaScript programmer.”
—
@andreas_madsen
‼️
Towards more interactive
#selfdriving
, we propose a new motion forecasting network based on the transformer architecture to explicitly model interaction among actors. Check out our recent IROS'20 paper, available on arXiv:
#SelfDrivingCars
@uber
@uberatg
Wondering about self-driving vehicles encountering unseen animals on the road? Our CoRL paper studies how to recognize unknown objects in the open world. Joint work with Kelvin Wong, Shenlong Wang, Ming Liang and Raquel Urtasun.
Most of our next generation will probably not understand why communicating with computers in human language was so non-trivial back in the days. Just like we are so used to take consumer electronics for granted.
In standard few-shot learning (FSL), an elephant is always an elephant no matter which episode it is. Check out our new paper that extends FSL to more flexible classification criteria -->
Check out our new paper Flexible Few-Shot Learning -- the same object can belong to different classes depending on context. We found unsupervised representation is better than supervised. A short version at NeurIPS metalearn workshop today at 10 EST.
Our recent paper features a natural few-shot learning paradigm that continually introduce new objects in a sequence, such as when wandering through a new environment like walking room-to-room in a new house. Code & data:
@mengyer
Continuous vs discrete, local vs centralized, exploration vs exploitation… They are not only pairs of opposite concepts and tradeoffs, but also pairs of one concept embedded in another repeated over many layers.
ICLR score ladder designers and reviewers, score ONE, it's the lowest positive integer out of 10, do you really mean it? To me, it's like saying this paper is absolutely trash.
It is scary if an entire population uses one proprietary chat software to have daily private conversations. Switching cost is too high for individuals. For cell phone, you can switch service provider and still talk to anyone. Shouldn’t we have open protocols for our digital life?
Imagine a cloak can make you invisible? We build a car-top that can hide cars from LiDAR detectors. Let's start making self-driving safer and more secure. Work by James Tu and other amazing collaborators!
Do we still need SGD/Adam to train neural networks? Based on our
#NeurIPS2021
paper, we are one step closer to replacing hand-designed optimizers with a single meta-model. Our meta-model can predict parameters for almost any neural network in just one forward pass. (1/n)
In 2023, the role of photography is getting redefined. Whereas AI generates near perfect images, Apple’s new Vision Pro lets us immerse in historical moments. To many people, photos are visual souvenirs that activates our memory. But how do we preserve our digital memory? A
gmail has been aggressively classifying emails as spam these days. A lot of surprise when I opened up my spam folder. Even for replies to my original email? Such a false positives rate seems like something is wrong...
Testing
#GPT4V
with the Stroop color-word test -- Impressive performance but not quite there. Making more sequential attentional mistakes rather than color-word interference.
Using national security as a disguise for making discriminatory laws is unacceptable. See Science article: New Florida law blocks Chinese students from academic labs
Why is today's authentication technology a disaster? To sit down at my desk, I
1. Transfer my coffee and food to my left hand, and complete the rest of the tasks with another hand
2. Fail to unlock my iphone with a mask for 3 seconds and enter password with my right hand
Can we make unsupervised learning not backprop end-to-end? In our paper , we propose LoCo that matches end-to-end BP using SIMCLR, unlocking the potential for model parallelism + memory savings + bio plausibility.
@mengyer
@RaquelUrtasun
@UberATG
@Uber
Just realized how boring our digital lives are: Everyone has the same GUI, same phone, same email, same calendar, same msg, same website, etc. the list goes on. The most you get is changing the bg img or colors. When will design be decoupled from tech products?
Protip: When plotting a training curve, do not include all of your 1,000,000 training step data points in a vectorized figure in the paper pdf -- it will clog up your pdf reader and your printer.
@savvyRL
Yeah, but I still think there is also the question of dataset size. I don't think the usage of the word has changed much from before -- pretraining on ImageNet classification and finetuned on some other domain specific image classification dataset also meant the same learning
New preprint on learning to reweight training examples w/ Wenyuan, Bin and Raquel. More robustness towards label noises and class imbalance by comparing the grad. direction of clean and noisy inputs.
Progress I made over
#quarantine
: Developed a taste for cheap instant coffee. Over the past 5 months (2 cups per day), the money I spend on coffee is less than a dozen
#Starbucks
.
TFW the reviewers say that there are minor flaws but not pointing out where are the flaws... that the idea is not novel and not SOTA but not giving the actual references... How about just have an option: "not in a good mood of accepting this paper"?
UofT: please cancel in-person classes. The Canadian government: please shut down entrance from countries undergoing outbreak. Why does Canada always have to be the slow mover, again?
1/ Spent the last couple weeks in quarantine obsessively coding a website for Virtual ICLR with
@hen_str
. We wanted to build something that was fun to browse, async first, and feels alive.
@ylecun
@MetaAI
@paperswithcode
Very cool work! Could Galactica potentially plagiarize existing papers, like in OpenAI’s codex that copied other people’s licensed code line-by-line? Or what if two people used the same tool and wrote two similar papers?
This blog introduces Uber ATG Toronto’s recent research on SBNet, leveraging block activation sparsity for speeding-up CNN inference. Our customized CUDA kernels are also open-sourced on GitHub (). via
@ubereng