Jerry Li
@jerryzli
Followers
3,455
Following
169
Media
14
Statuses
194
Senior researcher at @MSFTResearch . Machine learning, esp. robust and principled ML. @MIT Ph.D '18. More importantly, I once hit challenger in TFT 😎
Seattle
Joined December 2011
Don't wanna be here?
Send us removal request.
Explore trending content on Musk Viewer
Vance
• 2396724 Tweets
Vance
• 2396724 Tweets
#VPDebate2024
• 332546 Tweets
Pence
• 136253 Tweets
Minnesota
• 134115 Tweets
Tampon Tim
• 96367 Tweets
Tiananmen Square
• 65895 Tweets
महात्मा गांधी
• 64225 Tweets
LINGLING MYCHERIE AMOUR EP4
• 62996 Tweets
からくりサーカス
• 61958 Tweets
गांधी जयंती
• 59758 Tweets
#原神KFCコラボ
• 57910 Tweets
लाल बहादुर शास्त्री
• 56557 Tweets
#GandhiJayanti
• 55539 Tweets
ケンタッキー
• 51354 Tweets
Obamacare
• 32481 Tweets
ニンテンドーミュージアム
• 28322 Tweets
リネット
• 26305 Tweets
Timmy
• 23926 Tweets
Vladimir
• 23703 Tweets
宮崎空港
• 17545 Tweets
ネットオーダー
• 17181 Tweets
原神コラボ
• 13907 Tweets
#LalBahadurShastri
• 11043 Tweets
Pinned Tweet
Hello! For my biennial post here, a bit of a personal update: after 5 great years at
@MSFTResearch
, this fall, I'll be joining the CSE faculty at
@UW
.
Much love to the great folks at MSR I've had the pleasure of working with.
1/2
13
10
340
First lecture is online!
Lecture 1: Introduction to robust statistics
2
63
376
Hi! So I was coerced into writing this thread---er I mean let me tell you a little bit about quantum learning and testing, and about some of the papers
@sitanch
, Jordan Cotler,
@RobertHuangHY
and I just posted on arxiv:
1/
3
5
79
Lecture 2: Total variation, statistical models, and lower bounds
1
10
74
Lecture 3: Robust mean estimation in high dimensions
I'm pretty sure I bungled some constants in the proof, but oh well...
0
9
50
Understanding adversarial examples is not only an important theoretical problem, but they also pose security risks to systems using AI as a component. We consider the real world impact of adversarial ML, and how systems designers can mitigate them, and how researchers can help.
1
9
47
@ccanonne_
However, note that this effort may have been abandoned due to the author becoming addicted to various video games
2
0
46
When can you learn a simple transformation of a Gaussian in high dimensions? We give the first nontrivial end-to-end guarantees for this basic problem, which also has applications to learning deep generative models.
Massive props to
@sitanch
for hard carrying, see his thread ⬇️
0
0
43
Slightly belated, but happy to announce a new paper with
@sitanch
, Brice Huang, and Allen Liu. We give tight lower bounds for basic quantum property testing problems without quantum memory.
tl;dr Non-adaptivity is all you need, a thread 🤪 1/n
2
5
40
First time back in the office and immediately having technical issues with my computer.
So nostalgic 🥲
1
0
35
Obivous bias aside: this talk was one of a number of exciting talks at the joint
@MLFoundations
/
@SimonsInstitute
workshop co-organized by Adam Klivans, Tselil Schramm, and myself. Check out the full list of talks and speakers here:
Just watched an incredible talk by
@AlexGDimakis
at the Simons Institute, highly recommended. Their Iterative Layer Optimization technique to solve inverse problems with GANs make a LOT of sense!
The empirical results on the famous blurred Obama face speak for themselves!
1/4
3
78
457
0
6
33
Lecture 5: Efficient filtering from spectral signatures.
The first "payoff" lecture! We give a super simple (~8 lines!) poly-time algorithm for robust mean estimation, with a fully self-contained analysis.
0
3
34
New video: A short proof of the spectral signatures lemma.
Alternative title: Jerry learns that manim is a thing, thinks it's amazing, and obsessively messes around with it all week. But hey, this time, all the constants are correct!
1
5
32
Lecture 4.5: Finite sample concentration with bounded second moments.
Sorry for the weird timing, youtube hates me sometimes! Also: can you spot all of the bungled constants??? 🤔
3
5
30
Lecture 4: Spectral signatures and efficient certifiability.
More bungled constants...it seems that this will be a trend ¯\_(ツ)_/¯.
0
6
29
for unknowable reasons sometimes youtube decides that it wants to upload exactly 67% of my videos...sorry about the delay, but the next video lectures are coming!
2
0
27
If you're a talented student interested in a nonempty subset of {TCS, quantum, machine learning, AI}, please consider UW! It's a very exciting place for all of these areas!
2/2
0
3
27
I'm generally planning on posting my lectures twice a week, but for my own sanity I'm only planning to do 1 per week (hopefully on Mondays) until the NeurIPS deadline.
1
0
25
A bit late, but let it be written here for the record that a couple of weeks ago,
@SebastienBubeck
promised, in front of witnesses (
@TheGregYang
@hadisalmanX
), to dye his hair after his third child
@ilyaraz2
3
2
19
I'm planning to focus only on robust statistics, so I won't cover everything I covered in the class. However, this allows me to cover more topics in this area than I could during the class, including robust covariance estimation, linear time algos, and SoS-based methods. (2/3)
1
2
18
@thegautamkamath
how odd, I spent the whole day reading physical copies of papers I have to review and yet my stress level has increased dramatically 🤔🤔
1
0
15
@thegautamkamath
@eytyang
Whoa I won't stand for such blatant lies. I definitely provided citations.
0
0
13
Big shout-out to all my co-authors, they are all insanely talented. Fun fact: Brice and Allen didn't know any quantum at all, and within a month or so of working with them, we had a working proof of this lower bound, which had eluded
@sitanch
and me for nearly 2 years...🙃 9/9
0
0
12
First off, what is a quantum distribution? They're known as "mixed states", and they are any d-dimensional PSD matrix with trace 1. Notice the eigenvalues of this state form a probability distribution. You can think of them as rotations of distributions in d-dim space.
2/
1
0
10
We also show that limited amounts of quantum memory are not sufficient for some of these tasks, and we show hierarchies for the power of entangled measurements. The techniques are surprisingly elementary, and we think may have application to classical learning!
10/
1
0
9
Big shoutout to my coauthors
@sitanch
, Jordan Cotler, and
@RobertHuangHY
, they're all super great and you should hire them (although
@hseas
already claimed
@sitanch
).
11/end
1
0
8
How do you interact with a mixed state? In the classical world, we get samples from our distribution. Quantumly, one designs a measurement scheme known as a POVM, and them measures the mixed state in this POVM. The outcome of this is a draw from a classical distribution.
3/
1
0
8
In our new papers, we show *exponential* separations for a number of learning and testing tasks, including shadow tomography, purity testing, and some channel testing tasks. In fact, sometimes there are exponential separations even with very mild entanglement.
9/
1
0
8
We show that the answer is no: adaptive algorithms still need Θ(d^3/2) samples. We construct a new hard instance based on Gaussian perturbations. We do this because unlike previous instances, the likelihood ratio for this instance has a really elegant self-similar structure. 5/n
1
0
7
Previously,
@SebastienBubeck
,
@sitanch
and I showed that for the task of mixedness testing, there are polynomial gaps between the power of entangled and unentangled measurements. But are there tasks with much larger separations?
8/
1
0
7
In the same way that n draws from a classical distribution can be thought of as one draw from the joint distribution, n copies of a mixed state can be thought of as one big tensored up mixed state. One can then apply an arbitrary POVM to this entire tensor!
5/
1
0
7
This allows us to do measurements that we can't do just by applying POVMs to each individual state. Such entangled measurements are quite powerful, many algos use them to get optimal statistical rates see e.g.
@BooleanAnalysis
and John Wright's amazing papers.
6/
1
0
6
@thegautamkamath
mfw I remember twitter exists and decide to check it for the first time in weeks
0
0
6
But they also come at a cost: they require the algorithm to store many copies of the mixed state in memory simultaneously. They're also complex to form. In our papers, we ask if they're truly necessary for a number of basic quantum learning tasks.
7/
1
1
6
There are two key differences between classical and quantum learning. First, notice that the measurement operation is inherently interactive, so measurements can be chosen adaptively, which is not true classically. Also, measurements can be entangled.
4/
1
0
6
For state certification, we generalize the prior instance-optimal bounds of
@BooleanAnalysis
,
@sitanch
, and myself, and show that they hold for arbitrary adaptive measurements, not just non-adaptive ones. Here too, you seem to get the same bounds with and without adaptivity. 7/n
1
0
5
This lets us boil the question down to controlling the behavior of a certain matrix martingale, which we can do using well-known techniques from matrix concentration inequalities. 🤯 6/n
1
0
5
It's "folklore" that if these measurements are specified ahead of time, that you need Θ(d^3/2) samples. More recently,
@SebastienBubeck
,
@sitanch
, and myself showed that even adaptive algorithms require Ω(d^4/3) samples.
But this left the question: does adaptivity help? 4/n
1
0
5
@thesasho
@BooleanAnalysis
this reminds me of when I was learning probability theory my prof tried to use Av notation for Markov chains and at some point literally threw his notes in frustration and walked out of class...good times
0
0
5
We consider mixedness testing and state certification, which are the natural quantum analogues of uniformity and identity testing, respectively. For mixedness testing of a d-dim state,
@BooleanAnalysis
and John Wright showed that Θ(d) samples are sufficient and necessary. 2/n
1
0
4
But their tester needs heavily entangled measurements, which makes their tester impractical to implement on existing quantum devices.😭
So what if we only consider testers that don't use entangled measurements, i.e., only measure one copy of the state at a time? 🤔3/n
1
0
4
It's pretty fascinating: even though proving lower bounds against adaptive algorithms is often really hard, I don't know of any natural quantum state learning setting where it actually gives any advantage. Hence, non-adaptivity is all you need????? 🤔🤔🤔 8/n
1
0
4
(whoops, sorry, made it public!)
(but really, maybe consider watching it on Thursday?)
1
0
3
Estimating Coronavirus Prevalence by Cross-Checking Countries by Jacob Steinhardt
0
1
3
@thegautamkamath
@ccanonne_
yeah, what
@thegautamkamath
said. Surprisingly, the results appear to achieve incomparable sample complexities, and the techniques are quite different!
0
0
3
@suriyagnskr
As is so often the case, the solution involved "lifting" the problem and, in this case, the chair arm.
1
0
3
@ccanonne_
this is not quite matrix Bernstein's but you can often do the following: since Frobenius is self-dual, you just need that <M, U> concentrates, if M is your matrix, and U is any unit Frobenius matrix. Get a good tail bound for a fixed, U, then union bound over all 2^(d^2) of them.
1
0
3
@thegautamkamath
@tetraduzione
wow good thing I randomly decided to check twitter today, outlook decided that this email was junk...
0
0
2
@ccanonne_
Yeah, what I suggested is exactly the analog of the proof technique used there. See for instance the discussion of Cor. 2.1.12 in my thesis (although it's almost certainly been written down before).
1
0
2
0
0
1
@kwon_youngjoon
bruh couldn't you have taught that lesson without scarring me with that image...like what is that on your face
1
0
1