Constantinos Daskalakis
@KonstDaskalakis
Followers
7K
Following
542
Media
5
Statuses
121
Scientist. Computer Science Professor @MIT. Studying Computation, and using it as a lens to study Game Theory, Economics, and Machine Intelligence.
Cambridge, MA
Joined November 2018
Min-max optimization (used among other applications in GANs, and adversarial training more broadly) is empirically challenging. We show why min-max optimization is hard in the following paper with Stratis Skoulakis and Manolis Zampetakis:.
3
69
374
There are many algorithms for learning Ising models. Yet there is no sample-efficient and time-efficient one that works without assumptions, even for tree-structured models. We obtain the first such result in fresh work w/ @PanQinxuan:.
1
18
154
Congratulations to the ACM Turing Award winners-live from San Francisco
0
8
95
Identity testing between high-dimensional distributions requires exponentially many samples in the dimension---aka good luck GANs targeting high-dimensional distributions. Joint work with @FeiziSoheil and M. Ding shows how to escape these lower bounds if the target is a Bayesnet!.
Q: How to design a GAN for distributions on Bayesian networks? A: check out our paper introducing "SubAdditive GANs": Joint work with Constantinos Daskalakis @KonstDaskalakis and @umdcs student Mucong Ding, started at @SimonsInstitute
4
18
90
Congratulations to my student, Dr. @NishanthDikkala, who successfully defended today, having done innovative and deep work on statistical inference using dependent data!.
Congratulations to my academic brother Dr. Nishanth Dikkala (@NishanthDikkala) who just defended his thesis on statistical inference from dependent data! Another student lucky to be advised by Costis (@KonstDaskalakis)! Check out his work on his website:
1
0
91
Honored to give a lecture in memory of Paris Kanellakis!.
Professor @KonstDaskalakis of @MIT will deliver the 19th annual Paris C. Kanellakis Memorial Lecture ("Learning from Censored and Dependent Data") at 4 PM on December 5 in CIT 368. Details at:
0
8
90
Can you train a generative model using only noisy data? If you can, this would alleviate the issue of training data memorization plaguing certain genAI models. In exciting work with @giannis_daras and @AlexGDimakis we show how to do this for diffusion-based generative models.
Consistent Diffusion Meets Tweedie. Our latest paper introduces an exact framework to train/finetune diffusion models like Stable Diffusion XL solely with noisy data. A year's worth of work breakthrough in reducing memorization and its implications on copyright 🧵
1
12
70
Congratulations to Manolis Zampetakis for this richly deserved honor!.
The @AcmSIGecom Dissertation Award for 2020 goes to .Manolis Zampetakis for his thesis "Statistics in High Dimensions without IID Samples: Truncated Statistics and Minimax Optimization" advised by @KonstDaskalakis at @MIT. Congratulations!.More details:
2
3
68
Samples from high-dimensional distributions can be scarce or expensive to acquire. Can we meaningfully learn them from *one* sample?!? In new work w/ @YuvalDagan3, @NishanthDikkala, &Vardis Kandiros, we show how to learn Ising models given a single sample.
1
5
66
How fast does player regret grow in multi-player games? Standard no-regret learners give regret growing as √T in T rounds of interaction. Recent work w/ @MFishelson & @GolowichNoah brings this down to a near-optimal poly(log T)!.
2
8
61
A great line-up for the July 13-14 FODSI workshop on ML for Algos! @AlexGDimakis-Yonina Eldar-@annadgoldie-@HeckelReinhard-Stephanie Jegelka-@tim_kraska-Benjamin Moseley-David Parkes-@AlgoSvensson-Tuomas Sandholm-@vsergei-Ellen Vitercik-David Woodruff!.
1
18
63
Exciting work w/ @YuvalDagan3 @MFishelson @GolowichNoah on efficient algos for no-swap regret learning and, relatedly, correlated eq when the #actions is exponentially large/infinite. While classical works point in the opposite direction, we show that this is actually possible!.
1
9
59
Congratulations Manoli! Proud for the beautiful work in this dissertation!.
Congratulations to my academic brother Manolis Zampetakis on successfully defending his PhD thesis! The latest amazing student of @KonstDaskalakis. Huge turnout for his defense (larger than expected). Going to UC Berkeley for a postdoc next. Check him out:
2
3
59
How to use subadditivity properties of probability divergences in GANs to decrease the effective dimensionality of your problem?. Check out our oral paper with @FeiziSoheil and M. Ding at AISTATS in a couple of hours!.
Q: how can we improve the design of GANs if the underlying independence graph of variables (as a Bayes net or an MRF) is known?. A: check out our paper: which will be presented as an oral talk @aistats_conf . Joint work with M. Ding and @KonstDaskalakis
0
12
55
While coarse correlated equilibria (CCE) are very tractable in one-shot games, @GolowichNoah, @KaiqingZhang and I show that, surprisingly, stationary (Markov) CCE are intractable in stochastic games, and thus Multi-Agent Reinforcement Learning (MARL):.
2
11
52
Great lineup of invited speakers at the NeurIPS workshop on the important interface between ML and Game Theory! Submit your interesting papers!.
Game theory and deep learning workshop at NeurIPS!. Invited talks:.Éva Tardos! Asu Ozdaglar! David Balduzzi! Fei Fang! . Plus more contributed talks, posters, panels and discussions. Submit your extended abstracts by September 16th!.
0
5
38
Happy advisor moment! Congratulations to Matt Weinberg and the other fellows!.
The @SloanFoundation fellows for 2020! Non-exhaustive list of some names I recognize in CS&Math: @mcarbin, Sanjam Garg, @zicokolter, Yin Tat Lee, @polikarn , Aaron Sidford, Matt Weinberg, Hao Huang, @weijie444, and Cynthia Vinzant. Congrats to all winners!
1
0
37
Proud of my brother's research!.
Very proud and energized for 2025 that our paper on 🧠 molecular pathology in #PTSD and #MDD made it in #NIH Director’s 2024 #ScienceHighlights!.
0
4
37
Looking forward to this event on Thursday!
June 20th in Athens 🇬🇷- the Lyceum Project: AI Ethics with Aristotle. With speakers including @alondra, @KonstDaskalakis, @yuvalshany1, @kmitsotakis, Josiah Ober, @mbrendan1, and @FotiniChristia. Places are rapidly filling up. Register now.
0
4
31
Standard no-regret learners, e.g. online GD, converge to equilibria of monotone games like the moon converges to the earth, i.e. their average converges, at a best achievable rate of 1/T. At what rate can their last-iterate converge? See 👇 & find @GolowichNoah at NeurIPS today.
New paper with Sarath Pattathil & Costis Daskalakis: We answer the question: at what rate can players' actions converge to equilibrium if each plays according a no-regret algorithm in a smooth monotone game?.
0
5
27
In follow-up work, joining forces with Ioannis Anagnostides, @gabrfarina and Tuomas Sandholm, we obtain similar bounds for no-internal regret, no-swap regret, and convergence to correlated equilibrium.
1
1
15
Thus, while min-max equilibria of two-player zero-sum games with convex-concave objectives are tractable, even approximate and local min-max equilibria of nonconvex-nonconcave objectives are as intractable as Nash equilibria in general-sum normal-form games.
1
0
15
Many thanks to Kush Bhatia and @CyrusRashtchian for their excellent article on this talk.
4/n Next up, we have an interview with @KonstDaskalakis and coverage of his keynote talk(. Written by Kush Bhatia and @CyrusRashtchian. Thanks a lot to @boazbaraktcs for hosting this one!
0
5
15
The question: How to use ML+data to design algorithms w/ better performance on non worst-case instances, e.g. better data structures, online algos, streaming and sketching algos, market mechanisms and algos for combinatorial optimization, similarity search and inverse problems?.
2
1
14
In his beautiful thesis he expands Econ-CS research in fundamental & timely directions, pertaining to statistical estimation, optimization & learning from (i) incomplete (ie biased, censored, or truncated) data & (ii) data that is subject to adversarial or strategic manipulation.
1
0
14
Our work falls in a long line of research, motivated by the profound applications of the Ising model in a range of disciplines, including Statistical Physics, Information Theory, Computer Vision, Computational Biology, and the Social Sciences.
1
1
10
Such approximate local min-max equilibria are guaranteed to exist for small enough delta, and are equivalent to fixed points of the projected gradient descent (in x) - ascent (in y) dynamics, i.e. they are first-order solutions of min_x max_y f(x,y).
1
0
11
We study the complexity of computing approximate local min-max equilibria of f, i.e. points (x,y) such that no jiggling of x within a delta neighborhood can decrease the function by more than epsilon, and no delta-jiggling of y can increase the function by more than epsilon.
1
0
10
Our learning algorithm improves on prior work whose time depends exponentially on the number of agents (known as the "curse of multi-agents") as well as V-learning, which beats the curse of multi-agents but computes non-Markovian CCE.
1
2
8
Many congratulations to the other awardees as well!.
0
0
9
To control the error accumulation, we combine this hierarchical classification with delicate uses of the subadditivity of the squared Hellinger distance over graphical models from:.
0
1
9
To be precise, we study min_x max_y f(x,y) where (x,y) is constrained to lie in a compact convex set and f is smooth and Lipschitz, but not necessarily convex in x and concave in y. In the absence of the latter, classical min-max/convex-programming duality theorems don't apply.
1
0
9
Nice!.
Adding airline pilot to my CV.
0
0
8
Said differently, computing approximate local min-max equilibria of two-player zero-sum games with smooth and Lipschitz objectives is PPAD-complete, i.e. as hard as (i) computing Nash equilibria in general-sum normal-form games; and (ii) computing approximate Brouwer fixed points.
1
0
8
Brought to you by Foundations of Data Science Institute:
0
0
8
This oracle lower bound is a byproduct of a complexity-theoretic result, showing that the problem is PPAD-complete, which means that no algorithm (first-order, second-order or whatever) can compute approximate local min-max equilibria in polynomial time, unless P=PPAD.
1
0
8
This line of work includes a recent renaissance of learning algorithms, including e.g. the following.
2
0
7
In particular, we show that the celebrated Chow-Liu algorithm, using the plug-in estimator for mutual information, learns arbitrary n-variable tree-structured models to within eps-total variation distance from an optimal O(n log n/ eps^2) samples.
1
1
7
Contemporaneous work by Jin, Muthukumar and Sidford shows a related lower bound:.
0
2
6
On the technical front, we establish novel concentration and anti-concentration bounds for functions of the Ising model, a fascinating topic on its own right. We go beyond high temperatures! Learn more by attending a Simons workshop tomorrow@.
0
1
7
Even more recent work by Georgios Piliouras, Ryann Sim and Stratis Skoulakis provides an implicit method with constant regret (w.r.t. T) albeit with polynomial dependence on the dimension.
0
0
7
In the oracle optimization model, computing an approximate local min-max equilibrium of a L-smooth G-Lipschitz function f using queries to f and its gradient requires exponentially many queries in at least one of 1/approximation, G, L, the dimension.
1
0
6
Our work, whose talk was yesterday and poster is tomorrow at NeurIPS'21, advances a long line of work on the benefits of optimistic learning in games. We show that a simple variant of Hedge has near-optimal regret dependence in T and still logarithmic dependence in the dimension.
1
0
6
@ccanonne_ @FeiziSoheil Inspired by prior work with Qinxuan Pan this new paper shows how to use subadditivity theorems for probability distances/divergences (including Wasserstein distance) in GAN architectures to improve statistical/computational aspects of GAN training.
1
1
6
As a corollary, we show near-optimal Õ(1/T) convergence to coarse correlated equilibrium. We prove our result by studying higher-order smoothness properties arising when players use optimistic Hedge to update their strategies against each other.
1
0
5
@ccanonne_ @FeiziSoheil Namely, subadditivity theorems over Bayesnets imply that the discriminator can be broken into multiple discriminators whose job is only to enforce that small set marginals of the generated distribution match those of the target distribution.
0
1
5
Our work advocates for *non-stationary* Markov policies/equilibria as viable optimization targets in general MARL and we provide a decentralized learning algorithm converging to non-stationary Markov CCE in time that is polynomial in the states, actions, agents and approximation.
1
0
4
These works learn Ising models assuming upper bounds on the strengths of the interactions in the model, a.k.a. lower bounds on the temperature of the model. Our work removes these assumptions, estimating arbitrary tree-structured models.
1
1
4
@RadioMalone @thegautamkamath @NPR @planetmoney @nycmarathon It's always fun to talk to you too @RadioMalone and I'm happy how mechanism design was thoroughly illustrated through a single example!.
0
1
4
@marilynika @AthensSciFest Thank you so much Marily for seeding and moderating the conversation!.
0
0
3
Our guarantees unify, extend and interpolate between two separate strands of works: recent work in CS on estimating MRFs from multiple independent samples under mild assumptions; and a growing literature in Probability on estimating them from one sample under more assumptions.
3
0
3
@roydanroy Nice! Or should I say scary? Maybe she is used to looking at bars as something good like earnings?!?.
1
1
2
The algorithm may learn some edges incorrectly but we control the impact of its mistakes on the TV distance. Indeed, our proof involves a hierarchical classification of the true model's edges into layers with different reconstruction guarantees, depending on their strength.
1
2
3
0
0
2
At a technical level, our upper bound employs a tree structure on a collection of external regret learners. Each external regret learner updates its action distribution only sporadically, allowing us to avoid polynomial dependence on the number of experts.
1
0
2
Congratulations to the ACM Turing Award winners-live from San Francisco
0
0
2
More generally, we obtain that any class of finite Littlestone dimension (or seq fat shattering dimension) has a no swap regret learner. We also derive an extension to the bandit setting, for which we show a nearly-optimal swap regret bound for large action spaces.
1
0
1
@NishanthDikkala @roydanroy I agree. Less jokingly, the purpose is pretty clearly manipulation of the public.
1
1
1
We quantify the error in estimating the model w.r.t. the metric entropy of possible interaction matrices. If multiple samples are available, we handle them by viewing them as one sample from a larger model. We thus provide one method for learning given one, a few, or many samples.
1
0
1
Note that concurrent work by Peng & Rubinstein establishes a similar set of results:
0
0
1
They also provide algorithms for computing low-rank and sparse correlated equilibria.
1
0
1
Our intractability result holds even if the game has two players, the discount factor is 1/2, and we are shooting for an absolute constant approximation, and lies in sharp contrast to MDPs (aka single-agent RL) as well as normal-form games where optimal policies/CCE are tractable.
1
0
1
The number of rounds in our upper bound depends exponentially on the inverse of the approximation parameter. We present a lower bound showing that this dependence is necessary, even when the adversary is oblivious.
1
0
1
0
0
1
@thegautamkamath @mraginsky @SebastienBubeck @ilyaraz2 @ShamKakade6 @aleks_madry @Andrea__M Guilty as charged :).
0
0
1
We show that no-swap regret learning is possible for any function class which has a no-external regret learner. As an example, in the setting of learning with expert advice with N experts, only polylog(N) rounds are needed to obtain swap regret bounded by an arbitrary constant.
1
0
1
As a consequence of our main results and the standard connection between no-swap regret learning and correlated equilibria, we obtain a polynomial-time algorithm for finding an approximate correlated equilibrium in extensive-form games, for constant approximation parameter.
1
0
1