Jeffrey Wang
@jeffreygwang
Followers
4K
Following
2K
Media
34
Statuses
403
Researcher @OpenAI; on leave from @Harvard.
San Francisco, CA
Joined July 2020
This is a few months late, but I suppose there's no time like 2025 to share. I took leave from Harvard and joined @OpenAI as a researcher! . It's so exciting to hold a🕯️to the unknown every day alongside so many brilliant coworkers! I'm thrilled for what's next.
45
23
1K
@brianryhuang I think this is true for a lot of quants? In my experience they are almost always extremely humble, because they're in an extremely intelligence-dense pocket of the world.
2
0
156
TIL the first place winner of the 2024 science talent search is a ****MECH INTERP**** project.
3
4
86
Really proud of my first paper 🗞️ coming out of a college research project! We create a SOTA membership inference attack on language models and show the importance of second-order loss curvature info for membership determination. Read my PI @SethInternet's 🧵for more!.
🚨 New Paper appearing in @emnlpmeeting '23 on detecting if a passage x appears in the training data of a large language model. Our method MoPe works by perturbing the model and measuring the resulting drop in the log-likelihood:
10
7
65
excited to be part of this group!!.
Meet the 2025 Neo Scholars!🎉. Congrats to these 32 exceptional students who will embark on a year-long program of events and mentorship to accelerate their paths as future tech leaders. (1/2)
2
0
48
I have just learned that @perplexity_ai is giving a YEAR'S worth of Pro out for FREE ($200 value) to @harvard students if 500 emails sign up by 9/15 -- harvard people please sign up! .
3
4
40
. labor to ChatGPT. Hard reasoning used to be a strategy-proof eval for technical ability, and now it completely fails. How can we align this power with our pedagogy? . (I have so many more thoughts here, which I would like to find the time to crystallize soon!).
1
0
35
MyHarvard, Harvard's official course registration tool, is stunningly slow. Excited to have contributed to a course search tool that is 100x faster! With @ekzhang1 and others :) .
1
0
33
This week, @WSJ ran a series of perspectives on crypto's state post-FTX. And I got published!
3
1
30
I worry that the removal of AA + elimination of standardized testing will exacerbate a system where people who know "how to tell a story" (ie play the game) do better, which may lead to more equity in surface statistics but hide more insidious outcomes underneath
3
2
27
Amidst the twitter froth about o3 today, I think it's particularly important to ask: what is the future of technical education in a world where every reasoning-heavy question has an answer at the tip of a student's fingers?.
1
0
27
@willdepue pretty sure at least @mercor_ai and @haizelabs have meaningful product & revenue (the two closest startups in my mind to your description— totally possible you’re thinking of others!).
1
0
25
My first scientific paper got published! (w/ @ferhatay and @abhijitchak23) . Check it out here: .
7
5
24
@zdeborova Take a look at this paper, which provides a theoretical explanation for why training without overparameterizing is impossible: .
0
1
21
Larry Summers?!.
We have reached an agreement in principle for Sam Altman to return to OpenAI as CEO with a new initial board of Bret Taylor (Chair), Larry Summers, and Adam D'Angelo. We are collaborating to figure out the details. Thank you so much for your patience through this.
4
0
18
Some freshmen at Harvard have a delicious lack of shame. I appreciate it; it’s refreshing to extend grace to others that I so often don’t extend to myself.
4
0
18
i've met few people as genuinely curious, talented, and kind as @ayushnoori at harvard. #proud friend moment here:.
Looking ahead to 2025, I’m grateful and excited to share that I was selected as a U.S. Rhodes Scholar. Supported by @rhodes_trust, I will pursue graduate study at @UniofOxford this fall to continue researching AI for neuroscience and precision medicine.
1
0
18
#ICML2024 Come by Straus 1 on Saturday to see @marvin_li03 and I present our project on the impact of biased (multimodal) embeddings on diffusion models! . We're giving a lightning talk sometime around 3:40pm and presenting our poster in the hour afterward.
2
1
16
today's ABSOLUTELY PACKED #harvard ml foundations seminar: "sparks of AGI: Experiments with an early version of GPT-4"
3
1
14
Some of you might not know this, but I've writing on @Quora since 2016. 📝. Last month, I encountered 2 statistical paradoxes so fascinating that I had to write pieces about them. And they went viral! I've been blown away by the response since I honestly can't remember. (1/6).
2
3
13
Had the chance to speak with @FrankWilczek today at #RegeneronSTS finalist week. Super excited and can't wait to read my signed copy of "Fundamentals: Ten Keys To Reality"!
0
3
13
flashes of the future, today ⬇️. awed and excited by what folks here @openai are doing!.
Summary of all the CRAZY benchmark results from OpenAI's most advanced model, o3!. SWE-Bench Verified: 71.7%.Codeforces rating: 2727.Competition Math: 96.7%.PhD level science (GPQA): 87.7%.Frontier Math: 25.2% (previous best was 2%).ARC-AGI: 87.5%. TRULY SUPERHUMAN.
1
0
14
Last year, I was the head Teaching Fellow for Harvard's flagship data structures & algorithms course, a class where struggling through hard HW formed the backbone of the student learning experience. Even then, I often saw students in my office hours outsource their cognitive. .
1
0
14
One day I too will finish a PhD with 93/100 hope in 4 and a half years
1
1
12
I am getting sold on @perplexity_ai. Today, I was searching for the name of @TheNotableApp, a Markdown editor with a pencil-like logo that I once used. "What markdown editors have a pencil logo and begin with the letter n?" is useless on Google, but perfect in Perplexity.
0
1
11
This is what everyone's been begging for . cc @alexkaplan0 @Kalshi . Monetary market on the superconductor:
1
1
11
@kbarley66 sleep researchers call this the “wake maintenance zone.” Pre-historically, this surge of wakefulness occurred right after dusk—as it was evolutionarily helpful to be more awake at the time of max vulnerability. Today with your shifted sleep cycle it’s right after midnight!.
1
0
7
The second starts with this question: "When waiting for a bus that comes on average every 20 minutes, how long is your average waiting time?". The intuitive answer is 10, but the actual answer is 20. Which was so mind-bending I had to write about it: (4/6).
1
2
11
spotted @georainbolt's billboard on a trip out of harvard square today. (visiting the new boston soup dumplings!)
1
0
11
a common problem plaguing membership inference papers: the benchmarks are broken! . A detailed and diplomatic write up by @pratyushmaini and @iamgroot42 ⬇️.
1/6 A lot of us are grappling with peer review these days, but its worst manifestation is when prestigious conference awards overlook critical flaws. Case in point: #EMNLP2024’s Best Paper Award. I & @iamgroot42 wrote a blog on what went wrong: 🧵
0
1
9
There are tons of unanswered questions in privacy/inference right now --- and answering them is made possible by @AiEleuther's Pythia Suite. Our latest paper (referenced in @BlancheMinerva's tweet below) is entirely based on it!.
It also suggestions a potential solution a conflict between most of the literature (which finds late training data easier to extract) and the ones reported in Pythia and "Emergent and Predictable Memorization" (which doesn't). Really high quality paper.
0
1
9
nowadays, instead of simply searching some question Q, i will either:.- google Q followed by "reddit".- go to and ask my transformer of choice.
3
0
8
mike was my TA for advanced macro in my freshman year and he remains, to this day, the best teacher I’ve had in college. So happy he’s going places 🥹.
Super excited that Michael Droste from Harvard will be joining our macro group as an Assistant Professor!. (First hire while chair = very happy chair!).
1
0
8
One of the brightest groups of people I have ever worked with, building the foundation of decentralized finance. Amazing stuff ⚡️.
1
0
8
oppenheimer: a deeply engaging & suspenseful cinematic masterpiece with the historical content of a wikipedia stub.
2
0
7
happy askers 🤝 great answers 🤝 first $5 free!.
Launching Quidio’s new website! 🚀. We’re the knowledge marketplace for programmers 💻 We help people with coding questions get high quality answers, and we make sure experts get paid for their expertise. Check it out 👉
0
0
6
midnight mood is listening to @jmschreiber91 give an awesome talk about ML pitfalls in genomics and spending the evening manipulating mock data to recreate those pitfalls #RECOMB2022.
1
0
7
Congrats to all the winners! It's a great day for science.
🏅We're excited to announce the winners of the 2024 @Regeneron Science Talent Search. Congratulations to these extraordinary young scientists! #RegeneronSTS
0
1
4
Check out this awesome paper on diffusion models by @marvin_li03 & @sitanch --> it provides a theoretical explanation for this fascinating empirical effect: when you generate images, they seem to "emerge" in a narrow window during generation. .
In diffusion models, the features of generated images emerge in narrow time intervals of the reverse process 😮—can we provably characterize these “critical windows” in which features are decided?. W/ @sitanch we describe critical windows for a rich family of distributions. (1/n)
1
1
7
anyone seen my jaw? I dropped it a few minutes ago and can’t find it.
Greg Brockman (@gdb) of OpenAI just demoed GPT-4 creating a working website from an image of a sketch from his notebook. It’s the coolest thing I’ve *ever* seen in tech. If you extrapolate from that demo, the possibilities are endless. A glimpse into the future of computing.
0
0
5
this allows you to make a custom ETF like "S&P 500 without big oil": it takes your prompt, creates a custom basket, which you can then buy! super cool.
Today is launch day! 🚀🎉🔥. Investing is hard. We make it super easy to build wealth based in your individual ideas and values. Want to invest in "EVs without Elon". you say it, we build it, you invest. Download at
0
1
6
The fact that @Quora just quote rt'ed me with a witty Bayes joke has ME spinning at 100krpm!.
That sound you hear is the Reverend Thomas Bayes spinning in his grave at 100krpm.
0
0
4
TIL that @kbarley66 is a stellar social change-maker creating accessible education through . (also, that she has springtime allergies)
1
1
5
1) used @obsdmd's api to migrate a notion database today. 2) wrote routines to tag and categorize data into markdown notes. 3) the result:
3
1
6
@OpenAI The most impressive thing here IMO is ARC-AGI and FrontierMath—where o3 is a step function improvement (essentially 0 -> 1). We've known the other benchmarks have signs of life and therefore just had to be hill-climbed (the hill was climbed shockingly well & fast though!).
2
0
6
boarding PHL to BOS flight. in middle seat. 4 of 8 people around me are on business calls.
1
0
5
the state of Harvard's IT infra: scheduled maintenance takes >48 hours 🥲
1
0
4
Thank you to 1) @boazbaraktcs, whose Sp' 23 ML seminar helped me enter this area of research, 2) @ericmitchellai @chelseabfinn whose DetectGPT paper inspired this idea, and 3) @haileysch__ + the @AiEleuther team for helping us so much over Discord as we got Pythia online!.
0
0
4
the modern peace sign ☮️ has its origins in the 1950s; it combines the semaphore letters for "N" and "D" (Nuclear Disarmament)
0
0
4
FZ back at it again.
Since the discovery of CRISPR-Cas systems, people have wondered if a similar type of system exists in eukaryotes. Today, we report in @Nature the characterization of Fanzor, a eukaryotic RNA-guided system that can be reprogrammed for human genome editing.
0
0
4
Setting security questions for a financial account of mine right now, and the options are:.1. name of first movie you ever watched.2. first thing you learned to cook.3. first beach you visited. Are these things people remember??.
0
0
5
TWITTER: If I want to whip up a quick blog-type site, not as formal as a substack but cleaner than wordpress, what platforms do you have in mind?.
2
0
4
UMMMMM OKAY THIS SEEMS LIKE A HUGE NATURAL EXPERIMENT?? . and it shows causality (which is so, *so* hard to find)??.
Biggest thing to ever come out of my little group. Pls help spread this finding!. We found clean, CAUSAL evidence that the shingles vaccine prevents a good chunk of dementia cases. So, could a virus cause Alzheimer’s->YES!. Hear me out & see preprint: 🧵1/
0
0
2
What if you could solve problems, when they arise? Know something, when it comes up? . Every knowledge platform out there gives you no agency over *your* needs as an asker. Quidio is designing a new marketplace. for freelancers and knowledge.
How does Quidio work? . Ask a question, add a bounty, and get a great answer 😎. Quidio works on a bounty system that better aligns incentives for question askers and answers. Why a bounty system? Keep reading 🧵.
0
0
4
special shoutout to @adhsu for asking me to code ChatGPT for him in 45 minutes 😌.
Congrats to 150 @Neo Scholar Finalists. We're delighted to connect you to each other and to awesome startups!. Neo had >1,000 applicants for 30 spots (more selective than @Harvard, @MIT, or @Stanford). The eval process has been extraordinary. 🧵 /1
0
0
4
@christinexye Is it a PR problem or a lack of a general consensus on what form the technology will take? The various trajectories that A(G)I can take each lead to different worlds => different messaging.
2
0
4
I think the hunger for these stories, math and all, reflects an inbuilt fear/fragility that we have. Despite being at the top of the evolutionary pyramid, our brains are SO easy to trick. Outside of @Quora, exactly how to fix that is a *question* I am still answering. (5/6).
2
0
3
Is there anything that reads @arxiv papers to you? . feeling hungry for a paper but eyes tired.
2
0
4
One of my good friends @Joseph_E_B authored a fascinating paper today (that leverages AlphaFold! first time I've seen that in the wild):.
0
0
3
Petition for loquacity to be the noun form of loquacious instead of loquaciousness.
1
0
3
One of my favorite classes this semester—would recommend anyone to check it out!.
Posted all slides and readings for my course on foundations of deep learning on its webpage , including the great guest lectures of @ShamKakade6 and @cHHillee. Thanks @michael_nielsen for many suggestions on historical readings for the last lecture.
0
0
3
Special shoutout to Jason and Marvin, my two fellow H'25 undergrad co-authors—who have truly been the dream team in getting this work online! And of course. .
1
0
3
1
0
2
(Shoutout to Harvard's @stat110 for catalyzing these pieces; the world is 50,000 people wiser for it. I would wholeheartedly recommend anyone interested in probability to check it out.).
1
0
3
@Seanjkernan It should be something on @Quora but @Medium is winning my heart: Why We Suck at Preparing For Pandemics
0
0
3
0
0
1
My jaw dropped when I read this. Everyone in CS (and beyond) should read it too—.
It's about time that I re-claim my identity in its entirety. Here is my post discussing my caste.
1
0
3
sunset fantasias, earth, 2021. stunning.
0
0
2
Going to name my kid Caliciviridae IgE Wang.
Are you an immunologist who's having a baby? Here are some #immunology-related names for your child: 🦠🔬. (Neutro)Phil.(Anti)Gen.Helminth.Plasma.IgE (pronounced "Iggy").Caliciviridae.(Organ)Elle.Cilium.
1
0
3
The first is about Sally Clark, a British mother falsely convicted of MURDER due to "prosecutor's fallacy," confusing P(A | B) with P(B | A). This simple switch convinced a jury Clark had murdered her two children. (3/6). Full post here:
1
0
3
@Stone_Tao the trick is to ruthlessly optimize for only journal publications—you can finish in ~4 years by doing that.
1
0
3
For the past two years, I’ve read the headline on Fox and CNN each day. Usually, they’re completely different. Today, for the first time in a long time, they match. #Ukraine.
1
0
3
this right here wins my "internet moment of the week" award.
So, we have a guess for this location that stumped @georainbolt. We think this is Trinity Park. Thanks to @_artseabra and others for leading us to check our photo collection. In the replies, we'll share some photos that we took in the mid-2000s.
0
0
1
my friend just released an incredible paper on visualizing attention mechanisms. check it out!.
Check out our paper, on visualizing attention patterns via joint key + query embeddings!.
1
0
2
Interesting post from @RoyRinberg on overlooked considerations for differentially private fine-tuning:.
New Blog Post: 🔐"When is Differentially Private Finetuning actually Private?"🔐. TLDR: To provide meaningful DP guarantees in finetuning, you should also report: .1️⃣ Model capacity/size.2️⃣ How in-distribution the finetuning dataset is. Check it out here:
0
0
3
why is the area in new orleans around @NeurIPSConf.a food desert? i feel like i haven't seen fresh food in days. .
1
1
3
chatGPT is surprisingly useful for coding, *especially* in unfamiliar frameworks. rather than struggling to make incremental progress, it gives you large chunks of semi-functional code. this gives you a coup d'œil of what you need to do and helps you iterate faster.
1
0
3
Here, we show two intuitive conclusions hold, theoretically and empirically: 1) if the input embedding to your diffusion model is biased, your generations will be too; 2) if your multimodal embedding is biased, audits done with that embedding will be biased too.
1
0
3
TIL that the most widely-circulated photos from the James Webb Telescope, like this one in the NYTimes, were not released by NASA. Instead, they were created by Judy Schmidt, an amateur editor of the cosmos.
1
0
2
TIL Bill Clinton (who took office almost 31 years ago) is 4 years younger than Joe Biden.
0
0
3
My funny 🧵of the day: Generative AI thinks @elonmusk is a conman. @metaphorsystems integrates GPT-3 and stable diffusion for more intuitive web-searching. For instance, you could ask for a "wikipedia page about the most Elon Musk-figure from the 19th century."
1
0
3
Another awesome STS finalist. And look. is that a Hi-C map :D? @Regeneron @Society4Science.
Neil Chowdhury finished fifth, creating a computer model to study the role of a histone protein implicated in colon cancer. He is exploring DNA. “My work is about computer simulation, (specifically) on DNA and the three-dimensional (3D) genome.”.
0
1
3