Andi Peng
@TheAndiPenguin
Followers
2,496
Following
755
Media
33
Statuses
162
Researcher @AnthropicAI | PhD @MIT_CSAIL (on leave) | formerly @MSFTResearch @Yale @WHOSTP | cats are dope.
Joined October 2019
Don't wanna be here?
Send us removal request.
Explore trending content on Musk Viewer
Colorado
• 133022 Tweets
Notre Dame
• 115160 Tweets
Kentucky
• 67807 Tweets
#仮面ライダーガヴ
• 56513 Tweets
The Weeknd
• 55885 Tweets
#AEWAllOut
• 52537 Tweets
Northern Illinois
• 38507 Tweets
Nebraska
• 35411 Tweets
Abel
• 29945 Tweets
Carti
• 29475 Tweets
#ブンブンジャー
• 25101 Tweets
Auburn
• 24168 Tweets
Talleres
• 24055 Tweets
#UFCVegas97
• 18724 Tweets
Medina
• 15765 Tweets
Willow
• 15755 Tweets
Bama
• 13979 Tweets
#ニノさん
• 13680 Tweets
Skullduggery
• 13293 Tweets
ショウマ
• 12907 Tweets
Herrera
• 12867 Tweets
ひろプリ
• 11876 Tweets
Day 3: Officially entered the job market of my local anarchist commune. Dismayed to learn that an honors thesis on Thucydides would only net two la croix and a jar of cheez whiz per day.
#chazseattle
44
74
518
Life update: I've joined the Frontier Red Team
@AnthropicAI
for the summer!
Building safe models is the *highest* priority for our field, and I'm thrilled to be working with an amazing team to secure that future.
I'll be in SF Jun-Aug, and excited to see friends old and new!
7
2
215
Very excited to announce our workshop on 'Interactive Learning from Implicit Human Feedback' at
@ICML2023
this year!
1
12
109
Humans communicate preferences by providing rich linguistic feedback. Yet, preference-learning algorithms do not always take this social learning view into account.
We leverage pragmatic communication for RLHF in our
#ICML2024
paper!
Paper: 🧵⬇️
1
21
98
Introducing LGA (Language-Guided Abstraction) at ICLR 2024! 🧵
📰 Paper:
🌐 Website:
🗞️ MIT News:
State abstraction is key to generalizable learning, but how do we know which features are task-relevant?
2
22
80
Humans use abstractions for data-efficient learning. We wish for neural networks to do the same.
In our proposed human-in-the-loop framework, we automatically generate a spectrum of abstractions and allow users to deploy task-appropriate ones.
To appear at
#NeurIPS2023
! [1/n]
2
20
71
I'm excited to share our
#ICML2023
paper: we develop a user-informed framework for eliciting feedback to diagnose and fix policy failures.
Project page: . [1/8]
2
9
67
very unclear if currently living in seattle, east berlin, or book 3 of the hunger games rn. do i still have to pay rent in the capitol hill autonomous zone?
6
11
56
It’s a beautiful day to UNIONIZE!!
#MITGradUnion
#MITGSU
I stand in solidarity with
@MITGradUnion
. All workers, students or not, should have the chance to organize and fight for better benefits, affordable housing, and more. Putting power in the hands of workers is how we level the playing field.
74
299
2K
2
2
41
Excited to present an encore of our AAAI oral at
#CHI2022
#TRAIT2022
today!
As we continue to deploy real AI recommender systems in the world, it is important to understand their impact on human BIAS and ACCURACY.
2
7
32
On the ground for
#ICLR2024
!
I'd love to talk about:
1. Personalized preference learning
2. Building abstractions from language
3. The future of AI safety (and policy)
I'll also be presenting the following work👇
2
2
31
On the plane to
#NeurIPS2023
!
Excited to talk Human-Guided Complexity-Controlled Abstractions on Thurs + help host GCRL Workshop on Fri ().
I'd love to chat about:
1) abstraction in RL
2) learning world models from humans
3) the future of AI safety!
Humans use abstractions for data-efficient learning. We wish for neural networks to do the same.
In our proposed human-in-the-loop framework, we automatically generate a spectrum of abstractions and allow users to deploy task-appropriate ones.
To appear at
#NeurIPS2023
! [1/n]
2
20
71
0
4
29
Amazing talk by
@mark_ho_
at
@corl_conf
on how humans construct value-guided construals for planning!
@andreea7b
0
3
26
Excited to see our work featured on the front page of MIT News!
I'm excited to share our
#ICML2023
paper: we develop a user-informed framework for eliciting feedback to diagnose and fix policy failures.
Project page: . [1/8]
2
9
67
1
5
22
Can changes in user behavior tell us anything meaningful about their implicit preferences? Our
#HRI2024
paper suggests yes!
Paper:
[1/n]
2
5
20
Check out our new survey on challenges and open problems in RLHF!
I had a blast working on this with (many) diverse co-authors :)
New paper: Open Problems and Fundamental Limitations of Reinforcement Learning from Human Feedback
We survey over 250 papers to review challenges with RLHF with a focus on large language models. Highlights in thread 🧵
15
168
714
0
2
19
An oldie but a goodie - representation alignment is important! Even more nowadays in the age of scaling up human feedback for task learning.
Come chat w/us at HRI about it :)
What does it mean for humans and robots to align their representations of their tasks and how do current approaches fare? Come see at
#HRI2024
in the Tuesday 17:10 session!
Paper: w/
@TheAndiPenguin
,
@pulkitology
,
@julie_a_shah
,
@ancadianadragan
1
7
65
0
3
16
This is a block from where I live. The protest was peaceful for 4 hours, as it was yesterday, and as it was the day prior. There are elderly and children on my street, which is now filled with tear gas. The only riot here was the one started by
@SeattlePD
.
@seattleprotests
1
7
14
"Reliable carbon accounting won't solve the climate crisis, but it is essential for implementing strategies that could." Check out our paper out in
@Nature
co-led by
@amyluers
and
@LeehiYona
!
On the critical need for reliable carbon accounting, this week in Nature:
@amyluers
@lucasjoppa
@theNASEM
@MSFTResearch
#ClimateCrisis
#GHG
1
6
7
1
0
12
I don't often get to meet folks doing *radically* cool things outside of my field nowadays -- but this community continues to amaze and inspire me.
If you're a technologist between 18-23 - considering applying!
Applications are now open for the 2022 Interact Fellowship!
Learn more and apply here by February 15th: .
6
67
317
1
3
13
We're excited to have you!!
Very excited to announce that I'll be joining
@MIT
's AeroAstro department as a Boeing Assistant Professor in Fall 2024. I'm thankful to my mentors and collaborators who have supported me during my PhD, and I look forward to working with students and colleagues at
@MITEngineering
.
43
10
487
0
0
12
Human representations help us learn better robot representations! Submit to our workshop and join us in New Zealand in December :)
How can robots align their representations of the task with their human partners? Really excited to be organizing a
#CoRL2022
workshop exploring this topic!
We have an amazing lineup of speakers, and we're now accepting submissions until 10/21:
1
11
68
0
1
10
I had a blast working on this project with collaborators
@avivnet
,
@mark_ho_
,
@tianminshu
,
@andreea7b
,
@julie_a_shah
,
@pulkitology
!
Paper: [8/8]
0
2
11
For additional details, including our call for papers: see: .
With
@AkankshaSaran
@andreea7b
@tengyangx
@ancadianadragan
@pyoudeyer
@JohnCLangford
0
1
11
Honored to speak alongside
@RepBillFoster
at today’s AI event 🤖
“We’re going to have to embrace our shared humanity if we’re going to get through this.” Closing remark from IL congressman
@RepBillFoster
(the only physics PhD in Congress) at
@Aon_plc
’s AI event in the context of our society getting through the inevitable AI onslaught
0
0
2
2
0
11
Fam calls from Wuhan offering reassurance that they’re ok and to get tested early since is free and US healthcare best in world. Did not have heart to explain that in nation of best healthcare in world, I would neither be able to get tested early nor for free.
#CoronaVirusSeattle
0
1
10
Thanks to JHU for covering our
#ICML2024
work!
Into the woods:
#AI
goes mushroom foraging to learn how humans make choices. A new machine learning framework by
@tianminshu
,
@TheAndiPenguin
,
@dabelcs
, & more promises to make systems more personal and ethical. Learn more:
0
3
6
4
1
10
Excited to be organizing a great workshop! Consider submitting by Oct 4!
Announcing the NeurIPS 2023 Workshop on Goal-Conditioned Reinforcement Learning (GCRL).
We welcome either a 5-minute video or 2-page paper by October 4th, 2023.
More info:
1
8
19
0
0
8
Representation alignment is critical in *not just robotics*, but all sorts of other biological and artificial systems! Check out our preprint below.
🧵🎉 Our new preprint is up, and we’d love your feedback! We're "Getting Aligned on Representational Alignment" - the degree to which internal representations of different (biological & artificial) information processing systems agree. 🧠🤖🔬🔍
#CognitiveScience
#Neuroscience
#AI
7
125
454
0
1
8
We have an amazing lineup of speakers scheduled, including
@DorsaSadigh
@daniel_s_brown
@_jessethomason_
@jgrizou
@dabelcs
and more!
1
0
9
1. LGA () on Tuesday morning
2. PLGA () in the LLM Agents workshop on Saturday
3. I'll also be a panelist in the RepAlign workshop () on Saturday
Please email or DM if you'd like to meet up!
1
0
7
To automatically construct diverse abstractions, we use a discrete information bottleneck approach to trade off complexity, informativeness, and utility of neural representations. The key idea is that penalizing complexity allows us to induce more general abstractions. [2/n]
1
0
6
As always, awesome collaborating with
@MycalTucker
@EoinKNNy
@NogaZaslavsky
@pulkitology
@julie_a_shah
!
0
1
6
Our goal is to foster interdisciplinary discussions on topics such as learning with multimodal human feedback, learning without tagged rewards, interaction-grounded learning, personalized interactive learning, applied and theoretical implications for HCI and embodied learning.
1
0
6
We are excited about future directions that leverage human knowledge in tandem with cognitively-motivated training objectives!
Paper:
Code:
Reach out! We're friendly and happy and love taking walks at conferences!
2
0
5
if this ain't blatant language model misalignment i don't know what is
0
0
5
When humans give preferences, there are "abstractions" at play, or in other words, there are "features" that most directly contribute to their preferences.
Ex: a mushroom forager🍄 may prefer Mushroom A over B because A is more colorful. We call this a "feature preference".
2
1
4
Excited to talk AI ethics, bias, and effective human collaboration tomorrow at
@YaleCyberForum
@JacksonYale
TOMORROW: Join us for our final session, AI Ethics and Safety, tomorrow 9-11:45 a.m. (ET). Open to the public! Pre-registration is required:
@ekvochko
@TheAndiPenguin
@TedWittenstein
@brianchristian
@scottjshapiro
@oonahathaway
0
7
6
0
1
5
In a crowdsourced study where we collect hundreds of thousands of biographies from the Internet, then pair them with over 38,400 human annotations on a hybrid decision-task, we find that while a better AI always improves human accuracy...
1
0
5
We find that policies deployed with our framework result in (1) significantly more accurate user feedback compared to seeing behaviour alone, and (2) higher performance on desired test tasks with fewer human demonstrations. [6/8]
1
0
4
In our framework, given a human demonstration, we search through the concept space for observations that would have resulted in the policy succeeding *had a specific concept changed*. This can be seen as a *contrastive explanation*, helping isolate the cause of failure. [5/8]
1
0
4
But how do we reliably elicit this feedback? Problematically, humans are not always reliable identifiers of feature-level black box model failures. Inspired by the interpretability literature, we formulate a *counterfactual* approach. [4/8]
1
0
4
To see how LGA can be further extended to infer implicit human preferences, see our followup work, PLGA (Preference-Conditioned Language-Guided Abstraction) from HRI 2024!
1
0
4
Next up -
@jacobandreas
talks about how we can learn better robot representations from natural language!
@andreea7b
0
1
4
~10 days til the submission deadline!!
Human representations help us learn better robot representations! Submit to our workshop and join us in New Zealand in December :)
0
1
10
0
0
3
I'm particularly excited by the direction of incorporating interpretability-based tools to help fix model failure, in the hope of leveraging end users to more efficiently perform interactive alignment of robotic policies at test-time. [7/8]
1
0
4
Importantly, LGA complements traditional supervised learning methods like behavior cloning (BC), WITHOUT relying on pre-trained skills, additional environment interaction, large multitask datasets, or even the ability to exhaustively describe behavior in language.
1
0
4
[7/n] I’m super excited about utilizing pretrained models (such as LMs) in conjunction with human feedback to interactively learn human-aligned representations for decision-making.
1
0
3
Excited to talk about improving the ML publication process via strengthening subcommunities
@iclr_conf
#smiles
tomorrow! Joint with
@jefrankle
@in4dmatics
@yonashav
3
2
4
As always, extremely grateful to awesome collaborators
@andreea7b
@belindazli
@jacobandreas
@cocosci_lab
@sucholutsky
@tedsumers
@nishanthkumar23
@julie_a_shah
0
0
4
LGA improves sample efficiency and distributional robustness in both single- and multi-task settings, matching the performance of human-designed state abstractions while requiring a fraction of the human effort. See Moana (our Spot robot) in action!
2
0
4
[8/8] Many thanks to awesome collaborators!
@andreea7b
@belindazli
@sucholutsky
@tedsumers
@nishanthkumar23
@cocosci_lab
@julie_a_shah
Paper:
0
1
4
We propose a pedagogical framework for modeling feature preferences.
Our key insight is that humans communicate preferences pragmatically: when they describe which features are important to their preference, they are also implicitly revealing which features are NOT important.
1
0
3
In a user study, we found that pragmatic feature preference queries did NOT cause users to experience more frustration with providing labels vs. RLHF queries. 😡
This is an important finding, as it suggests we should continue exploring ways to learn from natural human feedback.
1
0
3
LGA begins by querying the user for high-level task descriptions, then uses a LM to translate these descriptions into task-relevant state abstractions.
Intuitively, this can be thought of as language-guided attention, allowing strong human priors to steer representation learning.
1
0
3
An amazing session in store today at the CoRL ARRH Workshop!
WS
@corl_conf
discussing representations that enable robots and humans to learn, reason & act. If these representations are inherently different, how can we align them computationally to make our interaction with robots efficient, fluent and transparent.
@andreea7b
@TheAndiPenguin
4
0
10
0
0
3
Our insight is that *end users are uniquely positioned to recognize which concepts are irrelevant for their desired task*. If we had a way to reliably query for irrelevant concepts, then we could use data augmentation to quickly finetune the policy. [3/8]
1
0
3
Policies deployed in the world face different sources of distribution shift. Data augmentation can help models be more robust by varying *task-irrelevant* concepts. But how do we know what is task-irrelevant vs. -relevant? [2/8]
1
0
3
@minsuk_chang
@roboticwrestler
@HsseinMzannar
Pretty math sadly isn't always reflected in real human studies :/ hence, one must sometimes make a choice on what to get "working" for a publication (e.g. make the math work or make the human work), and the choice of venue, it seems to me, reflects that choice in priority
0
0
2
In bandit experiments, we find that learning from pragmatic feature preferences outperforms learning either only example-level preferences or pragmatic-augmented features, verifying both elements are important for making use of contextual information contained in descriptions.
1
0
3
[3/n] Our key insight is that changes in human behavior tell us something meaningfully about these implicit preferences, or in other words, that different demonstrations for the same task implies there must be different *task-relevant features* at play.
1
0
2
[4/n] We extend previous work () to, given two contrastive demonstrations, also query language models for these hidden preferences. Our method utilizes off-the-shelf segmentation and captioning models to construct preference-conditioned-abstractions.
1
0
2
In computational experiments, we show that tuning to the "right" complexity supports the greatest finetuning accuracy for a small number of labels. [3/n]
1
0
2
[2/n] LMs have been deployed in robotics as general-purpose task specifiers and planners. But what happens if the user’s utterance does not convey a potentially hidden preference?
For example: the user may prefer Spot to avoid electronics but not clothes on the ground.
1
0
2
[6/n] Importantly, the LM is able to model its own uncertainty when faced with “ambiguous” preferences, and proactively ask the user for their true preferences when queried preferences are high entropy.
1
0
2
We introduce our full data, comprised of 38,400 individual human judgements over 9,600 prediction tasks, as a first-ever large-scale dataset for studying human-AI collaborative decision-making trained, collected, and evaluated on real data.
1
0
2
We contribute a pragmatic approach to data augmentation: we use feature-level preference data to synthesize new examples based on which features are not considered relevant. ✅
1
0
2
[5/n] Policies trained with PLGA are able to produce policies that generalize to new environments, such as Spot successfully avoiding new objects like laptops at test time.
1
0
2
Next up -
@yayitsamyzhang
talks about attending to what matters in representation learning!
@corl_conf
ARRH Workshop
@andreea7b
0
1
2
0
0
1
@iclr_conf
@jefrankle
@in4dmatics
@yonashav
Excitingly, along with
@SchmidtFutures
, we will also be awarding prizes for reviewers + best papers! Come check out the workshop here:
0
0
2
More experiments, discussion, and our user study can be found in the paper:
I had a great time on this paper with awesome collaborators
@dabelcs
@tianminshu
Yuying Sun!!
Reach out if you'd like to chat more!
0
0
2
0
0
2
1
0
1
In a user study, participants were able to select this "right" complexity level to best support a specified downstream task (for various tasks). [4/n]
1
0
1
@erichorvitz
@WHOSTP
Congrats Eric :) couldn't imagine anyone better contributing to such a critical place
0
0
1
0
0
1
@harmankkaur
@UMNComputerSci
@grouplens
So excited for you Harman!! Minnesota is lucky to have you :)
1
0
1