Mina Lee
@MinaLee__
Followers
3,558
Following
477
Media
14
Statuses
180
Assistant Professor at @UChicagoCS ✍️ AI-assisted writing, Human-AI interaction, Language models
Chicago
Joined March 2018
Don't wanna be here?
Send us removal request.
Explore trending content on Musk Viewer
Ricardo
• 112263 Tweets
Karime
• 97486 Tweets
#TheStatement7
• 81600 Tweets
重陽の節句
• 73201 Tweets
#LISAxVogueKorea
• 63456 Tweets
昭和99年
• 43630 Tweets
シーザリオ
• 43160 Tweets
BTS is 7
• 41725 Tweets
高市さん
• 32424 Tweets
Didi
• 27679 Tweets
BTS IS SEVEN
• 25373 Tweets
Stafford
• 22701 Tweets
MSSP
• 21093 Tweets
Happy New Week
• 20302 Tweets
PRABOWOJKW TetapSELARAS
• 19070 Tweets
ADUdomba UdahGAKJAMAN
• 18416 Tweets
UPICK ADD SUGA
• 17126 Tweets
कारगिल युद्ध
• 16576 Tweets
Priyadarshan
• 14268 Tweets
#NiceGuy_NEXTDOOR
• 13370 Tweets
落語心中
• 13158 Tweets
#9Eylül
• 11318 Tweets
Pinned Tweet
A Design Space for Intelligent and Interactive Writing Assistants
#CHI2024
👩🏻✏️🤖
What writing assistants do you use? What else are out there and how do they differ? What do we need to consider when designing new writing assistants?
🔗 (1/6)
4
51
217
CoAuthor: Human-AI Collaborative Writing Dataset
#CHI2022
👩🦰🤖 CoAuthor captures rich interactions between 63 writers and GPT-3 across 1445 writing sessions
Paper & dataset (replay):
Joint work with
@percyliang
@fabulousQian
🙌
5
91
443
I'll present and defend my work on writing with AI over the past 6 years 👩🦰🤖✍️
If you are interested in the topic and happen to be free this upcoming Tuesday at 4-5PM PT, you are welcome to attend my defense over zoom!
7
49
435
Language models (LMs) are already deployed in many real-world applications and used to interact with users 👩🦰, but these models are primarily evaluated non-interactively.
How can we evaluate LMs interactively and why is it important? (1/8)
5
57
336
I'll be joining the University of Chicago (
@UChicagoCS
and
@DSI_UChicago
) as an assistant professor in the summer of 2024. 👩🏻✨
Excited to start the Communication & Intelligence group with Ari Holtzman (
@universeinanegg
) and Chenhao Tan (
@ChenhaoTan
)!
Announcing the Communication & Intelligence (C&I) group at UChicago!
Comprised of
@universeinanegg
,
@MinaLee__
,
@ChenhaoTan
—C&I will tackle how AI and communication co-evolve as LLMs break long-held assumptions.
We're recruiting PhDs & Postdocs for 2024!
1
17
62
14
23
291
We are open-sourcing the CoAuthor interface for those who are interested in using a text editor to record and replay human-LM collaborative writing sessions!👩🦰🤖✍️
Code: (joint work with
@vishakh_pk
)
CoAuthor: Human-AI Collaborative Writing Dataset
#CHI2022
👩🦰🤖 CoAuthor captures rich interactions between 63 writers and GPT-3 across 1445 writing sessions
Paper & dataset (replay):
Joint work with
@percyliang
@fabulousQian
🙌
5
91
443
8
61
275
I got selected as one of the Korean Innovators Under 35 by MIT Technology Review! 🇰🇷🏆
Happy and honored! Will do my best to continue to learn, grow, and do good research. ☺️
11
14
218
Honored to be on the list! 🙏
I’m actively recruiting students who are interested in AI and writing to understand how AI will change the way we communicate. ✍️ Please consider applying to
@UChicagoCS
and
@DSI_UChicago
!
More info:
Here's a list of NLP superstars 🌟 who are beginning their journey 🚀 in the 2023/24 academic year.
@jieyuzhao11
@GabrielSaadia
@acbuller
@Lianhuiq
@ManlingLi_
@yuntiandeng
@rajammanabrolu
@YueDongCS
@tanyaagoyal
@MinaLee__
@yuntiandeng
@alsuhr
@wellecks
@hllo_wrld
@Xinya16
19
42
293
1
33
216
Can we simulate both a user👩🦰 and system 🤖 and learn to autocomplete in an unsupervised way?
Yes! We frame the autocomplete task as a cooperative communication game.
#StanfordNLP
Talk: Dec 14 9:45-10AM (West 118)
#NeurIPS2019
2
44
184
CoAuthor is featured in the Stanford Graduate School of Education's article on "A New Class of AI Tools" 🤖✍️💡
0
32
174
How to fill in the __blanks__ using language models
1. Download your favorite language model ❤️
2. Fine-tune the model on infilling examples 🤖
3. Use the model to fill in any number of blanks in text! 😮😮😮
#StanfordNLP
#ACL2020NLP
@chrisdonahuey
1
27
155
📢 Intelligent & Interactive Writing Assistants Workshop ()
#In2Writing
#ACL2022
We invite NLP and HCI researchers as well as industry practitioners and professional writers to build, improve, and evaluate AI-powered writing assistants! 👩🦰🤖✏️
1
35
120
Swords ⚔️: Stanford word substitution benchmark
#StanfordNLP
Can your model & writing assistant system find appropriate synonyms for a word in context? ✍️
June 9 (Wed) @ 9-10:20 AM PT
#NAACL2021
w/
@chrisdonahuey
@robinomial
@alexanderiyabor
@percyliang
0
24
102
Finally releasing my old book on how to apply to grad school 📕 (written in Korean) as a PDF file 😊
미국 대학원에 관심있는 분들이 주변에 있다면 공유해주세요 ❤️ 블로그 글도 계속해서 쓰려고 하는데 잘 써지지가 않네요 🫠
1
14
92
Curious how writers write with language models / how industries build writing assistants / how researchers evaluate their performance?
Workshop on Intelligent and Interactive Writing Assistants
#In2Writing
is happening tomorrow at
#ACL2022
! 🤖👩🦰✏️
0
19
60
@kchonyc
@nyuniversity
As
@percyliang
said, it's necessary to check Twitter to ensure your talk is up to date... up until the very last minute 🤦♀️
0
2
54
CoAuthor is featured on
@StanfordHAI
News and Announcements! ✏️
Joint work with
@percyliang
@fabulousQian
0
3
37
If you are planning to attend
#CSCW2023
, please consider submitting your questions for the panel. Also, let's grab coffee together! 😊☕️
With Sherry Wu, Mina Lee, Ken Holstein, Vera Liao, Hari Subramonyom, and Min Kyung Lee, I am organizing a CSCW panel to discuss how LLMs could influence CSCW and social computing research & vice versa.
What would you want to ask the panel?
0
18
75
1
1
36
When building AI writing support tools, what are the most important factors to consider? How can we make them safe and inclusive? How will they change the way we write?
Join the conversation at our workshop on Intelligent and Interactive Writing Assistants!
#CHI2023
✍️👩💻
The 2nd
#In2Writing
workshop will be at
#CHI2023
!
This year we invite 🔥2-page position papers🔥 that portray thoughts on writing assistants (see CFP). Submit a paper to join the in-person event!
🗓️Submission Deadline: 2/23
🗓️Workshop Date: 4/23 (Sun)
🔗
2
20
61
0
8
30
In a classroom setting, how can we harness the potential while minimizing the risks of large language models such as ChatGPT? 👩🏫👨🎓
Check out this article by Inside Higher Ed:
1
5
27
How can we synthesize programs with functional correctness? 🤖✔️
We release SPoC dataset (18K programs + human-authored pseudocode). Poster
#169
at 5-7PM today!
#NeurIPS2019
SPoC: Search-based Pseudocode to Code (
@sumith1896
,
@IcePasupat
et al.)
2
5
26
TL;DR: Standard (non-interactive) benchmark performance is an imperfect proxy for evaluating human-LM interaction. For more details:
Joint work with
@megha_byte
,
@amelia_f_hardy
,
@jwthickstun
,
@esindurmusnlp
,
@AshwinParanjape
, (cont.) (7/8)
2
5
25
Valentine's Day + AI = AI in Dating Apps? ❤️💔
It was both interesting and concerning to think about the use cases of AI for dating. Thanks
@mollyglick
for interviewing me!
For
@inversedotcom
’s Future of Love package, I explored how AI could change dating forever — from machine learning matchmakers to increasingly monotonous convos:
1
1
4
1
4
18
For example, datasets can answer:
✏️Can GPT-3 contribute new ideas to one's writing?
✏️Does this ideation capability fluctuate in different writing contexts?
✏️To what extent does it fluctuate when decoding parameters change?
Find answers to these questions in our paper!
2
3
15
Thank you
@fabulousQian
! Check out our paper and dataset here: ✍️
#CHI2022
By the way, I seem to have closed my eyes for all the photos for some reason 🫠
PC:
@joon_s_pk
Fantastic talk -
@MinaLee__
’s
#chi2022
presentation of our Best Paper Honorable Mention paper “CoAuthor” (in collaboration with
@percyliang
)— A much needed discussion on the capabilities and limits of LLMs when interacting with users!
4
11
123
2
0
15
Language models have highly context-dependent capabilities that are often subjectively interpreted 🤔
We argue that datasets can be one way to understand language models' generative capabilities in relation to interaction design, under various definitions of good collaboration!
2
0
14
We release the reviewed papers along with their design space annotations to help you navigate existing writing assistants:
🔗
This is a living artifact and we invite you to add new papers, features, and discussions to track future developments. (3/6)
2
0
13
We propose a design space as a structured way to explore the space of writing assistants.
By systematically reviewing 115 papers from HCI and NLP, we identify key design considerations around task, user, technology, interaction, and ecosystem. (2/6)
1
0
12
We design five tasks, ranging from goal-oriented to open-ended, to capture a variety of different interactions. Then, we evaluate three variants of OpenAI’s GPT-3 and AI21’s Jurassic-1 by constructing interactive systems that connect LMs and users via an interface. (3/8)
1
1
12
@tal_august
@aviinashbhat
Madiha Zahrah Choksi
@senjutidutta2
@Jin_LC_Guo
Md Naimul Hoque
@hai_yewon
@seyyedparsa
Agnia Sergeyuk
@ShibaniAntonett
@DishaShrivasta9
@LilaShroff
Jessi Stark, Sarah Sterman
@sitong0926
@ABosselut
@DBuschek
@josephcc
(5/6)
1
0
11
From 1000+ human-LM interactions, we first find that non-interactive performance and interactive performance can diverge. For example, a model with the worst accuracy achieved the best performance as an interactive LM assistant in certain contexts. (4/8)
1
2
11
This work was possible through a large community collaboration with 36 researchers across 27 institutions.
Here are the heroes behind the work ❤️
@katyilonka
@john_jyc
@sbuckshum
@_vipulraheja
@huashen218
Subhashini Venugopalan
@WThiemo
David Zhou
@Emad_A_Alghamdi
(4/6)
1
0
10
Second, users sometimes perceived LMs to be more helpful than they are. For instance, users were easily deceived by misinformation from LMs, especially when they were fluent. We further observe that short prompts tend to exacerbate misinformation and toxicity. (5/8)
1
2
10
Ines Gerard-Ursin,
@XiangLisaLi2
,
@faisalladhak
,
@frieda_rong
,
@rose_e_wang
,
@MinaeKwon
,
@joon_s_pk
,
@CaoHancheng
, Tony Lee,
@RishiBommasani
,
@msbernst
, and
@percyliang
. Work from
@StanfordCRFM
&
@StanfordNLP
. (8/8)
2
1
10
Paper: Learning Autocomplete Systems as a Communication Game (
@MinaLee__
,
@tatsu_hashimoto
,
@percyliang
)
Poster:
2
0
10
@ffpaladin
@maxkreminski
Joonsuk Park
@rdp_life
@eugeniarho
@shannonzshen
@Siangliulue
(6/6)
Feedback is greatly appreciated and welcomed!
0
0
9
Lastly, we observe a discrepancy between metrics based on third-party and first-person perspectives, suggesting that what users find helpful in interacting with and improving these models over time might be not captured by standard benchmarking practices. (6/8)
1
1
9
Congrats
@WanyuDu
and coauthors! Look forward to learning more about your work at the
#In2Writing
workshop next week! 🏆✍️
If you are interested in human-machine interactive text revision, please check out our paper, which won 🎉the best paper award🎉 at
#In2Writing
workshop at
#ACL2022
! We attach more details below.
Paper:
Demo:
3
7
38
1
1
8
We develop a framework, HALIE, that captures (i) the interactive process, not only the final output; (ii) the first-person subjective experience, not just a third-party assessment; and (iii) notions of preference beyond quality. (2/8)
1
3
8
@tatsu_hashimoto
@percyliang
Talk: (from 50 min / from slide 87 / from 24. Learning Autocomplete Systems as a Communication Game) 🎙️
0
0
6
@timfong888
Here is the abstract for the talk! My thesis is yet to be written... coming soon! ✍️
0
0
2
@AliceAlbrecht
@percyliang
@fabulousQian
There has been big power outage at Stanford which is affecting the server that’s hosting the website ☠️ Hopefully it will get fixed soon! Meanwhile, you can access it via — happy researching! 😊
1
0
2
@m_guerini
@percyliang
@fabulousQian
2/n During data collection, we didn't analyze factuality of GPT- or human-written text as we were not experimenting any kind of interventions.
However, I'm quite certain that GPT-3 must have hallucinated, which might have made humans generate invalid arguments as a result.
0
0
1
@m_guerini
@percyliang
@fabulousQian
3/3 In fact, one participant reported "there was not a single source cited that was accurate--It had quotes from unrelated authors, studies, websites" 😂 It would be great future work to analyze how GPT-3 impacts humans' factuality (similar to named entity analysis in the paper)!
1
0
1
@m_guerini
@percyliang
@fabulousQian
1/n Enjoyed reading your paper! It is amazing that your annotators underwent thorough training for two weeks and were asked to check veracity of model outputs 😮
0
0
1
While thinking about the topic, I came across "AI and education: guidance for policy-makers" by UNESCO (), which includes benefit-risk assessments and policy recommendations for using AI in education.
What other reading resources would you recommend?
2
1
1