It's official! I will join
@SCSatCMU
@cmuhcii
as an Assistant Professor in Fall 2022, with a courtesy appointment
@LTIatCMU
. Toughest decision ever, but I'm excited (albeit scared) to see what comes next :) THANK YOU everyone for all your support!
So I finally did what I promised 3mo ago – We wrote a reflection paper on using LLMs to replicate crowdsourcing pipelines, based on an assignment in our Human-Centered NLP course!
20+ students implemented 7 crowd. pipelines from prior research & find…
1/
Best gift for the Children's Day in China: I defended👧🏻👩🏻🎓! Thank you my advisors
@jeffrey_heer
@dsweld
, committee members
@marcotcr
@nlpnoah
, Mari Ostendorf, collaborators, family, & everyone I encountered in my life for making this possible❤and thx
@SarahHShen1
for the flower🌸
RIP 😢 Yifan tragically took her own life shortly after leaving ETH, and, like many others on my timeline, I've been hearing some unsettling rumors her experiencing mistreatment while at ETH. I hope ETH can look into this and provide justice for Yifan and all parties involved.
Aww last day of lecture & these notes from students made my day ❤️They gave me the courage to share the course material – Check out our new Human-Centered NLP course if you're interested!
1/
New preprint alert!
*Tailor: Generating and Perturbing Text with Semantic Controls*
Title says it all: we perturb sentences in semantically controlled ways like how a tailor changes clothes 🪡.
w/
@alexisjross
,
@haopeng01
,
@mattthemathman
,
@nlpmattg
1/n
Just arrived at NOLA so I'm gonna join the "say hi to me at
#CHI2022
” party! Fun fact: this is my first AND last in-person conf as a phd student, where I have a full paper to present. I missed ALL the others, either bc of visa or pandemic😅 So be kind to a senior and junior me :)
Ever surprised by users’ unexpected behaviors? Wondered why they prefer one app over another that’s powered by the same model? Come seek for answers in our
#EMNLP
2023 tutorial: “Designing, Evaluating, and Learning from Human-AI Interactions”! w/
@Diyi_Yang
@SebastinSanty
1/2
hihi Twitter friends! I'm in the middle of greencard application which has proven very challenging🥹If you have included my work in your courses (e.g. as readings or in lectures), could you please DM me the details like syllabus? It will enhance my app greatly! Thanks a ton!
BEST OVERALL PAPER AT
#acl2020nlp
:
🎉🎉🎉
Beyond Accuracy: Behavioral Testing of NLP Models with CheckList
Marco Tulio Ribeiro, Tongshuang Wu, Carlos Guestrin and Sameer Singh
🎉🎉🎉
Slides available! Thx everyone for coming and for all the fruitful questions! Online participants -- again very sorry for the poor/non-existent WIFI connection...We will try to post a recorded version afterwards, stay tuned!
Ever surprised by users’ unexpected behaviors? Wondered why they prefer one app over another that’s powered by the same model? Come seek for answers in our
#EMNLP
2023 tutorial: “Designing, Evaluating, and Learning from Human-AI Interactions”! w/
@Diyi_Yang
@SebastinSanty
1/2
We're so excited to have Sherry Tongshuang Wu (
@tongshuangwu
) with us as a new member of the
@cmuhcii
and
@LTIatCMU
faculty this year! Here, she talks about creating human-centered solutions for debugging and correcting errors in AI.
I feel proud but also weirdly shocked seeing so many gov depts (including CDC!) actually rely on the data crowdsourced by the volunteers and developers at
@1p3aDev
lol A big shout out to them for their reliable daily updates on confirmed case growth, test status, policies, etc.!
Got >10 inquiries about PromptChainer () in the last month- amazing to see the interest! Unfortunately it was my Google internship work (with my amazing collaborators), and now that I'm not affiliated, I don't know if/when it'll be public. Sorry! 1/
Wow totally shocked that I'd have to pay $200 *per
#CHI2022
workshop* (even as a co-organizer😅), on top of the $500 student registration lol Did I just forget how expensive in-person conferences are? Has it always been this price??
1/ I usually try to avoid posting anything politically related on social media, but as the president of a developed country, please, PLEASE have the decency to call it by its official name, COVID-19.
Thread ↓
The United States will be powerfully supporting those industries, like Airlines and others, that are particularly affected by the Chinese Virus. We will be stronger than ever before!
Heading to
#emnlp2022
? Interested in NLP research but even more in the stories behind research? Please join us at our Story Shared and Lessons Learned (SSLL) workshop, happening on *Dec 8* at *Capital Suite 12B* (or on Zoom - Pls check Underline)! (1/)
Check out our new blog post on our
#acl2019nlp
paper! A ~10min story on how to correctly debug your ML systems, with the help of our Errudite tool. Joint work with
@marcotc
,
@jeffrey_heer
and
@dsweld
; Posted on the
@uwdata
blog.
Excited to host Penn State CSRAI Young Achievers Symposium this semester! Our first speaker is
@tongshuangwu
(Ph.D. student at UW), who will be giving a talk on January 18th, 4 - 5pm EST.
My Twitter feed now explodes with amazing NLP preprints on every 15th. I “like” & bookmark many many of them, but when to actually read them is a 1M question…
Adding my two cents as a grad student — I genuinely think people here are very supportive & diversity is an active topic here, & Pedro D’s behaviors have not gone unnoticed. Before he retired, most faculty members used to openly disagree and challenge him on multiple issues, 🧵
Not gonna amplify that other tweet but I’ll instead point to this one. Speaking as a relatively newish member, I’ll say that I’ve found
@uwcse
to be a supportive place for women, w/ a strong commitment to diversity and inclusion from our leadership.
I consider
@marcotcr
my unofficial advisor & I survived my PhD largely by stealing his research style😀 Now he decided to share his wisdom! Definitely read if you ever wonder about e.g. what project to do, how to flesh it out, etc. I testify they are super helpful!
We are thrilled to have Dr. Wei Xu (
@cocoweixu
) from GeorgiaTech and Dr. Sherry Tongshuang Wu (
@tongshuangwu
) from CMU as our morning and afternoon keynote speakers at HEAL
@CHI24
Thank you both for sharing the insights from
#HCI
and
#NLProc
perspectives
Debugging is becoming more critical everyday with “AI pair programmers” contributing bugs, but no one remembers it taught in class…We built an LLM-based tutor to help & want to see if you like it - If you know basic Python, pls sign up for our study! 🧵
Excited that Sherry Tongshuang Wu (
@tongshuangwu
) from
@SCSatCMU
/
@cmuhcii
will join us as a keynote speaker at
#NAACL2022
! Sherry will be talking about the model's role in model-in-the-loop data collection particularly in the contexts of CheckList, Polyjuice and Tailor 🚀
Checkout our new interesting work w.r.t. the "merge conflict" of the external input vs. LLM internal knowledge! Even irrelevant info may sway how LLMs answer questions 🤯 I like projects that surprises me and this def. counts :)
📢 Thrilled to announce our latest paper!
"Merge Conflicts!" Exploring the Impacts of External Distractors to Parametric Knowledge Graphs:
🤖 Experience the CLASH of external knowledge and LLM's parametric knowledge through its inner mechanism!
(1/n)
A team of collaborators from ALL different institutes? 5 female researchers + 1 high school student? I am excited that our fairness work "Pretty Princess vs. Successful Leader: Gender Roles in Greeting Card Messages" is conditionally accepted by
#CHI2022
! Stay tuned for details!
Super happy about this collaborative effort! I especially like the unique integration of rich metadata for datasets including identifiers, data attributes, & provenance. I'm hopeful that they will enable us to understand, compare and pick among the otherwise scattered datasets!
📢Announcing the🌟Data Provenance Initiative🌟
🧭A rigorous public audit of 1800+ instruct/align datasets
🔍Explore/filter sources, creators & license conditions
⚠️We see a rising divide between commercially open v closed licensed data
🌐:
1/
We all interact with
#LLMs
every day, but how to design better human-LLM interaction?
Come to our
#NAACL2024
tutorial: 🌟Human-AI Interaction in the Age of LLMs 🌟 w/
@tongshuangwu
Marti Hearst. We will cover 3 aspects from a HCIxNLP perspective:
🎨🎨Design: Why should they
...I'm very behind on writing descriptive tweets, but hey our CHI LBW 2022 paper is cool, check it out! TL;DR: a sequel to our AI Chain () w/ many
@GoogleAI
PAIR folks, on helping developers/designers prototype AI-infused apps with a chain of LLM prompts.
We all interact with
#LLMs
every day, but how to design better human-LLM interaction?
Come to our
#NAACL2024
tutorial: 🌟Human-AI Interaction in the Age of LLMs 🌟 w/
@tongshuangwu
Marti Hearst. We will cover 3 aspects from a HCIxNLP perspective:
🎨🎨Design: Why should they
CMU HCII is hiring tenure track faculty & teaching track faculty:
Ping me if you have any questions. Technical HCI is one of the areas we are interested in. //cc
@sigchi
@ACMUIST
@cmuhcii
Milestone: First-ever virtual poster presen at
#RisingStars2020
lol Big shout out to the committee & the mentors for such a rewarding event! I was thrilled to take a peek at the faculty application procedure & to learn about all the great work from the cohort :)
@Berkeley_EECS
So, here’s the preprint I was talking about:
“Polyjuice: Automated, General-purpose Counterfactual Generation”
The title says it all: We develop a “polyjuice” transforming sentences into counterfactuals.
w/
@marcotcr
,
@jeffrey_heer
,
@dsweld
(1/5)
We have a diverse PC & reviewer pool with expertise in HCI, ML, NLP, and AI, and we all look forward to your submissions! And we have an entire half-day for group exercises and activities so come have fun with us 🙌
Not sure about "successful", but happy to share stories! Please feel free to join us if you'd like to hear me rant about applications and interviews (j/k)
Interested in working as a professor in HCI? We invite
@tongshuangwu
and
@ThijsRoumen
to share their experience as recently-successful
#HCI
job-seekers this Friday (9/23) at 11am PST on Zoom. Register and send us your questions here 🙌
📢
#AAAI
#HCOMP2024
will take place in Pittsburgh, PA. The theme for this year could not be more urgent and relevant: "Responsible Crowd Work for Better AI." See the call for papers and more details here 👉 ✨ Do consider submitting your work! 🌟
A direct instantiation of my profile pic! And apparently it makes your talk memorable. Cute flower from
@yanghci
's lab, and cool pic from
@a_a_cabrera
(if you zoom in enough you see the flower🌸)
New skill unlocked🔓: making bubble tea during the (remote) social event w/
@GoogleAI
pair team! Yummy🧋, except that I prepared wayyyyy too much syrup and I don't know what to do with them...esp. since I don't even drink my tea sweet lol
And our next speaker is...
🗣Tongshuang Wu (
@tongshuangwu
) to talk with us about "Principles and Tools for Evaluating and Improving NLP Models"
🗓April 28th, 00:00 UTC
📝Sign up here:
NAACL 2024 (1/4)
@naaclmeeting
- Combating Security and Privacy Issues in the Era of Large Language Models. Muhao Chen, Chaowei Xiao, Huan Sun, Lei Li, Leon Derczynski and Anima Anandkumar.
- Human-AI Interaction in the Age of LLMs. Diyi Yang, Tongshuang Wu and Marti A. Hearst
#chi2020
papers going to
#tweetchi
, NO.2: "Local Decision Pitfalls in Interactive Machine Learning: An Investigation into Feature Selection in Sentiment Analysis"! with
@dsweld
&
@jeffrey_heer
, we show rapid iterations with IML systems can be dangerous! ↓
Come chat about generating counterfactual examples with me!
#ACL2021NLP
🕗 8/4 8:00 am PT
📍15A: LANGUAGE GENERATION
#3
Polyjuice: Generating Counterfactuals for Explaining, Evaluating, and Improving Models
Or, you know, please catch me on gather town! :P
📢🌟Just 2 more days for our
@hcomp_conf
and
@ci_acm
conference!
Meet the speakers of the final day 8th Nov!
Looking forward to the talk by Sherry Wu
@tongshuangwu
-
As PhD application deadlines approach, we are super excited to announce , created by
@zhaofeng_wu
@alexisjross
@ZejiangS
💻
is a platform with statements of purpose generously shared by previous applicants to CS PhD programs
🧵(1/n)
THANK YOU
@marcotcr
for writing all these because this means I'd just share them with my students lol
AND Bonus: I'm told that I'm "featured" somewhere in this post, so see if you can spot me :)
With Sherry Wu, Mina Lee, Ken Holstein, Vera Liao, Hari Subramonyom, and Min Kyung Lee, I am organizing a CSCW panel to discuss how LLMs could influence CSCW and social computing research & vice versa.
What would you want to ask the panel?
Happy to announce that I've formed a company, Inspired Cognition () together with
@stefan_fee
and
@odashi_en
!
Our goal is to make it easier and more efficient to build AI systems (particularly NLP) through our tools and expertise. 1/2
I first got to know
@lxieyang
in 2017 as his UW PhD visit day scheduler but CMU won him over lol Time flies...Still, we stayed in touch & it's great to see him succeed as a researcher! He writes solid code, chooses projects with taste, makes cool figs, & is just a great person!
I’m also on the job market for industry research positions this coming 2023! I design and build systems that accelerate online sensemaking for developers and facilitate human-AI interactions for end-users.
Please get in touch if you know of any relevant opportunities! 😃
Many people asked for an extension, so here it is -- you now have another 13 days to prepare your submission to
#chi2022
TRAIT workshop✍️ Accepted submissions are required for attendance, so make sure to submit & join us!
Yayyyy!! Now ALL we need to do is just to actually write this paper...lol😅😅
Why cant imaginary papers just...write themselves??
Loved the discussions with my teammates, and a big shout out to the organizers!
#HCXAI
#CHI2021
I'm fascinated by cross-doc reading / annotation & I want it on news but also on research papers that make similar or contrastive claims :) Also I think
@jerelev
vision of ref. free verification is super valuable and deserves a lot of attention!
Ever wish you could read multiple articles at once? easily grasp consensus & disagreement in a cluster of documents? or get updated info as you read?
Excited to share NewsSense, a tool that offers “reference-free verification” (soon at
#EMNLP2023
):
1/🧵
I practiced a talk that’s by design half Chinese and half English with my mom.
Mom goes “your English presentation is much better than your Chinese parts”… bilingual=bye-lingual lol
Vera is accepted to
#EMNLP2023
!! 🎉🎉
We’re releasing a community-contributed benchmark for commonsense evaluation, derived from user interactions with Vera in the past months. (Thanks everyone for using our demo!)
This new challenging dataset is on HF:
Jim is one of my closest friends in grad school and a great researcher/human being -- Almost all of my projects benefited from random chats with him😛 If you are hiring you won't regret interviewing him!
Hi Twitter/X folks!
Excited to announce I am going on the job market this cycle! (industry & academia)
I work on building uncertainty-aware tools and workflows that support capturing and defining socially-constructed concepts at scale.
Here are some examples of my work:
(1/n)
2/ If you think "nicknaming" the virus with its place of origin is not racism, please checkout this video:
Also, all those virus named after some countries are from acient times, NOT the 20th centry. No one called H1N1 in 2009 "American/Mexico virus."
So, I decided we need an NLP course for HCI people & an HCI course for NLP people. Making the course material from scratch was a nightmare (seriously!), but I'm glad I did it -- I learned so much from prep. and student Q&As. Shoutout to those who helped me make it happen! 📷3/3
I created this course because this quote from
@fabulousQian
's Sketching NLP paper keeps hunting me: “HCI people design useful things that NLP people cannot build; NLP people make things that nobody uses.” Working at the intersection I can say it’s just…too true.
2/
My student Kayo Yin needs your help. Her visa has been unnecessarily delayed, which would prevent her from coming to UC Berkeley to start her studies. Despite bringing all required documents, the
@StateDept
refused to process the visa and it could take months to re-process.
11/ For people read all the way to here (thx!!), some cool videos showing what the Chinese have been doing:
Fun in Quarantine (in Eng):
An MV (w/ Chinese captions):
end :P
@tongshuangwu
et al developed an automated QA generation model architecture using a newly released book QA dataset. The model extracts candidate answers from a given storybook passage, generates appropriate questions, and ranks top QA pairs using another
#QAmodel
#MachineLearning
Thx
@huashen218
for allowing me to tag along this amazing reflection! It's always fascinating to see how people use a variety of metrics and wording to capture the essential elements of a research field😄
If you need detailed lists of "evaluation metrics" and "survey questions", we surveyed SOTA co-writing systems and summarized in this table🤗! We further describe "how to use Parachute🪂 in practice" with a case study. Lmk if you’re interested in chatting more abt our work🥰!
Heading to
#emnlp2022
? Interested in NLP research but even more in the stories behind research? Please join us at our Story Shared and Lessons Learned (SSLL) workshop, happening on *Dec 8* at *Capital Suite 12B* (or on Zoom - Pls check Underline)! (1/)
@kayo_yin
Talk to people who are not affiliated with any of your lab-of-interest, but are familiar with them/ in your field. You'd be surprised how gossip the field is, and can sometimes get unbiased suggestions on research match, group environment xD
#chi2020
papers that miss Hawaii deserve a stage on
#tweetchi
! Excited to share NO.1/2: "Tempura: Query Analysis with Structural Templates"! We show how structural-template-based query groups reveal the dataset distribution & ML model error patterns.
Demo:
Today at
#chi2022
in the 2:30 Natural Language session
@tongshuangwu
will present work from her PAIRminternship w/
@Carryveggies
& Michael Terry on "AI Chains: Transparent and Controllable Human-AI Interaction by Chaining Large Language Model Prompts"
Google PAIR is hiring a postdoc in Human-AI Interaction! Systems-building (e.g. prototyping) & HCI publication experience (e.g. CHI/UIST/CSCW/IUI) preferred. This is a 1-2 yr full-time role based in Mountain View CA, Seattle WA, or Cambridge MA, starting fall/summer '22. (1/2)
We've thought a lot about how explanations help us understand AI models, but how can AI-gen expls. help us understand...the world?
@ChengleiSi
's work will be an important add-on to the future of AI-infused search! Spoiler alert: unsurprisingly overreliance remains a headache...
How can we humans verify the truthfulness of LLM outputs (or any claims you see on the Internet)? Should we ask ChatGPT (
#LLMs
)? Search on Google (retrieval)? Are they complementary?
Tldr: LLMs Help Humans Verify Truthfulness - Except When They Are Convincingly Wrong!
1/n
Overall, it was fun writing a paper with students in courses! And def. the best way I can imagine wrapping up my first-ever course. Thanks
@Haiyi_Zhu
for inspiring me on the assignment design and all the students on their amazing work w.r.t assignment and paper revision! 5/5
9/ It's ok if you disagree with me, but the disagreement cannot erase the amount of effort that Chinese citizens have devoted into the battle with COVID19. We've experienced great loss, but we've never hesitated to share valuable experience with the rest of the world.
Tomorrow is the first day of ChinaVis 2020 and
@tamaramunzner
will give a keynote speech titled "Problem-Driven Visualization Through Design Studies". The conference dates were rescheduled due to the COVID-19 situation. Currently, there are over 560 registered attendants.
10/ Till today, Chinese government has supplied mountains of medical supplies to Iran, Korea, Japan, and etc. Showing some respect to such a generous friend during this battle between mankind and COVID19 is the least we could all do.
#STOPFINGERPOINTING
4/ If you think China could have stopped the outbreak months ago: Without knowing anything about the virus at the beginning (i.e. infection/fatality rate), the Chinese government has put a tremendous amount of effort in stopping this virus from spreading.
7/ If you think the Chinese brought the coronavirus because we eat bat: Regardless of the fact that the primary source of this virus has yet been confirmed, it's not an uncommon thing for countries to have a unique food culture.