Exciting personal update!! I am thrilled to share that I defended my PhD
#PHDone
🎉
Next steps: Young Investigator/Postdoc
@allen_ai
. I’m super excited to work with
@YejinChoinka
and the
@ai2_mosaic
team, as well as to join the vibrant NLP community in Seattle 🗻✨🧋☕️
Come intern with us
@ai2_mosaic
next summer!!
I am recruiting interns interested in
📋evaluating generative models
🔥robustness
🔎model interpretability
🥸 privacy
If this interests you, please apply by **November 6th**!
Retweets/shares appreciated 🥰
The Mosaic team at AI2 is looking for interns for Summer 2024. The deadline for applying is November 6th. If building and researching machines with commonsense intelligence while working with awesome mentors sounds exciting to you, apply at
✈️I'm in Seattle for
#NAACL2022
! Would love to meet and chat about NLU evaluation, robustness, trustworthy NLP, interpretability and model analysis, NLP with Friends, grad school, coffee, or anything else☕️🧋. I love to make friends, please say hi if you see me! :)
I was selected as a "Rising Star in Data Science", and was at the workshop at
@DSI_UChicago
last week! The campus was beautiful, and lovely to meet old friends and make some new ones😊 Check out their work here:
I am recruiting a PhD research intern at
@ai2_mosaic
!
Topics of interest include:
📋generative model evaluation
🔥robustness
🔎model interpretability
🥸 privacy
✨Deadline: Nov 6 (today!!)✨
The Mosaic team at AI2 is looking for interns for Summer 2024. The deadline for applying is November 6th. If building and researching machines with commonsense intelligence while working with awesome mentors sounds exciting to you, apply at
What a great
#NAACL2022
!! I couldn't say a proper goodbye to everyone, but it was so meaningful to meet in-person after the long pandemic years.
(Also got
#dalle2
access last week!)
✈️🇸🇬I'm in Singapore for
#EMNLP2023
!
Would love to meet and chat about NLU evaluation, robustness, trustworthy NLP, interpretability and model analysis, privacy, coffee, life, or anything else🍜☕️
I love to make friends, please say hi if you see me! :)
This peaceful room was one of my favorite places
@SCSatCMU
. This is not ok.
Image description: CMU SCS reading room with "No Women Allowed" scrawled on the glass door
Our NLP systems are getting more accurate—are they also getting useful?
To properly evaluate, we need to consider context(s) of use. New
#EACL2021
paper on QA evaluation w/
@siddalmia05
@maria_ryskina
Florian Metze
@ehovy
Alan Black
All the details: 👇
It was a delight to host
@johnhewtt
today
@NLPwithFriends
. Thanks for a very *clarifying* talk and interesting discussion. And thanks to all our friends for attending!🤗
We are skipping next week, but join us on Sep 23 for
@davlanade
's talk on NLP for African Languages! 🌍🥳
In the coming days, I’ll be joining the
@aclmeeting
microbloggers on the new “Interpretability and Analysis of Models for NLP” track! Looking forward to covering thoughtful work aimed at *understanding* model behavior🤖
#acl2020nlp
#acl2020
Happy 2022!🎉 Do join us for this talk, where I will discuss:
🔎 My research on how we should analyze and evaluate NLP models
🗣️ My experiences doing a Ph.D. in NLP
You can ask me anything in the QA session!
#NLProc
#Women_in_NLP
I'm excited to announce the first
#women_in_nlp
talk in 2022 with Abhilasha Ravichander (
@Lasha1608
). She is going to tell us about her research and share her experience as a Ph.D. student
@SCSatCMU
. Join us on 21/01/2022 at 18:00 GMT at .
#NLProc
@dair_ai
(Very) late to the Twitter party, but excited to present work w/i
@boknilev
and Ed Hovy
“Probing the Probing Paradigm: Does Probing Accuracy Entail Task Relevance?”
looking at the information probes recover as they relate to task usefulness
#EACL2021
Our work on building EQUATE, a new benchmark evaluation platform for numerical reasoning in NLI is finally out! This is joint work with
@arnaik19
, Carolyn Rose and Ed Hovy, to appear at
@conll2019
. Paper: . Code/data: (1/9)
Catch our work on context-driven QA evaluation at
#EACL2021
! We study the effect of ASR, MT and Keyboard interfaces to QA systems.
Link:
Time: Apr 21, 9-11 AM EST (Gather-1A) or 8 AM EST (Zoom-2C)
Website:
Discussion welcome!
Check out our new work
@allen_ai
on analyzing web-scale corpora!
Super excited about this large-scale effort to actually understand what is in the data that powers language models🔍🧐
I won't be attending
#EMNLP2022
in-person due to visa issues (🥲), but if you are interested in negation, robustness and contrastive data collection, catch
@amarasovic
presenting CondaQA at Poster Session 10, Saturday, Dec 10 at 11 AM.
If you are applying to 🎓🤖grad programs in AI🤖🎓, here are three great resources:
1. Student perspectives on applications:
2. Example SoPs from recent applicants:
3. The CMU application mentorship program:
New paper "EIGEN" (), a framework that leverages large-scale language models to automatically generate event influences.
An incredible effort led by
@aman_madaan
, starting from a *course project*
(P.S. He's applying to PhD programs soon 😉)
Our new paper "EIGEN: Event Influence GENeration
using Pre-trained Language Models" with
@aman_madaan
@dheerajgopal
Yiming Yang,
@Lasha1608
& Ed Hovy explores the use of language models for generating event influences.
Paper:
Code:
Thanks to
@probablyNitika
for a very interesting conversation at
@NLPwithFriends
last night!
If you missed the talk, you can catch it here:
Don't miss
@sjmielke
next week, telling us about comparing generative LMs and anger-driven development! 💯
We are very excited to announce our next speaker!!
🗣Nitika Mathur(
@probablyNitika
), telling us about "Tangled up in BLEU: Reevaluating the Evaluation of Automatic Machine Translation Evaluation Metrics"
🗓August 26th, 23:59 UTC
📝Sign up:
Hello
#NLProc
twitter! We are NLP with Friends, a friendly online NLP seminar series where students present ongoing work.
Join us to meet our amazing speakers, learn about cool research, and collectively think about what’s next:
See you soon! ❤️
The "Interpretability and Analysis of Models for NLP" track at
#EMNLP2023
is in need of emergency reviewers!
If you are able to review at least one paper by Aug 10th, please DM me 🙏
#NLProc
New quarantine hobby: since I spend so much time on
@arxiv_cscl
, I decided to make a periodic thread on recent interesting papers (i.e past 24 hours🙃). Largely biased towards natural language understanding (May 7, 2020):
@kelina1124
@LTIatCMU
@ETH
I am really sorry to hear this happened, and that proper support for handling the situation institutionally was not available. Thank you for sharing this story. I hope our community can discuss how to do better
#NLProc
Can we sanity-check our NLI Models? New Paper at
#COLING2018
on "Stress Test Evaluation for Natural Language Inference" : . Joint work with
@LTIatCMU
@isrcmu
researchers, evaluation exercises neural NLI models on large-scale stress tests of their failures:
@rctatman
Non-exhaustive list of studies I liked: 1) BERT Rediscovers the Classical NLP Pipeline by
@iftenney
et. al, 2) Still a Pain in the Neck by
@VeredShwartz
&Dagan 3) How Contextual are contextualized word representations by
@ethayarajh
and a nice survey:
#emnlp2020
COGS: A Compositional Generalization Challenge Based on Semantic Interpretation
Najoung Kim, Tal Linzen
Paper:
Talk:
@najoungkim
,
@tallinzen
/1
We are very excited to announce our next speaker!!
🗣Tom McCoy(
@RTomMcCoy
), telling us about "Universal Linguistic Inductive Biases via Meta-Learning"
🗓August 12th, 14:00 UTC
📝Sign up:
Keep up to date with talks at
Thanks to
@emilymbender
@ojahnn
@zibuyu9
for this initiative, particularly for emphasizing first-time authors. I would like to add that if you're from any underrepresented group in NLP and would like coverage of your paper, please reach out! I would love to amplify your work ❤️
This talk was SO GOOD. Thanks to
@SebastinSanty
and
@Roprajo
for speaking with us, and to all friends who joined in for a fascinating discussion!! The video is now up
We are off next week for EMNLP, but we'll be back on Nov 25th with
@feralvam
#NLProc
We are super excited to announce our next speakers!!
🗣️ Pratik Joshi(
@Roprajo
) and Sebastin Santy(
@SebastinSanty
), telling us about "The State and Fate of Linguistic Diversity and Inclusion in the NLP World."
🗓️ November 11th, 14:00 UTC
📝 Signup here:
We are two days from our first talk with the amazing
@lena_voita
!!
Help us make this the friendliest space in NLP — check out our community guidelines and code of conduct before the talk 🥳
I am very excited about our initiative to promote the work of marginalized researchers in our community at
#NAACL2022
! If this applies to you, we would *really really really* love to feature your work.
Please fill in your details here by ✨July 3✨:
NAACL D&I is starting a new initiative to promote the work of marginalized researchers in our community. If this is applicable to you and you are presenting at NAACL 2022, we would love to feature your work! Please fill in your details here by July 3:
Thanks
@pmichelX
for joining us
@NLPwithFriends
, and for a very interesting conversation! In case you missed it, the recording is now up:
Next week we have a double feature,
@Roprajo
and
@SebastinSanty
telling us about linguistic diversity in NLP
#NLProc
Another week, another friend. Next Wednesday we will be joined by Paul Michel!!
🗣Paul Michel (
@pmichelX
), telling us about "Modeling the Second Player in Distributionally Robust Optimization"
🗓November 4th, 14:00 UTC
📝Sign up:
Nominations open for the next edition of
@NLPwithFriends
! 🎉
Nominate yourself, your friends, your colleagues, your students, or anyone else you would like to hear speak🤗♥️
NLP With Friends is opening nominations for our Fall 2021 series!!
We want to hear from YOU — nominate people you’d like to hear from, self-nominations accepted and encouraged!
🗓️Nominations close July 15
📝Nomination form:
🎙️Talks return on Aug 11!
Super excited that we'll be hosting John (
@johnhewtt
) next week! Come join us, friends 🥳
#NLPWithFriends
#NLProc
P.S. We won't be recording this session, so remember to register if you plan to attend! Link:
Very thankful for the chance to give this talk! Students interested in understanding neural representations of language, I’d love if you came and gave your thoughts and perspectives on this ongoing work on the probing methodology.
It is somehow hard to say enough good things about Ana ❤️ Future grad students: I think Ana will be an amazing and kind advisor, and you should definitely consider applying to her group!
I'm recruiting students!
My interests include measuring usefulness of explanations for human-AI collaboration, addressing human factors that confound such measurements, & modeling interactive explainability (multimodality, few/zero-shot learning, dialogs, personalization, etc)
#ACL2023
#ACL2023NLP
track "Interpretability and Analysis of Models for NLP" is missing 20% of assigned reviews 🥲
Please please please, if you can review even 1 paper until March 16, fill out this form: 🙏
Attending
#EACL2021
? Please come chat if you are interested in probing, or model diagnosis, or probing as a tool for model diagnosis!
🕑Time: 14:00 CET (on Zoom), or 15:00-17:00 CET (on Gather)
📜Paper:
🚀Rocketchat:
(Very) late to the Twitter party, but excited to present work w/i
@boknilev
and Ed Hovy
“Probing the Probing Paradigm: Does Probing Accuracy Entail Task Relevance?”
looking at the information probes recover as they relate to task usefulness
#EACL2021
Nominate speakers for the next edition of
@NLPwithFriends
!
Thanks, friends--- the 2020 edition was great, and we hope to have a diverse set of excellent speakers for 2021 as well. Please feel free to DM us if you have any questions about nominations, or feedback on the form❤️🤗
🗣️Who is our next speaker?
We want to hear from YOU — we’re opening nominations for the 2021 edition of NLP with Friends!! We encourage you to nominate people you’d like to hear from, and also yourselves❤️🤗
🗓️Nominations close Sunday, January 10
Aakanksha Naik
@arnaik19
& Abhilasha Ravichander
@Lasha1608
: "Stress Test Evaluation for Natural Language Inference". NLI requires modelling of quantification, compositionality and many more. Paper:
#coling2018
#NLProc
I've been debating just how candid I should be about my excitement for this upcoming talk. I landed on being forthright.
@sulin_blodgett
's work is IMO some of the most important work that's being done in
#NLProc
these days. I literally cannot wait for this conversation.
Introducing TLDRs on Semantic Scholar, now available in beta for ~10M Computer Science papers!
These auto-generated extreme summaries help you decide which papers are most relevant to your work.
Learn more:
#TLDR
#sciencetwitter
🗣️Who is our next speaker?
We want to hear from YOU — we’re opening nominations for the 2021 edition of NLP with Friends!! We encourage you to nominate people you’d like to hear from, and also yourselves❤️🤗
🗓️Nominations close Sunday, January 10
This work began in a wonderful internship
@allen_ai
. I could *not* ask for more brilliant or empathic mentors than
@nlpmattg
@anmarasovic
.
Many more results and analysis in the paper. We welcome discussion!
📝Paper
🚀Data [8/8]
Finally, please please DM me if you are based in Seattle and would like to chat about things research-related or otherwise! I recently moved to the city and would love to meet people here!🤗
A nice article on some risks of marginalizing applications research "They(ML researchers) evaluate a model’s performance using metrics that don’t translate to real-world impact, or they choose the wrong target altogether."
AI/ML tweeps, here is a must read.
It’s a wonderful article questioning whether we are spending our valuable brain cycles on the right research questions.
If you disagree, I’d love to hear why. If you agree, what can be done to accelerate this?
@NLPWithFriends
is coming back this spring, and will continue to be a friendly student-run space to discuss research. I am so excited about this organizing team!🥳
We’re thrilled to announce NLP with Friends will return this spring with an all-new cast of student organizers!! 🎓🥳
@zhaofengw
@sumanthd17
@BrihiJ
@kayo_yin
Stay tuned for news very soon… 👀
Excited for this talk!! Come join us, and check out their
#acl2020nlp
paper if you haven't already-
The State and Fate of Linguistic Diversity and Inclusion in the NLP World
Pratik Joshi*, Sebastin Santy*, Amar Budhiraja*, Kalika Bali, Monojit Choudhury
We are super excited to announce our next speakers!!
🗣️ Pratik Joshi(
@Roprajo
) and Sebastin Santy(
@SebastinSanty
), telling us about "The State and Fate of Linguistic Diversity and Inclusion in the NLP World."
🗓️ November 11th, 14:00 UTC
📝 Signup here:
This job search was complicated for multiple reasons. I am deeply grateful to everyone who shared their stories with me, and took the time to give me advice that was helpful for making decisions. I promise to pass it forward.
The deadline for submitting an affinity workshop proposal at NAACL has been extended to ✨Jan 31✨! Do consider submitting. We are excited for this initiative, and its potential to help support and grow marginalized research communities.
Announcing call for affinity workshops for NAACL 2022. The goal here is to establish research communities out of affinity groups and provide opportunities for folks from these communities. Deadline Jan 20. More details here:
I think there are lots of cool contributions here:
- Contrastive data
- An evaluation for negation
- Lots of careful “data work” (collecting good data is tons of effort!)
- Crowdsourcing of nuanced linguistic phenomena (“change the scope of the negation”)
- Strong baselines [7/8]
Come catch our oral session at
#EACL2021
, April 23 14:00 CET, or poster at 15:00 CET. This work fits into a broader question of how to understand and diagnose models, so please reach out if you are interested in creating better evaluations and new model analysis techniques!
I’m absolutely buzzing after the
@NLPwithFriends
social! It was great to see a very diverse group of people (in terms of geography, academic progression and nationality) talking about research, issues of diversity and the politics of tech and research! 🥰
Our first speaker will be Lena Voita (
@lena_voita
), talking about Information-Theoretic Probing with Minimum Description Length
#NLProc
Paper :
Event details: 👇
We are very excited to host our first speaker in August!!
🗣Lena Voita (
@lena_voita
), telling us about Information-Theoretic Probing with Minimum Description Length
🗓August 5th, 14:00 UTC
📝Sign up:
Keep up to date with talks at
I thought the first thing I would do with DALLE-2 would be evaluation design. There are a lot of questions here.
Turns out,
#dalle
as a 🖌️stochastic paintbrush🖌️ is really fun😄 I never know what I'm going to get but I'm here for it!
Anyway, here's The Great Wave off New York.
It is absolutely heartwarming to see names added to this spreadsheet. Makes me feel really happy about our community 🤗❤️ (And now I think we should have this even for non-virtual conferences).
If you are a faculty member in NLP/CL and would be willing to talk to students during ACL2020, please consider adding your information here. Hopefully, this will make faculty more approachable for new members to the field.
(h/t to
@zehavoc
for the priming)
#acl2020nlp
Thanks also to my wonderful thesis committee Ed Hovy, Norman Sadeh, Alan Black, and
@sameer_
. I learned so much from you and am honored you gave me your time and mentorship!
@TharinduDR
@coling2020
Hi sorry for reaching out unsolicited, but I think this may potentially de-anonymize your paper since reviewers generally have access to paper ID. I just wanted to let you know in case it caused any issues, hope it's not disrespectful!
Important announcement!
Today we launch a new Twitter-like space for the AI community - Sigmoid Social.
We hope to ensure the thriving AI Twitter community can live on by maintaining this Mastodon instance going forward.
Join here:
My recent musings on (aka: a brief critical review of) probing classifiers, their promises, their shortcomings, and some alternatives:
Comments welcome.
Yupei Du on his work about Exploring Human Gender Stereotypes with Word Association Test with Yuanbin Wu and Man Lan at
#emnlp2019
@emnlp2019
#NLProc
Paper:
Code:
Spoiler: surprisingly, accuracy≠usefulness. When picking the "best" QA system to use in practice, evaluation must be done holistically considering context(s) of use, including how users interface with the system, and not just based on benchmark performance. 👇4/n
Help amplify diverse papers by signing up to livetweet at
#NAACL20222
! If you have never livetweeted at a conference before,
@rctatman
wrote an excellent blog:
It's not too late to help with live tweeting for
#NAACL2022
! We're still looking for more volunteers to help tweet about papers from different sessions! Fill out this form by 7/7 + we'll follow up with more info on how you could help amplify diverse papers. Please retweet!
Due to volume I won't be able to respond to individual DMs/emails or comment on application materials, but please apply today if these areas are of interest!
I am very excited about this; negation (e.g. when we talk about things *not* happening) is fundamental to how we use language and pretty crucial for NLU, but reasoning about negated language is not easy for machines. [2/8]
Thanks to wonderful collaborators for being such a fun team and working hard. Check the paper out for full details! We welcome comments/questions/feedback/discussion🤗
Paper:
Dataset:
Website:
New quarantine hobby: since I spend so much time on
@arxiv_cscl
, I decided to make a periodic thread on recent interesting papers (i.e past 24 hours🙃). Largely biased towards natural language understanding (May 7, 2020):
There has been disagreement about how this changes with newer models.
There is no substitute for systematic testing. The first step to building systems that process negation is a high-quality evaluation.
Key idea: contrastive data collection. [3/8]
@sleepinyourhat
Could you say more about the evidence for your claim about negation? We created an NLI dataset of negation interacting and not interacting with lexical entailment pairs that determine the labels, and the examples seem very hard for davinci-002 across a range of prompt types.
Swabha: Let's teach our models how they can do better, instead of presenting them "gotcha" instances. Less hate, more love.
🧵End of thread🧵 See you at the next
#NAACL
!