Jieyu Zhao
@jieyuzhao11
Followers
3K
Following
525
Media
22
Statuses
263
Assistant Prof. @CSatUSC, @USC || Postdoc @ClipUMD || PhD from @UCLANLP, @UCLA. #NLP, #ML, #Fairness, #TrustworthyNLP
Los Angeles, CA
Joined February 2012
After #NAACL2024, got held up in LAX CBP for 3 hours even with a valid H1B visa. Yeah, what a welcome.
28
9
308
Still feel unrealistic, but I finished my PhD defense! Thank you to my advisor @kaiwei_chang, my committee members @WeiWang1973, @YizhouSun and Cho-Jui Hsieh! Thanks to everyone joining me today and to my collaborators making this happen! And big big thank you to my family! ❤️
22
5
243
I’m glad to share our awesome list of NLP (NLU & NLG) fairness papers/tutorials This "incomplete" list mainly covers papers from #nlpproc conferences/workshops.
3
49
187
Very grateful to the opportunity and looking forward to the collaborations!.
Postdoc @jieyuzhao11 is teaming up with @haldaume3 to mitigate bias in machine learning. They're developing interpretation tools that can provide a method for early intervention, instead of waiting on the output for analysis. Read more:
1
2
95
Thank you for the invitation! Looking forward to it!.
For this week’s NLP Seminar, we are thrilled to host @jieyuzhao11 to talk about Building Accountable NLP Models for Social Good!. When: 10/24 Thurs 11am PT.Non-Stanford affiliates registration form (closed at 9am PT on the talk day):
3
7
95
Can we intervene in a model’s behavior by natural languages? Check our #ACL2021 Findings “Ethical-Advice Taker: Do Language Models Understand Natural Language Interventions?” (. w/ @DanielKhashabi, Tushar Khot, Ashish Sabharwal, and @kaiwei_chang. 1/n
2
15
81
Thanks for the invitation! Let's chat! #NAACL2024.
📢 We are also delighted to have Jieyu Zhao (@jieyuzhao11) as our panelist for our NAACL 2024 SRW. 🌟 Jieyu is an Assistant Professor at USC specializing in fairness in ML/NLP models. Don't miss the opportunity to ask her your questions:
2
3
81
1/3 Happy to share our new paper "Gender Bias in Multilingual Embeddings and Cross-Lingual Transfer" with @subho_mpi,@sagharh, @kaiwei_chang and Ahmed Hassan Awadallah at #acl2020nlp
1
12
79
Missed the group group photo @nlp_usc @CSatUSC here #EMNLP2024, but I have my LIME 🍋🟩 lab group photo! .Let’s keep doing great research! .And #Our goal is ____!
2
3
77
Biases in NLP models can cause potential harms. But what harms and how can they be evaluated? In our #aacl2022 paper (, we discuss major types of representational harms --.
3
6
69
🔊 Please consider submitting to our Secure and Trustworthy Large Language Model Workshop at #ICLR2024! Submission ddl is Feb 12.
Submission ddl is Feb 12. We welcome any topics related to LLM security and trustworthiness. Look forward to your submission!.
1
7
61
🥳🥳Taiwei is doing a fantastic job and finished the project within the first 3 months of his PhD! Check our paper here and feel free to reach out for any type of discussions!.
Excited to get Safer-Instruct accepted to NAACL 2024 🥳! You don’t want to miss it if you want to reduce cost and boost efficiency in preference data acquisition 🚀. Check out our framework and dataset here:
1
1
49
Ziyi did a great job to understand to what extent LLMs can show social intelligence in a game other than just win rate. She has enthusiasm to study/interpret LLM behaviors. Check the paper for more interesting findings and we appreciate your feedback!.
🤔Are LLMs capable of demonstrating social intelligence in a complex interactive game environment? Check out our recent work: We introduce ‘InterIntent’ to study social intelligence via intention understanding in the Avalon game. [1/x]
1
4
48
Prior studies often test model robustness by applying semantic-invariant perturbation on a given test set. In our #NAACL2021 “Double Perturbation: On the Robustness of Robustness and Counterfactual Bias Evaluation”, we propose a new framework for robustness verification. 1/n
1
2
43
There has been some improvement in LLM reasoning. However, Ziyi's work shows that there can be conflicts in LLM reasoning and its predictions. She and other collaborators did some detailed analysis on this. Please check the details ⬇️⬇️. We welcome suggestions and comments!.
Can LLM detect reasoning fallacies? Check out our latest work: We ask GPT-4 to detect different reasoning fallacies using multiple predictors. We find that LLMs still struggle with detecting problematic reasoning, performing much worse than humans.
0
4
42
[1/3] Our EMNLP17 paper ( demonstrates bias amplification in model’s top predictions. How about in distribution? Check our #acl2020nlp paper "Mitigating Gender Bias Amplification in Distribution by Posterior Regularization".
2
8
38
While we are making progress in LLMs+clinical tasks, we found that the medically adapted models sometimes can be worse than their predecessors (vanilla pretrained version) in some clinical-related bias evaluation. Check the paper for more!.
🚀 Excited to share our latest work: "CLIMB: A Benchmark of Clinical Bias in Large Language Models" 🏥🤖! .We delve deep into intrinsic (within LLMs) and extrinsic (on downstream task) biases in LLMs, unveiling how they impact clinical decision-making.
0
3
37
Before that, I will do my postdoc at UMD @ClipUmd. Feel free to ping me if you are interested in working on fairness or NLP for social good (and of course, many other great ideas).
1
2
32
Check our paper if you are also interested in building efficient pipeline for alignment! Comments and suggestions are appreciated!.
🤔Enhancing LLM with RLHF is powerful, but ever wondered how to reduce costs and boost efficiency in preference data acquisition? 💰.🚀Introducing Safer-Instruct, a groundbreaking pipeline that complements humans to construct large-scale preference datasets efficiently. 🧵1/5
0
6
33
Wait a sec, adding me and @jonathanmay . Should ask an AI agent to do this image editing for me next time.
0
0
32
My first conference talk done! Thank you all for helping me to finish this! @kaiwei_chang @ryandcotterell @bluevincent @yatskar Tianlu and all my lab mates!.
3
3
30
Enjoyed the #WiML2021 journey with everyone! You are all amazing!! Let’s keep in touch!! .@mariyaivasileva @Boyiliee @linh_trd @MeeraDesai18 @siri_r @AmakaOraekwe @akikoe_ @NanaYaaSally @geeticka1, Soomin Aga Lee. And of course @nmervegurel !
3
6
31
This work is led by our first-year PhD student @taiwei_shi. I am very impressed by his knowledge and capability. He is now looking for a summer intern position. If you are hiring, do not miss him!.
Learn more in our paper: "Safer-Instruct: Aligning Language Models with Automated Preference Data", an awesome collaboration with @jieyuzhao11 and @kaichen23 ! For our code implementation and dataset, see
1
2
26
👏 I enjoyed it a lot! Thanks everyone!.
🏝️And that’s a wrap! Thank you everyone for travelling or driving to Los Angeles/@ucla and #SoCalNLP2023! It was a fun day with great discussions, networking and some gossip strewn in from recent news 🤭. See you all next year!!!
0
0
26
📢📢USC is hiring! The ddl is Jan 5th. Do check the link!.
We're hiring! @CSatUSC is growing fast, with multiple openings for tenure-track and tenured positions. 🔍Security/privacy, AI, machine learning, data science, HCI, but exceptional candidates in all areas considered. 📅Deadline: Jan 5.🔗Details:
0
0
26
🗳️ 🗳️ vote to make your opinion count! And of course vote for Kai-Wei!.
I am honored to be nominated by SIGDAT (the org that oversees EMNLP) to run for VP-elect with other awesome candidates who share the goal of improving our community. Please check your email to vote by 3/24.🗳️ See details:
0
2
24
So sad to learn this. Although our interactions were limited, I were able to feel her vitality. May she rest in peace.
Deeply saddened to hear this tragedy though I don't know her personally. Hope she rests in peace.
1
0
24
1/2 Existing studies on measuring bias often consider performance gap between cohorts over the entire test set. But, does it show the whole story? In our #EMNLP20 paper “LOGAN: Local Group Bias Detection by Clustering” (, joint work with @kaiwei_chang,.
1
1
24
The afternoon session started at 1pm. We have a great list of speakers!
0
3
22
😎✌️.
The USC NLP faculty is 2x of its size in two years! We're thrilled to have @DaniYogatama and @swabhz joining us this Fall (@jieyuzhao11 starting in Fall'23), along with five other CS new hires! 🙌. 🔖
0
0
21
I'm not physically in Singapore, but my spirit has been there already! 😆.
We are at #EMNLP2023 this week! 👋. Explore the latest @CSatUSC @USC_ISI research spanning social bias in name translation, AI tools for journalists, ambiguity in LMs, and much more ⬇️. @USCViterbi @emnlpmeeting @USCResearch #NLP.
1
0
21
Thank you!😄 And also great thanks to all the organizers! #KGBIAS.
Cool live notes of the KG-BIAS workshop at AKBC!! Also, congrats @jieyuzhao11 for delivering her first invited talk ;-).
1
0
21
👏 Check this new work about fairness in sequential decision making! Glad to be part of this team!.
🔥 We uncover new insight into #AI fairness, 🤯a hidden pitfall: "Fair Now" doesn't mean "Fair Forever". Stop treating fairness as a one-time check! .👉 A dynamic approach to long-term fairness in sequential decision-making #AIBias #LongTermFairness .A 🧵
0
2
19
We accept both archival and non-archival submissions! 😀.
🚀Interested in #NLProc + Social Good? The Call for Papers of the #EMNLP2022 Workshop *NLP for Positive Impact* @NLP4PosImpact is coming up!.1⃣️ Direct submission: by Aug 15 (~20 days to go!).2⃣️Commit from ARR: by Oct 2.Check for details! @emnlpmeeting
0
0
16
See you next time!.
Thanks to our steering committee, co-organizers for their dedication in making the 4th NLP4PI Workshop at ACL 2025 possible! @katwellll @lbiester23 @AnganaBorah2 @iamdddaryna @OanaIgnatRo @Neemakot @liuziyi93 @Ruyuan_Wan @SteveWilsonNLP @jieyuzhao11 @radamihalcea @Tetreault_NLP
0
1
17
Special thanks to @kaiwei_chang, @bluevincent, @haldaume3 and Yulia Tsvetkov! And thanks to everyone who helped me during the job hunting.
2
0
13
Congrats!!.
GLIP is listed as a finalist for CVPR best paper. Congrats @LiLiunian and coauthors. 👏.
0
0
11
If you plan to attend icml, do stop by and chat with Yuancheng!.
@icmlconf Joint work with @chenghao_deng @sun_yanchao @ruijie_zheng12 @XiyaoWang10 @jieyuzhao11 and @furongh .ICML Poster: #2306 Hall C 4-9, 1:30 pm - 3:00 pm CEST on July 25th.
0
3
12
Thank you! Looking forward to seeing more discussions on related topics!👀.
1/ 📢 In this week’s TrustML-highlight series, we are glad to feature Jieyu Zhao @jieyuzhao11 . Jieyu is a PhD student at UCLA. Her recent research focus has been detecting gender bias in modern machine learning systems and proposing different bias mitigation methods.
0
1
11
The initial list is collected together with @ewsheng @sunipa17 @houyu0930 @VioletNPeng and @kaiwei_chang from Pluslab & @uclanlp. Please share your comments/suggestions or submit a pull request.
0
0
10
Do not forget your #WiML2021 submission! 👏.
*** Last three days!!! ***. The submission deadline for #WiML2021 @NeurIPSConf is approaching soon! Submit new, previously, or concurrently published work (1-page abstract) by Sep 29 11:59pm AoE! . Submit here 👉: Full CFP here:
0
1
9
2/3 We examine gender bias in multilingual word embeddings (fastText, focused on EN, ES, DE, FR) from both intrinsic and extrinsic perspectives. We collect occupation lists and bios dataset for each language for intrinsic and extrinsic analysis respectively.
1
1
8
Please submit to our Socially Responsible Machine Learning workshop! (ddl: 06/10 midnight AoE).
0
2
8
-- and introduce heuristics using which delineations of harms measured by different bias measures in NLP can be identified. We also propose methods of documentation to aid proper usage of measures in different scenarios.
1
0
9
🥳🥳 👏👏.
Honored to be awarded Sloan Fellowship for our work on fairness, robustness, inclusion in Human Language Technology at #UCLANLP. The award belongs to my students and collaborators. Thank the NLP/ML/AI community and @UCLAengineering to support our work
0
0
9
Appreciate your support to #trustnlp!.
📚Interested in becoming a reviewer for #NAACL2024 @trustnlp ? .⬇️Sign up using the form below!.
0
0
7
3/3 We find that 1) bias in the multilingual word embeddings changes differently when we align embeddings to different target spaces; 2) the alignment direction can also have an influence on the bias in cross-lingual transfer learning.
1
1
7
It is a great opportunity! Do fill in the form if you are interested in #trustnlp Workshop at #NAACL24.
We are excited to announce that our trustnlp workshop at this year’s NAACL is awarding DEI awards on behalf of our sponsor, Amazon. You can nominate yourself here (. While the award is intended to cover virtual registration fees, you can indicate.
0
0
7
👏👏.
Led by Hal Daumé III (@haldaume3), the new Institute for Trustworthy Artificial Intelligence in Law & Society (TRAILS) aims to transform AI by prioritizing ethics, human rights, and input from communities who have been previously marginalized. Read more:
0
0
7
Great opportunities to ask questions!.
#WiML2021 will be hosting a panel discussion with Emily Denton (@cephaloponderer), Devi Parikh (@deviparikh), Adriana Romero-Soriano (@adri_romsor), and Bo Li! Feel free to comment on this thread with any questions you would like to ask to our panelists.
0
0
7
Had it been just a little bit more sympathetic.
So many layers of irony here …….
0
0
5
The work is done by @choniong, @jieyuzhao11, @huan_zhang12, @kaiwei_chang and Cho-Jui Hsieh @uclanlp. The paper is available at: See detail at Talk to you all in just 2 days! n/n.
0
0
5
So many interesting topics! Please feel free to drop by if you are interested!.
Our second #WiML2021 virtual Mentorship Roundtable session is starting shortly, at 20:15 - 21:55 UTC. We will be hosting a total of 29 roundtables, in addition to sponsor tables! Check out the full list of topics here: #NeurIPS2021 @NeurIPSConf.
0
0
6
Work led by @sunipa17, @ewsheng, Aubrie Amstutz, with collaboration @sunjiao123sun_, @houyu0930, Mattie Sanseverino, Jiin Kim, Akihiro Nishi, @VioletNPeng and @kaiwei_chang. We also thank various people for providing great suggestions!.
1
0
6
We propose a new language understanding task, Linguistic Ethical Interventions (LEI), which studies the ability of models to understand interventions and amend their predictions accordingly. 2/n.
1
1
6
‘66 - Gender Bias in Coreference Resolution: Evaluation and Debiasing Methods, with Jieyu Zhao’ on #SoundCloud #np
0
0
4
📢📢.
Hello World! 🙌🏼.#SoCalNLP2023 is back this year, to be held @UCLA! 🏝️☀️. Keep an eye here for more announcements, but for now, our CfP is open! More details ⬇️. 🔗 📆Nov. 17, 2023
0
0
5
We also have a very young attendee today!.
❤️ Standing-room only at our @NLP4PosImpact workshop at #EMNLP2024 — very happy to see all this interest in pursuing NLP research with a positive societal impact.
0
0
5
We are about to start the #nlp4pi workshop in 20mins. On 3rd floor, Foster!.
@NLP4PosImpact is starting in 20 minutes! Come visit us on the 3rd floor of conference center in room Foster. #EMNLP2024 #NLP4PI #AIforGood #NLProc.
0
0
5
@uclanlp Because I was so lucky to start from such a great place!! 🥳 I enjoyed every minute here (except sometimes I locked myself outside the office)! I’m so thrilled about the next journey! 🤩.
0
0
5
🤩.
#SoCalNLP2023 is this Friday!!! 🏝. Check out our schedule of invited speakers and accepted posters! 👉🏽
0
0
5
Thank you all!!👏.
#WiML2021 would like to thank our volunteers and supervolunteers @mennatullahSiam @linylinx @MandyZong @Julythe17th @SharvVadgama @silviapaglia @liyue_shen @disha_shur @naitisb @wsanda89 @PriyaBannur @nicemakr and others for their amazing effort in putting the workshop together!
0
0
5
Same here! Great thanks to everyone for making this happen! 👏.
was an honor to moderate this panel at @WiMLworkshop :)
0
0
4
We evaluate recent pre-trained language models on LEI, and find even the best technology of today fails to appropriately respond to the interventions. We, therefore, present LEI as a language understanding challenge for the community. n/n.
0
0
4
After a long time, I'm still confused about the Tweets' timestream.
0
0
4
2/2 we revisit the coarse-grained metric and propose a new clustering-based method, LOGAN, to group input instances and highlight bias hidden in the local regions. Come to join us in Gather Session 1 today (18:00-20:00 PST, Nov 16th) if you have any questions or suggestions.
0
0
4
Also keeping an eye on this!👀.
🙋♂️🙋♀️Dear Twitter academics, are there any Zoom interview tips for faculty positions? I'm pretty sure there are many Ph.D. students like me who are in dire need of such advice right now #interviewtips #facultypositions.
0
0
4
[3/3] 2) We propose a new bias mitigation method based on posterior regularization which can successfully remove the bias amplification in posterior distribution. Watch our video:
1
1
4
👏.
Now you can sign up to be a reviewer for the NLP for Positive Impact Workshop. Link: Website: Paper Submission Deadline: May 7 2021!.
0
0
4
@sunipa17 @ewsheng @sunjiao123sun_ @houyu0930 @VioletNPeng @kaiwei_chang Found Aubrie @aubrieamstutz!.
1
0
4
Thank you!😄 Enjoyed the discussions with different people about fairness @NAACL2019. Let’s keep in touch!.
Thanks, @emilymbender's awesome live tweeting. The full talk “Gender Bias in Contextualized Word Embeddings (NAACL 2019)” is on YouTube now. Great job @jieyuzhao11!.
0
0
2
Vote ✅.
This Thanksgiving, I’m deeply grateful for opportunity to run for NAACL Board Member alongside so many incredible candidates who share a passion for making a difference! .I hope to earn your support as we work together to shape the future of our community! #NLProc #Gratitude
0
0
3
😀
#emnlp2017 best long paper 2: Jieyu Zhao, Tianlu Wang, Mark Yatskar, Vicente Ordonez, @kaiwei_chang. Reducing Gender Bias Amplification
0
0
3
We applied our framework in both testing model robustness and counterfactual fairness. We found that our approach can fool a robust training model and identify vulnerable instances that cannot be identified in the past. 3/n.
1
0
3
1
0
3
0
0
3
Thanks!😄.
1/3 Two great papers from 2017 (@jieyuzhao11 et al's 'Men also like shopping . ' and @aylin_cim's 'Semantics derived automatically . contain human biases' drew attention to gender & other biases in ML approaches to languag3.
0
0
3