Shujin Wu
@shujin_wu
Followers
358
Following
125
Media
10
Statuses
21
Graduate Student @USCViterbi . Natural Language Processing & AI Alignment
Los Angeles, CA
Joined July 2022
Don't wanna be here?
Send us removal request.
Explore trending content on Musk Viewer
#2024MAMAVOTE
• 2759166 Tweets
I VOTE
• 1524142 Tweets
EP6 U STEAL MY HEART
• 427164 Tweets
#Jin_Happy
• 304834 Tweets
JIN IS COMING
• 285295 Tweets
Choice - Female Award
• 216095 Tweets
Libya
• 213435 Tweets
Columbus
• 204581 Tweets
HAPPY IS COMING
• 191593 Tweets
seokjin
• 180562 Tweets
Nobel
• 168845 Tweets
KSJ1 IS COMING
• 146078 Tweets
MY ROYALTY HEESEUNG DAY
• 98394 Tweets
YOU ARE LOVED
• 95195 Tweets
Bill Clinton
• 78303 Tweets
Daron Acemoğlu
• 76970 Tweets
Thanksgiving
• 74892 Tweets
#Number_i_1stAnniv
• 34323 Tweets
Europeans
• 27053 Tweets
Leire
• 25154 Tweets
Laken Riley
• 22867 Tweets
kesha
• 22128 Tweets
Native Americans
• 20297 Tweets
Bret
• 19971 Tweets
La Oreja de Van Gogh
• 17311 Tweets
Amaia
• 17304 Tweets
Reece James
• 16713 Tweets
Billy Maximoff
• 15301 Tweets
Nasser
• 12955 Tweets
Europa Clipper
• 12665 Tweets
Is your Vision-Language Model really helpful at all times?
Can we instruct them to interact with users during conversations to avoid hallucinations or biased responses?
🍰Take some bites of PIE and MACAROON! We present a benchmark to evaluate LVLMs’ proactive engagement
4
31
189
🧵(1/n) Existing Large vision-language models (LVLMs) invariably generate detailed responses even when questions are ambiguous or unanswerable, leading to hallucinations and bias issues. Thus, it is essential for LVLMs to proactively engage with humans to ask for clarifications
1
2
8
Really appreciate the help and guidance from my best mentor
@YiFung10
, advisor
@hengjinlp
, and of course all the amazing collaborators
@ZoeyLi20
@yixin_wan_
@kaiwei_chang
.🥰🫶
0
0
4
🧵(5/n) Our case study further proves that MACAROON not only successfully gains proactive engagement capabilities, but also exhibits promising multi-turn conversational skills.
1
0
4
🧵(2/n) We create a new benchmark PIE, which consists of 853 questions across six distinct, fine-grained question types based on a three-tiered hierarchy of model’s interactive behaviors: challenging invalid question settings, seeking clarifications, and uncovering latent human
1
0
3
🧵(4/n) Experimental results on PIE and other general benchmarks such as MME show that MACAROON demonstrates significantly better proactive engagement capabilities while maintaining the general visual-language performance.
1
0
3
🧵(3/n) We then present MACAROON, self-iMaginAtion for ContrAstive pReference OptimizatiON. MACAROON avoids using extensive human or teacher model supervision via self-imagined desirable and undesirable responses based on human-written criteria. The contrastive response pairs,
1
0
2