Zorik Gekhman
@zorikgekhman
Followers
242
Following
325
Statuses
132
PhD student @ Technion | Research intern @Google
Joined August 2019
Does Fine-Tuning LLMs on New Knowledge Encourage Hallucinations? New preprint!๐ฃ - LLMs struggle to integrate new factual knowledge through fine-tuning - As the model eventually learns new knowledge, it becomes more prone to hallucinations๐ตโ๐ซ ๐ ๐งต1/12๐
Google presents Does Fine-Tuning LLMs on New Knowledge Encourage Hallucinations? Highlights the risk in introducing new factual knowledge through fine-tuning, which leads to hallucinations
5
57
187
Our work on abductive reasoning across multiple images has been accepted to #ICLR2025! Congrats, @mor_ventura95 !
Weโre excited to share that our paper, "NL-EYE: Abductive NLI for Images" has been accepted to ICLR 2025!๐ฅณ Our benchmark reveals that VLMs struggle with abductive reasoning across multiple images (worse than random!) Think you can solve it? Check out these examples! Linkโฌ๏ธ
0
4
10
Our work on the intrinsic representation of hallucinations in LLMs has been accepted to #ICLR2025! Congrats, @OrgadHadas
Our work "LLMs Know More Than They Show: On the Intrinsic Representation of LLM Hallucinations" was accepted to #ICLR2025! Looking forward to discussing our findings in person. Project page >> Code >>
0
2
21
RT @NitCal: Our paper: "On Behalf of the Stakeholders: Trends in NLP Model Interpretability in the Era of LLMs" has been accepted to NAACLโฆ
0
25
0
Cool work by @NitCal. If you ever wonder if LLM-based labels work for your use case, now youโve got a (simple) way to find out.
Do you use LLM-as-a-judge or LLM annotations in your research? Thereโs a growing trend of replacing human annotators with LLMs in researchโthey're fast, cheap, and require less effort. But can we trust them?๐ค Well, we need a rigorous procedure to answer this. ๐จNew preprint๐
0
1
10
RT @goldshtn: Today we published FACTS Grounding, a benchmark and leaderboard for evaluating the factuality of LLMs when grounding to the iโฆ
0
8
0
@_jasonwei Great work! I couldnโt find the QA prompt used for evaluating the models in Table 3. Was it the same for all models? If possible, could you share it? Thanks!
0
0
1
RT @YonatanBitton: ๐จ Happening NOW at #NeurIPS2024 with @nitzanguetta ! ๐ญ #VisualRiddles: A Commonsense and World Knowledge Challenge for Vโฆ
0
7
0
Thank you @MilaNLProc for featuring our work!
For this week's @MilaNLProc reading group, Yujie presented "Does Fine-Tuning LLMs on New Knowledge Encourage Hallucinations?" by @zorikgekhman et al. Paper: #NLProc
0
0
11
RT @roireichart: My students, collaborators and I have just published blog posts describing our work on NLP for the human sciences (pursuedโฆ
0
4
0
RT @NitCal: Do you think LLMs could win a Nobel Prize one day? ๐ค Can NLP predict heroin addiction outcomes, uncover suicide risks, or simuโฆ
0
8
0
At #EMNLP2024? Join me in the Language Modeling 1 session tomorrow, 11:00-11:15, for a talk on how fine-tuning with new knowledge impacts hallucinations.
I'll be at #EMNLP2024 next week to give an oral presentation on our work about how fine-tuning with new knowledge affects hallucinations ๐ตโ๐ซ ๐
Nov 12 (Tue) 11:00-12:30, Language Modeling 1 Hope to see you there. If you're interested in factuality, letโs talk!
0
5
14
And now, you can also use our implementation to probe LLMs for truthfulness in your research
Our code for "LLMs Know More Than They Show: On the Intrinsic Representation of LLM Hallucinations" is now available! Utilize our implementation to probe the internal representations of LLMs and explore the insights found in our work. Check it out here:
0
0
1
RT @mor_ventura95: ๐ Our paper "๐๐๐ฏ๐ข๐ ๐๐ญ๐ข๐ง๐ ๐๐ฎ๐ฅ๐ญ๐ฎ๐ซ๐๐ฅ ๐๐ก๐๐ฌ๐ฆ๐ฌ: ๐๐ฑ๐ฉ๐ฅ๐จ๐ซ๐ข๐ง๐ ๐๐ง๐ ๐๐ง๐ฅ๐จ๐๐ค๐ข๐ง๐ ๐ญ๐ก๐ ๐๐ฎ๐ฅ๐ญ๐ฎ๐ซ๐๐ฅ ๐๐๐ ๐จ๐ ๐๐๐ฑ๐ญ-๐ญ๐จ-๐๐ฆ๐๐ ๐ ๐๐จ๐๐๐ฅ๐ฌ" is accepted tโฆ
0
12
0
I'll be at #EMNLP2024 next week to give an oral presentation on our work about how fine-tuning with new knowledge affects hallucinations ๐ตโ๐ซ ๐
Nov 12 (Tue) 11:00-12:30, Language Modeling 1 Hope to see you there. If you're interested in factuality, letโs talk!
Does Fine-Tuning LLMs on New Knowledge Encourage Hallucinations? New preprint!๐ฃ - LLMs struggle to integrate new factual knowledge through fine-tuning - As the model eventually learns new knowledge, it becomes more prone to hallucinations๐ตโ๐ซ ๐ ๐งต1/12๐
0
6
16
RT @alon_jacovi: "Is It Really Long Context if All You Need Is Retrieval? Towards Genuinely Difficult Long Context NLP" was accepted to EMNโฆ
0
20
0
RT @AdiSimhi: LLMs often "hallucinate". But not all hallucinations are the same! This paper reveals two distinct types: (1) due to lack ofโฆ
0
187
0
RT @itay__nakash: Breaking ReAct Agents: Foot-in-the-Door Attack ๐ช๐งโ๐ปโ ๏ธ ๐จ New preprint! ๐จ FITD attack subtly misleads LLM agents ๐ง . Our reโฆ
0
12
0
RT @omer6nahum: ๐ Excited to share our research: "Are LLMs Better than Reported? Detecting Label Errors and Mitigating Their Effect on Modeโฆ
0
11
0
Repeated model sampling boosts the chance of producing the right answer, which motivates scaling the inference compute. Yet this work shows that repeated guessing achieves the same, raising questions about whether the model is "right for the right reasons" in these improvements.
Scaling inference compute by repeated sampling boosts coverage (% problems solved), but could this be due to lucky guesses, rather than correct reasoning? We show that sometimes, guessing beats repeated sampling ๐ฒ @_galyo @omerlevy_ @roeeaharoni
0
0
2
RT @omarsar0: LLMs Know More Than They Show We know very little about how and why LLMs "hallucinate" but it's an important topic nonetheleโฆ
0
134
0