Daniella Ye
@Daniella_yz
Followers
357
Following
158
Media
4
Statuses
12
PhD NLP student @UniofOxford Prev: intern @Cohere
Oxford, England
Joined February 2021
Don't wanna be here?
Send us removal request.
Explore trending content on Musk Viewer
LINGORM PANTENE TRIP
• 600057 Tweets
محمد
• 392359 Tweets
KAOPP 1ST FAN MEETING
• 80208 Tweets
Kindiki
• 78194 Tweets
#Mステ
• 60854 Tweets
NUNEW BRAVE TOGETHER
• 53973 Tweets
Hayırlı Cumalar
• 50743 Tweets
APO DELIGHTS AND SURPRISES
• 43396 Tweets
新刀剣男士
• 31965 Tweets
LMSY AFFAIR FINAL EP
• 31168 Tweets
オースティン
• 21951 Tweets
#bebekkatilleri
• 16301 Tweets
BINI ANGEL MILK GIRLIES
• 11212 Tweets
国連脱退
• 10031 Tweets
Beyond their use in assisting human evaluation (e.g. CriticGPT), can critiques directly enhance preference learning? During my
@Cohere
internship, we explored using synthetic critiques from large language models to improve reward models.
📑Preprint:
5
57
322
Scaling experiments show that high-quality critiques significantly enhance data efficiency, especially with limited data. 1 critique roughly equals 40 vanilla preference pairs. Our method, using open-source models, is accessible and budget-friendly.
1
0
15
This project is a collaboration with
@FraserGreenlee
,
@max_nlp
, Phil Blunsom,
@jaa_campos
and
@mgalle
. I am grateful for the support from the incredible team at
@cohere
.
The paper is on arXiv:
0
0
13
@cohere
Instead of solely relying on better/worse annotations, we enrich reward models with synthetic critiques from LLMs, dissecting completion features. We then train the model to predict scalar rewards, conditioned on these critiques.
1
0
8
Results show that adding critiques improves reward model accuracy. We found that critique quality matters: high-quality critiques boost performance, while low-quality ones can hinder it.
1
0
8
Thank you
@stefan_fee
,
@JinlanFu
, and Professor
@gneubig
for all your guidance and support! I'm interested in exploring the use of performance prediction in many more scenarios.
How can we reliably estimate a system's performance without performing experiments? Check out our work (EACL)
1. Formulate performance prediction as Tensor Completion problem 2. Establish a set of reliability analysis mechanisms (confidence, calibration)
3
6
41
0
0
5
@Zhuang_Li_NLP
Thank you for your comment! Yes, the RM is trained on instruction-response-critique triplets. We haven't done experiments to train LLM using the critiques yet, but in practice yes this means passing the critique context.
0
0
1