RT @ZhiyuChen4: 🤖 Can LLMs effectively assist in cognitive behavior therapy (CBT)? 🔗New paper: We present the fir…

Many thanks to my wonderful collaborators: Zhiyu (@ZhiyuChen4), Xi(@xiye_nlp), Xianjun ( @Qnolan4), Lichang (@LichangChen2), William Wang ( @WilliamWangNLP) and Linda Ruth Petzold.

4/🧩 Further analysis revealed that coding data generally provides similar task-specific benefits across model families. While most optimal proportions of coding data are consistent across families, no single proportion enhances all task-specific reasoning abilities.

3/🔍 Diving into each domain, we found that coding data uniquely impacts different reasoning abilities. Consistent trends within each domain across model backbones and sizes suggest the benefits of coding data transfer effectively during the IFT stage.

2/ ✨ Overall, we observed a consistent and gradual enhancement in the LLMs' reasoning performance as the proportion of coding data used for fine-tuning increased.

1/📊 We created IFT datasets with increasing coding data proportions, fine-tuned six LLM backbones, evaluated performance across twelve tasks in three reasoning domains, and analyzed outcomes from overall, domain-level, and task-specific perspectives.

Check out our medical instruction dataset, MedInstrcut-52k, the clinician-crafted evaluation test set, MedInstrcut-test, and our models AlpaCare-7B/13B on the GitHub link:

@Qnolan4 @LichangChen2 @ZekunLi0323 6/🔔 Takeaway: Our study shows the power of tuning LLMs with diverse & domain-specific instructions. High-quality, diverse data with domain knowledge boosts domain-specific capacity & generalization, even with small quantities.

@Qnolan4 @LichangChen2 @ZekunLi0323 5/ 🏥 In-depth comparisons of AlpaCare-13B vs. its 13B instruction-tuned LLM counterparts reveal a consistent edge in medical proficiency and adaptability.

@Qnolan4 @LichangChen2 @ZekunLi0323 4/🔍 Evaluation on AlpacaFarm reveals AlpaCare's robust generalization abilities in both medical and general domains. Training with a diverse, domain-specific instruction dataset also enhances generalizability.

3/🦙 AlpaCare: Leveraging a 7B-LLaMA with our 52k med dataset. Evaluations reveal AlpaCare's superior medical prowess, outperforming other 7B instruction-tuned models even those trained on much larger datasets.

@Qnolan4 @LichangChen2 @ZekunLi0323 2/🔍 How do we create a diverse med dataset? Begin with expert-crafted tasks spanning medical topics, types & levels. Use #GPT4 for generating diverse tasks, ensuring depth! Tasks are then input into #ChatGPT sequentially for detailed output. See our seed example:

1/📊 Many medical LLMs utilize vast amounts of data, but diversity is key! To fill this gap, We created a dataset of 52k unique med instructions. See our language diversity compared to the baseline below. The left is ours, and the right is one of baseline.

🧵6/6 A big shoutout to our fantastic team of authors: @ShiyangLi6, @Qnolan4, Chenxin Tian, @YaoQinUCSD, and Linda Ruth Petzold! Don't miss out on more insightful results in our paper! 📚🔍

🧵 5/6 It also demonstrates impressive generalizability and broad applicability by OOD testing and experiments in two general domain datasets. 💡

🧵4/6 Our method achieves up to a 22.57% increase in absolute accuracy compared to SLM fine-tuning w/o context and sets SOTA results in two medical tasks within privacy-restricted scenarios.🚀🩺