Akari Asai @ COLM
@AkariAsai
Followers
12,651
Following
669
Media
107
Statuses
1,295
Ph.D. student @uwcse & @uwnlp . NLP. IBM Ph.D. fellow (2022-2023). Meta student researcher (2023-) . ☕️ 🐕 🏃♀️🧗♀️🍳
Seattle, WA
Joined December 2017
Don't wanna be here?
Send us removal request.
Explore trending content on Musk Viewer
NUNEW JACKIE XX THIRD
• 146221 Tweets
#プロフェッショナルランキング
• 74957 Tweets
オロルン
• 57465 Tweets
世界さん
• 37666 Tweets
ミクトランのコウモリ
• 37434 Tweets
チャスカ
• 35902 Tweets
ビビデバ
• 26049 Tweets
SOTA
• 22000 Tweets
ランクイン
• 20895 Tweets
芸能界ダンスNo.1
• 20510 Tweets
すいちゃん
• 20338 Tweets
平野紫耀
• 20061 Tweets
#嘘解きレトリック
• 16964 Tweets
#ヘタコイ
• 12422 Tweets
ラウール
• 10801 Tweets
Pinned Tweet
Introducing Self-RAG, a new easy-to-train, customizable, and powerful framework for making an LM learn to retrieve, generate, and critique its own outputs and retrieved passages, by using model-predicted reflection tokens.
📜:
🌐:
24
169
744
日本(の学部)からアメリカのコンピューターサイエンス博士課程に出願を検討している方向けにブログを書きました。
日本の学部からアメリカのコンピューターサイエンス博士課程に出願する - あさりさんの作業ログ
3
212
810
New paper 🚨
Can we train a single search system that satisfies our diverse information needs?
We present 𝕋𝔸ℝ𝕋 🥧 the first multi-task instruction-following retriever trained on 𝔹𝔼ℝℝ𝕀 🫐, a collections of 40 retrieval tasks with instructions! 1/N
12
106
578
Introducing 𝗔𝗧𝗧𝗘𝗠𝗣𝗧, a new modular, multi-task, and parameter-efficient approach to combine knowledge from multiple tasks to solve a new task using a small trainable of parameters 🔥 while keeping the original LM *frozen* 🧊 [1/9]
Paper 📜:
7
103
532
Can we solely rely on LLMs’ memories (eg replace search w ChatGPT)? Probably not.
Is retrieval a silver bullet? Probably not either.
Our analysis shows how retrieval is complementary to LLMs’ parametric knowledge [1/N]
📝
💻
15
95
535
Don't miss our
#ACL2023
tutorial on Retrieval-based LMs and Applications this Sunday!
with
@sewon__min
,
@ZexuanZhong
,
@danqi_chen
We'll cover everything from architecture design and training to exploring applications and tackling open challenges! [1/2]
6
106
504
（便乗してみる...） 東大に文科で入学して一度経済学部に進学しましたが工学部電子情報工学科を卒業してアメリカのCS博士課程でNLP/機械学習の研究をしています。特にプログラミングはもっと早く始めたかった（20歳まで未経験でした）とたまに思いますが楽しいです😀
すごく今更だけど、昔から平均以下の数学の才能しかなくて（東大数学も二完以下だったし）高二まで文系行くかなと思ってた人間がMITの博士課程に進学するとか無謀過ぎて笑えてくる。どうしてこうなったんだっけ....。
7
96
1K
1
39
441
🚨We all complain about LLM "hallucinations", but what are they? We study Automatic Fine-grained Hallucination Detection, with a novel taxonomy, a benchmark, and a 7B LM, surpassing ChatGPT in hallucination detection and editing
9
76
435
New work with Kazuma Hashimoto,
@HannaHajishirzi
,
@RichardSocher
, and
@CaimingXiong
at
@SFResearch
and
@uwnlp
! Our trainable graph-based retriever-reader framework for open-domain QA advances state of the art on HotpotQA, SQuAD Open, Natural Questions Open. 👇1/7
8
99
408
𝗛𝗼𝘄 𝗰𝗮𝗻 𝘄𝗲 𝗯𝘂𝗶𝗹𝗱 𝗺𝗼𝗿𝗲 𝗿𝗲𝗹𝗶𝗮𝗯𝗹𝗲 𝗟𝗠-𝗯𝗮𝘀𝗲𝗱 𝘀𝘆𝘀𝘁𝗲𝗺𝘀? Our new position paper advocates for retrieval-augmented LMs (RALMs) as the next gen. of LMs, exploring the promises, limitations, and a roadmap for wider adoption.
🧵
4
82
359
A deadline life pro tip: now you can use Grammarly with Overleaf ✍️
2
32
324
New paper 🚨
Can LLMs perform well across languages?
Our new benchmark BUFFET enables a fair eval. for few-shot NLP across languages in scale.
Surprisingly, LLMs+Incontext learning (incl. ChatGPT) are often outperformed by much smaller fine-tuned LMs
🍽️
8
57
296
Retrieval-augmented LMs have made great progress& been adapted to real-world applications. Yet we still face major challenges.
We
@AkariAsai
@sewon__min
@ZexuanZhong
@danqi_chen
will be giving an ACL 2023 tutorial on retrieval-LMs! Join us to learn more about this exciting area.
2
39
295
Recently I gave a lecture about retrieval-augmented LMs like RAG, covering their advantages, an overview of diverse methods, and current limitations & opportunities, based on this position paper.
video:
Feedback is welcomed :)
𝗛𝗼𝘄 𝗰𝗮𝗻 𝘄𝗲 𝗯𝘂𝗶𝗹𝗱 𝗺𝗼𝗿𝗲 𝗿𝗲𝗹𝗶𝗮𝗯𝗹𝗲 𝗟𝗠-𝗯𝗮𝘀𝗲𝗱 𝘀𝘆𝘀𝘁𝗲𝗺𝘀? Our new position paper advocates for retrieval-augmented LMs (RALMs) as the next gen. of LMs, exploring the promises, limitations, and a roadmap for wider adoption.
🧵
4
82
359
6
52
286
New
#acl2020nlp
paper "Logic-Guided Data Augmentation and Regularization for Consistent Question Answerings"!
We show SOTA QA models produce inconsistent predictions and introduce logic-guided data augmentation & consistency-based regularization.
1/
2
47
265
Can we build a *single* open-domain QA model that works in *many* languages?
We’re excited to present 𝗖𝗢𝗥𝗔 using a single retriever and generator, showing SOTA results in 26 diverse languages on XOR QA & MKQA, including the unseen languages. [1/5]
3
60
263
Our paper got the ACL 2023 best video award (at EMNLP) 🥳🎉
The video by
@alextmallen
is available at
This 5 mins video summarizes the interesting findings on (1) when LLMs hallucinate (and scaling may not help) how retrieval-augmented LMs alleviate it.
Our work "When Not to Trust Language Models: Investigating Effectiveness of Parametric and Non-Parametric Memories" will appear in
#ACL2023
!!
This is my first NLP conference paper and I'm very happy I got to pursue this project with these amazing people at UW!
2
4
49
5
25
266
This is a comprehensive list of the must-read papers on the recent progress of self supervised NLP models (or impressive capabilities of LLMs) and great summary slides!
I also love the role-playing paper-reading seminar fromat! 👩🏽🔬 ()
For my first course at
@jhuclsp
, I am leading a class on recent developments in "self-supervised models." Here is the list of the papers and slides we cover: Would love to hear Twitter's suggestions for additional exciting developments to discuss!🤗
6
69
384
3
56
258
A powerful retriever+pre-trained generator (eg. DPR+T5) often relies on spurious cues / generates hallucinations.
Our 𝕖𝕧𝕚𝕕𝕖𝕟𝕥𝕚𝕒𝕝𝕚𝕥𝕪-guided generator learns to focus and generate on the right passages and shows large improvements in QA/fact verification/dialogue👇
2
39
249
Our
#ICLR2020
camera-ready version, code, and blog are now available!
paper:
code:
blog:
You can train, evaluate, and run an interactive demo on your machine. We also release the models for reproducibility.
New work with Kazuma Hashimoto,
@HannaHajishirzi
,
@RichardSocher
, and
@CaimingXiong
at
@SFResearch
and
@uwnlp
! Our trainable graph-based retriever-reader framework for open-domain QA advances state of the art on HotpotQA, SQuAD Open, Natural Questions Open. 👇1/7
8
99
408
2
49
240
How GPT-4 triggers tools (eg web) seems to be similar to how Self-RAG works—fine-tuning an LM on augmented corpora with special tokens to call tools on demand during generation. We trained 7/13B LMs (open sourced) in this way.
@nbonamy
It emits special words, e.g. <|BROWSE|> etc. When the code "above" the LLM detects these words it captures the output that follows, sends it off to a tool, comes back with the result and continues the generation. How does the LLM know to emit these special words? Finetuning
16
89
998
1
24
233
📢 Thank you so much for attending our tutorial! 🙌
🔗All the materials are available online
Our slide:
The live Q&A on :
If you registered for ACL, you can see the recorded Zoom video on Underline.
Don't miss our
#ACL2023
tutorial on Retrieval-based LMs and Applications this Sunday!
with
@sewon__min
,
@ZexuanZhong
,
@danqi_chen
We'll cover everything from architecture design and training to exploring applications and tackling open challenges! [1/2]
6
106
504
5
40
235
Our work on scaling RAG with a 1.4T token corpus was accepted at
@NeurIPSConf
This work led by
@RulinShao
has many interesting findings eg.,
- RAG with massive corpus for better compute-optimal scaling
- thorough analysis on modeling / analysis choices at scale
Check it out!
We’ve known the importance of scaling pre-training data—what if we scale data used at “test time”?
Our new paper shows retrieval-augmented LMs can significantly benefit from our 1.4T token datastore on both up and downstream tasks & lead to better compute optimal scaling curve!
1
13
110
2
31
220
Our BPR (ACL 2021, code is publicly available at Github) substantially reduces the index size (e.g., DPR consumes 65 GB while BPR only uses 2GB) without accuracy drop, and enables us to scale up to *billions of article* 📚
Great blog post on building a search engine using BPR!
A great article on how to construct a 𝘣𝘪𝘭𝘭𝘪𝘰𝘯-𝘴𝘤𝘢𝘭𝘦 vector search engine using
@vespaengine
and our 𝗕𝗣𝗥 model!
1
19
102
1
43
203
Our "Learning to Retrieve Reasoning Paths" paper has been accepted at
#ICLR2020
!!! Joint work with amazing co-authors at
@SFResearch
and
@uwnlp
, Kazuma Hashimoto,
@HannaHajishirzi
,
@RichardSocher
, and
@CaimingXiong
:)
New work with Kazuma Hashimoto,
@HannaHajishirzi
,
@RichardSocher
, and
@CaimingXiong
at
@SFResearch
and
@uwnlp
! Our trainable graph-based retriever-reader framework for open-domain QA advances state of the art on HotpotQA, SQuAD Open, Natural Questions Open. 👇1/7
8
99
408
3
26
179
Knowledge is not uniformly distributed across languages so an ideal open-retrieval QA model should search and retrieve multilingual resources.
We propose a new task 𝐗𝐎𝐑 𝐐𝐀 with a new dataset 𝐗𝐎𝐑-𝐓𝐲𝐃𝐢 QA(40k newly annotated Qs in 7 languages)
👇
4
46
178
Had a great time at the NeurIPS instruction workshop and we are honored to get the Best paper Honorable Mention Award!
Check the details of the paper here:
I will present Self-RAG at the instruction workshop (Room 220-222).
- Poster sessioonn: 1 PM
- Oral talk: 4:50 PM
Come say hi!
#NeurIPS2023
0
9
49
5
10
168
Accepted at ACL2023 🇨🇦 as an oral paper! The updated camera ready version is available at
LLMs hallucinate more in long tails, which scaling may not help while retrieval does. Adaptively combining retrieval w/ LLMs improves performance and efficiency.
Can we solely rely on LLMs’ memories (eg replace search w ChatGPT)? Probably not.
Is retrieval a silver bullet? Probably not either.
Our analysis shows how retrieval is complementary to LLMs’ parametric knowledge [1/N]
📝
💻
15
95
535
0
24
154
Self-RAG is now available on
@llama_index
🥳
Also some update: Self-RAG is accepted as an Oral at
#ICLR2024
, and we are now working to update the draft & improve code bases now. Stay tuned!
Self-RAG in
@llama_index
We’re excited to feature Self-RAG, a special RAG technique where an LLM can do self-reflection for dynamic retrieval, critique, and generation (
@AkariAsai
et al.).
It’s implemented in
@llama_index
as a custom query engine with
6
80
457
4
16
151
I am very excited to announce that new work with
@HannaHajishirzi
has been accepted to
#acl2020nlp
@aclmeeting
.
We investigate the inconsistencies of SOTA QA models' predictions and introduce FOL logic-guided data augmentation & consistency-based training regularization. [1/2]
4
4
141
Super excited that our XOR QA paper has been accepted at
#NAACL2021
🎉🎊
We are updating the paper given the insightful feedback from reviewers. Stay tuned for the final version!
Knowledge is not uniformly distributed across languages so an ideal open-retrieval QA model should search and retrieve multilingual resources.
We propose a new task 𝐗𝐎𝐑 𝐐𝐀 with a new dataset 𝐗𝐎𝐑-𝐓𝐲𝐃𝐢 QA(40k newly annotated Qs in 7 languages)
👇
4
46
178
5
13
136
I will be attending
#NeurIPS2023
in person from Dec 11-15 to present
1. RealtimeQA (Dataset track)
2. Self-RAG (Workshop on instruction tuning )
Come say hi! DM or email me if you wanna chat 💬
0
24
133
If you are planning to apply for CS PhD program this year, you should definitely sign up for PAMS!
You’ll be matched with one CSE PhD student, have zoom meetings with them & get lots feedback on your SOP.
I mentored some via this program, one of whom is now at UW CSE PhD 😎
7
24
134
This work has been accepted at the first
@COLM_conf
!
We introduce a novel taxonomy for diverse LLM hallucination & 1k human annotations for hallucinations of multiple LLM outputs, and train a new RAG for this challenging task (all open sourced).
Work led by
@mishrabhika
🥳
🚨We all complain about LLM "hallucinations", but what are they? We study Automatic Fine-grained Hallucination Detection, with a novel taxonomy, a benchmark, and a 7B LM, surpassing ChatGPT in hallucination detection and editing
9
76
435
0
12
128
One of my favorite posters today. It has so many great Japanese examples 😂
#EMNLP2022livetweet
In Japanese, "me" can be 俺 (masculine and casual), 僕 (less masculine and casual) and 私 (gender-neutral but more often used by women and official)
2
8
128
Knowledge Graphs @ ICLR 2021 by Michael Galkin
A great summary of knowledge graph-related papers @ ICLR2021! Thank you for featuring our MultiModalQA 😉
2
25
123
Our RealTimeQA has been accepted at the NeurIPS (dataset)! Kudos to
@jungokasai
🚀
LLM parametric memories get obsolete quickly. RealTimeQA is one of the earliest work demonstrating the effectiveness of retrieval augmentation + GPT3 to address this.
RealTime QA will be presented at
#NeurIPS2023
(Datasets and Benchmarks Track) in New Orleans! See you there!
2
3
14
2
12
119
Amazing follow-up of Self-RAG by
@ocolegro
, which fine-tunes Self-RAG on top of Mistral-7b. Even stronger performance from the original LLama2-7b-based Self-RAG!
Also, a nice API to retrieve documents and generate using the Self-RAG model is provided!
We encountered issues with accuracy in synthetic data generation. Fortunately, a great approach called self-rag was introduced by
@AkariAsai
et al.
Introducing SciPhi-Self-RAG-Mistral-7B-32k
I fine-tuned the SciPhi model with this and more. The result is powerful + efficient.
4
50
318
5
19
118
Heading to ICLR 2024!
I’ll present Self-RAG on Tue at 4:15pm Oral (Hall A8-9) and help presenting my amazing friends’ work, BTR on Friday
(details in threads 🧵)
Feel free to reach out to me via email or Whova if you wanna chat!
Introducing Self-RAG, a new easy-to-train, customizable, and powerful framework for making an LM learn to retrieve, generate, and critique its own outputs and retrieved passages, by using model-predicted reflection tokens.
📜:
🌐:
24
169
744
4
11
114
We’ve known the importance of scaling pre-training data—what if we scale data used at “test time”?
Our new paper shows retrieval-augmented LMs can significantly benefit from our 1.4T token datastore on both up and downstream tasks & lead to better compute optimal scaling curve!
🔥We release the first open-source 1.4T-token RAG datastore and present a scaling study for RAG on perplexity and downstream tasks!
We show LM+RAG scales better than LM alone, with better performance for the same training compute (pretraining+indexing)
🧵
18
90
389
1
13
110
How can we inject entity knowledge into BERT? In our
#EMNLP2020
work, we introduce a new entity-aware self-attention mechanism and a new pre-training objective, advancing SOTA on many tasks such as Entity Typing / QA / NER. Paper, code, and trained models are now available 👇
Our
@emnlp2020
paper “LUKE: Deep Contextualized Entity Representations with Entity-aware
Self-attention” is now available on arXiv! We present new pretrained contextualized representations that achieve SOTA on five datasets including SQuAD and CoNLL-2003.
1
5
64
0
14
103
SOTA models still struggle in information-seeking QA datasets such as NQ while they suppress humans on many MRC datasets. Our long paper
@aclmeeting
quantifies modeling challenges and suggests several ways to improve future dataset collection. [1/6]
3
16
104
Want to reduce the passage index size for your open-domain QA system? We propose 𝗕𝗶𝗻𝗮𝗿𝘆 𝗣𝗮𝘀𝘀𝗮𝗴𝗲 𝗥𝗲𝘁𝗿𝗶𝗲𝘃𝗲𝗿 (𝗕𝗣𝗥) that learns to hash passage embeddings into compact binary codes (65GB --> 2GB!!🥰) without losing final open QA performance.
#ACL2021NLP
🚀Neural passage retrieval with substantially reduced memory size🚀
BPR presented in our
#acl2021nlp
paper drastically reduces the memory size of the SOTA retriever (DPR) without a loss of QA accuracy
Paper:
Code/Model:
👇Threads
5
40
194
0
17
104
Super excited to host the first large-scale cross-lingual open-retrieval QA shared task in *14* languages (with prizes💰) at
#NAACL2022
MIA!
We release all models&preprocessing codes, training data eg., neg/pos passages to train multilingual retrievers/readers, and results [1/2]
#NAACL2022
We released the test data for our cross-lingual open-domain QA shared task!
All baseline codes, models, training corpora, and intermediate & final results are now available at
Sign up today at !
0
8
16
2
27
100
New paper🚨 dense retriever often suffers from in domain (supervised) and out of domain (zero shot) tradeoff. While prior work tries more complex arch, scaling models or costly data generation, DRAGON trained with progressive multi-teacher training achieves SOTA with Bert-base!
🎉 New Paper 🎉 Introducing DRAGON, our Dense Retriever trained with diverse AuGmentatiON.
It is the first BERT-base-sized dense retriever (DR) to achieve state-of-the-art effectiveness on both supervised and zero-shot evaluations.
Link:
1/7
7
56
286
1
14
97
I'll be attending
#EMNLP2022
in-person from Dec 7-12. Say hi if you're around!
2 of my papers will be presented.
Happy to chat about research (QA, multilingual NLP, retrieval-augmented LMs... etc) as well as PhD applications if you need help :)
🧵
2
1
93
I will be attending the upcoming
#ACL2023NLP
#ACL2023
in person! I will be presenting a tutorial and a paper on retrieval-based LMs. If you have any interest in this area or simply want to chat, please feel free to reach out to me. [1/4]
4
3
92
I'm a big fan of this work! Retrieval-based Multimodal Modeling seems to be quite an exciting area with many interesting open questions & applications!
#ICML2023
Generative AI is hot🔥 We introduce *Retrieval-Augmented* Multimodal Modeling, a new technique that boosts text & image generation accuracy (prevents hallucination) while reducing training cost by 3x!
I’ll be presenting the poster on Tuesday 11am. Happy to chat!!
2
35
208
0
13
91
Just started my summer internship at
@MetaAI
in Seattle with
@scottyih
@riedelcastro
🥳 I’m SO exited to work with fantastic folks at Meta Seattle and London!
If you are in Seattle during summer or coming to Seattle to attend NAACL in person, let's hang out 🏔☀️
1
0
86
2 years ago I changed my major from Econ to EECS. Today I finally graduated, with The Dean’s Award (1st in class) and Best Bachelor Thesis Award. Cannot thank enough for lots of help and encouragement from my family and friends. I’m embarking on my PhD journey this fall in US :)
4
5
82
Thanks,
@jerryjliu0
for highlighting Self-RAG!
Retrieval+LLM pipelines are powerful yet several issues remain (eg irrelevant context, unnecessary/insufficient retrieval, unsupported generations).
More work to address them can further improve retrieval-guided LLM systems.
Advanced RAG architectures need to implement dynamic retrieval 🔎
A *big* downside with naive top-k RAG is the fact that retrieval is static:
⚠️ Always retrieves a fixed number (k) regardless of query
⚠️ What if query requires more context (e.g. summarization) or less context
12
75
457
1
10
82
Did you know that more than 50% of questions were unanswerable in both Natural Questions and TyDiQA?
In a new preprint with
@eunsolc
(), We carefully analyze unanswerable questions in both datasets and attempt to identify the remaining headrooms. 👇 1/3
1
10
80
We've updated the TART paper with some new results
💡 FLAN-T5-based TART-full improves previous TART (T03B)
💡Negative sample ablations: our new instruction-unfollowing negatives help a lot in cross-task retrieval
🤗ckpts:
(code will be available soon!)
New paper 🚨
Can we train a single search system that satisfies our diverse information needs?
We present 𝕋𝔸ℝ𝕋 🥧 the first multi-task instruction-following retriever trained on 𝔹𝔼ℝℝ𝕀 🫐, a collections of 40 retrieval tasks with instructions! 1/N
12
106
578
1
3
77
Cool work benchmarking instruction-following retrieval systems! While our TART-dual () performs well despite only having 110M params (best robustness and second best NDCG among models<7B), still these’s large room for improvements eg instruction sensitivity
Can retrievers follow 📝instructions📝, including your intentions and preferences? 🧐
Introducing INSTRUCTIR, a benchmark for evaluating instruction following in information retrieval. [1/N]
1
37
150
1
9
77
Our 𝕖𝕧𝕚𝕕𝕖𝕟𝕥𝕚𝕒𝕝𝕚𝕥𝕪-guided generator was accepted to appear at the NAACL 2022 main conference:
#NAACL2022
@naaclmeeting
🥳 Huge thanks to my co-authors
@HannaHajishirzi
@nlpmattg
!!
Stay tuned for the final camera-ready version
A powerful retriever+pre-trained generator (eg. DPR+T5) often relies on spurious cues / generates hallucinations.
Our 𝕖𝕧𝕚𝕕𝕖𝕟𝕥𝕚𝕒𝕝𝕚𝕥𝕪-guided generator learns to focus and generate on the right passages and shows large improvements in QA/fact verification/dialogue👇
2
39
249
0
11
76
I was trying to fix the autogenerated captions for my ACL presentations and these are my favorite ones: (1) well, technically my first name sounds like "a curry" in English, (2) ninja training data sounds super cool 🥷
1
0
74
We are proposing a workshop on 𝐌𝐈𝐀: 𝐌𝐮𝐥𝐭𝐢𝐥𝐢𝐧𝐠𝐮𝐚𝐥 𝐈𝐧𝐟𝐨𝐫𝐦𝐚𝐭𝐢𝐨𝐧 𝐀𝐜𝐜𝐞𝐬𝐬 for improving multilingual knowledge-intensive NLP.
If you're interested, we'd love your support in the survey, due November 3rd!
1
10
74
CodeRAG has so much potential yet is underexplored. Our new benchmark and corpora enable rigorous evaluations of code retrieval and RAG! Key takeaway: Even SOTA models like GPT-4 or Claude 3 can significantly benefit from retrieving from open, large-scale corpora as a data store.
Introducing 🔥CodeRAG-Bench🔥 a benchmark for retrieval-augmented code generation!
🔗
- Supports 8 codegen tasks and 5 retrieval sources
- Canonical document annotation for all coding problems
- Robust evaluation of retrieval and end-to-end execution
6
34
146
2
15
73
I’m super excited to give an invited talk virtually at SIGIR workshop REML(retrieval-enhanced machine learning)🎙️on July 27th!
@SIGIRConf
@841io
@bemikelive
@danqi_chen
@MohitIyyer
In the afternoon, REML will host four invited talks by
@AkariAsai
(University of Washington),
@lateinteraction
(Stanford),
@lauradietz99
(UNH), and Anirudh Goyal (University of Montreal).
#SIGIR2023
1
1
5
1
8
71
I’ll be at
#NAACL2022
in-person!
- presenting at Wednesday QA oral session (2:15 PM-3:45 PM)
,
- organizing an workshop on multilingual NLP this Friday
Hit me up! So excited to attend conference in person for the first time!
0
5
70
Introducing Wikipedia2Vec, an efficient and easy-to-use toolkit to learn and visualize the embeddings of words and entities from Wikipedia.
Paper:
Site:
Demo:
#emnlp2020
(demo) led by
@ikuyayamada
[1/3]
2
11
71
An up-to-date curated list of BERT-related papers (downstream task such as QA, probing, multilingual, model compression, domain-specific... etc) by
@stomohide
1
16
67
We're releasing a new multimodal QA dataset, MultiModalQA! MultiModalQA consists of about 30k multimodal questions, some of which require complex multihop reasoning across modalities (image, text, and table). Check the cool video by
@AlonTalmor
and come to our poster
@iclr_conf
!
When was the painting with two touching fingers completed? We present MultiModalQA: Complex Question Answering over Text, Tables & Images!
@iclr_conf
@OriYoran
,
@amnoncatav
,
@LahavDan
,
@yizhongwyz
,
@AkariAsai
,
@gabriel_ilharco
,
@HannaHajishirzi
,
@JonathanBerant
2
29
115
0
9
69
Really cool work on multimodal retrieval! In TART, we propose a new formulation of retrieval with instructions and show the effectiveness of instruction tuning for generalization and alignment w/ users intents. Excited to see it works well in multimodal ✨
We found that the instruction tuning is crucial for not only boosting models' performance but also enabling the model to generalize to unseen information retrieval tasks. The gain is dramatic in most of the tasks. This project was partly inspired by
@AkariAsai
's TART paper.
2
0
15
0
13
64
Hyper collocation is a nice tool to search example sentences from 800k Arxiv papers.
Compared to other engines like Ludwig, it focuses on examples from scientific papers and provides rankings, which might be really useful esp for non-native writers :)
1
6
61
Our first MIA (Workshop on Multilingual Information Access)
#MIA_2022
will be held at NAACL!
We're excited to invite amazing speakers
@aliceoh
@gneubig
@seb_ruder
@aviaviavi__
Holger Schwenk, Kathleen McKeown (and more to come) as well as posters & a shared task.
Stay tuned 🤩
We're excited to announce the 🌎🌟
#MIA_2022
🌟🌍Workshop on Multilingual Information Access will be held at
#NAACL2022
in Seattle!
Shared Task Competition + Speaker info coming soon!
1
8
22
0
16
61
Top Trends of Graph Machine Learning in 2020 by Sergei Ivanov in
@TDataScience
Happened to find our ICLR paper on reasoning path retrieval is mentioned in this great article :) Yes, "Knowledge graphs become more popular"!
0
8
59
I’ll give a remote talk at UCL to present our recent work including our soon-to-be-arXived position paper!
I’ll discuss
1. promises and limitations of current retrieval-augmented LMs
2. how to advance them beyond simple RAG and a longer-term roadmap for further progress
📢
#WITalk
with Akari Asai (
@AkariAsai
)
📕 Reliable, Adaptable, and Attributable LMs with Retrieval
🗓️ 23 February, Friday, 2024 at 4 PM BST
✍️Do not forget to sign up:
#WITalk
@ucl_wi_group
@ai_ucl
@uclcs
0
2
7
2
3
60
Many recent papers evaluate prompted davinci-002/003 or turbo (ChatGPT) on widely-used datasets as zero-or few-shot models. While they are definitely impressive, I've been wondering if they have been trained on such datasets as a part of training...
This is an important effort.
2
5
59
多くの研究論文でも使われているhugginface transformers (私も研究で使っています) の入門コースの日本語翻訳版が公開されています🎉
とてもわかりやすい内容なので自然言語処理をこれから始めたい方は是非チェックしてみてください
Hugging Faceから日本へのお知らせです！
Hugging Faceコースの日本語翻訳を始めました。東北大学のStudent Ambassadorsの皆さんのお陰で第一章の翻訳が終了しました。
今後もコツコツと翻訳していきます。
是非コースを読んでHugging Face Tranformersについて学んで、使ってみてください！
2
230
807
0
17
58
XOR QA
@NAACLHLT
2021 camera-ready version is online!
Website (&data):
Code:
We did some minor changes to the XOR-TyDi QA data and released a new version (v1.1). Please check&download the updated version 😉
Knowledge is not uniformly distributed across languages so an ideal open-retrieval QA model should search and retrieve multilingual resources.
We propose a new task 𝐗𝐎𝐑 𝐐𝐀 with a new dataset 𝐗𝐎𝐑-𝐓𝐲𝐃𝐢 QA(40k newly annotated Qs in 7 languages)
👇
4
46
178
1
14
56
Really important question! Multilingual datasets translated from English datasets are common, which often don’t reflect native speakers’ interests / linguistic phenomena.
Still many languages often only have translated data only (our survey on this )
Translating benchmarks has become an in-vogue way to evaluate non-English capabilities of language models (see, for example,
@MistralAI
's latest model …) but people who are using a LLM in another language probably have different concerns and interests.
2
2
16
2
6
53
We will present this work tomorrow, Tuesday at 17:00-19:00 GMT / 10:00-12:00 PDT (Tuesday session 4)& 20:00-22:00 GMT / 13:00-15:00 PDT (Tuesday session 5)!
Please come say hi to our virtual poster session :)
#ICLR2020
Our
#ICLR2020
camera-ready version, code, and blog are now available!
paper:
code:
blog:
You can train, evaluate, and run an interactive demo on your machine. We also release the models for reproducibility.
2
49
240
2
6
55
学部生での研究インターンでの学びと現在の博士課程留学 (MSR Internship アルムナイ Advent Calendar 2020)
@msraurjp
さんに声をかけていただき、学部4年次のMicrosoft Research Asiaでのインターンと博士課程留学の最初の1+α年振り返りを含めたブログ記事を書きました。
0
3
53
I will present Self-RAG at the instruction workshop (Room 220-222).
- Poster sessioonn: 1 PM
- Oral talk: 4:50 PM
Come say hi!
#NeurIPS2023
Introducing Self-RAG, a new easy-to-train, customizable, and powerful framework for making an LM learn to retrieve, generate, and critique its own outputs and retrieved passages, by using model-predicted reflection tokens.
📜:
🌐:
24
169
744
0
9
49
We are presenting at 1913!
1
14
105
1
2
47
As shown in the example, Self-RAG can perform diverse tasks / follow instructions as in standard instruction-tuned LMs + incorporate external knowledge when it enhances factuality & quality.
Model checkpoints are available at:
Self-RAG enables fine-grained on-demand control over the retrieval component, and it can be used together with instruction tuning to boost the factuality of a general-purpose model!
Please check this great work led by
@AkariAsai
!
0
3
37
0
8
44
This Friday I will give a talk about our
#NeurIPS2021
paper, “One Question Answering Model for Many Languages with Cross-lingual Dense Passage Retrieval”
Come join us :D
🚨 Only two more days till our next SEA meetup w/
@AkariAsai
(U. Washington) and Arnold Overwijk (Microsoft) about Dense Passage Retrieval!
Don't forget to sign up on our Meetup page to receive the Zoom link:
See you on Friday! 😁
0
2
13
2
6
45
This is a great news but the example is a bit wrong (but interesting).
するます is grammatically incorrect as when followed by an auxiliary verb ます (often added for politeness), する must be conjugated to し, so it should be します.
I wonder if it’s common in MT to Japanese…
Last week,
@MetaAI
introduced NLLB-200: a massive translation model supporting 200 languages.
Models are now available through the Hugging Face Hub, using 🤗Transformers' main branch.
Models on the Hub:
Learn about NLLB-200:
6
140
473
1
7
45
とても尊敬する友人の
@00_
がアメリカ大学院出願に至る敬意を書いてくれています。海外大学院博士課程留学を目指している方もそうではない方も、是非読んでみてください。
「私生活を多少犠牲にし、同級生が遊んだり、就活や院試をしている間も研究や勉強をする決意をしてください。」
留学を志した経緯を大学入学時から振り返りながら書きました。ご笑覧ください。
Posted to Hatena Blog
アメリカ博士課程留学 − 立志編 - yamaguchi.txt #はてなブログ
7
362
2K
0
3
44
博士課程の入学審査の時期ですが、毎年日本からの出願はかなり少ない印象を受けます。
出願に向けた準備（早い時期からの研究含め）は多くの労力を要しますが、所属学部での研究環境や機会など、それに見合う大きなリターンがあると感じています。
もう少し日本からの出願が増えるといいのですが…
1
3
44
明日の11時からAI王最終報告会で講演させていただきます。多言語での情報検索・質問応答を可能にするための大規模データセット・モデルの設計や今年NAACL2022ワークショップで行ったコンペティションの結果等についてお話しさせていたただく予定です。
12/2(金)に開催されます，第3回 日本語質問応答コンペティション
#AI
王 では，浅井 明里 氏（University of Washington)から『より多くの言語での情報検索・質問応答を可能にするために』の招待講演を頂きます．貴重な機会ですので是非ご参加ください！
事前登録はこちら：
1
5
10
0
8
43
𝐋𝐔𝐊𝐄, our deep contextualized word&entity representations, is now at huggingface transformers🤗
You can reproduce the SOTA results on tasks like Open Entity easily at the colab notebooks
Paper&Video:
LUKE has just been added to the master branch of Huggingface Transformers!🎉🎉 It should be included in the next release of Transformers! Huge thanks to
@NielsRogge
and
@huggingface
folks!
1
13
69
0
10
42
昨年RITで行なっていたプロジェクトをまとめたものが主著論文として自然言語処理の国際会議のLREC 2018にアクセプトされました
0
2
42
I found this is a great resource for those who are interested in or working on open-domain QA! You can learn the SOTA open-domain QA systems and smart techniques to reduce the model size. I also really enjoyed the analysis section and future suggestions on QA evaluations 😀
Happy new year!
#NeurIPS2020
EfficientQA organizers, together with participants, wrote a paper that includes systems, analyses, and lessons learned from the competition.
Thanks to everyone who took part in it!
2
35
122
0
3
40
IMO the scaling effects on factual knowledge memorization are interesting.
It helps to memorize popular concepts🟥, while the performance on less popular ones🟦 is almost plateau.
This may explain why even 100+B LLMs still struggle to memorize non-general (eg science) concepts.
Surprisingly, in long-tail distributions, scaling LLMs may not be as helpful as we believed: GPT-3 003 performs nearly as poorly as GPT-Neo 2B on less popular entities 🟦
Prior knowledge learning analysis often uses NQ/TriviaQA 🟥 may inflate the effectiveness of scaling [5/N]
1
1
34
2
6
41
ACL2020: General Conference Statistics
@aclmeeting
"To show how much our field has grown, ACL 2002 received 258 submissions total across all tracks." 😲
1
7
39
Being part of this project was a great experience! We developed AfriQA - the very first large-scale open QA dataset for 10 African languages, using our XOR QA (NAACL 2021) scheme, where a system is required to search and answer cross-lingually to overcome information scarcity.
1
4
39
Finally Japanese GLUE (JGLUE)!!🤩 This is a great effort by Yahoo Japan and Waseda University:
大変遅くなりましたが、ヤフーと早稲田大河原研の共同研究で構築した日本語言語理解ベンチマークJGLUEのv1.0を公開しました。
日本語自然言語処理における評価セット不足を少しでも解消できるように���築しましたので、広くご利用いただけると幸いです。
0
132
435
2
3
37
Our Q&A session is today at 11AM-12PM and 1-2 PM PDT (6-7 PM and 8-9PM GMT). I'll stay there between two sessions as well to chat about any topics (e.g., QA/MRC, KB, grad school applications) over lunch. Please come join us :)
#acl2020nlp
#acl2020en
New
#acl2020nlp
paper "Logic-Guided Data Augmentation and Regularization for Consistent Question Answerings"!
We show SOTA QA models produce inconsistent predictions and introduce logic-guided data augmentation & consistency-based regularization.
1/
2
47
265
1
3
36
I went to a MLB game for the first time and I’m so happy to see his MLB-leading home run
#33
with my own eyes 🤩
Definitely he’s one of the best Japanese-born MLB players ever
#ShoheiOhtani
3
0
36
言語処理学会年次大会にて若手奨励賞を受賞しました。学部3年次の演習をきっかけに取り組んできた「実行時機械翻訳による多言語機械読解」の研究によるものです。
原稿:
発表スライド:
英文原稿:
0
2
35
Surprisingly, in long-tail distributions, scaling LLMs may not be as helpful as we believed: GPT-3 003 performs nearly as poorly as GPT-Neo 2B on less popular entities 🟦
Prior knowledge learning analysis often uses NQ/TriviaQA 🟥 may inflate the effectiveness of scaling [5/N]
1
1
34
We will present CORA, our multilingual MANY-to-MANY Open QA model, at NeurIPS Poster Session 1, 8:30 am - 10 am tomorrow. Come say hi!
@HannaHajishirzi
@wittgen_ball
@XinyanVYu
Can we build a *single* open-domain QA model that works in *many* languages?
We’re excited to present 𝗖𝗢𝗥𝗔 using a single retriever and generator, showing SOTA results in 26 diverse languages on XOR QA & MKQA, including the unseen languages. [1/5]
3
60
263
0
5
33
Back in 2020, we started working on cross-lingual semantic retrieval+generation to overcome information scarcity in many languages.
(XOR QA)
(CORA)
Excited to see more work on cross-lingual semantic retrieval coming out recently!🌍
0
0
33
@_lewtun
@AiEleuther
We have instruction-tuned LMs that learn to retrieve *documents* following users' instructions (e.g., retrieve corresponding code/an answer to a technical question)
paper:
GitHub:
2
4
32
Another really helpful resource for CS Ph.D. application is cs-sop, where more than 100 successful applicants’ SoP across many areas, years (2018-) and universities are available.
Created by
@zhaofeng_wu
@alexisjross
@shannonzshen
!
3
7
34
7/7@東大の海外大学院留学説明会で登壇します(思い出深いの工学部2号館)😀
最近の研究とかPhD5年間振り返って思うことなどを多分色々話すと思います！
ぜひ来てください〜
拡散希望📣
7月7日(日)に東大で海外大学院留学説明会を開催します！！！
かっこいい先���たち4名に受験・留学中の生活・留学後のキャリアなどについてお話いただきます🔥
私は当日司会を担当するので何でも聞きにきてください🙌
ホームページから事前登録お願いします！
0
18
122
0
3
32
Our
#emnlp2022
(findings) paper conducts meta survey on hundreds of multilingual datasets, to understand the landscape, methodologies and bottlenecks. Super interesting findings from meta survey, meta analysis and MTurk experiments.
A great summary by amazing
@XinyanVYu
⬇️
New paper 🚨
We conduct the first meta survey on 156 labeled and publicly-available multilingual NLP datasets, and we find that simply counting # datasets in a target language can be misleading especially in low-represented languages. (1/N)
2
27
118
1
6
30
Many schools offer similar programs for CS Ph.D. applicants, particularly those from underrepresented groups!
- MIT:
- CMU:
- UCSB:
- Stanford:
2
4
30