Can #LLMs truly reason over loooong context? 🤔 NoCha asks LLMs to verify claims about *NEW* fictional books 🪄 📚 ⛔ LLMs that solve needle-in-the-haystack (~100%) struggle on NoCha! ⛔ None of 11 tested LLMs reach human performance → 97%. The best, #GPT-4o, gets only 55.8%.

RT @aclanthology: If you're working on an ACL paper and citing any papers with extremely long author lists, consider updating your acl_natb…

Test-time scaling has lead to seemingly better #llms such as #o1 or #r1. BUT the models remain vulnerable to prompt injection attacks. ➡️Injecting decoy tasks into the prompt significantly increases reasoning tokens without affecting answer accuracy. This means HIGHER cost💸& MORE computing resources 🌎😩& MORE time ⌛️to get THE SAME results!

RT @JaechulRoh: 🧠💸 "We made reasoning models overthink — and it's costing them big time." Meet 🤯 #OVERTHINK 🤯 — our new attack that forces…

RT @mykocyigit: Thrilled to share our latest findings on data contamination, from my internship at @Google! We trained almost 90 Models on…

Application link can be found at:

RT @mjpost: The Microsoft Translator group is looking for a Ph.D. student intern this summer to work with us in Redmond on machine translat…

RT @TuhinChakr: Really cool work on AI and writing with some great findings 1) Experts are infinitely better at detecting AI generated slop…

@srivatsamath @jxmnop seems so, check out the paper :)

Congrats @jennajrussell on the first paper!

@yoavgo i think i prefer benchmarks which tells us something about the models even if/when they saturate (other than we asked such a niche question that the models failed)

RT @MohitIyyer: Please consider submitting your research to our upcoming Workshop on Narrative Understanding! Deadline Feb 17.

RT @MaartenSap: CMU LTI is hosting predoc interns this summer, centered around "Language Technologies for All"! Please apply and circulate!…

RT @TuhinChakr: If you are looking to do a PhD in Generative AI , Creativity and Human Behavior , please apply to @sbucompsc PhD program by…

RT @jacobandreas: Are you an undergrad interested in NLP research? Intern with us through the MIT summer research program! Includes stipend…

@EhudReiter reminds me of this paper:

RT @EhudReiter: New blog: We need better LLM benchmarks Current benchmark (suites) for evaluating LLMs are disappointing. I describe the p…

@EhudReiter 3. also the pairwise design showed not only which llms have label bias, but also that a model can validate one claim but then totally contradict itself (hence shouldn't be given credit)