@nickcdryan The authors responded, it is indeed that the model did not output either yes or no. Strange indeed

@nickcdryan @omouamoua @stochasticchasm Great to hear! We are currently working on a multilingual version of CUTE, release date still TBD :)

@nickcdryan I was also wondering this, I thought maybe their model didn't output either "yes" or "no". I'll try emailing the authors

Thanks to everyone who stopped by my poster at #EMNLP2024 today! Had some really fun and interesting discussions with you all :)

I'm happy to announce that our LLM benchmark for measuring orthographic understanding, CUTE, was accepted to EMNLP main! Looking forward to seeing you all in Miami! #NLProc Arxiv: HF Leaderboard: Github:

RT @gsarti_: Very excited to present my work at @CisLmu tomorrow; thanks for having me! 😄

@vansteenkiste_s @jowenpetty We compared char-level to subword-level for NMT, and found the char models are generally on-par or better, and show word-level processing capabilities when needed. Indeed, the major issue seems to be that inference is ~5x slower. Link:

@AnthropicAI Proud to have grown up in a rabid wasteland

In case you missed it, you can check out the paper and models at and feel free to reach out!

@diwuNLP @gsarti_ @_atoral @gertjanvannoor1 @AriannaBisazza Thank you! I think from-scratch char-level models are still very useful for certain tasks (e.g. anything with morphology), but for translation they will perform similarly to subword-level models.

RT @gsarti_: After a year of restless development, I'm finally happy to announce Inseq, a new tool to democratize post-hoc interpretability…

Thanks everyone for coming to my poster at #EMNLP2022! Great to hear all your thoughts and to meet so many people that are interested in character-level #nmt and #nlproc :)